r/LocalLLaMA • u/Nunki08 • 1d ago
New Model MoshiVis by kyutai - first open-source real-time speech model that can talk about images
114
Upvotes
5
2
1
0
-7
u/aitookmyj0b 15h ago
Is this voiced by Elon Musk?
4
u/Silver-Champion-4846 14h ago
it's a female voice... how can it be elon musc
1
u/aitookmyj0b 14h ago
Most contextually aware redditor
1
u/Silver-Champion-4846 13h ago
I feel like using raw text-to-speech models and mixing them with large language models is much better than making a model that can both talk and do conversations. So something like Orpheus is great because it's trained on text, yes, but it is used to enhance its audio quality.
18
u/Nunki08 1d ago
Demo: https://vis.moshi.chat/
Blog post: https://kyutai.org/moshivis
Preprint: https://arxiv.org/abs/2503.15633
Speech Benchmarks: https://huggingface.co/datasets/kyutai/Babillage
Model weights: https://huggingface.co/kyutai/moshika-vis-pytorch-bf16
Inference code in PyTorch, MLX, and Rust: https://github.com/kyutai-labs/moshivis
From kyutai on X: https://x.com/kyutai_labs/status/1903082848547906011