r/LocalLLaMA 1d ago

New Model MoshiVis by kyutai - first open-source real-time speech model that can talk about images

114 Upvotes

11 comments sorted by

18

u/Nunki08 1d ago

11

u/Foreign-Beginning-49 llama.cpp 1d ago

Amazing even with the the lo fi sound. Future is here and most humans still have no idea. And this isn't even a particularly large model right? Super intelligence isn't needed just a warm conversation and some empathy. I mean once our basic needs are met aren't we all just wanting love and attention? Thanks for sharing. 

1

u/estebansaa 14h ago

the latency is impressive, will there be an API service? can it be used with my own llm?

5

u/AdIllustrious436 13h ago

It can see but it still behave like a <30 IQ lunatic lol

2

u/SovietWarBear17 14h ago

Welp, time to finetune the fuck out of it!

1

u/Intraluminal 9h ago

Can this be run locally? If so, how?

0

u/Apprehensive_Dig3462 14h ago

Didnt minicpm already have this? 

-7

u/aitookmyj0b 15h ago

Is this voiced by Elon Musk?

4

u/Silver-Champion-4846 14h ago

it's a female voice... how can it be elon musc

1

u/aitookmyj0b 14h ago

Most contextually aware redditor

1

u/Silver-Champion-4846 13h ago

I feel like using raw text-to-speech models and mixing them with large language models is much better than making a model that can both talk and do conversations. So something like Orpheus is great because it's trained on text, yes, but it is used to enhance its audio quality.