That was interesting. I asked it to try to say something in another language, and she read it in a thick American accent. No surprise. Then I asked her to sing, and she said something like "asterisk in a robotic singing voice asterisk...", and then later explained that she's just text to speech. Ah, ok, that's about what I expected.
But then I asked her to integrate sin(x) * e^x and got this bizarre answer that started out as speech sounds but then degenerated into chaos. Out of curiosity, why and how did she end up generating samples that sounded rather unlike speech?
This is pretty amazing. It's fast enough to converse with, and I can interrupt the model.
The underlying model is not voice trained -- she says things like "asterisk one" (reading out point form) -- but this is a great preview for when ChatGPT GAs their Voice Mode.
Fantastic demo. Do you know what's the difference between your stack and the livekit demo? [1] it shows your voice as text so you can see when you have to correct it.
Llama3 with ears just dropped (direct voice token input) which should be awesome with cerebras [2]
Zoom does use a custom protocol. This is why it doesn’t work nearly as well when you take a call in the browser client. Not because WebRTC isn’t up to the task, but because Zoom hasn’t invested in it.
Ignoring costs, while having someone host infra for you will always be easier than managing it yourself, I think we’ve really improved the DX of hosting your own WebRTC infra with LiveKit: https://github.com/livekit/livekit
HLS is a client-driven protocol, so it has high-scale but variable latency. You could build a mostly one-way webinar experience using it, but definitely not a conference call experience.
The primary issue with traditional WebRTC media servers and services is they didn’t horizontally scale. That’s changed recently. You can get pretty high numbers of users in a single WebRTC session now.
It really depends on the use case. In vanilla WebRTC, all media is transmitted directly between peers. In practice, this doesn’t scale beyond 5-10 users in a session. Most home internet connections can’t sustain that amount of upstream bandwidth.
There’s a big graveyard of products/companies that have tried to kill Larry over the years.
Meta has been trying since 2009. Back then, a former, well-known Facebook employee once told me not to join Twitter (thankfully, I didn’t listen). He said they, “have a wall at the office with a list of all the things Twitter does well. Every week someone checks another item off. We’re going to kill Twitter.”
This moment is probably Meta’s best chance. I’d say, if it doesn’t happen this try, it probably never will.
Here’s an AI voice assistant I built that uses it:
https://cerebras.vercel.app