The problem
Text chatbots stall the moment a task needs back-and-forth, and stitching speech onto them usually adds seconds of latency that kills the conversation.
The approach
- Streamed audio both directions over the LiveKit SDK to keep turn-taking natural.
- Ran ElevenLabs for voice synthesis and cloning, with Gemini handling voice and video analysis.
- Built AI personas on OpenAI with per-persona context and memory.
- Throttled and batched model requests to hold latency down under concurrent load.
The outcome
Users hold real spoken conversations with an agent that answers in near real time instead of typing at a chat window.