Real-Time Voice Coaching: Holding a Conversation Inside 2.5 Seconds
Speech to text to model to speech, in a loop tight enough that users forget they are talking to software. Every architectural decision was made against a latency budget.
- Client
- Consumer mobile app
- Industry
- Consumer mobile
~1–2.5s
End-to-end response
30–50%
Faster session completion
Hands-free
Full conversational loop
Stateful
Structured dialog control
The problem
The app's coaching experience was typed, which meant users could not use it while doing the thing they were being coached through. Voice was the obvious answer and the reason it had not been built: a voice loop that is too slow is worse than typing, because the silence is excruciating and users abandon mid-session.
The target was a conversation that felt like a conversation. In practice that means the gap between a user finishing a sentence and hearing a reply has to stay inside a couple of seconds, consistently, on a phone, on real networks.
The latency budget
We designed against a budget rather than optimising afterwards. Speech recognition, model inference and speech synthesis each got an allocation, and any design that could not fit was rejected before it was built.
The wins came from overlap rather than raw speed. Transcription streams rather than waiting for the user to stop talking, generation begins on partial input where the intent is already clear, and synthesis starts on the first sentence rather than the complete response. Users hear a reply beginning while the rest is still being produced, which is what makes 2.5 seconds feel like a pause rather than a wait.
Keeping the conversation stable
Latency is only half the problem. Free-form voice conversations drift — the model wanders off the coaching structure, loses the thread, or produces something too long to listen to.
We held the session in explicit dialog state rather than in the transcript, so the system always knows where it is in the coaching flow and what it is trying to accomplish. Prompt controls cap response length, because a paragraph that reads fine takes forty seconds to hear.
Results
End-to-end responses land in roughly one to two and a half seconds, which was the threshold for the interaction feeling conversational rather than transactional.
Drop-off caused by delayed replies fell substantially, and users completed sessions 30–50% faster than in the typed experience — partly because speaking is faster than typing, and partly because they stopped stopping.
What we'd flag
These numbers are network-dependent in a way that server-side benchmarks hide. We tested on real mobile connections including degraded ones, and the experience needs a designed fallback for when the budget cannot be met — silence with no explanation reads as a crash.
Voice also exposes model verbosity in a way text does not. Response length limits are a user experience requirement here, not a cost optimisation.
