Why model speed is only part of voice latency

There's a moment on every call that decides whether the thing you're talking to feels alive. You stop speaking. There's a gap. If it's short, the conversation just flows. If it stretches, you start wondering whether the line dropped, and you say "hello?" straight over the top of the reply.

That gap is the most important number in voice AI, and the most misunderstood. Almost every conversation about it opens with the same question: which model is fastest? Wrong question. Not because model speed doesn't matter, but because on a real call the model is rarely what's keeping the caller waiting.

First, what "fast" actually means

People reply to each other in roughly 200 to 500 milliseconds. That isn't a target someone invented, it's how human turn-taking works, fairly consistently across languages. Under 200ms you're interrupting. Past a second, the other person thinks something has gone wrong.

Where the time actually goes

When a caller stops speaking, four things happen before they hear anything:

Most people assume the thinking dominates. It doesn't.

That step sounds trivial and isn't. People pause mid-sentence constantly. They hesitate, they say "ummm", they stop while digging out a customer number. Treat every pause as an ending and the agent talks over people all day. Wait long enough to be certain and every single turn feels sluggish.

The other steps are cheaper than you'd guess. Transcription runs while the caller speaks, so barely any audio is left when they stop. And agents repeat themselves constantly, so most spoken responses come from a cache rather than being synthesised fresh.

The swap test

Here's the part that surprises people. If waiting dominates, a faster reasoning step is optimizing the wrong thing. You don't win this race by running it faster. You win it by running a shorter one.

Architecture changes the critical path

Three broad architectures, and what separates them is how many things must finish before the next can start.

Integrations are part of the latency budget

Here's the gap in most latency work. The platform gets measured carefully. The systems it calls during a turn don't. Most turns trigger a tool call. Some stay inside the platform and hide under the turn. Others leave to ask your systems something: a booking lookup, an account check, an eligibility test. Those are usually the single biggest contributor to a slow moment, and no model architecture touches them. A slow lookup delays an audio-native model exactly as much as a cascaded one.

The phone network adds its own delay

A voice app over the internet takes one fairly direct path. A phone call crosses carrier networks, wholesale routes and cloud telephony, and every hop buffers. Routes are typically chosen per phone number , so an unlucky number can pay a bad-route penalty on every call, permanently, until somebody notices.

Distance behaves differently on the two. Over IP it's close to physics: a continental crossing costs a few tens of milliseconds and stays predictable. Over the phone network it compounds, because crossing regions means more carrier handoffs, more buffering, often transcoding on the way. So geography isn't irrelevant. It's cheap on the internet and expensive on the phone network.

Three practical things follow. Terminate the call in the same region as the agent wherever you can. Cut the middlemen out of the telephony path, because every carrier, aggregator and handoff in the chain is another buffer. A call passed across several hops pays for all of them, on every turn, for the life of the deployment. And when one is slow, check the route on the number before you go looking at the data center.

Averages hide the calls people remember

Stack the three layers together and you can ask the honest question. Not "how fast is it?" but "how fast is it across a hundred calls?" Those are very different questions.

Nobody experiences an average. They experience the call they're on. A single latency figure, quoted without the distribution behind it, describes the calls nobody was going to complain about anyway. Always ask what shape sits behind the number.

Why more steps makes the tail worse

The architectural part is really just arithmetic. In a pipeline where each stage waits for the last, every stage gets its own chance at a bad moment. A slow transcription, a slow first token, a cache miss. Those chances add up. Four stages that are each occasionally slow produce a slow turn far more often than one stage that is. That holds even when every stage is well engineered.

Each stage, in other words, rolls its own dice. Remove stages and you remove dice.

This is the practical reason architecture beats model speed. A faster component narrows one distribution slightly. Removing a stage removes a whole source of variance, so the typical case improves a little and the worst case improves a lot.

How Dialog-RSN-1 changes the path

So we stopped making the pieces faster and changed the shape instead. Dialog-RSN-1 is our audio-native dialog model. It reasons over the call audio directly, so there's no transcript to wait for. It starts forming a response while the caller is still speaking. And it senses the turn ending as part of listening, instead of running a separate detector in front of the model. That last one matters most. It's the segment that dominated every chart above.

The thinking step isn't quite untouched either. Dialog-RSN-1 reasons automatically : it decides per turn whether a question actually needs a reasoning step, instead of spending one on every reply. That's still uncommon in audio-native models. And we keep that reasoning short on purpose, because a model that deliberates over "what's my balance" would quietly hand back the time the new shape just saved.

What it deliberately doesn't do is take over the voice. Dialog-RSN-1 reasons over audio but answers in text , and that one choice keeps the rest of the stack intact. Text is controllable. You can read it, log it, test it, and block it before anyone hears it. Every enterprise guardrail we run depends on exactly that, so all of them still apply, unchanged. Nothing had to be rebuilt or quietly relaxed to fit a new model.

The voice stays yours too. The same voice layer speaks every reply. You keep the voice you picked for your brand, instead of inheriting whichever one a speech-to-speech model ships with. The response cache keeps working for the same reason: there's still a text answer to cache. Those are the things a full speech-to-speech model asks you to give up. We didn't want to.

What this looks like in production

We run all three of these architectures in production. What follows is a sample of real traffic, drawn across a range of deployments over an extended period, not a benchmark assembled for the occasion. Everything is measured end to end, from the caller's voice leaving their mouth to the agent's voice coming back.

Read it as a sample, not a scorecard. Any deployment can land either side of these lines. It depends on the integrations behind it, the route its calls take, and how the agent is built. What makes it worth showing isn't the absolute numbers. It's that every sample sits on the same platform and behind the same voice layer, so architecture is the thing that varies.

The pattern is the one the whole piece points at. Fewer stages doesn't just shift the typical call left, it pulls the worst case in much harder. And the channel moves every row. The same model over a phone call pays a consistent penalty against itself in an app. And a turn waiting on an external API pushes the whole distribution right, whichever model sits behind it.

One caveat on the Realtime API figures. These are measured against the public API endpoint, and they probably understate what that model can do. Consumer apps on the same family feel quicker, plausibly nearer 500ms, most likely because they run on different serving tiers than the general API. The point isn't that a model is slow. It's that what you can build on a public API isn't always what you experience in the first-party app. That gap belongs in your latency budget from the start, along with the reliability of any third-party dependency. It's also why we're working with cloud distribution partners to make Dialog-RSN-1 deployable inside your own environment. Data residency aside, putting the model next to your own systems removes network legs from every turn, and removing legs beats speeding them up.

What to ask a vendor

Whoever you end up building with, us included, these are the questions that separate a real answer on latency from a marketing one.

None of these are about which model is fastest. That's the point. Speed on a phone call is a property of the whole system, and the biggest wins come from removing steps rather than accelerating them.

Existing PolyAI customers can enable Dialog-RSN-1 in their projects today. New customers can request early access .