Full-duplex voice agents: turn-based vs full-duplex, models, testing

Most voice agents still talk in strict turns. You speak, they wait for silence, they reply. Real conversations do not work that way. People interrupt, overlap, and murmur "mm-hm" mid-sentence. Full-duplex voice AI closes that gap by modeling both sides of the conversation at once.

That shift is now real. OpenAI launched GPT-Live in ChatGPT in July 2026 and in the API in September. Open models like Moshi and PersonaPlex ship weights. Yet 2026 benchmarks agree: these agents feel more natural, but task success and safety do not come free. This guide covers how they work, which options exist, the trade-offs, and how to evaluate them.

Turn-based vs full-duplex conversational AI: what is the difference?

Turn-based conversational AI waits for the caller to stop, then speaks. Full-duplex conversational AI keeps listening while it talks, so it can overlap, backchannel, and yield mid-sentence. The real difference is who decides when a turn ends, and how. A turn-based agent uses a detector. The detector waits for silence, or a semantic end-of-utterance guess, then hands the floor to the model. A full-duplex agent has no separate gate. The model itself hears the caller while speaking and chooses its next move frame by frame.

Human conversation sets the bar here. A cross-language study in PNAS by Stivers and colleagues found turn transitions cluster around very short gaps, typically within a few hundred milliseconds. Silence-based detectors usually wait longer than that by design. That is why turn-based agents often feel a beat slow, even when their raw model latency is low.

What is a full-duplex voice agent?

Full-duplex voice agent: a voice agent that processes incoming caller audio continuously while it generates its own speech. It decides many times per second whether to speak, keep listening, pause, or yield. It does not wait for a turn boundary.

The term comes from telecommunications. A full-duplex) link lets both ends transmit at once. A phone call is full-duplex. A walkie-talkie is half-duplex. In voice AI, the term describes the model's behavior, not the phone line. The question is whether the agent actually listens while it talks.

Keep three architectures apart. Blurring them leads to bad buying and testing decisions.

The middle row matters most. A speech-to-speech model is not automatically full-duplex. OpenAI's own GPT-Live launch post makes this distinction. It describes Advanced Voice Mode as a single audio model that "still operated through discrete turns." Silence-based turn detection meant a brief pause or background noise could trigger an unwanted reply.

Barge-in also exists outside full-duplex. A cascaded agent with voice activity detection can stop talking when the caller speaks. That is half-duplex with an interrupt switch. It usually cannot tell a real interruption from "yeah, go on." For pipeline options, see cascading vs speech-to-speech voice agents and the voice agent stack guide.

Inside a full-duplex model

Full-duplex models differ in detail, but the published designs share a core idea. They model the caller and the agent as two parallel, time-aligned streams. Kyutai's Moshi is the clearest public example, because the Moshi paper and GitHub repository document it fully.

Dual audio streams

Moshi models two streams of audio at once. One stream is the user. The other is Moshi itself. At inference, the user stream comes from the microphone. The Moshi stream is sampled from the model. Because both streams exist at every time step, there is no explicit speaker turn. Overlap, silence, and interruption are just patterns in the two streams.

Streaming audio tokens

Audio becomes tokens through Mimi, Kyutai's streaming neural codec. Per the repository, Mimi compresses 24 kHz audio to a 12.5 Hz token rate at 1.1 kbps. Each frame covers 80 ms. A small Depth Transformer models codebooks within a frame. A 7B-parameter Temporal Transformer models time. Moshi reports a theoretical latency of 160 ms, and about 200 ms in practice on an L4 GPU.

Inner monologue

Moshi also predicts time-aligned text tokens for its own speech before the audio tokens. Kyutai calls this the "inner monologue." It improves the linguistic quality of the generated speech. It also yields a free, word-aligned transcript of what the agent said. That transcript matters for evaluation, as you will see later.

Interaction layer plus delegation

Commercial systems add a second layer. OpenAI describes GPT-Live as two pieces. A continuous interaction model handles listening and speaking. A backend model handles deeper reasoning, search, and tools. At ChatGPT launch the backend was GPT-5.5. In the API you can pair GPT-Live-1 with GPT-6 Astra or a third-party model. The voice model can keep talking while the backend works.

So conversation timing lives in one model, and business logic lives behind it. That means two failure surfaces instead of one.

What full-duplex makes possible

The Full-Duplex-Bench paper (ASRU 2025) frames full-duplex behavior as four measurable skills. They are a useful checklist for any agent.

• Pause handling: staying quiet when a caller pauses mid-thought, instead of jumping in.
• Backchanneling: producing "mm-hm" or "got it" without taking the floor, and not stopping when the caller does the same.
• Smooth turn-taking: starting a reply quickly once the caller has actually finished.
• Interruption management: yielding fast when a caller barges in, then responding to the new content.

Two more behaviors show up in newer work. The first is in-turn adaptation. The Duplex Cue study (September 2026) argues "keep talking or stop" is too narrow. Humans often continue speaking while folding in what the listener just said. The second is tool use under disfluency. Full-Duplex-Bench-v3 tests chained API calls when callers hesitate and self-correct.

The business case is real but narrow. OpenAI reports that Speak saw interruptions during learners' thinking pauses drop by almost 80% versus earlier turn-based systems. That is a vendor-reported customer result, not an independent audit. Treat it as a signal, then measure your own calls.

Which voice APIs support full-duplex communication in 2026?

This is the most common question we see in search. The honest answer: few production APIs are natively full-duplex today. Several are fast speech-to-speech systems with configurable turn detection. The table reflects each vendor's own documentation as of September 2026.

On GPT-Live-1, the getting-started guide lists WebRTC, WebSocket, and SIP connections. It also lists partner paths for LiveKit, Twilio, Telnyx, and Daily/Pipecat. OpenAI also notes that GPT-Live-1 "natively supports turn detection." You can keep building around explicit turn boundaries while migrating.

On Grok, searchers often ask about "grok speech to speech." The xAI Voice Agent docs describe a real-time speech-to-speech API with `server_vad` turn detection. The `grok-voice-latest` alias currently points to `grok-voice-think-fast-2.0`. That is a turn-detection design, not a documented dual-stream full-duplex model. It can still handle barge-in well; test it rather than assume. See our xAI Grok voice agent guide for setup detail.

On open models, PersonaPlex builds on the Moshi architecture and weights. It adds text role prompts and audio voice conditioning. Open models are ideal for research and on-prem experiments. They lack the tool delegation and telephony plumbing that production agents need.

The trade-offs nobody puts on the demo slide

Full-duplex improves feel. It does not automatically improve outcomes. Five trade-offs show up repeatedly in 2026 research and docs.

Tool calling and task success

The tau-voice benchmark (March 2026) tested voice agents on 278 grounded tasks. A text GPT-5 reasoning baseline reached 85%. Voice agents reached 31–51% in clean audio and 26–38% with noise and accents. The authors attribute 79–90% of failures to agent behavior. FDB-v3 found GPT-Realtime led its field at Pass@1 of 0.600. The cascaded baseline had a perfect turn-take rate but the worst latency, 10.12 seconds. Self-correction handling and multi-step reasoning failed most consistently across all systems.

Controllability

In a turn-based system you control the gate. In a full-duplex model, turn behavior is learned. You steer it with prompts, not thresholds. Exact wording is harder too. GPT-Live's session docs say `session.commentary.append` "may paraphrase" your text. For disclosures, OpenAI recommends `session.instructions.append` and checking the generated audio before marking it delivered.

Hallucination and stale context

Delegation adds a timing risk. The voice model may speak before a backend result arrives. The docs note the model "may still respond before it has used the whole update." If a caller interrupts mid-lookup, your app decides whether the backend task finishes or cancels. Get that wrong and the agent confirms an action the caller just withdrew.

Safety under interruption

DuplexJail (September 2026) is the sharpest warning. Researchers injected fixed spoken prompts while four open full-duplex models were responding. On AdvBench, fixed-delay interruption raised attack success to 40.3% for PersonaPlex and 48.7% for PersonaPlex-RL. Those are increases of 33.8 and 39.3 points over baseline. Other models showed smaller effects or decreases. The takeaway: a refusal at the start of a turn does not guarantee safety through the turn.

Cost

Full-duplex sessions bill by connected time, not just by tokens. GPT-Live-1 costs $0.05 per minute for the voice layer, billed per second, plus backend model and tool usage. Idle but open sessions cost money. OpenAI's docs suggest closing sessions during long inactivity and resuming with saved context.

Architecture patterns: native full-duplex vs cascaded plus smart turn detection

You have two realistic patterns in 2026. Neither wins everywhere.

Pattern A, native full-duplex. A model like GPT-Live-1 owns listening and speaking. A backend model owns reasoning and tools. You get backchannels, pause tolerance, and fast yields out of the box. You give up fine control of turn behavior and some predictability.

Pattern B, cascaded or speech-to-speech with smart turn detection. You keep an explicit turn gate, but make it smarter. OpenAI's Realtime API offers a `semantic_vad` mode, documented in its VAD guide. It scores whether the caller has finished based on their words. An `eagerness` setting trades responsiveness against cutting people off.

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "audio": {
      "input": {
        "turn_detection": {
          "type": "semantic_vad",
          "eagerness": "low",
          "create_response": true,
          "interrupt_response": true
        }
      }
    }
  }
}

Here, `eagerness: "low"` lets callers take their time. `interrupt_response: true` stops the agent's audio when the caller starts talking. That gives you barge-in, but not backchannel awareness. A caller's "yeah" can still halt the agent mid-sentence. For silence-based tuning, `server_vad` exposes `threshold`, `prefix_padding_ms`, and `silence_duration_ms`. Our endpointing guide covers how to tune those.

The latency picture differs by pattern. The numbers below are illustrative, not measured. They show typical shape, and your stack will vary.

The last row is the quiet truth. Full-duplex does not make a slow backend fast. It lets the agent keep talking naturally while the backend works. FDB-v3's multi-second latencies on tool tasks make the same point.

Rule of thumb: pick native full-duplex when conversational feel drives outcomes, like tutoring or reservations. Pick smart turn detection when scripted accuracy dominates, like payments or identity checks.

Why do voice agents talk over users?

Talk-over is the signature failure of duplex systems. It also has more causes than most teams expect. The table maps each failure mode to its usual root cause and the test that surfaces it.

Notice that only the first three are pure timing problems. The last four are correctness problems caused by timing. A transcript review will miss most of them. Our stress-testing guide covers how these multiply under load and noise. The barge-in and interruptions guide and barge-in vs turn-taking explain the underlying mechanics.

How to evaluate a full-duplex voice agent

Evaluating full-duplex agents means measuring timing and correctness together, on audio. Text evals miss overlap entirely. Pure timing benchmarks miss whether the agent did the right thing. Here is the process we recommend.

1. Define the behaviors you actually need. List where your calls require backchannels, pause tolerance, and fast yields. A payments line and a tutoring app need different settings.

2. Script overlap into every scenario. For each scenario, specify when the synthetic caller interrupts, pauses, backchannels, or self-corrects. Vary the offset from agent speech onset, for example 0.3, 1, and 3 seconds.

3. Vary caller profiles. Mix speaking pace, accents, background noise, and speakerphone echo. Duplex behavior shifts sharply with acoustics, as tau-voice showed.

4. Capture stereo audio. Record the caller and agent on separate channels. GPT-Live's stored recordings already download as stereo WAV, input on the left and output on the right.

5. Compute timing metrics from the audio. Measure interruption response time, talk-over rate, false-yield rate, and the turn-gap distribution. Use the code below as a starting point.

6. Score task success separately. Check tool calls, arguments, and final outcomes against expected state. Pay special attention to interrupted tool calls and self-corrections.

7. Red-team the interruption channel. Time adversarial prompts right after refusals and disclosures. Confirm the agent holds policy through the whole turn.

8. Set thresholds and gate releases. Turn each metric into a pass or fail threshold per scenario. Rerun the suite on every model, prompt, or backend change.

The metrics, defined

Each metric needs a precise definition, or teams argue about numbers forever. The thresholds here are illustrative starting points, not industry standards. Tune them to your callers.

Two notes. The turn-gap distribution matters more than its mean. A long p90 tail feels worse to callers than a slightly slower median. And task success should be split by "interrupted" versus "clean" turns. That split reveals whether duplex behavior is costing you correctness. See our deeper guides on interruption rate, turn-taking evaluation, pause handling, and task success rate.

Measuring overlap from a stereo recording

The script below computes the core timing metrics from a two-channel recording. It uses a simple energy-based speech detector. That is enough for synthetic test calls with clean channels. For real calls, swap in a proper VAD and filter echo first.

import numpy as np
import soundfile as sf

FRAME_MS = 20
BACKCHANNEL_MAX_MS = 600   # caller bursts shorter than this count as backchannels
TALK_OVER_BUDGET_MS = 400  # agent must stop within this after a real interruption

def speech_mask(x, sr, thresh_db=-35.0):
    hop = int(sr * FRAME_MS / 1000)
    n = len(x) // hop
    frames = x[: n * hop].reshape(n, hop)
    rms = np.sqrt(np.mean(frames ** 2, axis=1) + 1e-12)
    return 20 * np.log10(rms) > thresh_db

def runs(mask):
    """Return (start, end) frame indices of contiguous speech runs."""
    edges = np.diff(np.concatenate([[0], mask.astype(int), [0]]))
    return list(zip(np.flatnonzero(edges == 1), np.flatnonzero(edges == -1)))

audio, sr = sf.read("recording.wav")  # stereo: left = caller, right = agent
caller = speech_mask(audio[:, 0], sr)
agent = speech_mask(audio[:, 1], sr)

irt, talk_overs, false_yields, backchannels = [], 0, 0, 0
for start, end in runs(caller):
    if not agent[start]:
        continue  # caller did not overlap the agent
    burst_ms = (end - start) * FRAME_MS
    rest = ~agent[start:]
    stop_ms = (np.argmax(rest) if rest.any() else len(rest)) * FRAME_MS
    if burst_ms < BACKCHANNEL_MAX_MS:
        backchannels += 1
        if stop_ms < burst_ms + 200:
            false_yields += 1  # agent stopped for a backchannel
    else:
        irt.append(stop_ms)
        if stop_ms > TALK_OVER_BUDGET_MS:
            talk_overs += 1

gaps = []
for _, end in runs(caller):
    if end < len(agent) and not agent[end]:
        nxt = np.flatnonzero(agent[end:])
        if nxt.size:
            gaps.append(nxt[0] * FRAME_MS)

print({
    "interruption_response_p90_ms": float(np.percentile(irt, 90)) if irt else None,
    "talk_over_rate": talk_overs / len(irt) if irt else 0.0,
    "false_yield_rate": false_yields / backchannels if backchannels else 0.0,
    "turn_gap_p50_ms": float(np.median(gaps)) if gaps else None,
    "turn_gap_p90_ms": float(np.percentile(gaps, 90)) if gaps else None,
})

A few caveats keep this honest. Burst length alone is a rough backchannel classifier. A short "No!" is an interruption, not a backchannel. Scripted scenarios solve that, because you know the caller's intent in advance. Also, "agent stopped" is not the same as "agent handled it." Pair every timing result with the transcript and the tool log for that turn. That is why we prefer audio-first evaluation over transcript-only review.

What public benchmarks tell you, and what they do not

The Full-Duplex-Bench family is the best public reference. Version 1 covered the four core behaviors. Version 1.5 focused on overlap handling. Version 2 added multi-turn evaluation with an automated examiner. Version 3 added real human disfluent audio and chained tool calls across four domains. It compared GPT-Realtime, Gemini Live 2.5 and 3.1, Grok, Ultravox v0.7, and a cascaded pipeline.

Benchmarks help you shortlist models. They are weak for release decisions, because your prompts, tools, callers, and telephony change results. Vendor-reported wins also use vendor-chosen configurations. OpenAI, for example, reports a 30-point Full-Duplex-Bench gain for GPT-Live-1 over GPT-Realtime-2.1. That may hold for you, or not. Rerun the same behaviors on your own scenarios before you trust any leaderboard.

Where Evalgent fits

Evalgent is an independent evaluation platform for voice agents. It tests full-duplex and turn-based agents the same way, over real audio. That lets you compare architectures on equal terms.

• Scenarios script interruptions, backchannels, thinking pauses, and self-corrections at precise offsets.
• Profiles vary caller pace, accent, noise, and how aggressively callers barge in.
• Metrics track interruption response time, talk-over rate, false yields, turn gaps, and task success, with thresholds you set.
• Evaluations run the suite as batches of synthetic callers before every release.
• Reviews let your team replay any failed exchange and hear the overlap, not just read about it.

Because Evalgent is platform-agnostic, the same suite runs against GPT-Live-1, a Realtime agent with semantic VAD, or a cascaded stack. That makes a migration decision evidence-based. For GPT-Live specifics, see testing GPT-Live voice agents. For the wider discipline, see the AI voice agent testing pillar and testing speech-to-speech voice agents.

Should you adopt full-duplex now?

Not always. Half-duplex is simpler and easier to control. For IVR flows, data capture, or regulated scripts, explicit turns can be safer.

A sensible path: baseline your current agent on the metrics above. Pilot full-duplex on one call type where feel matters. Compare talk-over, false yields, turn gaps, and task success side by side. Adopt only where it wins on outcomes, and keep the interruption red-team in your regression suite. For interruption-specific methods, see interruption detection evaluation.

Frequently asked questions

What is a full-duplex voice agent?

A full-duplex voice agent listens and speaks at the same time. It processes caller audio continuously while generating speech, so it can backchannel, wait through pauses, and yield instantly to interruptions. Turn-based agents instead wait for a silence or end-of-utterance signal before replying. Examples in 2026 include OpenAI's GPT-Live-1 and the open models Moshi and PersonaPlex.

What is the difference between turn-based and full-duplex conversational AI?

Turn-based conversational AI uses a detector, such as voice activity detection, to decide when the caller has finished. Only then does the model respond. Full-duplex AI has no separate gate. The model hears both sides continuously and decides frame by frame whether to speak, listen, pause, or yield. Full-duplex feels more natural but is harder to control and test.

Which voice APIs support full-duplex communication?

As of September 2026, OpenAI's GPT-Live-1 is the main production API documented as natively full-duplex. It launched in the API on September 10, 2026. GPT-Realtime and xAI's Grok Voice are speech-to-speech APIs that document VAD-based turn detection. Moshi and PersonaPlex are open, natively full-duplex models you self-host. Always verify current vendor docs.

How does GPT-Live's full-duplex architecture work?

GPT-Live splits the work into two layers. A continuous interaction model listens and speaks at once, deciding many times per second whether to talk, pause, interrupt, or call a tool. A backend model, such as GPT-6 Astra in the API, handles reasoning, search, and tools. The voice layer keeps the conversation going while the backend works.

What is Full-Duplex-Bench-v3?

Full-Duplex-Bench-v3 is an April 2026 benchmark for full-duplex voice agents. It uses real human audio with five annotated disfluency types and scenarios that require chained API calls across four domains. It scores tool-use accuracy, turn-taking, and latency. It compared GPT-Realtime, Gemini Live, Grok, Ultravox, and a cascaded pipeline. Self-corrections and multi-step reasoning were the most consistent failures.

Is Grok Voice full-duplex?

xAI documents Grok Voice as a real-time speech-to-speech API over WebSocket, with server-side VAD turn detection. That is a turn-detection design rather than a documented dual-stream full-duplex model. It can still support fast barge-in. It is also compatible with the OpenAI Realtime API. Test its interruption and backchannel behavior on your own scenarios rather than relying on labels.

Why do voice agents talk over users?

Voice agents talk over users when they detect interruptions too slowly or miss them entirely. Common causes include slow barge-in detection, echo from the agent's own audio, noisy lines, and overlapping speech the model cannot separate. Full-duplex agents add another cause: learned turn behavior that fails on unusual callers. Measure talk-over rate on audio, since transcripts hide it.

How do you test a full-duplex voice agent?

Test it on audio with overlap scripted into every scenario. Have synthetic callers interrupt, pause, backchannel, and self-correct at set offsets. Record caller and agent on separate channels. Then compute interruption response time, talk-over rate, false yields, and turn gaps. Score task success separately, including interrupted tool calls. Finally, red-team interruptions for safety.

The bottom line

Full-duplex voice agents make conversations feel human by listening and speaking at the same time. That naturalness only counts if the agent still yields correctly, completes the task, and stays safe through every overlap.

Book a demo to see how Evalgent evaluates full-duplex agents on real audio: book a demo.