How to test GPT-Live voice agents: the new failure modes of full-duplex

OpenAI's GPT-Live, launched on July 8, 2026, is not just a faster voice model. It is full-duplex: it listens and speaks simultaneously, decides many times a second whether to talk, keep listening, backchannel, or pause, and hands deeper reasoning to GPT-5.5 in the background while the conversation keeps flowing. That architecture is more human — and it breaks the assumptions most voice-agent testing is built on. Evalgent tests for exactly these new failure modes, and this guide walks through them.

Full-duplex voice agent: a voice agent that can listen and speak at the same time, continuously deciding whether to talk, stay quiet, backchannel, or interrupt, rather than waiting for a clean end-of-turn before responding.

The clean-turn assumption most testing breaks

Almost every voice-agent test written to date assumes a turn. The caller speaks, stops, and the agent responds. Endpointing detects the stop; the transcript captures the exchange; the test checks the reply. That model has worked because that is how half-duplex agents behave.

GPT-Live does not behave that way. There is no clean stop to wait for, because the model is making speak-or-listen decisions continuously. It can start responding before the caller finishes, hold the floor while it thinks, backchannel mid-sentence, or correct course as new words arrive. A test that waits for a turn boundary is testing a boundary that no longer exists. The OpenAI voice agents guide frames the architecture; the testing implication is that the old harness measures the wrong thing.

This matters now even if you have not adopted GPT-Live yet. Full-duplex is the direction the whole category is moving, and the failure modes below are already documented in the research. Our companion piece on building voice agents with GPT-Live covers the architecture side; this one covers what to test.

Six new failure modes of full-duplex voice agents

The academic benchmark Full-Duplex-Bench-v3 documents where full-duplex agents break, and the numbers are sobering. These are the six failure modes to build a GPT-Live test suite around.

Self-correction is the sharpest one. The same early processing that makes full-duplex fast also makes it pre-commit. When a caller says "book me to Rome — actually, make it Paris," a fast model can lock in the first value and call the tool with `destination=Rome`. Full-Duplex-Bench found even a strong realtime model succeeds on fewer than 59% of self-correction scenarios. The transcript reads correctly; the action is wrong. Only an argument-level assertion on the tool call catches it, which is the discipline from our tool calling guide applied to a moving target.

Backchanneling and barge-in are the overlap failures. A full-duplex agent is supposed to say "mhmm" naturally and yield instantly when interrupted. The benchmark found one model interrupted the caller nearly half the time — 47.9% — talking over them instead of listening. Testing has to script interruptions and backchannels and assert the agent handles both, exactly the concern of our barge-in guide.

The delegation gap is new to this architecture. When GPT-Live hands a hard question to GPT-5.5 in the background, there is a variable pause while it thinks. A good agent holds the floor naturally — "let me look into that" — instead of going silent. The benchmark found a "silent worker" pattern in 22% of cases, where the agent simply went quiet mid-task. Testing has to drive delegated queries and assert the agent fills the gap rather than leaving dead air.

Metrics that matter for full-duplex

Full-duplex needs metrics the turn-based world never measured. Task completion still matters, but it now has to be read alongside timing and overlap behavior.

Track barge-in rate and interruption-recovery: does the agent stop promptly and recover gracefully. Track end-of-turn accuracy under pauses, since pauses are the weakest category across systems. Track a false-interruption rate: how often the agent talks over the caller. Track delegation-gap latency and whether the agent holds the floor during it. And track self-correction pass rate — the share of scenarios where a mid-sentence correction actually reaches the tool call. These are the numbers that tell you whether a full-duplex agent feels human or chaotic.

How to test a GPT-Live voice agent

The method is the familiar one — synthetic callers, scenarios, outcome metrics — pointed at the new failure modes.

Build synthetic callers that do the things full-duplex breaks on: interrupt mid-response, correct themselves mid-sentence, add backchannels and fillers, pause in the middle of a thought, and ask questions that force delegation. Run them over real audio, not clean text, since overlap and timing only exist in audio. Assert at the level that matters: the tool call used the corrected value, the agent stopped on interruption, it held the floor during delegation, it did not talk over the caller. Then gate releases on those assertions, and re-run after any change. This is the stress testing mindset applied to an agent that is always half-listening.

Reasoning tiers need their own baselines

GPT-Live exposes reasoning tiers — Instant, Medium, and High — that trade latency against depth. Instant runs on the fast model; Medium and High invoke deeper thinking. Each tier is effectively a different agent, and each needs its own regression baseline.

A change that improves correctness on High can wreck latency on Instant, or a model update can shift the tradeoff on all three at once. Testing has to run the suite per tier and compare each against its own known-good run, not a blended average. Treat a latency regression on Instant and a correctness regression on High as separate, both release-blocking. Without per-tier baselines, a tier-specific regression hides inside an overall number that looks fine.

Building the synthetic callers full-duplex needs

Testing full-duplex is only as good as the callers you test with, and a scripted, one-utterance-at-a-time caller cannot exercise it. The synthetic callers have to behave like real people in the messy moments the architecture breaks on.

That means callers that interrupt the agent mid-response, not just after it finishes. Callers that start a request and correct it a few words later, so you can assert the correction reaches the tool call. Callers that scatter backchannels — "mhmm," "right," "go on" — while the agent talks, to see whether it holds its turn or stops. Callers that pause mid-thought, to test whether the agent waits or jumps in. And callers that ask questions hard enough to force background delegation, so you can watch the gap.

Each of these has to run over real audio, because overlap and timing do not exist in text. A synthetic-caller suite built only from clean, turn-based scripts will pass a full-duplex agent that would fail with real people. The callers are the test; design them to break the agent the way a real conversation would.

Testing GPT-Live voice agents with Evalgent

Evalgent is built for exactly the failure modes full-duplex introduces. Scenarios script the hard cases — mid-sentence self-corrections, interruptions, backchannels, pauses, and delegated queries — so the new failure modes are exercised rather than assumed. Profiles vary caller pace, accent, and interruption style over real audio, since overlap and timing only appear in sound. Metrics encode barge-in rate, interruption recovery, false-interruption rate, delegation-gap behavior, and self-correction pass rate, with thresholds you set. Evaluations run the suite per reasoning tier as automated batches of synthetic callers, so tier-specific regressions surface before release. Reviews let you replay a call and hear exactly where the agent locked in the wrong intent or went silent mid-thought.

The result is a full-duplex agent you can actually trust: one that yields when interrupted, follows a correction to the tool call, and holds the floor while it thinks. For the broader discipline, see the AI voice agent testing pillar, and for the build side see building voice agents with GPT-Live.

Conclusion

GPT-Live removes the clean turn that most voice-agent testing was built around, and with it the assumptions that made a transcript enough. Full-duplex agents fail in new ways — locking in outdated intent, talking over the caller, going silent mid-thought — that only show up in audio, in overlap, and at the tool call.

Test the six failure modes directly, with synthetic callers that interrupt, self-correct, and pause, and baseline every reasoning tier on its own. As full-duplex becomes the default, the teams that tested for it will be the ones whose agents feel human instead of chaotic.

Frequently asked questions

How do you test a GPT-Live voice agent?

Test it with synthetic callers that do what full-duplex breaks on: interrupt mid-response, correct themselves mid-sentence, add backchannels, pause, and force background delegation. Run them over real audio and assert at the outcome level — the tool call used the corrected value, the agent stopped on interruption, it held the floor while thinking — then baseline each reasoning tier separately and gate releases on the results.

Why does full-duplex need new tests?

Because full-duplex removes the clean end-of-turn that turn-based tests wait for. GPT-Live listens and speaks at once and delegates reasoning mid-conversation, so failures happen in overlap and timing, not in a tidy transcript. A test that checks a reply after a turn boundary misses interruptions, backchannels, silent delegation gaps, and mid-sentence corrections entirely.

What are the failure modes of full-duplex voice agents?

The main ones, documented by Full-Duplex-Bench-v3, are self-correction or intent lock-in, poor barge-in recovery, inappropriate backchanneling, the delegation gap where the agent goes silent while reasoning, reasoning-tier tradeoffs between speed and correctness, and weak pause robustness. Each shows up in audio and timing rather than the transcript, which is why turn-based testing misses them.

What is intent lock-in in a voice agent?

Intent lock-in is when a fast, full-duplex agent commits to a caller's first intent before they finish correcting it. If a caller says "book Rome — actually Paris," the agent may call the tool with Rome. The same early processing that makes full-duplex responsive causes it to pre-commit, so a mid-sentence correction never reaches the action.

How do you test barge-in on GPT-Live?

Script scenarios where the caller interrupts while the agent is speaking, and assert the agent's speech stops promptly and it recovers gracefully, using what the caller said. Separately, test backchannels and assert the agent does not treat them as full interruptions. Because full-duplex agents can talk over callers, measure a false-interruption rate alongside barge-in success.

How do you test the delegation gap?

Drive queries that force the agent to delegate hard reasoning to the background model, then assert it holds the floor naturally — with a filler or acknowledgment — instead of going silent. Measure delegation-gap latency and flag the "silent worker" pattern where the agent simply goes quiet mid-task. The gap is variable and mid-utterance, so test it in audio, not per turn.

What metrics matter for full-duplex voice agents?

Beyond task completion, track barge-in rate and interruption recovery, end-of-turn accuracy under pauses, false-interruption rate, delegation-gap latency and floor-holding, and self-correction pass rate. These capture the overlap and timing behavior that define whether a full-duplex agent feels human. Read them together, and per reasoning tier, rather than as a single blended score.

How do you test reasoning tiers in GPT-Live?

Treat each tier — Instant, Medium, and High — as a separate agent with its own regression baseline. Run the suite per tier and compare each against its own known-good run, since a change can improve correctness on one tier while hurting latency on another. Treat a latency regression on a fast tier and a correctness regression on a deep tier as separate, both release-blocking.