LLM-as-a-judge limits in voice AI: 9 failure modes and how to catch them

Your judge says 94% of calls passed. Your repeat-call rate says otherwise. One of those numbers is wrong, and it is rarely the repeat-call rate.

LLM-as-a-judge became the default for voice agent evaluation because it is easy. You write a rubric, send a transcript, and get a score. But ease and validity are different properties. A judge can be biased, noisy, and overconfident. On top of that, a transcript drops half of what a caller experienced.

This guide catalogs each failure mode with research evidence and a voice example. Then it shows how to measure your judge and what to fix. It is for engineers who need scores they can defend.

What is LLM-as-a-judge and what does it measure?

LLM-as-a-judge: the practice of prompting a language model to grade another model's output against a rubric, returning a score, a label, or a preference, often with a written rationale.

In voice AI, the setup is usually post-call. The speech-to-text layer produces a transcript. A judge model such as GPT or Claude reads it. The prompt asks whether the agent handled the call well.

That judge can assess four things well enough:

• Words and turn order. What each side said, as the ASR system heard it.
• Coherence. Whether replies make sense for the apparent intent.
• Register. Whether the agent sounds polite, patient, and on-brand.
• Apparent completion. Whether the dialogue looks like it wrapped up.

These are real signals. The founding paper on the method, Zheng et al.'s "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena", found GPT-4 matched human preferences over 80% of the time. That is about the rate at which humans agree with each other. The same paper also documented position, verbosity, and self-enhancement biases. Both findings are true at once.

So ask where the judge fails, and how you would know.

Why voice agents make judge errors worse

Text chatbots hand the judge the exact artifact the user saw. Voice agents do not. The judge reads a lossy copy of the call.

Three things compound the problem in voice:

1. The input is already an estimate. The transcript is the ASR system's guess. Errors in it look like facts to the judge.

2. The experience lives in timing and sound. Latency, interruptions, tone, and pronunciation shape the call. None of them survive as plain text.

3. Volume is high. A busy agent handles thousands of calls a day. Expensive judges push teams to score a sample, so rare failures hide.

A broad replication study makes the general point. Bavaresco et al. built JUDGE-BENCH from 20 human-annotated datasets and tested 11 LLMs as judges. Reliability varied widely by task and by property being judged. Their conclusion was blunt: validate LLM judges against human judgments before relying on them. Voice adds audio loss on top of that variance.

The 9 failure modes of LLM-as-a-judge

Failures one to seven come from the judge. Eight and nine come from the setup around it.

1. Position and order bias

Position bias: a judge's tendency to prefer an answer because of where it appears in the prompt, not because of its quality.

Evidence. In Zheng et al., GPT-4 gave the same verdict after swapping two answers in only 65% of cases. It favored the first answer 30% of the time. Other judges did worse. Wang et al., "Large Language Models are not Fair Evaluators", showed how far this goes. With ChatGPT as the judge, Vicuna-13B beat ChatGPT on 66 of 80 queries just by reordering. A later systematic study by Shi et al. ran 15 judges across over 150,000 evaluation instances. It found position bias is not random chance. It is strongly affected by the quality gap between candidates.

Voice example. You A/B test two prompts for a booking agent. The judge compares transcript A against transcript B. Version A is always listed first. The judge picks A more often, and you ship it. The "win" came from the prompt layout.

2. Verbosity bias

Evidence. Zheng et al. tested a "repetitive list" attack. They padded answers with restated content. GPT-3.5 and Claude-v1 were fooled 91.3% of the time. GPT-4 was fooled 8.7% of the time. Those were 2023 models. Test your own judge rather than assume newer ones are immune.

Voice example. On the phone, longer is usually worse. A caller asking for a pharmacy's hours wants one sentence. A judge reading text may reward the agent that also recites parking and holiday hours. That agent sounds thorough on paper. On a call, it talks over the caller's next question and inflates handle time.

3. Self-preference

Self-preference bias: a judge scoring outputs from its own model family higher than equal-quality outputs from others. Zheng et al. call it self-enhancement bias.

Evidence. Zheng et al. found GPT-4 favored itself with a 10% higher win rate. Claude-v1 favored itself with a 25% higher win rate. Panickssery, Bowman, and Feng went further. LLMs can recognize their own text. Stronger self-recognition correlated linearly with stronger self-preference. The G-Eval paper also flagged a bias toward LLM-generated text in general.

Voice example. Your agent runs on one model family. Your judge is the same family. You then compare it against a rival vendor's agent on another model. The judge quietly tilts toward the house style. Your vendor bake-off is not neutral anymore.

4. Inconsistency across reruns

Evidence. Many judges run at non-zero temperature by default. The same transcript can get a different score on each run. In LangChain's study "Can Jev Be a Better Agent Evaluator?", the team froze five captured agent runs. Each judge graded each run 100 times. The input never changed. Yet Claude Sonnet 4.6 agreed with the human label on only 80.0% of the 500 decisions. The misses were the same cases graded differently on replay. That is pure judge noise.

Voice example. A compliance rubric asks, "Did the agent read the recording disclosure?" On Monday the call passes. On Tuesday the same call fails. Your dashboard shows a compliance dip that never happened. Someone spends a day chasing it.

5. Prompt sensitivity

Evidence. Small wording changes shift judge output. Zheng et al. found a few-shot prompt raised GPT-4's position consistency from 65.0% to 77.5%. That is good news for mitigation. It is bad news for stability. If three example calls move the numbers that much, so will an innocent rubric edit. Ye et al.'s CALM framework catalogs 12 bias types. Several are triggered by surface features of the prompt rather than content.

Voice example. A teammate rewrites "resolved the caller's issue" as "fully resolved the caller's issue." Pass rates drop six points overnight (illustrative). The agent did not change. The ruler did.

6. Overconfident, uncalibrated scores

Calibration: the match between a stated confidence and the actual hit rate. A calibrated judge that says "90% sure" is right about 90% of the time.

Evidence. Guo et al., "On Calibration of Modern Neural Networks", showed modern neural networks are often poorly calibrated. LLM judges add another layer. Many return a bare score of 1 to 5 with no confidence at all. When asked for confidence, they produce a number as text. That number is a generated token, not a measured probability.

Voice example. Your judge says "Confidence: 0.95" on a call where the caller said "cancel" and ASR heard "can sell." The judge sounds certain. You have no way to route that call to a human, because every call says 0.9 or higher.

7. Rubric drift

Rubric drift: a gradual change in what a score means over time, caused by judge model updates, rubric edits, or new call types.

Evidence. Providers update hosted models. JUDGE-BENCH found agreement varies by model and by dataset. A version bump is effectively a new judge. Few teams re-validate when that happens.

Voice example. In spring, "empathy: 4" meant the agent acknowledged frustration. After a model update, the judge grades empathy harder. Your quarter-over-quarter chart shows an empathy decline. It is a measurement artifact. See how LLM updates cause voice agent regressions for the agent-side version of the same problem.

8. Blindness to audio

This is the failure that matters most for voice. A transcript-only judge cannot see:

• Tone. A flat, rushed "I understand" reads as empathy on paper.
• Interruptions. Barge-in and talk-over vanish. So does the agent that kept talking after the caller said "stop."
• Latency. A 4-second pause and a 400 ms pause produce the same text. ITU-T G.114 notes that one-way delay above roughly 150 ms starts to affect conversational quality.
• Pronunciation. The agent says a drug name or street name wrong. The transcript shows the correct spelling, because it came from the text-to-speech input.
• ASR errors. The caller says "fifty." ASR writes "fifteen." The agent processes fifteen. The judge sees a clean, consistent call.

Voice example. A banking agent confirms a $15 transfer. The caller asked for $50. The judge scores the call 5 out of 5 for accuracy. The error happened before the judge ever saw a word. For more on this split, read transcript vs audio voice agent evaluation and our guide to word error rate for voice agents.

9. Cost and latency force sampling

Evidence. In the LangChain study, the same 500-decision run cost $28.17 with Claude Sonnet 4.6 and $0.34 with Jev. LLM judges also take seconds per call. At production volume, that cost adds up fast.

Voice example. You handle 20,000 calls a day. Judging all of them with a frontier LLM is expensive, so you score a 2% sample (illustrative). A failure that hits 0.5% of calls shows up about twice a day in your sample. It looks like noise. It is actually a hundred unhappy callers a day.

LLM judge failure modes: detection and mitigation table

Use this table as a checklist. Each row maps a failure to a concrete test.

What one study shows about judge agreement and variance

The LangChain study is useful because it isolates judge noise. The team built a weather agent. They captured five runs and stored each as a fixed example in LangSmith. Each judge graded each fixed run 100 times. A human reviewer labeled every run against the same rubric.

Results on the binary `does_pass` decision:

On the continuous quality score, Jev's mean per-case variance was 0.0000149. The three LLM judges were 92x to 913x higher. The full run cost $0.34 for Jev and $28.17 for Claude.

Notice one detail. A judge can be near-perfect on pass or fail and still wander on the quality score. Agreement and variance measure different things. If you trend a quality score week over week, variance becomes fake movement in your chart.

Read it with care. This is one study on one agent type. It used a simple weather agent, not a voice agent. It used a binary pass criterion. The LLM judges ran at provider default settings, with no temperature or seed set. LangChain itself called the results promising but early. Treat it as a reason to measure your own judge, not a final verdict. For a deeper look at the calibrated option, see Jev as a judge for voice agents.

How to detect false positives in an LLM-as-a-judge

A false positive is a call the judge passed that a careful human would fail. It is the most dangerous judge error, because nobody looks at passing calls. Here is a repeatable way to find them.

1. Build a frozen golden set. Pull 200 to 500 real calls (illustrative size). Oversample hard cases: accents, noise, numbers, and escalations. Keep audio, transcript, timing, and tool logs together.

2. Label with humans first. Have two reviewers label each call against the same rubric. Measure their agreement too. If humans cannot agree, the judge cannot either.

3. Score every call several times. Run the judge 5 to 10 times per call. Keep every verdict, not just the last one.

4. Compute agreement and Cohen's kappa. Raw agreement looks high when most calls pass. Cohen's kappa corrects for chance.

5. Isolate false passes. List every call where the judge said pass and humans said fail. Read them. Group them by cause: ASR error, latency, tool failure, or judge bias.

6. Measure rerun flip rate. Count calls whose verdict changed across reruns. Those calls are coin flips, not measurements.

7. Check calibration by bucket. Group verdicts by stated confidence. Compare each bucket's confidence to its actual accuracy.

8. Run the bias probes. Swap order in pairwise tests. Pad a good answer with filler. Rewrite one rubric word. Count how often the verdict moves.

9. Set a gate and re-run it. Pick thresholds, such as kappa of at least 0.7 and a flip rate under 2% (illustrative). Re-run the whole set after any judge, model, or rubric change.

The script below does steps 4, 6, and 7. It is simplified for clarity.

# Simplified: judge validation against human labels.
# Each record: call_id, human label, and a list of judge runs.
# A judge run is (verdict, confidence) with verdict 1 = pass, 0 = fail.
from collections import defaultdict
from statistics import mean, pvariance

records = [
    {"call_id": "c1", "human": 1, "runs": [(1, 0.92), (1, 0.90), (1, 0.94)]},
    {"call_id": "c2", "human": 0, "runs": [(1, 0.88), (0, 0.61), (1, 0.85)]},
    {"call_id": "c3", "human": 0, "runs": [(0, 0.97), (0, 0.95), (0, 0.96)]},
    # ... load your full golden set here
]

def majority(runs):
    votes = [v for v, _ in runs]
    return 1 if sum(votes) >= len(votes) / 2 else 0

def cohens_kappa(a, b):
    n = len(a)
    p_obs = sum(x == y for x, y in zip(a, b)) / n
    pa, pb = sum(a) / n, sum(b) / n
    p_exp = pa * pb + (1 - pa) * (1 - pb)
    return 1.0 if p_exp == 1 else (p_obs - p_exp) / (1 - p_exp)

human = [r["human"] for r in records]
judge = [majority(r["runs"]) for r in records]

agreement = mean(h == j for h, j in zip(human, judge))
kappa = cohens_kappa(human, judge)
false_passes = [r["call_id"] for r, j in zip(records, judge)
                if j == 1 and r["human"] == 0]

# Rerun stability: share of calls whose verdict flipped, plus score variance.
flip_rate = mean(len({v for v, _ in r["runs"]}) > 1 for r in records)
mean_var = mean(pvariance([c for _, c in r["runs"]]) for r in records)

# Calibration: accuracy per confidence bucket, per individual run.
buckets = defaultdict(list)
for r in records:
    for verdict, conf in r["runs"]:
        b = min(int(conf * 10), 9) / 10  # 0.0, 0.1, ... 0.9
        buckets[b].append(verdict == r["human"])

print(f"agreement={agreement:.3f} kappa={kappa:.3f}")
print(f"false_passes={false_passes}")
print(f"flip_rate={flip_rate:.3f} mean_confidence_variance={mean_var:.5f}")
for b in sorted(buckets):
    acc = mean(buckets[b])
    print(f"confidence {b:.1f}-{b + 0.1:.1f}: accuracy={acc:.2f} n={len(buckets[b])}")

Kappa shows whether agreement beats chance. Flip rate shows which verdicts are noise. If the 0.9 bucket is only 70% accurate, the judge is overconfident.

Mitigations that hold up in practice

No single fix covers every failure. Layer them.

Run pairwise comparisons in both orders

Zheng et al. suggest a conservative rule. Call the judge twice with the order swapped. Declare a win only if both orders agree. Otherwise, call it a tie. Wang et al. proposed the same idea as Balanced Position Calibration. It doubles cost, but it removes most order effects from A/B tests.

Anchor the rubric with examples

Vague criteria invite drift. Replace "Was the agent empathetic?" with anchored levels. For example, "Score 3: acknowledges the problem once. Score 5: acknowledges it and adjusts the plan." Add two or three labeled example calls per level. Zheng et al.'s few-shot result shows anchors raise consistency.

Split big rubrics into small questions, too. "Did the agent confirm the date?" is easier to grade than "Was the booking handled well?"

Use a panel instead of one judge

Mix judge families. Take a majority vote. This dilutes self-preference and random noise. It costs more, so reserve it for high-stakes checks and vendor comparisons.

Put verifiable checks first

Many voice questions have exact answers. Did the booking API return 200? Is the appointment in the database? Did the agent say the required disclosure string? Code can check these with zero variance. Run them before any judge. Our comparison of verifiable rewards vs LLM judges vs Jev walks through the trade-offs. For tool calls that fail silently, see how to detect silent tool failures.

Use a calibrated scorer for scale

Judging every call with a frontier LLM rarely pencils out. A calibrated decision model can cover 100% of traffic cheaply. In the LangChain study, Jev returned typed answers with probabilities rather than generated text. It cost cents for the full run. The trade-off is real: it writes no rationale. Learn more in our overview of Jev for voice agent evaluation.

Send low-confidence calls to humans

Calibration only helps if you act on it. Set a threshold. Auto-accept confident verdicts. Queue the uncertain middle for a person. Feed those labels back into your golden set. The human-in-the-loop voice agent evaluation guide covers queue design and reviewer agreement.

Add audio and timing signals

A judge cannot hear. So measure what it misses directly. Log per-turn latency, barge-in events, silence gaps, and ASR confidence. Join them with judge scores. A call with a 5 out of 5 score and a 6-second silence deserves a look. Turn-taking deserves its own tests; see voice model turn-taking evaluation.

What no judge can see in a transcript

Some failures sit outside the transcript entirely. No prompt or model upgrade fixes them.

Abandonment deserves a note. A caller who hangs up during the greeting leaves little or no transcript. A crashed session may leave none. Those are often your least patient callers. A judge-only pipeline never scores them. That is one reason voice agents fail in production even when eval dashboards look healthy.

When an LLM judge is still the right tool

None of this makes LLM judges useless. They are the right tool in several jobs.

• Rationales. A calibrated scorer gives a verdict. An LLM judge explains why. For debugging a failed call, that explanation saves time.
• Nuanced review. Subjective qualities like clarity or helpfulness resist exact checks. A well-anchored judge handles them better than code.
• Triage at scale. A judge can surface the 3% of calls worth a human's attention (illustrative). Treat it as a filter, not a final grade.
• Compliance wording. Checking for prohibited phrases or required scripts works from words alone.
• Test design. An LLM can draft scenarios for synthetic callers and stress tests.

The rule is simple. Use an LLM judge for questions the transcript can answer. Validate it on human labels. Pair it with checks for everything else.

A layered scoring stack for voice agents

Put the pieces together in order, from cheapest and most exact to most expensive and most nuanced.

Each layer catches what the previous one cannot. The platform metrics view is where these signals come together on one call.

The bottom line

LLM-as-a-judge is a biased, noisy, audio-blind instrument that becomes useful only after you measure its agreement with human labels. Voice teams that trust judge scores should validate them with kappa, rerun variance, and calibration, then run verifiable checks and human review on everything the transcript cannot show.

Frequently asked questions

What is LLM-as-a-judge for voice agents?

LLM-as-a-judge means prompting a language model to grade voice agent calls against a rubric. The judge usually reads a post-call transcript and returns a score, label, or preference. It can assess coherence, wording, and apparent completion. It cannot hear tone, latency, interruptions, or pronunciation, and it treats ASR errors as facts.

How do I detect false positives in LLM-as-a-judge voice agent evaluations?

Build a frozen golden set of real calls labeled by humans. Score each call several times. List every call the judge passed but humans failed, then group them by cause. Also check rerun flip rate and accuracy by confidence bucket. Most false passes trace to ASR errors, latency, silent tool failures, or verbosity bias.

What are the main biases in LLM judges?

Research documents position bias, where answer order changes the verdict, and verbosity bias, where longer answers score higher. Self-preference bias means judges favor text from their own model family. Studies also show inconsistency across reruns, sensitivity to prompt wording, and poorly calibrated confidence. The CALM framework catalogs 12 bias types.

Why do LLM judges give different scores on reruns?

Most LLM judges generate text by sampling, often at a default temperature above zero. The same transcript can produce different reasoning and a different score each run. In LangChain's study, three LLM judges showed 92 to 913 times more score variance than a calibrated scorer on identical inputs.

How do I measure LLM judge agreement with humans?

Label a set of calls with two human reviewers. Run the judge on the same calls. Compute raw agreement, then Cohen's kappa, which corrects for chance agreement. Kappa matters when most calls pass, since raw agreement looks high even for a weak judge. Re-run the check after any judge, model, or rubric change.

Can an LLM judge evaluate tone, latency, or interruptions?

Not from a transcript. Text drops pauses, talk-over, vocal tone, and mispronunciations. A 4-second silence and a 400 ms reply look identical on paper. Measure those directly with pipeline timing logs, voice activity events, and audio review. Then join those signals with judge scores for each call.

How do I reduce position bias in an LLM judge?

Run each pairwise comparison twice with the order swapped. Declare a winner only when both orders agree, and call disagreements a tie. Few-shot examples also help. In Zheng et al., few-shot prompting raised GPT-4's position consistency from 65.0% to 77.5%. Avoid always listing the incumbent first.

When should I still use an LLM judge?

Use it when you need a written rationale, when judging nuanced qualities like clarity, and when triaging large call volumes for human review. It also works for checking scripted wording. Validate it against human labels first, and pair it with verifiable checks and audio metrics for anything the transcript cannot show.

Book a demo to see how Evalgent independently audits voice agent evaluations.