How to Choose an STT for Voice Agents (2026)

Speech-to-text is the front door of every voice agent. If the transcript is wrong, everything downstream is wrong too. The language model reasons over bad text. The response is confident and incorrect. So the STT decision carries more weight than most teams give it.

Yet most selection happens on the wrong evidence. A vendor cites a headline accuracy number. A demo runs on clean, scripted audio. Neither reflects your callers, your accents, or your noise. This guide gives you a neutral framework instead. It covers the capabilities that matter and how to test each one on real calls.

This post is about selection criteria — what to look for. It is the companion to our guide on testing STT accuracy, which covers the measurement methodology in depth. Read both together. One tells you what to choose; the other tells you how to measure it rigorously.

Why STT choice is different for voice agents

General transcription and live voice agents are not the same problem. A podcast transcript can be produced offline. Latency does not matter. You can re-run the file. A voice agent works in real time, mid-conversation, with a caller waiting.

That difference changes every criterion. Speech recognition for a voice agent must stream partial results as the caller speaks. It must resolve final text fast enough to keep the turn moving. And it must get the entities right that drive the outcome.

Consider a caller spelling a last name or reading a policy number. A batch transcription tool can be 96 percent accurate overall and still fumble that string. For a voice agent, that single error is the whole call. So accuracy has to be weighted, not averaged.

The criteria that actually matter

Not every STT feature is equally important. Below are the capabilities that separate a production-ready choice from a demo-ready one. Each is a criterion you can test.

Streaming and real-time support. The engine must emit interim results while audio flows. It must finalize quickly and expose stable endpoints. Batch-only transcription cannot power a live turn.

Weighted accuracy on critical entities. Overall word accuracy is a blunt tool. What matters is accuracy on names, numbers, dates, addresses, and domain terms. A single wrong digit breaks a verification step.

Accent and dialect robustness. Your callers are not all native speakers of one accent. The engine must handle the accents in your actual traffic. A US audience still includes enormous accent diversity.

Noise and channel robustness. Real calls have background noise, crosstalk, and low-quality codecs. Phone audio is narrowband and compressed. The engine must hold accuracy under those conditions, not just in a quiet studio.

Latency. Every millisecond of transcription delay adds to the caller's wait. It stacks on top of model and speech-synthesis latency. The engine must return final text fast and consistently.

Diarization. Speaker diarization separates who said what. It matters when calls have multiple speakers or agent overlap. Without it, transcripts blur the caller and the agent together.

Language and locale coverage. Even US-focused agents serve multilingual callers. The engine should cover the languages you need. It should also handle code-switching within a single utterance.

On-prem or private deployment. Regulated industries often cannot send raw audio to a public endpoint. An on-prem or private-cloud option can be a hard requirement. Check it early, before you fall for the features.

Total cost. Per-minute price is only the visible cost. Add latency-driven abandonment, error-driven escalation, and integration effort. The cheapest engine can be the most expensive in production.

How to choose an STT for your voice agent

Use this sequence to move from a long list to a defensible decision. It takes days, not months, and it runs on your own audio.

1. Write down your weighted criteria first. List the criteria above and assign weights. A collections agent weights number accuracy heavily. A support agent may weight accent robustness more. Do this before you look at any vendor.

2. Collect a representative call set. Pull real recordings that span your accents, noise levels, and entity types. Include the hard cases: spelled names, long digit strings, and noisy lines. Aim for coverage, not volume.

3. Transcribe the same audio through each candidate. Feed the identical files to every engine. This is the only way to compare fairly. Our guide on running the same test cases across vendors walks through the setup.

4. Score with weighted, entity-aware metrics. Do not rely on a single word error rate number. Break out error rate on names, numbers, and domain terms. Our word error rate deep dive explains why the weighting matters.

5. Measure streaming latency under load. Time the interim and final results, not just the batch turnaround. Test at your expected concurrency. Latency that looks fine solo can collapse at scale.

6. Verify the non-negotiables. Confirm the on-prem option, the language coverage, and the diarization quality. Check the contract terms and data handling. A missing hard requirement disqualifies an otherwise strong engine.

7. Decide on measured results, weighted by your priorities. Combine the scores using your weights from step one. Pick the engine that wins on your calls, not on a slide. Document the evidence so the decision survives scrutiny.

Criteria, why they matter, and how to verify

How to test accuracy without fooling yourself

The most common mistake is testing on easy audio. Vendors demo on clean, scripted speech. It flatters every engine. Your production traffic looks nothing like it.

Build your test set from real calls. Include the cases that break transcription: heavy accents, background noise, fast speech, and spelled-out entities. Keep a hand-verified reference transcript for each. That reference is your ground truth.

Then score against it with an entity-aware method. Compute overall error rate, but also break it out by category. A high overall score can hide a fatal weakness on numbers. The STT evaluation methodology guide covers scoring in detail.

Re-run the same set on every candidate. Same audio, same reference, same metrics. Anything else is a comparison you cannot defend. This discipline is the core of any credible voice agent stack decision.

How to test latency the right way

Latency) for a voice agent is not batch turnaround. It is how fast the engine streams usable text mid-turn. Two numbers matter: time to first interim result, and time to a stable final result.

Measure both under realistic conditions. A single test call tells you little. Concurrency changes everything. Run the test at the call volume you expect in production, then at your peak.

Anchor your target to human conversation norms. The ITU's G.114 recommendation sets guidance on acceptable one-way latency. STT delay is only one slice of the total budget. Leave room for the language model and speech synthesis too.

Watch the tail, not just the average. A good median with a bad 95th percentile still produces awkward pauses. Callers notice the worst turns, not the typical ones. Track the full distribution.

How to weigh cost honestly

The per-minute rate is easy to compare and easy to over-weight. It is rarely the largest cost. The real expenses hide downstream.

Transcription errors trigger escalations to human agents. Each escalation costs far more than a minute of transcription. High latency drives call abandonment. Poor accuracy erodes trust and repeat usage. All of that is cost.

Integration and maintenance count too. An engine that needs heavy tuning consumes engineering time. An on-prem deployment adds infrastructure. Model the full picture across a realistic call volume, not the headline rate.

Fold cost into your weighted scorecard alongside accuracy and latency. The cheapest engine on paper can lose on total cost of ownership. Decide on the whole number, not the sticker price.

Governance and risk considerations

STT selection is also a risk decision. The engine handles sensitive audio: names, account numbers, and health or financial detail. Data handling and residency belong in your criteria, not just accuracy.

Map your choice to a recognized framework. The NIST AI Risk Management Framework offers a structured way to reason about model risk, bias, and reliability. Accent bias in transcription is a real fairness concern. Document how you tested for it.

Keep evidence of your evaluation. Regulated buyers and auditors will ask how you chose. A weighted scorecard with real-call results answers that question. It also sets a clear production readiness bar for the engine before launch.

Choosing an STT with Evalgent

Evalgent is an independent testing and evaluation platform for voice agents. It lets you compare STT engines on your own calls, using consistent criteria, without relying on vendor benchmarks. The platform is built around five primitives.

• Scenarios capture the calls that matter: spelled names, long digit strings, noisy lines, and multilingual turns.
• Profiles model the caller variety in your traffic, so accents and speech patterns are represented, not idealized.
• Metrics score entity-weighted accuracy, streaming latency, and diarization quality, broken out by category.
• Evaluations run the identical audio through each candidate engine and compare results side by side.
• Reviews put humans in the loop to check edge cases and confirm the numbers reflect real call quality.

Together they turn STT selection into evidence you can defend. You choose on measured results, weighted by your priorities. To see it on your own calls, book a demo. It pairs naturally with a structured approach to evaluating any voice agent vendor.

Frequently asked questions

What is the single most important STT criterion for voice agents?

Streaming latency and entity accuracy tie for first. A live turn needs interim results fast, and one wrong name or number can break the call. Weight both heavily, then verify them on your own recorded audio rather than trusting a headline benchmark number.

Why not just pick the STT with the best accuracy score?

Headline accuracy is an average across easy audio. It can hide a fatal weakness on names or numbers. It also ignores latency, noise robustness, and deployment needs. Score entity accuracy separately and weight it, then combine it with the other criteria that decide call outcomes.

How is choosing an STT different from evaluating one?

Choosing is about criteria: what capabilities matter and which trade-offs fit your calls. Evaluating is the measurement methodology behind those criteria. This guide covers selection; the companion STT evaluation guide covers how to score accuracy rigorously. Use them together for a defensible decision.

Do I need diarization for a voice agent?

It depends on your calls. If turns are clean and single-speaker, diarization matters less. If calls have overlap, multiple speakers, or handoffs, you need it. Test diarization quality against a hand-checked reference before assuming the engine handles your traffic well.

How much does streaming latency really matter?

A great deal. Transcription delay stacks on top of model and speech-synthesis latency. Callers notice pauses, especially on the worst turns. Measure time to first interim result and time to stable final text, under realistic concurrency, and track the tail latency, not just the average.

Should I require an on-prem STT option?

Only if your data rules demand it. Regulated industries often cannot send raw audio to public endpoints. If that applies, treat on-prem or private deployment as a hard requirement and confirm it in writing early. Otherwise, a hosted option may be simpler and cheaper to run.

How many test calls do I need to compare engines?

Coverage matters more than raw volume. Include your accent mix, noise levels, and hard entities like spelled names and long numbers. A few dozen well-chosen, hand-verified calls beats thousands of easy ones. Run the identical set through every candidate for a fair comparison.

How do I stop vendors from gaming the comparison?

Control the audio. Use your own recorded calls, your own reference transcripts, and your own weighted metrics. Never accept vendor-supplied benchmarks or demo clips as evidence. Run the same files through every engine, score them the same way, and decide on those measured results alone.

The bottom line

Choose an STT on the criteria that decide call outcomes, verified on your own audio. The engine that wins on your calls, not on a slide, is the right one.