How to load test Deepgram's Voice Agent API

An agent that answers in 200 milliseconds in your demo can stall at two seconds once real traffic hits, and you will not see that in a single-session test. Load testing is how you find the ceiling before your callers do. Deepgram's Voice Agent API has its own shape — one WebSocket per conversation, concurrency limited per project — so load testing it means testing the things that actually break at scale. Evalgent runs this kind of test with realistic traffic, and this guide walks through how.

Load testing a voice agent API: driving many concurrent, realistic sessions at the API to measure how latency, error rate, and stability behave under production-like traffic, rather than at a single session.

Why load testing a voice agent API is different

Most API load testing thinks in requests per second: fire many short requests, count throughput. A voice agent API does not work that way. Each conversation is a long-lived, bidirectional stream, and the unit of load is a concurrent session, not a request.

Deepgram's Voice Agent API makes this concrete. As its voice agent docs describe, a single WebSocket connection carries the whole conversational loop — speech-to-text in, the LLM, and text-to-speech out. One caller is one open connection for the length of the call. So the question is not "how many requests per second," it is "how many simultaneous conversations can I hold before latency degrades or connections get refused." That reframing is the whole point of load testing a voice agent, and it is why generic API load tools miss the mark.

How Deepgram's Voice Agent API behaves under load

To test it well, you have to know how it meters and limits usage. Three facts shape the test.

First, usage is billed by connection time — one hour of open WebSocket connection equals one hour of usage — so a load test that holds many connections open is also spending real money. Second, concurrency limits apply per project, not per API key or account, and per Deepgram, creating extra projects under the same account does not grant more concurrency. Third, when you exceed your concurrency ceiling, new connections are throttled and you receive 429 responses rather than a silent slowdown. The official concurrency and rate limits docs are the source of truth for your plan's exact numbers.

Knowing this tells you what your load test is actually probing: the concurrent-connection ceiling for your project, and how latency behaves as you approach it.

What to measure under load

A voice agent load test produces a few numbers that matter far more than raw throughput. Track these as you ramp.

The single most revealing number is the gap between p50 and p99. A healthy median with a terrible tail means most calls feel fine while a painful fraction stalls — and at scale that fraction is a lot of real callers. Latency percentiles matter more here than any average, a theme our reduce latency guide covers in depth.

How to load test Deepgram's Voice Agent API

The method is a controlled ramp of realistic sessions, measured at the percentiles.

1. Use real audio — Stream actual speech, not silence or tones, so speech-to-text does real work under load.

2. Test from the right region — Run the load from a cloud VM near your users; a dev-machine benchmark predicts nothing about production latency.

3. Ramp concurrency in steps — Go 5, 25, 50, 100, and up, holding each level long enough to see steady-state behavior.

4. Measure at percentiles — Record p50, p90, and p99 latency and word emission latency at each level, not just the average.

5. Handle 429s honestly — Build retry logic and log throttling separately, so 429 responses do not skew your latency numbers.

6. Find the breaking point — Keep ramping until latency spikes or errors climb, and record the concurrency where it happens.

Concurrency and rate limits: what to know

Your load test only makes sense against your actual limits, so check them first. Per Deepgram, default concurrency was recently tripled across the Voice Agent API, with Growth-plan customers getting up to several times more — enough to run dozens of concurrent agents with headroom on default plans, and enterprise capacity reaching into the many thousands of concurrent connections. Pricing is a flat hourly rate for the agent connection.

The practical implications for testing: because limits are per project, test against the project you will actually run in production, and do not expect a second project to multiply your ceiling. If your target is, say, 200 concurrent calls, your test has to confirm both that your plan allows it and that latency stays acceptable there — the ceiling being high enough is necessary but not sufficient.

Reading the results

The point of the ramp is to find where the numbers turn. A useful real-world illustration: one streaming API held a comfortable 180 milliseconds at 5 concurrent sessions, then jumped to 2.3 seconds at 200 concurrent — the same system, a completely different experience. That is the curve you are hunting for.

Read the results as a curve, not a single pass or fail. Latency that stays flat as concurrency climbs is the goal. Latency that is fine until a threshold and then spikes tells you your safe operating ceiling sits below that threshold. A rising 429 rate marks your hard concurrency limit. Together they give you a capacity number you can plan around: the concurrency at which the agent still feels fast, which is almost always lower than the raw connection ceiling.

Common load-testing mistakes

A few mistakes make a voice agent load test lie to you, and they are easy to avoid once named.

Testing from a dev machine is the most common — local latency has nothing to do with production, so test from the region your callers are in. Testing with silence instead of real audio understates the work speech-to-text does, flattering your numbers. Ignoring 429 retries lets throttled connections pollute your latency stats, so you cannot tell a slow call from a refused one. And extrapolating from a single session is the cardinal sin: linear assumptions break exactly at the concurrency where queueing and contention kick in. The whole reason to load test is that scale is non-linear, which our stress testing guide explores further. Re-run the test after any change to your stack, plan, or expected traffic, since a new model, a region move, or a different pricing tier can all shift where the curve turns — a capacity number from three months ago is not a capacity number you can trust today.

Load testing Deepgram voice agents with Evalgent

Evalgent load tests voice agents the way real traffic hits them — many realistic sessions at once, over real audio. Scenarios drive concurrent conversations, not empty connections, so speech-to-text and the whole loop do genuine work under load. Profiles vary caller voice, accent, and pace across the concurrent sessions, since a load test on one clean voice hides the cohorts that struggle. Metrics record p50, p90, and p99 latency, word emission latency, error and 429 rates, and disconnects as you ramp, with thresholds you set. Evaluations run the ramp as automated batches and surface the concurrency where the numbers turn. Reviews let you replay a slow session at peak load and see which part of the loop stalled.

The result is a capacity number you can trust: the concurrency at which your Deepgram voice agent still feels fast, proven before launch rather than discovered in an incident. For the broader Deepgram testing picture, see our Deepgram STT testing guide, and for the wider discipline the AI voice agent testing pillar.

Conclusion

Load testing Deepgram's Voice Agent API is about concurrent sessions, not requests per second, because each conversation is one long-lived WebSocket carrying the whole loop. Ramp real audio, measure the p99 tail, and find the concurrency where latency turns before production finds it for you.

The number that matters is not the raw connection ceiling but the concurrency at which the agent still feels fast. Test to find it, plan below it, and re-test whenever your stack or traffic changes.

Frequently asked questions

How do you load test Deepgram's Voice Agent API?

Drive many concurrent, realistic streaming sessions at it and measure latency, error rate, and stability as you ramp. Use real audio, test from a cloud region near your users, and step concurrency up in stages. Record p50, p90, and p99 latency plus 429 rates at each level, and keep ramping until the numbers turn to find your safe ceiling.

What are Deepgram's concurrency limits?

Concurrency limits apply per project and depend on your plan. Per Deepgram, default concurrency was recently tripled across the Voice Agent API, with higher tiers getting several times more and enterprise capacity reaching thousands of concurrent connections. Because limits are per project, extra projects under the same account do not add concurrency. Check the concurrency docs for your plan's exact number.

How many concurrent agents can Deepgram handle?

It depends on your plan. Default plans now support dozens of concurrent agents with headroom after Deepgram tripled defaults, while enterprise capacity reaches into the many thousands of concurrent connections. The number your account can run is set per project, so confirm your specific limit, and remember that the concurrency where latency still feels fast is usually lower than the raw ceiling.

How do you measure voice agent latency under load?

Record latency at percentiles — p50, p90, and p99 — at each concurrency level, rather than a single average. For speech-to-text, also track word emission latency, the delay from a word being spoken to its transcript. The p50-to-p99 gap is the key signal: a good median with a bad tail means a real fraction of callers experience stalls under load.

Why does latency spike under concurrency?

Because real-time infrastructure contends for resources as sessions pile up. A system can hold low latency at a handful of sessions and then degrade sharply past a threshold — one streaming API went from 180 milliseconds at 5 sessions to 2.3 seconds at 200. The spike is non-linear, which is exactly why a single-session benchmark cannot predict production behavior.

How do you test Deepgram rate limits?

Ramp concurrent connections until you receive 429 responses, which is how Deepgram signals you have hit your project's concurrency ceiling. Log those 429s separately from latency, and add retry logic so throttled attempts do not skew your numbers. This tells you both your hard limit and how the API behaves as you approach it, which matters for capacity planning.

What metrics matter for load testing a voice agent?

The key ones are p50, p90, and p99 latency, word emission latency, the concurrency ceiling where 429s begin, the error and disconnect rate under load, and the connection-time cost of the test. Raw throughput matters less than for a normal API, because a voice agent's unit of load is a concurrent session, not a request per second.

How do you simulate concurrent voice agent sessions?

Open many WebSocket connections at once, each streaming real audio through a full conversation, and hold them open the way real calls stay open. Vary the voices and pacing across sessions so the test reflects a real caller mix. Run the load from the production region, ramp in steps, and measure at each level. Platforms like Evalgent automate this with synthetic callers.