Voice Agent Integration Testing: Wired Into Your Product

You built the voice agent inside your own product. It is not a separate vendor widget. Every tool it calls is an endpoint your team owns or depends on: create the order, look up the account, send the text, write the note.

That is where most surprises live. The model can transcribe perfectly and still send `"qty": "two"` to an endpoint that wants an integer. The endpoint can return `200` with an empty body. The agent then says "you're all set" and the caller hangs up happy.

This guide to voice agent integration testing is about those seams. It is deliberately not a whole-call guide. For that, read our end-to-end testing guide. Here we cover boundary tests that run in seconds, fail with a clear cause, and catch drift before a live caller does. The wider risk is covered in voice agent blast radius.

Voice agent integration testing versus end-to-end testing

Integration test: a test of one boundary between your application and something outside it, run with the rest of the system held constant.

Martin Fowler's practical test pyramid defines integration tests as tests of "the integration of your application with all the parts that live outside of your application." He advises pushing tests "as far down the test pyramid as you can," with fewer tests as you go up.

Apply that to voice. A whole-call test needs telephony, speech-to-text, the model, text-to-speech and every tool. When it fails, you get one red result and a long search. A boundary test fails on one seam and names it.

Whole-call tests still matter. They catch what boundaries cannot: barge-in, noise, accents and timing. We keep a small set of them at the top, which is the shape in the diagram below. The counts are illustrative, not a benchmark.

The seven seams that voice agent integration testing covers

Walk the path of one call that changes data. Each handoff is a seam. Each needs its own test, because a failure there looks like a normal call from outside.

Seam 7 is the one teams forget. If the agent runs across many of your customers, the same code reads different menus, hours and rules per tenant. A change that is fine for one tenant can break another.

Worked example (illustrative): pizza POS software with built-in AI phone ordering. One tenant has a "half and half" option. Another tenant's menu has none. A prompt change that teaches the agent to offer half-and-half is correct for the first and a silent error for the second.

Voice agent contract testing for every tool the agent calls

In voice agent API testing, a tool has two contracts. The first is what the model must send. The second is what the agent can rely on getting back. Test both.

Contract one: what the model sends

This is voice agent tool call testing at its simplest. Your tool definition is a JSON Schema. OpenAI's function calling guide shows its examples with `"additionalProperties": false`, `"strict": true` and every property listed in `required`. It also advises you to "use enums and object structure to prevent invalid states."

Closing the schema matters because JSON Schema allows extra properties by default. An open schema accepts an argument you never meant to support. The MCP specification also says servers must validate all tool inputs, and tools may declare an `outputSchema` so clients can validate results. That page is the 2026-07-28 revision of the spec. Under it, input validation failures are returned as tool execution errors with `isError: true`, so the model can retry with corrected arguments. Revisions change, so check the current one before you copy field names.

A minimal contract test, simplified:

import json, pathlib
from jsonschema import Draft202012Validator

SCHEMA = json.loads(pathlib.Path("contracts/create_order.args.json").read_text())

def test_schema_is_closed():
    assert SCHEMA["additionalProperties"] is False
    # our convention: no optional fields, so omissions cannot hide
    assert set(SCHEMA["required"]) == set(SCHEMA["properties"])

def test_golden_arguments_validate():
    v = Draft202012Validator(SCHEMA)
    for f in pathlib.Path("goldens/create_order").glob("*.json"):
        errors = [e.message for e in v.iter_errors(json.loads(f.read_text()))]
        assert not errors, (f.name, errors)

The goldens are real argument payloads captured from reviewed test calls. When someone edits the schema, every golden is replayed. A renamed field fails in CI, not on a call. Our tool-calling test cases post lists the argument cases worth capturing, and tool call accuracy covers how to score them.

Contract two: what the agent can rely on

The agent reads your API's response and turns it into speech. If a field is renamed, the agent may read `null` and invent a sentence. So pin the response shape.

def test_create_order_response_contract(client):
    r = client.post("/orders", json=VALID_ARGS)
    assert r.status_code == 201
    body = r.json()
    assert {"order_id", "status", "total_cents"} <= set(body)
    assert isinstance(body["total_cents"], int)
    assert body["order_id"]

Consumer-driven contracts

Your agent is the consumer. Your product API is the provider. Pact's documentation describes consumer-driven contract testing this way: the contract is generated while automated consumer tests run, and each case describes "a single concrete request/response pair." Only the parts the consumer actually uses are tested, so the provider can change everything else freely.

The Pact flow has three stages. The consumer tests produce a pact file. The provider replays each recorded request against the real service. A broker connects this to CI.

For you this gives one practical rule. The team that owns the order API must not be able to ship a change that breaks the voice agent's recorded calls. It matters most when the API team and the voice team are different people.

Know the limit. Pact's docs note that provider-only contract testing, such as checking against an OpenAPI document, gives no assurance that consumers call the provider correctly. Consumer-driven contracts fix that, but they check only the calls your tests record. So they complement schema tests on the agent side; they do not replace them.

If the agent reaches your systems through MCP, the same two contracts apply to each server tool. See MCP server testing.

Data mapping tests: from spoken to stored

Speech errors do not stay in the transcript. They become field values. The data mapping layer converts "five five five, oh one two three" into a phone number, "next Friday" into a date and "twelve fifty" into money. Test it like any parser, with a table.

This part of voice agent integration testing needs no model. Keep these as parametrized unit tests with no model in the loop. Feed the mapper the string the model would pass and assert the stored value. For codes and IDs, add the confusable-pair cases from alphanumeric capture.

Include time zone and daylight saving cases. A booking for 3:00 PM in the tenant's zone must be stored as that instant, not as 3:00 in the server's zone.

Dependency stubs and voice agent fault injection

You cannot call the real payment processor, SMS gateway and CRM on every pull request. So you stub them. Fowler's note on contract tests explains the catch: a double stops your own tests from being slow and unreliable, but it raises the question of whether the double is accurate. His answer is contract tests that compare the double against the real service, run on the real service's rhythm, roughly daily.

Once you have a stub, you can make it misbehave on purpose. Two open-source tools cover most of this. WireMock supports fixed and random delays, chunked dribble delays, and faults named `EMPTY_RESPONSE`, `MALFORMED_RESPONSE_CHUNK`, `RANDOM_DATA_THEN_CLOSE` and `CONNECTION_RESET_BY_PEER`. Toxiproxy sits in front of a real TCP service and adds latency, timeouts, connection resets, bandwidth limits and data limits that close the connection.

A WireMock stub for a slow success:

{
  "request": { "method": "POST", "url": "/orders" },
  "response": {
    "status": 201,
    "jsonBody": { "order_id": "ord_123", "status": "open", "total_cents": 2150 },
    "fixedDelayMilliseconds": 4000
  }
}

The fault matrix

For each tool, run these faults and assert what the caller hears and what gets stored.

The dangerous row is the fourth. A `200` that carries no usable result is the classic silent failure. We cover how to catch it in production in detecting silent tool failures. The diagram shows the pattern.

Worked math (illustrative): suppose each tool call has a 3 s timeout and one retry after 0.5 s. Worst-case silence is 3 + 0.5 + 3 = 6.5 s. A caller will not wait that long without a sound. So the test asserts a filler line by about 1.5 s and a handoff after the second failure. Pick your own numbers, then test them.

The fault matrix is small. With 7 tools and 5 injected faults (slow, timeout, 5xx, wrong body, empty or cut response) you have 35 cases. At roughly 3 seconds each in text mode, that is under two minutes. It fits in a deploy gate.

Environment parity, test data and sandbox limits

Stubs are fast. Sandboxes are closer to truth. Neither is truth.

Stripe's testing documentation is a useful example of what sandboxes hide. As of its current text, test transactions for ACH Direct Debit settle instantly, while live ones can take days. In live mode refunds are asynchronous and can appear to succeed then fail; test cards mostly succeed immediately. The rate limiter is stricter in test than in live, and the docs warn against load testing there. Card state from one test can also affect later tests. We read that page for those facts only. Your providers will have their own gaps.

So list the gaps for each dependency and decide which test covers each one. Then check parity on the things you control:

• Tool manifest: the tool names, schemas and descriptions in staging match production.
• Prompt and model version: pinned and logged on every call.
• Tenant config: at least three real shapes: simple, large and odd.
• Test data: synthetic, resettable and never a real caller's record.
• Idempotency: every test run can be repeated without leaving residue.

Our staging environment guide covers numbers, sandboxes and smoke tests in depth. Keep this post's focus on the contracts.

What to run, and when

Match each test to its cost and failure mode. Fast and deterministic tests gate every change. Slow or flaky ones run on a schedule.

The right-hand column is where audio matters. Contract and mapping tests cannot hear a caller who talks over the agent or a line with traffic noise. This is the layer where Evalgent applies. It generates test scenarios from the agent's prompt and flows, calls the agent over phone, SIP or WebRTC with caller profiles for accent, pace, interruption and background noise, and judges from the audio, not only the transcript.

How to set up voice agent integration testing

1. Inventory the seams. Voice agent integration testing starts with a list. List every tool, the API behind it, the fields it writes and the side effects it triggers. One row per tool.

2. Close every tool schema. Set `additionalProperties` to false and decide which fields are required. Version the schema in the repo.

3. Capture goldens. Save the argument payloads from reviewed calls, one file per scenario. Replay them against the schema on every change.

4. Write response contracts. For each tool, assert status, required fields and types. Add consumer-driven contracts where another team owns the API.

5. Table-test the mappings. Phone, date, money, name, quantity, code. Include time zones and confusable characters.

6. Stub dependencies and run the fault matrix. Use WireMock or Toxiproxy. Assert both the spoken result and the stored result.

7. Set the gates. Per pull request: schema lint, goldens, mapping tests. Per deploy: fault matrix and a few scripted text calls. Nightly: contracts against real sandboxes and a manifest diff. Weekly: a small audio set.

8. Monitor the same seams in production. Alert on tool-call failures and on any `200` with no usable result. See monitoring AI voice agents.

9. Feed incidents back. Every production failure becomes a golden or a fault case. The loop is in the production feedback loop.

The copyable CI gate list

• Schema files lint clean and are closed.
• Every golden argument payload validates.
• Response contracts pass against stubs.
• Mapping tests pass, including time zones.
• Fault matrix passes: slow, timeout, 5xx, wrong body, empty.
• Tool manifest in staging equals the release candidate.
• Nightly: real-sandbox contract run shows no drift.
• Weekly: audio set passes with noise and accent variants.

For production, Evalgent monitoring takes the call recordings your team pushes, transcribes them, maps them to scenarios and flags tool-call failures, latency spikes and policy failures with fix recommendations. That closes the loop on seams your stubs cannot reproduce.

What this will not catch

Be honest about the limits. Contract tests prove shape, not meaning. A response can match its schema and still hold the wrong customer. Stubs can drift. Sandboxes differ from live. And none of this tests the voice itself.

So keep a thin layer of whole-call tests, and keep production checks. Our overview of AI voice agent testing and the piece on why voice agents fail in production show how the layers fit together. Later in this series we look at multi-step workflow failures and at what the agent is permitted to write; see workflow testing and agent permissions.

Frequently asked questions

What is voice agent integration testing?

Voice agent integration testing tests the boundaries between a voice agent and the systems it touches: tool arguments, API responses, data mappings and write-backs. Each test targets one seam, so a failure names its cause. It sits below whole-call tests and above prompt unit checks in the test pyramid.

How is integration testing different from end-to-end testing for a voice agent?

End-to-end tests run a whole call through telephony, speech, model and tools. Integration tests exercise one seam with the rest held constant. They are faster, more deterministic and easier to diagnose. End-to-end tests catch timing, interruption and audio problems that boundary tests cannot hear.

What is a contract test for a voice agent tool?

It pins the agreement between the agent and a tool. One side checks the arguments the model sends against a closed JSON Schema. The other checks the response shape the agent relies on. Consumer-driven contracts, such as Pact, let the API owner verify the agent's recorded calls.

Should a voice agent be tested against real APIs or stubs?

Both, at different speeds. Stubs run on every change and allow fault injection. Real sandboxes run nightly to confirm the stubs still match. Fowler suggests contract tests against the real service on its own rhythm, about daily, to detect double drift.

How do you inject faults into a voice agent's tool calls?

Put a stub or proxy between the agent and the dependency. WireMock can add delays and return empty or malformed responses. Toxiproxy adds latency, timeouts and resets to a real connection. Then assert what the caller hears and what gets stored.

What should run on every deploy for a voice agent?

Run schema lint, replayed golden arguments, response contracts, mapping tests and the fault matrix. Add a few scripted text calls. All of it should finish in minutes and be deterministic. Slower sandbox and audio tests run nightly or weekly.

How many integration tests does a voice agent need?

There is no standard number. Start with one contract per tool, a golden per scenario, a mapping table per field type, and every fault against every tool. For seven tools and five injected faults that is 35 fault cases. Add one case per production incident.

How do you detect schema drift in a tool the agent calls?

Version the schemas and diff the tool manifest between staging and the release candidate. Replay goldens on every change. Run provider verification or a nightly real-sandbox contract test. A renamed or retyped field should fail a test before it reaches a caller.

The bottom line

Voice agents inside your product fail at the seams, so voice agent integration testing should hit the seams directly with contracts, mappings and injected faults. Keep whole-call tests for what only audio can reveal, and run the fast ones on every deploy.