Voice Agent Workflow Testing: Partial Failures and Sagas

A caller says "book me Thursday at three, and take the deposit." Your agent does not make one API call. It holds a slot, authorises a card, commits a booking, captures money, sends a text and writes the CRM record. Six steps, six systems, one phone call.

Most teams test each tool alone. That misses the bugs that cost money: a charge with no booking, a booking with no charge, a text that confirms something that never happened. Voice agent workflow testing is about the chain. It models a call as a state machine, borrows the saga pattern from distributed systems, and gives you a step-failure matrix and YAML scenarios.

It does not repeat earlier work. For duplicate rows after the call, read voice agent post-call write-back testing. For tool timeouts and retries, read voice agent MCP server testing. Here the question is: when step four of six fails, what state is the world in?

The example is hypothetical: a booking platform for salons and clinics with its own phone agent. The shape also fits order taking and claims.

A call is a workflow, not a tool call

Voice agent workflow testing differs from tool testing. Tool-call tests ask whether one function got the right arguments; see tool call accuracy for voice agents and voice agent tool-calling test cases. A workflow test asks whether the final state is consistent across several systems.

Our example workflow has six steps:

1. hold_slot reserves the time for five minutes.

2. authorize_deposit places a hold on the caller's card.

3. commit_booking writes the appointment.

4. capture_deposit takes the money.

5. send_confirmation texts the caller.

6. write_back updates the CRM.

Each step can succeed, fail cleanly, or time out with an unknown result. The last case is the dangerous one. The request may have landed, and you cannot tell.

Failures also stack. Take an illustrative 99% success rate per step. Six steps succeed together about 94% of the time, since 0.99 to the sixth power is 0.941. At 2,000 workflows a day, roughly 117 chains hit at least one failed step. These numbers are made up to show the math. Yours will differ, but the multiplication will not.

Voice agent workflow testing starts with a state machine

A state machine makes every outcome explicit. Give each step one of six states: `pending`, `in_flight`, `succeeded`, `failed`, `unknown` and `compensated`. Store them outside the model.

The `unknown` state is what most agents lack. Without it, a timeout becomes "succeeded" or "failed" by guess, and a guess is often wrong. The right move for `unknown` is to ask the downstream system, using the same key, before doing anything else.

Keep this state in a database row, not in the conversation. The language model should not decide whether the deposit was captured. Deterministic code reads the row and tells the model what to say. That is our design opinion, not a standard.

# Simplified step runner. Names are illustrative.
STEPS = ["hold_slot", "authorize_deposit", "commit_booking",
         "capture_deposit", "send_confirmation", "write_back"]

def run(workflow_id, ctx, store, systems):
    for step in STEPS:
        row = store.get(workflow_id, step)
        if row.state == "succeeded":
            continue                     # resume skips finished steps
        key = f"{workflow_id}:{step}"    # stable across retries
        store.mark(workflow_id, step, "in_flight")
        try:
            systems[step].do(ctx, idempotency_key=key)
            store.mark(workflow_id, step, "succeeded")
        except Timeout:
            store.mark(workflow_id, step, "unknown")
            return reconcile(workflow_id, step)
        except Rejected as e:
            store.mark(workflow_id, step, "failed", reason=str(e))
            return compensate(workflow_id, upto=step)

The `continue` on a succeeded step is what makes the loop safe to run twice. It is also what makes resume after hangup possible, which we cover below.

The saga pattern for a voice agent workflow

The saga pattern is the standard answer when you cannot wrap several systems in one database transaction. Microsoft's Azure Architecture Center describes it as coordinating "a sequence of local transactions across multiple services," where "a series of compensating transactions undoes the changes that the completed steps made" if a step fails (Azure Architecture Center, Saga design pattern). The microservices.io catalogue defines it as "a reliably coordinated sequence of local transactions" (microservices.io).

Your voice workflow is a saga whether you call it one or not. No system can roll back the others. Four ideas from the Azure page matter for a voice agent saga pattern test plan:

• Compensable steps can be undone by "other transactions with the opposite effect."
• A pivot transaction is "the point of no return." After it, later steps must be completed, not undone.
• A retryable step comes after the pivot. It is "idempotent and help[s] ensure that the saga can reach its final state."
• Compensations can fail: "Compensating transactions might not always succeed."

Azure also names two styles. In choreography, services react to each other's events. In orchestration, a central controller "stores and interprets the states of each task, and handles failure recovery." Orchestration fits a voice agent better, because one owner of the state makes the failure matrix testable. Azure lists the orchestrator as a point of failure as its cost.

Classify every step: compensable, pivot or retryable

Before voice agent workflow testing begins, label each step. The label decides what a failure should do.

Two choices here are design decisions. Make them on purpose.

Where is the pivot? We put it at commit_booking. Before it, a failure rolls back cleanly. After it, the caller has an appointment, so we push forward. If you captured money first, your pivot would sit earlier and your compensation would be a refund, which is slower and visible on a statement. Splitting authorisation from capture makes the undo a cheap cancel. Stripe's docs, as one example, say an uncaptured authorisation can be cancelled and that if it expires before capture "the funds are released" (Stripe, place a hold on a payment method).

How long can a retry run? The same page lists authorisation windows: 5 days for card-not-present Visa transactions classed as merchant-initiated, and 7 days for most other combinations. A capture that retries for a week can fail because the hold expired. Test that boundary, and check the current table for your processor.

For the payment-taking side of this flow, see taking payments over the phone with a voice agent.

The step-failure matrix for voice agent workflow testing

This is the core test artifact. For each step, inject each failure and record what must be true afterward. Four failure modes cover most of the ground.

Read each cell as a test with three assertions: stored step states, downstream system state, and the spoken line. A cell is not covered until all three pass.

The last column is the silent failure: a 200 with the wrong object. It is covered in detecting silent tool failures, and the write-back post handles the record diff.

Six steps and four modes give 24 cells. One hangup test per step boundary brings it to about 30 scenarios. That is more than most teams run by hand.

Compensating actions need their own tests

Compensation is code, and it fails. Azure's compensating transaction page says the steps should be "idempotent commands so that you can repeat them if the compensating transaction itself fails." It adds that "the system should record progress so that it can resume the compensating transaction from the point of failure" (Azure, Compensating Transaction pattern).

The same page says compensating steps "don't always reverse the original operation in the exact opposite order," because you may undo the more sensitive store first. For money, cancel the card authorisation (the authorization hold) before you release the slot. Here that happens to match reverse creation order; the rule is to undo the more sensitive store first. A held slot costs one appointment. A stuck authorisation costs trust.

Test these cases:

1. Compensation succeeds. Fail commit_booking, then assert the auth is cancelled and the hold released.

2. Compensation fails once. Fail the cancel call, then assert a retry with the same key.

3. Compensation keeps failing. Assert the workflow lands in a human queue (a dead-letter queue) with the reason. Azure advises an alert with "detailed information about the reason for the failure."

4. Compensation runs twice. Replay the message. Assert one cancel and no error.

5. Concurrent change. Another call took the slot meanwhile. Assert the compensation leaves it alone.

The last case matters because simple restoration "can overwrite changes from other concurrent application instances," as Azure puts it.

Keys: workflow ids and step keys are different things

Teams often use one identifier for everything. Two scopes work better.

A workflow id names the business process: this caller's booking. Temporal's docs call a Workflow Id "a business-process identifier, such as customer identifier or order identifier," and guarantee "at most one Workflow Execution with a given ID running at any point in time" (Temporal, Workflow Id). You do not need that product to use the idea. It stops a second call starting a second booking for the same intent.

A step key names one side effect inside the workflow. On its v1 API, Stripe saves "the status code and body of the first request made with a given idempotency key" and returns it on retries, including `500` errors. It errors if a reused key arrives with different parameters, and it may prune v1 keys once they are at least 24 hours old (Stripe, idempotent requests). The workflow id vs idempotency key split follows from this.

Three rules for your tests:

• Derive step keys from the workflow id and step name. A random key per attempt stops retries being retries.
• Change the key when the intent changes. "Make it four o'clock" after a failed charge is a new intent.
• Mind the pruning window. Test a resume at 25 hours against a short-lived key.

Retry mechanics live in the write-back post and the MCP testing post. This post adds only the scoping rule.

What the agent says at each failure

The spoken line is part of the state. If the system says "failed" and the agent says "you're all set," the caller will not call back, and nobody will fix it.

Give the agent honest lines per state, driven by the stored row, not by the model's guess:

"You have not been charged" is true only after compensation succeeded, so gate that line on the `compensated` state.

Transcript-only checks fall short here. The words can be right while the pause before them is eight seconds of dead air, or the agent talks over the caller's correction. Evalgent evaluates from the audio rather than only the transcript, and its caller profiles (accent, pace, interruption level, response delay) let you run one failure scenario against a caller who interrupts the "still confirming" line.

Resumability after hangup

Callers hang up mid-workflow. Mobile calls drop. The workflow must not die with the call.

The risk is architectural. If the saga runs inside the call session, it ends with the session. LiveKit's agents docs say shutdown hooks run after the session ends and "should complete within a short amount of time," with a default of 10 seconds before forced termination (LiveKit Agents, job lifecycle). A six-step saga with a slow payment API will not fit. Other frameworks differ, so check yours.

The safer design splits the work. The call handler writes the workflow row and the first step, then speaks. A separate worker advances the workflow from the row, with or without a live call. Retryable steps finish on their own, and a text goes out for anything that needs the caller.

Test it by cutting the call at each step boundary and mid-step. Assert that the worker finishes or compensates, no step runs twice, and the caller's message matches the final state. Run each cut twice, because the second resume is where double-charge bugs live.

Ordering and race conditions between steps

Steps run in order inside one workflow, but workflows run concurrently. Two callers want one slot. One caller phones twice. A staff member edits the booking while the agent works.

Azure's saga page names the anomalies: lost updates (one saga overwrites another's change), dirty reads (reading another saga's unfinished update) and fuzzy reads, where "updates occur between the reads." Each has a voice version:

• Lost update. The agent reschedules while a human cancels. The record shows the old time.
• Dirty read. The agent quotes a slot another call has only held.
• Fuzzy read. The caller deliberates for 40 seconds, and the slot is gone at commit.

Two defences are cheap to test. First, hold with an expiry and re-validate at commit. Second, use conditional writes so a stale update fails loudly. HTTP has this built in: `If-Match` "can be used to prevent the lost update problem," and a mismatch returns `412 Precondition Failed` (MDN, If-Match). Without ETags, add a version field to your own API.

A race condition needs real concurrency to show up. Fire two simulated calls for one slot within a second and assert exactly one booking. Then delay the "yes" past the hold expiry and assert the agent re-checks instead of committing.

How to test a voice agent workflow

Work through these steps in order.

1. Write the workflow as data. This is your voice agent state machine on paper. List steps, classes (compensable, pivot, retryable), compensations, key rules and the allowed spoken lines. Keep it in version control next to the prompt.

2. Build fault injection into your test environment. Force a reject, a 5xx, a timeout after commit and a wrong-object success at any step; see voice agent staging environment.

3. Generate the matrix. One scenario per cell, plus one hangup per boundary. Name them `step.mode` so failures read cleanly. Evalgent generates test scenarios from an agent's prompt and flows, covering states, branches, policies and coverage gaps, which gives you a list to compare with your matrix.

4. Run each scenario through a real call path. Use a simulated caller over phone, SIP or WebRTC so audio, timing and interruptions are in play.

5. Assert three layers. Partial failure testing is only as good as its assertions: Stored step states, downstream system state (hold, auth, booking, message, record), and the spoken line.

6. Replay each scenario twice. The second run catches non-idempotent compensation and resume bugs.

7. Add race and expiry scenarios. Two concurrent callers, a slow caller, a resume after the key window.

8. Gate releases on the matrix. Every prompt, schema or integration change reruns it before it ships to live callers.

Here are three scenarios in YAML. The format is illustrative, not a specific tool's schema.

- id: commit_booking.timeout_unknown
  inject: {step: commit_booking, mode: timeout_after_commit}
  caller: {profile: fast_talker, noise: street}
  expect:
    steps: {hold_slot: succeeded, authorize_deposit: succeeded,
            commit_booking: unknown}
    system: {bookings_for_key: 1, auths_open: 1}   # no second booking
    say_not: ["you're booked", "all set"]
    say_any: ["still confirming", "text you"]

- id: commit_booking.reject_then_compensate
  inject: {step: commit_booking, mode: reject_409}
  expect:
    steps: {authorize_deposit: compensated, hold_slot: compensated}
    system: {auths_open: 0, holds_open: 0}
    say_any: ["not been charged"]

- id: hangup.after_commit
  inject: {hangup_after: commit_booking}
  expect:
    worker: {finishes: [capture_deposit, send_confirmation, write_back]}
    system: {captures_for_key: 1, sms_sent: 1}
    replay_twice: true

Scoring the workflow

Score the workflow, not each call. Track matrix pass rate (100% for money steps), orphan count (mixed outcomes such as charged with no booking; target zero), compensation success without a human, and resume correctness. State assertions are deterministic, so one run each is enough. The spoken line is not, so repeat each scenario with varied caller profiles. For repeated-run math, see voice agent SLOs.

Watching workflows in production

Tests cover the failures you imagined. The cheapest production monitor is a query for workflows stuck in a non-terminal state.

-- Workflows with a step stuck in unknown or in_flight over 10 minutes.
SELECT workflow_id, step, state, updated_at
FROM workflow_steps
WHERE state IN ('unknown', 'in_flight')
  AND updated_at < now() - interval '10 minutes'
ORDER BY updated_at;

Run a second query for mixed outcomes, such as a captured deposit with no committed booking. Each row is a real caller in a bad state.

Call-level monitoring complements this. Evalgent's production call monitoring takes your call recordings, transcribes them, maps each call to a scenario, judges it, and flags tool-call failures, PII leakage, latency spikes and policy failures with fix recommendations. For wider guidance, see monitoring AI voice agents in production and why voice agents fail in production.

Where this fits in the series

Blast radius covers how one failure spreads, integration testing covers system boundaries, rollback covers undoing completed actions, and permissions covers what the agent may write. Whole-call checks live in voice agent end-to-end testing.

Frequently asked questions

What is voice agent workflow testing?

It tests the whole chain of steps a call triggers, not single tool calls. Model the chain as a state machine, force each step to fail, then check stored state, downstream systems, compensating actions and the spoken words. The goal is a consistent final state after every failure.

How does the saga pattern apply to a voice agent?

A call touching several systems cannot use one database transaction. A saga runs local steps in order and, on failure, runs compensating steps for the completed ones. Label each voice step compensable, pivot or retryable, then test every compensation, including its own failure.

What should a voice agent say when one step fails?

It should say only what the stored state supports. Confirmed means state it plainly. Unknown means say it is still confirming and offer a text. Failed and compensated means say nothing was completed or charged. Never say "booked" or "charged" until the step reads succeeded.

What is the difference between a workflow ID and an idempotency key?

A workflow ID names the whole business process, such as one caller's booking, and prevents a second copy starting. An idempotency key names one side effect inside it, such as one charge, and makes retries safe. Derive step keys from the workflow ID and step name so retries reuse them.

What happens if the caller hangs up mid-workflow?

If the saga runs inside the call session, it can die with the call. Persist each step's state and let a separate worker finish or compensate the workflow. Then test by cutting the call at every step boundary and replaying the resume twice to confirm nothing runs twice.

Which steps of a voice agent workflow need compensating actions?

Steps before the point of no return need them. Holds, card authorisations and reservations are compensable, so each needs a tested undo. Steps after the pivot need idempotent retries plus a human queue instead. Irreversible steps should run only after validations pass.

How do you test partial failure in a voice agent?

Inject a failure at each step: a clean reject, a server error, a timeout after the side effect, and a success with the wrong object. Run each through a real call path. Assert the stored step states, the downstream systems and the spoken line, then replay the scenario to catch repeat effects.

How many failure scenarios do you need per workflow?

Start with steps times failure modes, plus one hangup per step boundary. Six steps and four modes give 24 cells and about 30 scenarios with hangups. Add race and key-expiry cases on top. This count is a planning estimate, and the real list grows with branches.

The bottom line

A voice agent workflow is a saga, so test it as one: every step, every failure, every compensation and every spoken line. Teams that only test tools will find the half-finished chains in their customers' records first.