How to Measure Tool Call Accuracy for Voice Agents

# How to measure tool call accuracy for voice agents

> Quick answer: Tool call accuracy for voice agents is a composite metric. It measures whether the agent chose the right tool when one was needed, passed a valid schema and correct arguments, and the call executed. Score each stage, then roll up to one rate that never hides a stage-level failure.

A voice agent that only talks is easy to grade. One that acts is not. The moment an agent books an appointment, moves money, or updates a record, it calls a tool. That tool call is where the real work happens, and where the real failures hide.

Teams often report a single number for this: tool call accuracy. The problem is that one number rolls up at least four different things. Did the agent pick the right tool? Did it call one at all when it should have? Was the payload valid? Did the call succeed? A blended score can look healthy while one of those stages quietly fails every day.

This article shows how to measure tool call accuracy as a composite metric. We break it into stages, apply the right test to each, and roll the stages up into one rate you can gate on. Along the way, we connect it to the deeper metrics for arguments and execution that it depends on.

What tool call accuracy actually measures

> Tool call accuracy: the share of tool-calling decisions a voice agent gets fully right, judged across selection, schema, arguments, and execution against known-correct behavior. A call counts as accurate only when every stage passes.

A tool call is a chain, not a single event. The agent hears the caller. It decides whether to act. If it acts, it names a function) to call. It fills that function's parameters). The runtime sends the request to an API. The call returns, or it does not.

Tool call accuracy is the composite over that whole chain. It is not one test. It is a rollup of stage-level tests, joined by a strict rule: a call is accurate only if it passes every stage. A perfect tool choice with a wrong argument is still a failed call. A perfect payload that never executes is still a failed call.

This is why the metric is easy to misread. People treat it like a single measurement and quote it like a batting average. It is really an AND across four gates. The value of the number depends entirely on whether you kept the stages visible underneath it.

The four stages a tool call has to pass

Every tool call moves through the same four stages. Each one can fail on its own. Each one needs its own assertion.

The first stage is selection. Given what the caller said, did the agent call the right tool, and did it call one only when a call was warranted? This has two failure directions. The agent can call the wrong tool. It can also call a tool that should not have fired, or skip one that should have.

The second stage is schema. The chosen tool has a defined shape: required fields, types, and allowed values. Schema accuracy asks whether the payload is well formed against that contract. A missing required field or a wrong type is a schema failure, even before anyone checks the values.

The third stage is arguments. The payload is valid, but are the values correct? Did the agent pass the date the caller meant and the amount they said? This is a separate question from schema, and it is deep enough to deserve its own treatment, which we give in the piece on tool argument accuracy for voice agents.

The fourth stage is execution. The call is well formed and correct, but did it actually work? Did the API return success, or did it time out, error, or come back empty while the agent claimed success? Catching that class of miss is the subject of our guide to detecting silent tool failures.

Compose these four and you have tool call accuracy. Keep them separate and you can see which stage is dragging the score down.

Selection: the right tool, and only when needed

Selection is the stage teams measure least well, because they only look at calls the agent made. That view misses half the errors. A complete selection metric has to account for calls the agent should have made and did not, and calls it made that it should not have.

This is a classification problem, and it maps cleanly onto precision and recall. Frame each opportunity to call a tool as a decision. The agent either calls or does not. The ground truth either wanted a call or did not. That gives you the four cells of a confusion matrix.

A true positive is a warranted call the agent made with the correct tool. A false positive is a call it made when none was warranted, or with the wrong tool. A false negative is a warranted call the agent skipped. A true negative is a case where no call was needed and none was made.

Precision answers one question: of the calls the agent made, how many were warranted and correct? Recall answers the other: of the calls it should have made, how many did it make? A chatty agent that calls tools too eagerly has low precision. A passive agent that fails to act when the caller asks has low recall. You need both numbers, because a single accuracy figure hides the trade-off.

To combine them into one selection score, use the F-score, which balances precision and recall. Report the components too. The blend tells you the level; the parts tell you the direction of the error.

The tool-call stages, failure modes, and how to measure each

Each stage fails in a distinctive way and needs a matching test. Treating the whole chain as one pass or fail hides the failure mode. The table below maps the four stages to what breaks and the assertion that catches it.

Read the table as a sequence of gates. A case has to clear selection before schema matters, clear schema before arguments matter, and clear arguments before execution is meaningful. Measuring in that order tells you where the chain broke, not just that it broke.

How to measure tool call accuracy step by step

Tool call accuracy is measured against known-correct behavior, not estimated from a demo. You build labeled cases, run real audio through the live agent, capture the full tool-call trace, and score each stage. Here is the end-to-end method.

1. Enumerate every tool and its schema. List each tool the agent can call. For each, write down the required fields, types, allowed values, and constraints. This contract is what the schema and argument checks assert against. A field that expects one of three values is a different test from a free-text note.

2. Build labeled cases, including cases where no tool should fire. For each scenario, record or synthesize spoken audio. Label the correct tool, the expected payload, and the expected result. Include negative cases where the right action is to call nothing. Without those, you can never measure a false positive.

3. Run the audio end to end through the live agent. Play the audio into the real pipeline, not a text prompt. The decision has to travel through recognition, reasoning, and formatting, because that whole path is what you are grading. This mirrors how the agent behaves on a real call.

4. Capture the full tool-call trace. Log the tool the agent chose, the raw payload it sent, and the result the API returned. Instrument the tool boundary so nothing is lost. You need the actual bytes, not a paraphrase, to score schema, arguments, and execution honestly.

5. Score each stage with the assertion that fits it. Grade selection against the labeled decision. Validate the payload against the schema. Compare each argument to its known value. Check the returned result against the system of record. Record a pass or fail per stage, per case.

6. Compute precision and recall for selection. Using the labeled decisions, count true positives, false positives, and false negatives. Report precision, recall, and F-score. Keep the negative cases in the denominator so an over-eager agent is penalized, not rewarded.

7. Roll up per stage first, then to a gated composite. Report a rate for each stage. Then compute the composite: the share of cases that passed all four stages. Gate release on both the composite and the stage rates, so a single weak stage cannot pass by hiding in the average.

Run this on your own scenarios, not a generic script. Grading on the calls your agent will actually take is what makes the number trustworthy, a point we develop in the guide to benchmarking on your own data.

Rolling up to a single rate without hiding stage failures

A composite metric is useful only if it does not bury the parts. The safe way to roll up is to keep two views live at once. One is the composite pass rate: the share of cases that cleared all four stages. The other is the vector of stage rates that feeds it.

The composite has to use a strict AND. A case passes only when selection, schema, arguments, and execution all pass. Do not average the four stage scores into a mean. A mean lets a strong stage mask a weak one. An agent with 99% schema accuracy and 88% execution accuracy does not have a 93% tool call. It fails whenever either stage fails.

Watch for a specific illusion. Suppose selection is 96%, schema is 99%, arguments are 94%, and execution is 97%. Each looks fine alone. The composite, if the failures land on different cases, can drop well below any single stage. That gap between the stage rates and the composite is the signal. A wide gap means failures are spread across stages, and no single fix will close it. Treat these figures as illustrative, not measured.

Weight the gate by risk, not by volume. A rare tool that moves money deserves a stricter bar than a common tool that reads a FAQ. Set the pass bar per tool and per stage, weighted by what a mistake costs. This mirrors how a good voice agent metrics scorecard treats different metrics: one blended average is never the whole story.

Precision and recall for the "should it have called" question

The selection stage deserves a second pass, because it is where the composite metric most often goes wrong. Most teams score only the calls that happened. That measures precision and ignores recall entirely.

Consider an agent that never calls a tool. On the calls it did make, it is never wrong, because it made none. A naive accuracy score rewards it. Yet it is useless, because it skips every action the caller asked for. Recall exposes that failure. It counts the warranted calls the agent missed.

The opposite failure is just as real. An agent that fires tools on every turn will catch every warranted call, so its recall is high. But it also fires when it should stay quiet, which tanks its precision. Each spurious call is a false positive: an action taken with no mandate. In a system that moves money or changes records, a false positive is not harmless noise. It is an unwanted side effect.

This is why the negative cases in step two matter so much. If your test set only contains calls that should happen, you can measure recall but never precision. Add cases where the correct behavior is to ask a clarifying question or do nothing. Only then can you catch the over-eager agent. Research on API-calling models, such as the Gorilla work on connecting language models to APIs, treats both hallucinated calls and missed calls as first-class errors for the same reason.

Where tool call accuracy fits in your evaluation program

Tool call accuracy is one metric among many, and it works best inside a wider program. It answers a specific question: can the agent act correctly through its tools? It does not tell you whether the conversation was natural, whether the agent followed policy, or whether latency stayed low. Those live in other metrics.

It also depends on layers below it. Selection depends on the model reasoning well over a clean transcript. Arguments depend on recognition and resolution. Execution depends on the integration and the endpoints. When tool call accuracy drops, the stage breakdown tells you which layer to inspect. That is the practical payoff of measuring by stage instead of in aggregate.

There is one more reason to keep the scoring rules outside the team that built the agent. The assertions here involve judgment. How strict is the schema check? What counts as a warranted call? Where is the gate set? A team grading its own agent has an incentive to set those bars generously. An outside evaluator does not. Evalgent runs these checks as an independent, third-party audit, so the rules are set by someone who gains nothing from a passing grade. The case for that separation is laid out in our piece on why independent voice AI evaluation matters.

Finally, tool call accuracy is an evaluation metric, not a monitoring dashboard, and the difference matters. Evaluation measures the agent against labeled cases before and after release. Monitoring watches live traffic. The distinction, and why you need both, is covered in the guide on testing versus evaluation for voice agents. A full voice agent evaluation program uses tool call accuracy as one gate among several, not as the only bar.

Frequently asked questions

What is tool call accuracy for voice agents?

Tool call accuracy is a composite metric that measures whether a voice agent handled a tool call correctly across four stages: choosing the right tool when one was needed, producing a valid schema, passing correct arguments, and executing successfully. A call counts as accurate only when every stage passes. One weak stage fails the whole call.

How do you measure tool call accuracy?

Build labeled test cases, including cases where no tool should fire. Run the audio through the live agent and capture the full tool-call trace: the tool chosen, the payload, and the result. Score selection, schema, arguments, and execution separately. Then compute the composite as the share of cases that passed all four stages.

How do you decompose tool call accuracy by stage?

Split the metric into selection, schema, arguments, and execution. Selection checks whether the right tool fired when needed. Schema checks the payload shape. Arguments check the values. Execution checks whether the call succeeded. Score each stage with its own assertion, report a rate per stage, and roll up with a strict AND across all four.

How do you use precision and recall for tool calls?

Frame each case as a decision to call a tool or not. A warranted correct call is a true positive. An unwarranted or wrong call is a false positive. A skipped warranted call is a false negative. Precision measures how many calls were justified. Recall measures how many needed calls happened. Report both, plus the F-score.

How do you roll up tool call accuracy into one number?

Use a strict AND, not an average. A case passes the composite only when selection, schema, arguments, and execution all pass. Report the composite alongside the four stage rates. A wide gap between the stage rates and the composite means failures are spread across stages, so no single fix will raise the score.

Why did my voice agent call the wrong tool?

Wrong-tool selection usually comes from ambiguous intent, overlapping tool descriptions, or a prompt that does not separate similar functions clearly. The agent maps the caller's request to the closest-looking tool. Measure this with the selection stage: label the correct tool per case and score choices as true or false positives against that label.

How do you test whether an agent should have called a tool?

Add negative cases to your test set where the correct behavior is to call nothing or to ask a clarifying question. Score the agent's decision against that label. A call made in a negative case is a false positive. Without negative cases, you can measure recall but never precision, so over-eager calling goes undetected.

How is tool call accuracy different from tool argument accuracy?

Tool call accuracy is the composite across selection, schema, arguments, and execution. Tool argument accuracy is one stage inside it: whether the values passed to the tool were correct. An agent can pass the argument stage yet fail the composite by choosing the wrong tool or failing to execute. Measure arguments separately, then roll them into the composite.

The bottom line

Tool call accuracy is a composite metric, not a single number. Measure selection, schema, arguments, and execution separately, then gate on a strict rollup that never lets one weak stage hide.

Ready to see where your agent's tool calls break? Book a demo and Evalgent will audit your tool call accuracy stage by stage, as an independent third party.