Evaluating Tool-Argument Accuracy from Speech

# Tool argument accuracy for voice agents, measured from speech

> Quick answer: Tool argument accuracy voice agents care about is whether the agent passed the right values to a tool: the correct date, amount, name, or ID. It is measured per argument, end to end from speech, against known-correct answers. The tool can execute perfectly and the task still fails.

A voice agent can pick the right tool and still get the job wrong. It hears a caller, decides to book an appointment, and calls the booking function. That part looks healthy. But it passes the 15th when the caller said the 50th, or $150 when they said $115. The tool runs. The confirmation email goes out. The task has already failed.

This is the argument layer, and most evaluation programs skip it. Teams check whether the agent called the correct tool. They watch for tool calls that error out. Neither check catches a clean call carrying a wrong value. That is the gap this article closes. We cover what argument accuracy means, why it fails, and how to measure it against known-correct answers from real audio.

What tool-argument accuracy actually measures

> Tool argument accuracy: the share of tool-call arguments the agent extracts correctly from speech, judged per argument against a known-correct value. It is scored on the value passed to the tool, not on whether the tool ran.

A tool call has two parts. There is the tool itself, and there are the arguments) it receives. The function `book_appointment` might take a date, a time, a service, and a customer ID. Each of those is an argument. Argument accuracy asks a narrow question. For each argument, did the agent pass the value the caller actually meant?

This is stricter than it sounds. A call can be right about the tool and the intent and still be wrong about three of four arguments. The agent understood that the caller wanted to book. It got the date, the time, and the ID wrong. From an intent view, that is a success. From the caller's view, it is a broken booking.

The unit of measurement matters. Call-level success hides argument errors. If you score a call as passed when the right tool fires, a wrong date never shows up. Argument-level scoring makes each value its own test. That is the only way to see the errors that reach the caller.

Why the right tool with the wrong arguments still fails

Speech is noisy, and small errors change meaning. Consider the paths a value takes before it becomes an argument. The caller speaks. Speech-to-text transcribes. The language model reads the transcript, decides on a tool, and fills the arguments. Each hop can corrupt the value.

Numbers are the most fragile. "Fifty" and "fifteen" differ by one sound. "Fifteen fifty" could mean $15.50, $1,550, or the year. Amounts get transposed, rounded, or split. A misheard digit in an account number routes money to the wrong place.

Names carry a different risk. Many names are homophones: Sean and Shawn, Catherine and Kathryn, Steven and Stephen. They sound identical and spell differently. The agent has to pick a spelling, and the wrong one can fail a lookup or create a duplicate record.

Dates and times invite format errors. "Next Tuesday" needs a calendar. "The 3rd at 4" needs a year, a month, and a time zone. The agent has to resolve a spoken phrase into a structured value. A resolution mistake books the wrong day.

None of these are tool failures. The tool is fine. The argument is wrong. That distinction is why argument accuracy needs its own evaluation, separate from checking that a tool executed.

How this differs from STT entity accuracy and silent tool failures

Three layers get confused here, and they need separate tests.

The first is speech-to-text entity accuracy. That asks whether the transcript captured the spoken entity correctly, at the word level, using metrics like word error rate. Our companion piece on STT entity accuracy for voice agents covers that measurement in depth. It stops at the transcript.

Argument accuracy sits one layer up. It judges the value that actually reached the tool, after transcription, resolution, and formatting. The transcript can be perfect and the argument still wrong, because the model resolved "next Friday" to the wrong date. The transcript can also carry an error the model quietly corrects. So the two scores can disagree, and both are worth knowing.

The third layer is silent tool failures. That is whether the tool executed at all: whether the call returned, timed out, or failed without surfacing an error to the caller. That is an execution question, not a value question. Argument accuracy assumes the tool ran and asks only whether the inputs were right.

Keeping the three apart tells you where to fix things. A transcript error points at the recognizer. An argument error with a clean transcript points at the model's resolution logic. An execution error points at the integration. Bundling them into one pass rate tells you none of that. For the full picture, an independent accuracy audit scores each layer on its own.

Argument types, failure modes, and how to test each one

Different argument types fail in different ways, and each needs its own assertion. A date needs format-aware comparison. An account ID needs an exact match with no normalization. Treating them the same either misses real errors or flags harmless ones. The table below maps the common argument types to their failure modes and the test that catches each.

The pattern is consistent. Free-form values like names need fuzzy rules with an allow-list. Identifiers need exact matching. Numbers need type-aware comparison that respects magnitude and units. Set your assertion to the argument type, not to a single global rule.

How to run a tool-argument accuracy evaluation

Argument accuracy is measured, not estimated. You build cases with known-correct values, run real audio through the live agent, and compare what the tool received to what it should have received. Here is the end-to-end method.

1. List the tools and their argument schema. Write down every tool the agent can call and the arguments each takes. Note the type, format, and constraints for each argument. This schema is what you assert against. A date argument that expects ISO 8601 is a different test from a free-text note.

2. Build test cases with known-correct arguments. For each scenario, record or synthesize spoken audio and label the exact values the tool should receive. A booking case might specify a date, a time, a service, and a customer ID. These labels are your ground truth.

3. Run the audio end to end through the live agent. Play the audio into the real pipeline, not a text prompt. The value has to travel through recognition, resolution, and formatting, because that whole path is what you are testing. Audio-in beats transcript-in for this reason, as our note on transcript versus audio evaluation explains.

4. Capture the arguments the agent passed. Log the actual tool call and its arguments. You need the values the tool received, byte for byte, not a summary or a paraphrase. Instrument the tool boundary so nothing is lost.

5. Score each argument against its expected value. Apply the assertion that fits the argument type from the table above. Record a pass or fail per argument, with the expected and actual values side by side. Store the mismatch so you can see the failure mode later.

6. Compute per-argument error rate, not just call success. Report accuracy for each argument type separately. A 4% error rate on dates and a 0.2% error rate on names are two different problems. A single blended number hides both.

7. Gate on the arguments that carry risk. Set a pass bar per argument, weighted by the cost of getting it wrong. An account ID that moves money gets a strict bar. A free-text note can be looser. Block release when a risk-bearing argument misses its bar.

Run this on your own scenarios, not a generic script. Benchmarking on the calls your agent will actually take is what makes the result trustworthy, a point we develop in the guide to benchmarking on your own data.

Scoring: exact match, normalization, and per-argument error rate

The scoring rule decides what counts as correct, so it needs care. Some arguments demand exact string equality. An account identifier either matches or it does not; there is no partial credit. Normalizing it would hide a swapped digit.

Other arguments need normalization before comparison. A date can arrive as "2026-09-18", "September 18, 2026", or "9/18/26" and still be correct. Normalize both the expected and the actual value to a canonical form, then compare. The same holds for phone numbers, currency, and casing on names. Normalize what should be normalized, and never normalize an identifier.

Report the result as per-argument error rate. Count each argument as a trial. Divide the misses by the trials, per argument type. This borrows from the logic of precision and recall: you care both about wrong values passed and correct values dropped. A dropped required argument is a failure, not a blank.

Avoid one common trap. Do not average argument accuracy into a single headline number and stop there. The blend of a strong date score and a weak amount score can look acceptable while the amount errors keep failing tasks. Keep the breakdown visible.

Handling homophones, formatting, and normalization edge cases

The hard cases live at the edges, and they decide whether your scoring is fair. Homophones are the clearest example. When the caller says a name aloud, several spellings are plausible. If the agent had no way to know the spelling, penalizing a reasonable choice is unfair. If the record already exists, the agent should have matched it, and a wrong spelling is a real miss. Score against context: use an allow-list of valid variants for known records, and require confirmation for new ones.

Formatting is the next edge. A date is right in substance but wrong in shape. The value means the correct day but arrives in a format the tool rejects. That is still a failure, because the tool cannot use it. Test the shape as well as the meaning. Both have to be right for the call to work.

Normalization cuts the other way. Over-normalizing hides real errors. If you strip leading zeros from an ID before comparing, you will pass a call that dropped a zero the tool needed. Match the normalization to the field. Loose for spoken numbers that a human would read flexibly. Strict for anything a system parses exactly.

These decisions are judgment calls, and they are easy to get wrong when the team that built the agent also grades it. This is one reason an outside evaluator helps. Evalgent runs these checks as an independent, third-party audit, so the scoring rules are set by someone who does not benefit from a passing grade. Independent evaluation, covered in our piece on why independent voice AI evaluation matters, keeps the argument bar honest.

Gating release on the arguments that carry risk

Not every argument deserves the same bar. Argument accuracy becomes useful when you weight each value by what a mistake costs. A wrong free-text note is an annoyance. A wrong transfer amount is a financial incident. The gate should reflect that difference.

Start by ranking arguments by blast radius. Money, identifiers, dates that trigger commitments, and destinations for a transfer or escalation sit at the top. Set a strict pass bar for those. A single miss on a money argument in a test set should block release, not average out.

Lower-risk arguments get looser bars. A note field or an optional preference can tolerate a higher error rate, because a mistake is recoverable and cheap. Spending your strictness budget evenly wastes it. Concentrate it where errors hurt.

This weighting also shapes the agent's behavior. If a high-risk argument cannot be verified, the right move is to confirm with the caller, not to guess. "I have your transfer as one hundred fifteen dollars, is that right?" is cheaper than a wrong transfer. Reward confirmation on risky arguments and penalize silent guesses. That policy is part of a broader voice agent evaluation program, and it is where argument accuracy connects to real outcomes.

Argument accuracy is not the same as intent accuracy, and the two need separate scoring, as the guide on intent versus entity in voice agents lays out. An agent can nail the intent and still fumble the arguments. Gating on both is what production readiness requires.

Frequently asked questions

What is tool argument accuracy for voice agents?

Tool argument accuracy is the share of tool-call arguments an agent extracts correctly from speech. It is scored per argument against a known-correct value: the right date, amount, name, or ID. It measures the value passed to the tool, not whether the tool ran. A clean call with a wrong value still counts as a failure.

How do you evaluate tool arguments from speech?

Build test cases with known-correct argument values, play the audio through the live agent, capture the actual tool call, and compare each passed argument to its expected value. Score per argument using a rule matched to the type. Report a per-argument error rate rather than a single call-success number.

Why does a voice agent call the right tool with wrong arguments?

Speech passes through recognition, resolution, and formatting before it becomes an argument. Each hop can corrupt a value. A digit is misheard, a date resolves to the wrong week, or a name gets the wrong spelling. The model still selects the correct tool, so intent-level checks pass while the argument is wrong.

How do you measure per-argument error rate?

Treat each argument as a trial. For every test case, compare the passed value to the known-correct value using a type-aware rule. Count misses per argument type, then divide by trials for that type. Keep the breakdown by type; do not blend dates, amounts, and IDs into one average that hides where errors concentrate.

How do you test date and amount arguments in voice agents?

For dates, resolve spoken phrases to a canonical format like ISO 8601 and assert the value and time zone. For amounts, compare the numeric argument to the known amount to the cent and reject silent rounding. Both need format-aware comparison, because a value that means the right thing can still arrive in a shape the tool rejects.

How do you handle homophones in tool arguments?

Homophones like Sean and Shawn sound identical and spell differently. Score against context. For an existing record, use an allow-list of valid spellings and treat a wrong match as a miss. For a new record, require the agent to confirm the spelling with the caller. Penalize silent guesses on names the agent could have verified.

How is tool argument accuracy different from STT entity accuracy?

STT entity accuracy judges whether the transcript captured a spoken entity at the word level. Argument accuracy judges the value that reached the tool, after resolution and formatting. A transcript can be correct while the argument is wrong, or the reverse. Measure both, because they point at different fixes: the recognizer versus the model's resolution.

How do you gate a voice agent on argument accuracy?

Rank arguments by the cost of an error. Set a strict pass bar for high-risk values like money, identifiers, and transfer destinations, and a looser bar for recoverable fields. Block release when a risk-bearing argument misses its bar. Reward the agent for confirming risky values with the caller instead of guessing silently.

The bottom line

A voice agent can call the correct tool and still pass the wrong value. Measure argument accuracy per value, from real audio, and gate release on the arguments that carry the most risk.

Ready to see where your agent's arguments break? Book a demo and Evalgent will audit your tool-argument accuracy end to end from speech, as an independent third party.