Evalgent
Back to Blog
Voice AI Testing

Answering Machine Detection for Voice Agents: How It Works, Where It Fails, How to Test It

Deepesh Jayal
14 min read
Answering Machine Detection for Voice Agents: How It Works, Where It Fails, How to Test It
On this page

An outbound voice agent makes its riskiest decision before it says a word. A person picked up, or a machine did, and the agent has seconds to tell which. Guess wrong one way and a customer hears dead air or a hang-up. Guess wrong the other way and your agent pitches a mailbox for a minute.

This guide is the generic explainer. It covers how AMD works, why it fails and how to test it. It is not a framework tutorial. LiveKit and Pipecat setup is in our voicemail detection guide. Trunk wiring is in the LiveKit and Twilio outbound guide. The wider outbound plan is in outbound voice agent testing.

~4 s
Typical time to a Twilio AMD result, default settings (Twilio FAQ)
2 s
Window after a completed greeting before a call counts as abandoned (47 CFR 64.1200)
3%
Abandoned share of live-answered telemarketing calls allowed per campaign, 30 days (47 CFR 64.1200)
99.3% vs 95.4%
One voicemail classifier: expert set vs held-out production set (Saurav, 2026)

What is answering machine detection?

Answering machine detection: software that labels the callee of an answered call as human, machine, IVR, fax or unknown. Teams also call it voicemail detection, answer machine detection or, when sold as a product, answering machine detection software.

In short, AMD is a classifier plus timers. The classifier says who answered. The timers decide how long the callee waits.

Voicemail answers with the same signal a person does. The network only reports "answered." So the agent must infer the callee from audio.

Who can pick up

An AMD voice agent faces more than two classes. A binary human-or-machine view hides most bugs.

Who answeredWhat it sounds likeWhat the agent should do
Person on a mobile"Hello?" then silenceSpeak within about a second
Person at a business"Thanks for calling Acme, this is Dana."Speak, not treat as machine
Default carrier voicemailStock phrase, then a toneWait, then leave the approved message
Personal voicemail greeting2 to 30 seconds, sometimes music, sometimes no beepWait for the recording to start
Mailbox full or not set upCarrier announcement, no recordingEnd the call, retry later
IVR or auto attendant"For sales, press 1."Navigate or end, per policy
Call screenerPhone asks who is calling and whyGive a short, specific answer
Silent pickupLine noise, or nothingProbe once with a short prompt

Two rows are traps. A personal greeting that sounds like a person fools timing rules. A screener waits for your reply like a person but speaks like a machine.

How answering machine detection works

Every detector combines three kinds of evidence. Production systems mix them.

1. Speech and silence timing. People say something short and stop. Machines talk longer. Twilio's FAQ gives reference durations. Personal greetings are "typically pretty short and less than 1800 ms." Business greetings run about 1,800 to 3,000 ms. Machine greetings are "typically longer, >3000 ms." The detector times the first speech block, waits for silence, and decides.

2. Tones. Beeps and fax tones are strong evidence when present. They are weak as the only signal. Twilio's FAQ says a machine beep "varies greatly between different calling destinations." Some beeps overlap standard call progress tones.

3. Words. A transcript makes "you've reached the voicemail of" unambiguous, even when timing looks human. The cost is speech-to-text plus a model call while a person waits. LiveKit's documented design runs a fast heuristic and an LLM classifier in parallel. The first to conclude sets the result.

Decision tree: first speech length, silence, transcript cues and tones lead to human, voicemail, IVR, screener or unknown verdicts, each with the agent action

What a verdict looks like

Providers return a small label set. Know it before you write a test.

  • Twilio Enable: human, machine_start, fax, unknown. It returns as soon as the callee is identified.
  • Twilio DetectMessageEnd: human, machine_end_beep, machine_end_silence, machine_end_other, fax, unknown. For machines it returns only once the greeting ends.
  • LiveKit: human, machine-ivr, machine-vm, machine-unavailable, uncertain. The docs say to treat uncertain as human.

Sources: Twilio AMD docs and LiveKit's AMD page, read November 17, 2026. Neither has a label for "screener."

Sync, async and the cost of waiting

Twilio's AMD is synchronous by default. The call is blocked until detection completes. With AsyncAmd=true the call connects at once and the result arrives on a callback. The docs add a limit: async AMD "consumes one forked audio stream from the per-call limit of four forked streams."

The FAQ puts typical result time near 4 seconds with defaults. Webhook round-trip latency is the most common cause of slow results. Four seconds of silence after "Hello?" is long enough for people to hang up.

What research says about accuracy

Treat vendor accuracy claims as claims. Twilio's FAQ says DetectMessageEnd is "close to 100% accurate in US destinations with default settings." It publishes no test set. It adds that international accuracy may drop, because voicemail tones differ by country.

Independent work is more useful. Saurav (2026) classifies 15 timing features from a voice activity detector. It reports 99.3% on a 140-call expert-labeled set but 95.4% on 624 held-out production calls. It also reports 0.3% false positives and 1.3% false negatives over 77,000-plus production calls. Altwlkany et al. (2024) report over 96% test accuracy for a recurrent model, above 98% with a silence detection step. These are other teams' results on their data. Yours will differ.

Where AMD fails

Failures cluster. Knowing the clusters lets you build a test for each.

AMD false positives: humans labeled as machines

The AMD false positive is the costly one, so define it carefully. AMD false positive: a live person classified as a machine, so the agent hangs up, stays silent or leaves a message. Some vendors flip the naming. Write your own definition into the test plan.

Common causes:

  • Long human greetings. A business line with a name, department and welcome runs 3 seconds or more. Twilio treats speech longer than MachineDetectionSpeechThreshold (default 2,400 ms) as machine-leaning.
  • Slow or repeated hellos. "Hello... hello?" has a pause. Twilio's docs warn that lowering the speech end threshold makes pauses look like the end of speech.
  • Screeners. Apple's "Ask Reason for Calling" asks unknown callers for a reason before the phone rings. Google's Call Screen answers and asks who is calling and why. A timing detector may read the screener as a person, or as a machine. If it reads a machine, the agent talks or hangs up and the phone never rings.
  • Over-tuned timers. Twilio's FAQ warns that one configuration for every destination can be inaccurate. Aggressive timing also causes unknown results.

AMD false negatives: machines labeled as humans

A false negative means the agent talks to a recording. Causes:

  • Short voicemail greetings. Twilio's docs say a very short two-second greeting may be interpreted as a human picking up.
  • Pauses inside a greeting. A lower MachineDetectionSpeechEndThreshold speeds up human detection. It can also mistake a pause for the end of speech, so human returns early.
  • Beep ambiguity. If the greeting ends in a tone the detector cannot place, Twilio can return machine_end_other. Your voicemail drop may then start in the wrong place.
  • Silent greetings and music. No speech inside the silence timeout ends in unknown or uncertain. What your agent does with that value is part of the test.
Two by two grid of detector verdict against who actually answered, with the cost of each cell: correct human, false positive hang-up, false negative, correct voicemail

Integration failures that look like AMD failures

Some errors are plumbing:

  • Trunk type. Twilio's FAQ states that "Elastic SIP trunking calls cannot utilize AMD." Teams dialing through SIP trunks need another detector. See the LiveKit and Twilio guide.
  • Slow callbacks. A webhook that takes a second adds a second of silence for the callee.
  • Verdict ignored. The detector says machine_start and the agent keeps talking, because the handler listens to the wrong event.
  • Early speech. The agent speaks on connect, before any verdict. Time to first audio matters here.
  • Load. Latency tails grow when servers saturate. See concurrency failures before trusting a quiet-hour result.

What each error costs

The two errors are not symmetric. A false negative wastes a minute of agent time. A false positive hangs up on a person who wanted to talk.

For US telemarketing calls, the rule puts a number on the second. Under 47 CFR 64.1200(a)(7), a call is abandoned if it is not connected to a live representative "within two (2) seconds of the called person's completed greeting." Callers may not abandon more than three percent of live-answered telemarketing calls over 30 days for a single campaign. A detector that hangs up on people spends that budget. The rule has more conditions, and it covers telemarketing only. This is not legal advice. Ask counsel how it applies to your calls. We left voicemail-drop law out, because we did not verify it.

For the compliance side, see outbound voice agent testing. For the funnel, see outbound sales voice agent metrics. Inbound lines use the word differently, as abandonment rate for voice agents explains.

A worked example (illustrative numbers)

Say you dial 10,000 numbers and 3,000 are answered by people. Assume the detector mislabels 1% of them as machines. That is 30 people hung up on or sent to voicemail. A 1% rate uses a third of a 3% abandoned-call budget before any other failure counts. These are assumptions for the arithmetic, not measurements of any detector.

What to measure

Score every test call against a label you set in advance. Then compute these.

MetricFormulaWhat it tells you
False positive ratehumans treated as machine / all human answersHang-ups and drops on people
False negative ratemachines treated as human / all machine answersAgent talking to a recording
Unknown rateunknown or uncertain / all answersHow often the detector punts
Unknown handlingcorrect action after unknown / all unknownsWhether the fallback is safe
Decision latencytime from answer to verdictHow long the callee waits
Time to first word (human)answer to first correct agent audioDead air on people
Message start accuracyvoicemail messages that start after the greeting / all dropsClipped or early messages
Screener pass ratescreeners that ring through / screener callsLost connections

Track by class, not just overall. A detector at 97% overall can fail every screener, because screeners are rare in the average mix.

Decision latency is two numbers

For a person, latency is felt as dead air. For a machine, the message must start after the greeting. Report both, and report the tail, not just the average. Twilio's FAQ quotes about 4 seconds for defaults. The slowest 5% of calls are the ones people hang up on.

Illustrative timelines of two decision latency clocks: dead air between a person's hello and the agent's first word, and a voicemail message that must start after the beep

How many calls you need

You need enough human answers to bound the false positive rate. With zero errors in n trials, the 95% upper bound on the true rate is about 3 divided by n. To show a rate below 1% takes about 300 human answers with no errors. Below 0.3% takes about 1,000. Use a Wilson interval when errors appear.

# Simplified: score AMD results against labels. Rows: (label, verdict)
from math import sqrt

def wilson(k, n, z=1.96):
    if n == 0:
        return (0.0, 0.0)
    p = k / n
    d = 1 + z * z / n
    c = p + z * z / (2 * n)
    m = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n))
    return ((c - m) / d, (c + m) / d)

def score(rows):
    humans = [r for r in rows if r[0] == "human"]
    machines = [r for r in rows if r[0] == "voicemail"]
    fp = sum(1 for _, v in humans if v.startswith("machine"))
    fn = sum(1 for _, v in machines if v == "human")
    return {
        "false_positive": (fp, len(humans), wilson(fp, len(humans))),
        "false_negative": (fn, len(machines), wilson(fn, len(machines))),
    }

The AMD scenario set

A scenario set for AMD lists callee behaviors, each with an expected action and a pass rule. Start from the answerer table, then add the timing variants that break detectors.

ScenarioCallee behaviorExpected agent actionPass criteria
Human, quick hello"Hello?" then waitsSpeak and ask for the personAgent speaks within your latency budget
Human, slow helloPause, then "Hello?", then pauseSpeak, do not hang upNo machine verdict; no hang-up
Human, long business greeting3 to 5 second greetingSpeak after greetingNot treated as machine
Human, wrong person"Who?" then "Wrong number"End politelyCorrect end reason
Voicemail, carrier defaultStock phrase then toneWait, then leave messageMessage starts after the tone, nothing clipped
Voicemail, short personal2 second greeting, pause, toneLeave messageNot treated as human
Voicemail, long personal20 to 30 secondsWait for the recordingNo speech over greeting
Voicemail, no beepGreeting then silenceLeave message after silenceMessage starts at a sensible point
Mailbox fullCarrier announcement, no recordingEnd the call, mark for retryNo message attempt
Silence on pickupNothing for 5 to 10 secondsProbe once, then follow policySafe fallback, no endless wait
IVR menu"Press 1 for sales"Per policyCorrect branch, no script read to the menu
Screener"State your name and reason"Short specific answerCall rings through; no pitch
Personal greeting that sounds like a person"Hey, it's Sam. Go ahead."Treat as voicemail if it recordsMessage is left, no conversation attempt
Greeting then humanVoicemail greeting cut off by a personSwitch to conversationAgent handles the late human

Run each under realistic conditions: different speech pace, accent, response delay and background noise. Clean synthetic audio passes detectors that fail on real handsets. Synthetic callers can play the human, IVR and scripted voicemail roles. Record carrier voicemails and screeners from real handsets, since their timing is hard to fake.

How to test answering machine detection before go-live

1. Write the label set — Fix the classes. Define false positive and false negative in one sentence each, before any call.

2. Build the corpus — Record carrier voicemails, personal greetings and screeners from several carriers and handsets. Script the human and IVR cases.

3. Add callee conditions — Vary pace, response delay, accent, noise and language. A clean "hello" is not enough.

4. Run over the real path — Use the same PSTN or SIP route and trunk type as production. A path that cannot run AMD shows up here.

5. Judge from the audio — Mark what the callee heard. Did the agent talk over the greeting? Did the message start after the beep? Transcripts miss this.

6. Score by class — Compute error rates and decision latency per class, with intervals.

7. Tune one timer at a time — Change one threshold and rerun the set. Twilio's FAQ recommends A/B testing configurations and avoiding over-tuning to one carrier or device.

8. Gate releases — Fail the build when false positives, unknown handling or the latency tail pass your limits. Rerun after every prompt, model or trunk change.

Why judging from audio matters

AMD errors are timing errors. A transcript shows the agent said "Hi, this is Acme calling." It does not show whether that landed over a voicemail greeting, or two seconds after a person said hello.

Evalgent is a testing and evaluation platform for voice agents. Its premise is that most tools judge calls from transcripts, so the judge is deaf to the audio where these failures live. Evalgent scenarios script the callee, caller profiles set accent, pace, interruption, response delay and noise, and the simulator reaches your agent over PSTN, SIP or WebRTC. Calls are judged from the audio, with latency and call end reason among the metrics, and humans review the verdicts. We cover the wider gap in transcript versus audio evaluation.

After launch, the same scenarios anchor monitoring. Production recordings can be mapped to them and judged, so a prompt change that slows the opener shows up in review, not in complaints. For vertical examples, see dental recall calls and leasing follow-up calls. Evalgent works with teams to set up evals for their first agent.

Frequently asked questions

What is answering machine detection?

Answering machine detection is software that listens to the start of an answered call and labels the callee as a person, voicemail, IVR, fax or unknown. It uses speech and silence timing, tones and sometimes transcripts. Outbound voice agents use it to decide whether to talk, wait for a recording or hang up.

How does answering machine detection work?

It measures the first speech block, then the silence after it. Short speech followed by a pause leans human; long continuous speech leans machine. Beep detection and transcripts add evidence. Twilio's reference lengths are under 1,800 ms for personal greetings and over 3,000 ms for machines. The detector returns a label and the agent acts on it.

How accurate is answering machine detection?

It depends on your call mix. Twilio says DetectMessageEnd is close to 100% accurate in US destinations with default settings, without a published test set. One academic detector scored 99.3% on an expert-labeled set and 95.4% on held-out production calls. Measure your own accuracy by class on real audio.

What is an AMD false positive?

An AMD false positive is a live person classified as a machine. The agent then hangs up, stays silent or leaves a voicemail. Definitions vary by vendor, so write yours into the test plan. It is usually the costlier error, because it wastes a live answer and can count as an abandoned call.

Why does AMD hang up on real people?

Common causes are long business greetings, slow or repeated hellos, call screeners and timers tuned too aggressively. Twilio treats speech longer than its speech threshold, 2,400 ms by default, as machine-leaning. Test each case with real callee timing, not just a clean "hello" from a script.

Does answering machine detection work with call screening?

Not reliably. Screening on iPhone and Pixel asks the caller for a reason before the phone rings. A detector may label the screener as a person or a machine. Either way, the agent must answer with one short, specific sentence. Test screeners from real handsets and treat them as their own class.

How long should answering machine detection take?

As short as accuracy allows. Twilio's FAQ puts typical results near 4 seconds with default settings, and says slow webhooks are the most common cause of delay. Set a latency budget for people and another for machines, then measure the tail. The slowest calls are the ones people abandon.

How do I test answering machine detection?

Build a labeled set of human, voicemail, mailbox-full, IVR, screener and silent answers. Place calls over the production route, judge the audio, and score false positives, false negatives, unknown handling and decision latency per class. Change one timer at a time, and rerun after every release.

The bottom line

Answering machine detection is a classifier plus timers, and its errors hit people first: a false positive hangs up on a customer, a false negative talks to a recording. Prove yours works the only way that counts, with labeled real-handset audio, per-class false positive and false negative rates, and a measured latency tail.

Related Articles