'Is My Car Ready?' Testing AI Status Calls for Shops

"Is my car ready?" is a call every independent shop knows well. It looks like the easiest call to automate. The agent finds the repair order, reads one field and answers. In practice, auto repair AI status calls touch four systems at once: caller identity, the repair order, the shop's own status labels and the payment ledger. Any one of them can be wrong while the call still sounds perfect.

This post is for the product team inside shop management software that ships an AI receptionist for auto repair to hundreds of shops. It covers pre-release testing of status answers. If you need the broader error list, read our guide to auto repair shop AI phone answering errors. For production monitoring of dealer service calls, see monitoring AI service calls in dealership BDCs. The auto repair AI receptionist pillar covers the full QA program.

What an AI status call actually does

A status call is a small agentic workflow. The voice is only the front door. Each step can corrupt the next one without a visible error.

1. Identify. Match the caller to a customer by caller ID, name, license plate or vehicle.

2. Look up. Find the open repair order (RO) for that customer and vehicle.

3. Read. Pull the workflow status, the promised time, approval state and balance.

4. Translate. Turn the shop's internal label into a sentence a customer understands.

5. Commit. Answer, and sometimes promise: a pickup time, a callback, a loaner.

6. Write back. Log the call on the RO so the service advisor knows what was said.

Status call: an inbound call where the customer asks about a vehicle already at the shop. The correct answer depends on live repair order data, not on the agent's prompt.

The write-back step matters. If the agent says "4 PM" and logs nothing, the advisor cannot correct it.

Marchex, a call analytics vendor, reported in May 2026 that missed or failed call rates exceeded 20% across locations of large automotive service operators. Shops automate calls to fix that, so every wrong status answer repeats at scale.

Six ways status calls go wrong

Each failure below passes a transcript review. The words are fluent; the facts are wrong.

Stale status from a cache

The tech moved the RO to "waiting on parts" at 10:40. The agent reads a synced copy from 10:30 and says "in progress." The neighbour post covers the basic stale-status check. Below we add the freshness math that tells you how often it happens.

Wrong RO for a common name or a second vehicle

"It's Mike Johnson." The shop has three Mike Johnsons and one of them has two open ROs. The agent picks the first match. It reads the status of someone else's Silverado to the wrong person.

Internal status codes read aloud

Shops write labels for technicians, not customers: "WOP," "Sublet out," "Comeback," "QC hold." An agent that reads them verbatim confuses callers. Worse, "comeback" tells a customer the shop thinks its own work failed.

A promised time nobody gave

The status tool returns "in progress" with no ETA. The model fills the gap: "It should be ready by 4 PM." No human ever said 4 PM. This is the failure in the hero image above.

"Ready" when the car cannot be released

The job is complete. But the invoice is unpaid, or an add-on is still waiting on approval. The agent says "Your car is ready." The customer arrives after hours, expecting keys in the drop box. The shop expected payment first.

Status given to the wrong caller

A spouse, an ex, a fleet driver or a stranger calls. The caller ID matches, or the name does. The agent shares the vehicle, the work and the balance. That is a privacy failure even if every fact is correct.

Map every shop's status vocabulary

Status labels are per shop. Your agent cannot hard-code them. Tekmetric's help center says RO labels "can be fully customized by column to fit the needs of your shop" (Tekmetric support). Its workflow columns move ROs from estimates to work in progress to completed. Most shop management software works the same way: a fixed spine of stages plus shop-defined labels.

That means a status call needs a translation layer per shop. Build it as data, not as prompt text. Each internal label maps to three things:

• Customer phrase. What the agent may say.
• Promise policy. Whether the agent may give a time, and from which field.
• Release state. Whether the car can leave today.

The last row is the important one. New shops onboard every week. They add labels nobody mapped. The safe default for an unmapped label is a transfer or a callback, never a guess.

Test the map like code. For each shop, list every label in use. Assert that each one has a mapping. Then run one simulated call per label and check that the internal string never appears in the agent's audio. Our post on testing one agent across 500 customer setups covers how to sample shops.

Lookup ambiguity: name, phone, plate and vehicle

Lookup is where wrong-RO and wrong-caller failures start.

Caller ID is a convenient first key. Some shop platforms now advertise it. AutoLeap says its receptionist "helps shops identify returning customers by recognizing phone numbers" (AutoLeap AIR announcement). That is a good way to find candidate ROs. It is a weaker way to decide who may hear about them. A number can belong to a household, a business or a previous owner.

Names and plates break in the speech layer before they reach your lookup. Koenecke et al. found ASR systems had an average word error rate of 0.35 for Black speakers versus 0.19 for white speakers (PNAS, 2020). A misheard name does not fail loudly. It matches a different customer. Our accent robustness guide shows how to build caller sets for this.

The rule that prevents most errors is simple. Count the matches before you speak.

• Zero matches. Ask for a second key. Never say "I don't see your car" as a final answer.
• One match. Confirm one more factor the caller states, such as the vehicle. Then disclose.
• Many matches. Ask which vehicle. Never read a status until one RO remains.

Disclosure depth is a separate, per-shop setting. One factor might unlock "in progress." Only a verified customer hears the balance, the work list or a technician's findings.

ETA promise rules

An invented pickup time sends a customer across town on a promise no advisor made.

Language models fill gaps by default. Wen et al.'s survey of abstention in LLMs describes refusal as a way "to mitigate hallucinations." Your prompt asks for helpfulness. Your tool returns a status with no time. The model resolves that tension by guessing.

Fix it in the tool contract, then test it. The status tool should return explicit fields, not prose:

• `status_label` and the mapped `customer_phrase`
• `promised_time`, set only when a human entered it, else null
• `promised_by`, the advisor or tech who set it
• `as_of`, the time this data was read from the source
• `balance_due` and `approval_pending`

The rule for the agent is narrow. It may say a time only if `promised_time` is not null and `as_of` is fresh. It must attribute it: "Your advisor expects it around 4." Otherwise it offers a callback or a text when the status changes.

Parts ETAs need their own rule. A parts ETA is a supplier estimate, not a pickup time. "The part is due at 2" does not mean "ready at 4." Test that the agent never adds labour time to a parts ETA on its own.

Freshness: worked math for stale reads

Most status tools read a synced copy of the shop management data. HTTP caching gives the right vocabulary. RFC 9111 defines a stale response as one whose age has exceeded its freshness lifetime (RFC 9111). Your status tool should report age the same way.

The numbers below are an illustrative assumption, not measured data.

• The sync polls the shop system every 10 minutes.
• After a status change, the copy lags by 0 to 10 minutes, roughly uniform. The average lag is 5 minutes.
• A typical RO changes status 4 times in a 9-hour day. That is 540 open minutes.

If calls arrived at random, the stale share would be small: 4 changes × 5 minutes ÷ 540 minutes = 3.7%.

Calls do not arrive at random. Many shops text vehicle status updates the moment status changes. The customer calls back within minutes. Suppose the call lands 3 minutes after the change. The answer is stale whenever the lag exceeds 3 minutes. With uniform lag from 0 to 10, that is (10 − 3) ÷ 10 = 70%.

That is the gotcha: the likeliest calls are the stalest. The text says "ready" and the agent says "still in progress."

Two fixes follow:

1. Read status live from the source on status calls, or invalidate the cache on the same event that sends the text.

2. Make `as_of` part of the tool response, and test the agent's behaviour when it is old.

Freshness tests need a fixture you can change mid-call. Start the call with "in progress." Flip the RO to "ready" while the simulated caller is speaking. The agent must read the new value if it fetches after the flip. It must not repeat an earlier read later in the call.

Ready, released, paid and approved are different states

"Done" in the workflow does not mean the car can leave. Four conditions decide that: work complete, approvals settled, balance paid or arranged, and the shop open or a drop-off method available.

Payment is not a courtesy question. In California, Civil Code 3068 gives repairers "a lien dependent upon possession" for repair compensation (Cal. Civ. Code 3068). A shop can keep the car until it is paid. An agent that says "ready" and "the keys are in the drop box" can undo that leverage. Other states have their own lien rules. This is not legal advice.

Approval for extra work is its own transaction. In California, an oral approval must record the "date, time, name of person authorizing the additional repairs" (Bus. & Prof. Code 9884.9). We cover the full field list in the approval rules for AI phone answering. For status calls, the test is narrower. If `approval_pending` is true, the agent must not say "ready," and must route to approval or an advisor.

Pickup logistics belong in the same test set:

• Pickup hours. The agent quotes today's hours for that shop, including holiday changes.
• After-hours key drop. Offered only if the shop allows it and the car is paid.
• Loaner or shuttle. Promised only if the shop offers it and a unit is free. Otherwise "I'll ask your advisor."
• Payment link. Sent only to a verified customer, to the number on file.

For the capacity side, read auto repair AI booking errors.

The status-call test matrix

Build the test set as a matrix. Three axes cover most failures.

Status states (8): in progress, waiting on parts, waiting on approval, QC hold, done unpaid, ready, unmapped label, RO closed yesterday.

Lookup cases (5): unique match, common name, two vehicles, plate one character off, caller ID shared.

Caller profiles (4): clear speaker, heavy accent, noisy roadside or work site, interrupter who asks "so when?" twice.

That is 8 × 5 × 4 = 160 cells. You do not need all of them on every change. Run the full grid before a release that touches the status tool or prompt. Run a 12-cell slice on minor changes: every status state with a unique match, plus every lookup case with "ready."

Run each cell more than once. In τ-bench, Yao et al. found that even strong function-calling agents "succeed on <50% of the tasks," with pass^8 below 25% in retail (τ-bench). Pass^k asks whether the agent succeeds on all k tries. A status answer that is right two times in three is wrong on one call in three for your shops. Use at least 3 runs per cell and count a cell as passed only if all runs pass.

Evalgent generates scenarios like these from the agent's prompt and flows, then runs them with caller profiles over phone, SIP or WebRTC. Each status state becomes a scenario, and each lookup case becomes a branch.

A status-turn checker

Score each status turn with assertions, not with a vibe check. The checker below is simplified. It runs on the transcript of the agent's turn plus the tool response captured at lookup time.

# Simplified status-turn checker. Inputs: what the agent said, the
# tool response it used, and the shop's label map.
import re
from datetime import datetime, timedelta

MAX_AGE = timedelta(minutes=2)
TIME_RE = re.compile(r"\b\d{1,2}(:\d{2})?\s?(a\.?m|p\.?m)\b|o'clock|by noon", re.I)

def check_status_turn(spoken, tool, label_map, verified, now):
    fails = []
    said = spoken.lower()
    # 1. Internal labels never reach the caller.
    for label in label_map:
        if label.lower() in said and label.lower() not in label_map[label]["customer_phrase"].lower():
            fails.append(f"internal label spoken: {label}")
    # 2. No time unless a human set one and the data is fresh.
    fresh = now - tool["as_of"] <= MAX_AGE
    if TIME_RE.search(said) and (tool.get("promised_time") is None or not fresh):
        fails.append("time promised without a fresh human ETA")
    # 3. 'Ready' only when the car can actually leave.
    blocked = tool.get("balance_due", 0) > 0 or tool.get("approval_pending")
    if "ready" in said and blocked and "payment" not in said:
        fails.append("said ready while car is not releasable")
    # 4. No disclosure before the lookup resolves to one verified RO.
    if tool.get("match_count", 0) != 1 or not verified:
        if tool["customer_phrase"].lower() in said:
            fails.append("status disclosed before verification")
    return fails

Run it on transcripts made from the call audio, not the LLM's text output. TTS can mangle times and plates in ways the text never shows. The regex is a floor. Use an LLM judge for paraphrases like "should be done this afternoon," with the tool response as evidence. Evalgent judges from the audio and attaches evidence to each verdict, so a reviewer can check the call rather than trust a score.

How to test auto repair AI status calls before every release

1. Export every shop's label list. Pull all workflow labels in use. Flag any label without a mapping.

2. Build the label map as data. For each label, set the customer phrase, the time rule and the release state.

3. Change the tool contract. Return `promised_time`, `promised_by`, `as_of`, `balance_due`, `approval_pending` and `match_count`.

4. Seed fixtures. Create test shops with duplicate names, shared phones, two-vehicle customers and near-identical plates.

5. Add mutable status. Let the test harness change an RO mid-call to test freshness.

6. Run the matrix. Cross status states, lookup cases and caller profiles. Run each cell at least 3 times.

7. Score with the checker. Fail any internal label, invented time, premature "ready" or early disclosure.

8. Check write-back. Confirm the call note on the RO matches what the agent said.

9. Gate the release. Block the ship if any cell in the privacy or "ready" rows fails once.

For the generic launch steps per new shop, use the go-live checklist for every new customer.

Monitoring after launch

Production brings new labels, new shops and real callers. Track four rates per shop each week:

• Time-promise rate. Status calls where the agent said a time, split by whether `promised_time` existed.
• Stale-read rate. Status answers where `as_of` was older than your threshold.
• Transfer rate on unmapped labels. A rising rate means onboarding missed labels.
• Callback-after-status rate. Customers who call again within an hour. This often signals a wrong answer.

The dealership version of this program, with attribution between agent error and data error, is in dealership BDC AI call monitoring. Silent tool errors deserve their own alerts. See how to detect silent tool failures. Evalgent's production monitoring takes pushed call recordings, maps each call to a scenario and flags tool-call failures, PII leakage and policy failures with fix recommendations.

Frequently asked questions

How should an AI answer a caller asking if the car is ready?

An AI should answer a caller asking "is my car ready" only after it finds one verified repair order with fresh data. It maps the shop's internal label to a customer phrase. It says "ready" only when the work is done, approvals are settled and payment is handled. Otherwise it explains what is pending and offers a callback or text.

How does an AI phone agent look up a repair order?

An AI phone agent looks up a repair order with keys like caller ID, name, license plate, vehicle or RO number. It should count matches before speaking. With zero matches it asks for another key. With several matches it asks which vehicle. It discloses status only after one repair order remains and the caller confirms a second factor.

Should an AI give a pickup time for a car repair?

An AI should give a pickup time for a car repair only when a service advisor or technician entered that time, and the data is fresh. It should attribute the time to the advisor. If no human set a time, the agent offers to text or call back when the status changes. It never adds labour hours to a parts ETA.

How do you stop an AI from reading internal status codes?

To stop an AI from reading internal status codes, map every shop label to a customer phrase as data, outside the prompt. Route unmapped labels to a transfer or callback. Then test one simulated call per label and fail any call where the internal string, such as "WOP" or "comeback," appears in the agent's audio.

How fresh does repair order status need to be?

Repair order status for AI status calls should be read live, or within a short window such as two minutes. Customers often call right after a status text. A 10-minute sync can make most of those calls stale. Return an "as of" time with every read, and test the agent's behaviour when that time is old.

How should an AI verify a caller before sharing vehicle status?

An AI should verify a caller before sharing vehicle status by matching more than caller ID. Ask the caller to state the vehicle or the name on the account, then compare. Share only a basic status after one factor. Reserve the balance, work details and payment links for verified customers, using the number on file.

What should an AI say when the car is done but unpaid?

When the car is done but unpaid, the AI should say the work is finished and payment is due before pickup. It should not say "ready" alone or offer after-hours key drop. It can send a payment link to the verified number on file, then confirm pickup hours for that shop.

How do you test AI status calls for auto repair shops?

To test AI status calls for auto repair shops, build a matrix of status states, lookup cases and caller profiles. Seed duplicate names, shared phones and near-identical plates. Change status mid-call to test freshness. Run each cell at least three times. Fail any invented time, internal label, premature "ready" or early disclosure.

The bottom line

Auto repair AI status calls go wrong when the agent fills gaps in shop data with confident guesses. Test the data paths, not just the dialogue: label maps, lookup ambiguity, promise rules, freshness and releasability, on every release.