Device Repair AI Intake: Is It Logging the Right Model?

The call sounds perfect. A caller says her screen is cracked. The agent quotes a price, books a 2 p.m. drop-off and says the part is in stock. The ticket reads "iPhone 13 · Screen." At check-in the technician turns the phone over and sees three rear cameras. It is a 13 Pro Max. The reserved screen doesn't fit, the quote was too low, and the 2 p.m. slot is now a refund conversation.

This guide is for product teams at repair shop POS software that run one voice agent for hundreds of shops. Device repair AI intake is now a shipped feature. RepairDesk, for example, says its built-in receptionist ARIA can "provide a repair quote" and book repair jobs. It attaches call transcripts to the ticket (RepairDesk blog). Once an AI receptionist for cell phone repair writes tickets and reserves parts, a misheard suffix becomes a stock error. Below you'll find a failure taxonomy, a disambiguation flow, a price-list oracle and a test matrix. For the wider repair-and-service playbook, start with our AI receptionist QA guide for auto repair shops.

What a device repair intake call actually writes

A repair intake call is the front door to a four-step workflow. Capture, act, confirm, write back. Each step can fail while the conversation sounds fine.

1. Capture. The agent resolves "my iPhone, the big one" to a canonical model. It records storage, color, the issue and any water exposure.

2. Act. It looks up the price row for that model, repair type and part tier. It checks parts inventory and may reserve a part.

3. Confirm. It reads back model, repair, price and turnaround time. It books a walk-in slot or starts a mail-in repair.

4. Write back. It creates the repair ticket in the POS with the price row ID, the reserved SKU and the call transcript.

Model capture accuracy: the share of intake tickets whose stored model matches the model a technician confirms at check-in. It is the one metric that ties the voice layer to parts and revenue.

A transcript review sees step 3 and nothing else. The caller heard "iPhone 13 screen, $189" (an illustrative price) and agreed. Was $189 the right row? Does the reserved SKU match? Those answers live in the tool calls. Our guide to tool argument accuracy from speech covers that layer in general. Here is what it looks like for repair tickets.

Repair ticket creation is where small errors compound. A wrong model changes the price row, the reserved SKU and sometimes the turnaround promise. One speech error corrupts three fields and raises no error.

Why model names break on phone audio

Model names are short, dense and nearly identical. That makes them the hardest entities any phone repair shop AI receptionist hears.

The audio is narrower than you think

Phone calls commonly reach the agent as G.711 audio sampled at 8,000 Hz (ITU-T G.711). Nothing above 4 kHz survives that sample rate. Much of the energy that separates "s" from "f" sits above that line. The weak endings that split "fourteen" from "forty" suffer too. Add a TV in the background, and the model suffix is often the first thing lost. Our breakdown of phone audio quality metrics shows how codecs and packet loss compound this.

A failure taxonomy for model capture

The last two rows matter most. They are not speech recognition errors. A perfect transcript still produces a wrong ticket. That is why transcript scoring alone undercounts this failure.

What the research says about entity errors

Three findings shape the fix.

• Context prompts barely move average WER. PROFASR-Bench tested Whisper-Small on entity-dense speech. WER went from 9.98% with no prompt to 9.92% with the gold transcript as prompt (Piskala, arXiv 2512.23686). The audio was synthetic and single-turn. Still, generic context doesn't rescue rare entities.
• Revision with the right list helps. An LLM step that sees the correct entity names cut named-entity WER by up to 30% relative (Trinh et al., arXiv 2506.10779). On repair calls, that list is the shop's device catalog.
• Biasing gains shrink on real lists. One contextual biasing study gained 35.6% relative on a rare-words set. On a real apps-and-contacts set it gained 13.32% (Jalal et al., arXiv 2401.13146).

The keyterm budget gotcha

Keyterm prompting is the obvious lever. Deepgram's Nova-3 and Flux accept `keyterm` parameters, limited to 500 tokens per request. The docs advise focusing on "the most important 20-50 terms" and avoiding "Excessive keyterms" (Deepgram keyterm docs). A repair shop price list can hold hundreds of models across phones, tablets, laptops and game consoles. You can't boost them all.

The working pattern is staged. Boost brand and family names at the start of the call. Once the caller says "iPhone," send a mid-stream update with that family's suffixes. Think "Pro," "Pro Max," "Plus" and "mini." Test both stages. A stale keyterm list left over from the last caller is its own bug. Our post on STT entity accuracy covers how to measure entity recall separately from WER.

Callers don't know their model: ask a discriminating question

The agent can't fix a caller who is confidently wrong by listening harder. It needs a question whose answer separates the candidates.

Apple's own specs give you the discriminators. The iPhone 13 has two rear cameras. The 13 Pro and 13 Pro Max have three. The 13 and 13 Pro both have 6.1-inch displays, but only the Pro has ProMotion. The 13 Pro Max is 6.7 inches. Only the Pro models came in 1 TB (Apple: Identify your iPhone model). Same screen size, different screen part. So "it's the 6.1-inch one" doesn't settle a screen replacement order.

The fallback is a model number, not a guess. Apple's path is Settings, General, About, then "tap the part number" to see the model number (Apple). US model numbers split the family cleanly. The iPhone 13 is A2482, the 13 Pro is A2483 and the 13 Pro Max is A2484 (Apple). Samsung's path is Settings, then "About phone," next to the IMEI (Samsung). A caller with a shattered screen may not be able to read it. Then the rule is simple. Quote a range, and hold no part until check-in.

The read-back should name the discriminator, not just the model. Try "iPhone 13 Pro Max, the large one with three cameras." That gives the caller something to correct. "iPhone 13, got it" does not.

Wrong price row, wrong part reserved

A phone repair price quote is only as good as the row it came from. The agent's job is to resolve three keys and read one row: model, repair type and part tier.

Build a price-list oracle

Each shop has its own price list. Your test oracle should be that list, not an LLM's opinion of a fair price. Compute each test call's expected row from its true model, repair and tier. The test passes only if the quoted price, stored row ID and reserved SKU all match. Our lawn and cleaning quote accuracy tests use the same approach with square footage as the key.

# Simplified: score one intake call against the shop's price list and ticket.
from dataclasses import dataclass

@dataclass
class PriceRow:
    sku: str      # e.g. "IP13PM-SCR-OEM"
    model: str    # canonical, e.g. "iPhone 13 Pro Max"
    repair: str   # "screen" | "battery" | "charging_port" | "water_damage"
    tier: str     # "oem" | "aftermarket"
    price: float

def oracle_row(price_list, model, repair, tier):
    rows = [r for r in price_list if (r.model, r.repair, r.tier) == (model, repair, tier)]
    return rows[0] if len(rows) == 1 else None  # 0 or 2+ rows = config bug

def score_call(expected, ticket, quoted_price, reserved_sku, price_list):
    fails = []
    if ticket["model"] != expected["model"]:
        fails.append(f"model: true {expected['model']}, stored {ticket['model']}")
    if ticket["repair"] != expected["repair"]:
        fails.append("repair type mismatch")
    row = oracle_row(price_list, expected["model"], expected["repair"], expected["tier"])
    if row is None:
        fails.append("no unique price row: shop config bug, not agent bug")
    else:
        if abs(quoted_price - row.price) > 0.005:
            fails.append(f"quoted {quoted_price}, price list says {row.price}")
        if reserved_sku != row.sku:
            fails.append(f"reserved {reserved_sku}, expected {row.sku}")
    if "passcode" in ticket.get("notes", "").lower():
        fails.append("passcode mentioned in ticket notes")
    return fails

The `None` branch matters on a multi-shop platform. Duplicate or missing rows are configuration bugs. They should fail a shop's go-live check, not the agent's score. See testing one agent across 500 customer setups for that split.

Charging port vs battery vs water damage

Issue capture fails differently. "It won't charge" can mean a charging port, a battery or a board fault. The agent shouldn't diagnose. It should map symptoms to a category and flag water exposure. Then it quotes only what the price list allows. If a shop prices water damage as a diagnostic fee, the agent must say so. Never quote a battery price for a phone that "fell in the pool."

OEM vs aftermarket parts wording

OEM parts and aftermarket parts often sit in separate price rows. The tier the caller agreed to must match the tier reserved. The wording carries legal weight too.

California's SB 244 covers repair providers not authorized by the manufacturer. They must give customers written notice of that status. They must also disclose "any used replacement parts" and parts from "a supplier other than the manufacturer." That is Public Resources Code § 42488.2(e), operative July 1, 2024 (chaptered bill text). The law also makes manufacturers stock parts for seven years on products priced $100 or more.

On iPhone 12 and later, Parts and Service History can label a display, battery or camera "Genuine," "Unknown," "Unverified" or "Used" (Apple). An agent that promises "you won't see a difference" on an aftermarket screen may be contradicted by the phone itself. Test the exact phrases your shops approve. This is general context, not legal advice. Each shop should confirm its own disclosure duties.

Passcodes and data the agent should never collect

Repair calls invite oversharing. Callers offer passcodes "so you can test it." The agent should decline and say why.

The best-practice case is simple. Apple tells customers to never give a "device passcode, or account security details to anyone else" (Apple: Prepare for service). It also says to stay present while a technician has access to the device. A spoken passcode lands in the recording, the transcript and the ticket notes. It then syncs everywhere those go. None of those systems need it.

What the agent should say instead is also in Apple's guide. Back up the device first. On iOS 26 or earlier, turn off Find My. On iOS 27 or later, turn on "Ready for Repair or Trade-In" (as of October 2026). Apple warns it may not be able to service a device otherwise. Shops set their own policies, but the read-back script should carry these prep steps.

Test callers who volunteer a passcode, who insist, and who slip it into the issue description. Pass means the passcode appears nowhere: not in the read-back, not in the ticket, not in the summary. Evalgent's production monitoring flags PII leakage on recorded calls, which is how you find the cases your scripted tests missed. See our guide to PII leakage in spoken readbacks and logs for redaction checks.

A test matrix: model names × caller profiles

A single happy-path call per model proves nothing. Build a matrix of confusion sets against caller conditions. Each cell is a scripted call with a known true model, repair and tier.

Each cell passes on the ticket, not the transcript. Run each cell several times, since one prompt can pass once and fail the next. Evalgent's caller profiles cover these columns: accent, pace, interruptions, background noise and language, Spanish included. Scenarios are generated from the agent's own prompt and flows. New suffix rules get coverage the day they ship.

Sample size for a real error rate

How many calls do you need? Use the normal approximation for a proportion. To estimate a model mismatch rate near 3% within ±1 point at 95% confidence:

n = z² × p × (1 − p) / e²
n = 1.96² × 0.03 × 0.97 / 0.01²
n = 3.8416 × 0.0291 / 0.0001 ≈ 1,118 tickets

That is a worked example, not a benchmark. The lesson is that 50 test calls can't tell 2% from 5%. Pre-release tests catch whole classes of failure. Only production volume gives you a rate.

How to test device repair AI intake before every release

Run this sequence before any prompt, flow or STT change reaches your phone repair shop AI receptionist.

1. Export each shop's price list and device catalog. Flag duplicate and missing rows as config failures first.

2. Generate confusion sets per brand. Group models that share a family name, a number or a screen size.

3. Script calls per confusion set. Include a true model, a stated model (sometimes wrong), a repair type, a part tier and a caller profile.

4. Place the calls over the real phone path. Use PSTN or SIP so the codec and noise conditions match production.

5. Score the ticket, not the transcript. Check model, storage, issue category, price row ID, quoted price and reserved SKU against the oracle.

6. Score the read-back from the audio. Confirm the agent said the suffix and the discriminator aloud, and didn't speak a passcode back.

7. Check safety rules. Passcode declined, prep steps given, approved parts wording used, no part held on low confidence.

8. Compare with the last release. Block the release if any confusion set regresses. Our guide to shipping prompt changes safely covers the gating mechanics.

New shops get the same suite before their first live call. The go-live checklist for each new customer's agent adds the shop-specific checks.

Monitoring: ticket model vs technician-confirmed model

Production gives a phone repair shop AI receptionist a free label. A technician looks at every device that arrives. Make "confirmed model" a required check-in field, and compare it with what the agent stored.

Track three rates per shop and per agent version:

• Model mismatch rate = tickets where confirmed model ≠ stored model, divided by tickets with a confirmed model.
• Wrong part reserved rate = reservations released or swapped at check-in, divided by reservations made by the agent.
• Quote drift rate = tickets whose final price differs from the agent's quoted row for reasons other than diagnosis.

Then split mismatches by cause. Was the suffix in the audio but missed by STT? Was it in the transcript but dropped by the LLM? Or did the caller state the wrong model, with no discriminator asked? Each cause has a different fix. Only the audio separates the first two. Transcript-only review blames the wrong layer.

This is where Evalgent fits for a POS team without an eval function. You push call recordings. Each call is transcribed, mapped to a scenario and judged from the audio. Tool-call failures and policy failures are flagged with fix recommendations. Each confirmed mismatch then becomes a test case in the pre-release suite. The sibling post on AI repair estimate accuracy applies the same reconciliation to vehicle repair quotes.

Frequently asked questions

Can an AI receptionist give phone repair quotes?

An AI receptionist can give phone repair quotes when it reads them from the shop's price list. Accuracy depends on resolving three keys first: exact model, repair type and part tier. Test each quote against the price list row, not against what sounds reasonable. If the model is uncertain, the agent should quote a range and confirm at check-in.

How do you test an AI receptionist for a repair shop?

Script calls with a known true model, repair and part tier for each confusion set. Place them over the real phone path with varied accents, noise and interruptions. Score the ticket the agent writes: model, price row, quoted price and reserved SKU. Compare results with the previous release before shipping any prompt or STT change.

Why does the AI log the wrong phone model?

AI intake logs the wrong phone model for three reasons. Narrowband phone audio blurs suffixes like "Pro Max" and number endings like teen versus ty. The language model may snap a partial name to the nearest catalog entry. And callers often state the wrong model themselves, which a perfect transcript can't fix without a discriminating question.

Should a repair shop ask for a phone passcode over the phone?

A repair shop should not collect device passcodes over the phone. Apple's service guidance tells customers never to give a device passcode to anyone. A spoken passcode ends up in recordings, transcripts and ticket notes. The agent should decline politely. Then it gives prep steps: back up, and turn off Find My or turn on Ready for Repair.

What should an AI intake call write to a repair ticket?

An AI intake call should write the canonical model and model number if known. Add storage, color, issue category and water exposure. Then part tier, price row ID, quoted price and any reserved SKU. Finish with intake mode and turnaround promise. Attach the recording and transcript. Never write passcodes. Storing the price row ID makes every quote auditable later.

How should an AI agent describe OEM vs aftermarket parts?

An AI agent should describe OEM and aftermarket parts using the shop's approved wording and tier names. It must quote from the matching tier row. In California, SB 244 requires independent providers to disclose used or non-manufacturer parts in writing. The agent should never claim an aftermarket part is indistinguishable, since iPhone part history may label it.

How do you measure model capture accuracy on repair calls?

Measure model capture accuracy by comparing the agent's stored model with the technician-confirmed model at check-in. Divide matching tickets by all tickets with a confirmation. Split misses by cause: STT miss, LLM normalization, or caller error. Estimating a rate near 3% within one point needs about 1,100 confirmed tickets.

Does right to repair law affect what an AI receptionist says?

Right to repair law can affect what an AI receptionist says about parts. California's SB 244 took effect July 1, 2024. Unauthorized repair providers must give written notice and disclose used or third-party parts. Other states have their own rules. Shops should confirm their duties with counsel and approve the agent's parts wording.

The bottom line

A phone repair shop AI receptionist is only correct when the ticket, price row and reserved part match the device that walks in. Test suffixes against a caller matrix, score quotes against each shop's price list, and reconcile every ticket with the technician's confirmed model.