Home Warranty Claims AI: Catch Wrong Coverage Calls

Home warranty companies are putting voice agents on the claims line. In October 2026, Liberty Home Guard announced two AI voice agents. The homeowner agent verifies the caller by phone and address, asks diagnostic questions, checks for duplicate claims, creates the service ticket and takes the service fee on the call. Claims software platforms are building the same home warranty claims AI flow for their carrier customers.

That flow is a chain of system writes. The voice call is only the front door. If the agent hears "water heater" when the caller said "water softener," every later step is correct for the wrong item. Nobody sees the error until a technician is standing in the garage.

This post is for the product team that built its own claims agent. It covers the claim flow, three silent failures, a test matrix across plan contracts and the production checks that catch drift.

Why home warranty claims AI differs from insurance claims

A home warranty is usually sold as a service contract, not an insurance policy. Federal rules draw that line. Under 16 CFR 700.11, an agreement bought for consideration beyond the product price is a service contract. Some agreements are "sold and regulated under state law as contracts of insurance." Which bucket applies depends on the state.

States regulate home warranties very differently:

The NAIC's Service Contracts Model Act chart shows wide variation. Ohio's entry, for example, says a "home service contract is not insurance."

This matters for your agent in two ways. First, state rules can change what a correct coverage answer is. Florida's section 634.346 says a home warranty "may not exclude coverage because of the presence of rust or corrosion" alone. An agent that denies a Florida claim on "it looks rusty" breaks state law. A national prompt will not know that. Second, the generic claims patterns in our insurance voice agent testing guide assume adjusters and policies. Warranty claims run on trade calls, covered item limits and contractor networks instead.

This is not legal advice. Confirm the rules for each state you sell in with counsel.

The claim call is an agentic workflow

A typical service-request call has seven steps. Each one writes something that a later step trusts.

The item code from step 3 drives the trade, the contractor and the fee. A tool call can succeed while carrying a wrong argument. Our guide to tool-argument accuracy from speech covers that class of bug in depth.

Three failures that create a bad dispatch

A misheard item

Look at one public contract. American Home Shield's 2023 Nevada sample agreement covers "all parts and components of the water heater unit including the tank." The same agreement excludes "water softener/filtration/purification systems." One word separates a covered claim from an excluded one.

On an 8 kHz phone line, "heater" and "softener" can blur when the caller is in a noisy utility room. The model then fills the gap with the more common item. The agent says, "I've opened a claim for your water heater." The caller hears "water" and agrees.

A wrong plan tier or missing add-on

Many plans split systems from appliances. Pools, wells and second refrigerators are often add-ons. If the agent reads the wrong tier, it can approve an item the homeowner never bought. It can also deny one they did. Both outcomes look like clean calls in the transcript.

A coverage promise

The most expensive failure is a sentence. "Don't worry, that's covered" is a statement the company may have to answer for. In Moffatt v. Air Canada, 2024 BCCRT 149, a tribunal held the airline liable for its chatbot's wrong refund advice. It wrote, "It makes no difference whether the information comes from a static page or a chatbot." That is a Canadian small-claims ruling, not US law. Still, regulators look at what phone reps say. In February 2026, Arizona's Attorney General announced an $11.8 million settlement with Choice Home Warranty. The state alleged phone sales reps often did not disclose exclusions and sometimes misstated coverage. The company denies the allegations and admitted no wrongdoing.

A claims agent is a phone rep that never gets tired. If its prompt drifts toward reassurance, it can repeat the same misstatement on every call.

Why speech makes coverage errors worse

Three mechanisms stack up on a claims call.

1. Narrowband audio. Phone audio arrives as 8 kHz μ-law, band-limited to about 3.4 kHz. Much of the energy in "s" and "f" sits above that band. Those are the sounds that tell "softener" apart.

2. Prior-driven fill. When the transcript is unclear, the LLM picks the item that fits its prompt and examples best. If the prompt is full of water heater examples, the guess leans one way, every time.

3. Relative dates. Pre-existing condition and waiting-period checks need an onset date. Liberty Home Guard's waiting period guide says waits often run 30 to 90 days. "It started acting up a couple weeks back" must become a date relative to the plan start. A 30-day wait turns "two weeks" versus "two months" into a coverage decision.

Research shows the gap is real. The SpokenWOZ benchmark (Si et al., NeurIPS 2023) found the best dialogue state tracker "only achieves 25.65% in joint goal accuracy" on spoken conversations. The best end-to-end model completed the user request in 52.1% of dialogues. Amazon researchers (Kim et al., 2021) found value prediction F1 dropped 5% and 6% once ASR errors were added to user turns. Spoken slots are harder than typed ones, and an item code is a slot.

Accent and pace make this worse. See our work on accent robustness for US calls and on STT evaluation.

Separate the coverage decision from the coverage sentence

The fix is architectural. The rules engine decides coverage. The agent only reads the decision back. Every coverage sentence the agent speaks should map to a decision record.

Coverage statement: any spoken phrase that tells the homeowner an item is or is not covered, or what they will pay.

Define three allowed statement types, tied to decision states:

Here is a simplified checker that compares agent turns to the decision record. It runs on transcripts after each call.

# Simplified: flag coverage statements that disagree with the decision record.
import re

PROMISE = re.compile(r"\b(that'?s|it'?s|you'?re|is) (fully )?covered\b", re.I)
FEE = re.compile(r"\$(\d+(?:\.\d{2})?)")

def check_call(turns, decision):
    """turns: list of (speaker, text). decision: dict from the rules engine."""
    flags = []
    for speaker, text in turns:
        if speaker != "agent":
            continue
        if PROMISE.search(text) and decision["state"] != "covered":
            flags.append(("coverage_promise", text))
        for amount in FEE.findall(text):
            if float(amount) != decision["service_fee"]:
                flags.append(("fee_mismatch", text))
        item = decision["item_name"].lower()
        if "claim for your" in text.lower() and item not in text.lower():
            flags.append(("item_readback_mismatch", text))
    return flags

decision = {"state": "needs_diagnosis", "service_fee": 100.0,
            "item_name": "water softener"}
turns = [("caller", "My water softener is leaking."),
         ("agent", "I've opened a claim for your water heater. It's covered."),
         ("agent", "Your service fee is $125.")]
print(check_call(turns, decision))

This prints three flags: a coverage promise, an item readback mismatch and a fee mismatch. Regex is a floor, not a ceiling. Paraphrases like "you're good on that one" need a model-based judge. The structure stays the same: utterance in, contract truth in, verdict out. Our guide to knowledge grounding tests covers the judge side.

A home warranty claims AI test matrix across plan contracts

One home warranty claims AI prompt serves many contracts. Your carrier customers sell different tiers in different states. Build the matrix from contracts, not from the prompt.

Scenario columns that catch most coverage bugs:

1. Covered item, clear audio. Baseline.

2. Excluded near-homophone. Water softener, well pump on a plan without well cover, outdoor sprinkler line.

3. Add-on item without the add-on. Pool heater on a base plan.

4. Inside the waiting period. Onset given as a relative date.

5. Pre-existing hint. "It's been making that noise since we moved in."

6. State rule. Florida rust on an HVAC unit.

Then cross the scenarios with caller conditions: noisy room, fast speech, a heavy accent, a mid-sentence correction ("sorry, the softener, not the heater").

Worked math (illustrative). Take 4 tiers × 12 common items × 6 scenarios. That is 288 contract cases. Add 3 caller conditions and you get 864 calls. A pairwise design cuts the caller crossing, so a few hundred calls covers most interactions. For release gates, use the rule of three. If 300 test calls show zero coverage promises, the 95% upper bound on the true rate is about 3 ÷ 300, or 1%.

Cost math (illustrative assumptions, not industry data). Suppose a platform handles 10,000 claims a month. Suppose 2% get the wrong item or trade, and each wasted trip costs $90 in contractor fees. That is 200 bad dispatches and $18,000 a month, before refunds and complaints. Plug in your own numbers. The point is that small item-code error rates compound per claim.

Evalgent generates scenarios like these from the agent's prompt and flows, including branches and hard constraints. It runs them as simulated callers with set accents, pace, noise and interruption levels over phone, SIP or WebRTC.

How to test a home warranty claims voice agent

1. Export the truth. For each plan tier and state, build a table: item, covered or excluded, add-on needed, covered item limit, service fee, waiting period.

2. List the near-homophones. Pair every excluded item with the covered item it sounds like. Add brand names that confuse ASR.

3. Script the scenarios. Use the six columns above for every tier. Give onset dates as spoken phrases, not ISO dates.

4. Call over the real phone path. Text tests skip the 8 kHz codec, which is where "softener" gets lost.

5. Assert on the records, not the transcript. Check the item code, decision state, fee charged and work order trade. A clean transcript can sit on a wrong record. See detecting silent tool failures.

6. Assert on every coverage sentence. Run the checker and a judge on agent turns. Zero coverage promises outside the `covered` state.

7. Test corrections. The caller fixes the item after readback. Pass means the work order changes, not just the next sentence.

8. Rerun after every change. Run the full matrix after any prompt, model, contract or rules-engine change, before it reaches all carriers. The go-live checklist covers onboarding each new carrier.

Suggested starting thresholds. These are assumptions to tune, not standards.

Monitoring coverage statements against contract truth

Tests catch what you thought of. Production catches the rest, so home warranty claims AI needs live scoring too. Score every live call against the same rules.

Track four signals per carrier and per tier:

• Coverage statement mismatch rate. Coverage sentences that disagree with the decision record.
• Item-code reversal rate. Work orders where the technician recodes the item on site. Join the dispatch system back to the call.
• Denied-after-dispatch rate. Claims denied after a technician visit. A jump here often traces to step 3 or 4.
• Fee dispute rate. Disputes linked to a fee the agent quoted.

The second and third signals are lagging. They arrive days after the call. Join them back to the call ID so each denial points at a recording. Then turn each confirmed failure into a regression test, as in our production feedback loop guide. Compliance teams will also want the policy test cases for regulated calls.

Evalgent's production monitoring takes pushed call recordings, transcribes them and maps them to scenarios. It judges each call and flags policy failures, tool-call failures and latency spikes, with fix recommendations. Because it judges the audio, not only the transcript, it can catch a "softener" the transcript turned into "heater."

Frequently asked questions

Is a home warranty considered insurance?

A home warranty is usually a service contract, not insurance, but states differ. California regulates home protection companies through its Department of Insurance. Texas licenses residential service companies through the Texas Real Estate Commission. Florida regulates home warranty associations under Chapter 634. Federal rule 16 CFR 700.11 separates service contracts from insurance. Check each state you sell in.

Why do home warranty companies deny claims?

Common reasons are pre-existing conditions, missing maintenance, code violations or improper installation, items not covered by the plan, and reached coverage limits, according to This Old House's guide to denied claims. For a voice agent, the risky ones are items not covered and pre-existing conditions. Both depend on details the agent captures by ear on the first call.

Can an AI agent tell a homeowner a repair is covered?

An AI agent should only state coverage the rules engine has already decided. Most claims need a technician's diagnosis first. Safe wording is "a technician will diagnose it, and coverage depends on what they find." A flat "that's covered" before diagnosis is a coverage promise. It creates risk if the claim is denied later.

How do you test a home warranty claims voice agent?

Build a truth table per plan tier and state. Script covered, excluded, add-on, waiting-period, pre-existing and state-rule cases. Pair each excluded item with a covered item that sounds similar. Call over the real phone path with noise and accents. Then check the item code, decision, fee and work order, plus every coverage sentence the agent speaks.

What is a home warranty service fee?

A home warranty service fee is the flat amount a homeowner pays per service request or trade call. NerdWallet reported an average of $108.45 per service call in March 2026, with a range from none to $200. The fee depends on the plan, so the agent must read it from the plan record, never from the prompt.

How does a waiting period affect a home warranty claim?

A waiting period delays when coverage starts after purchase. Liberty Home Guard's guide says waiting periods often range from 30 to 90 days. Problems that start during the wait are usually treated as pre-existing. A claims agent must turn spoken onset phrases like "a few weeks ago" into a date and compare it to the plan start.

How to monitor AI coverage statements on claim calls?

Score every call against the decision record. Flag coverage promises outside a covered state, fees that differ from the plan and item readbacks that differ from the work order. Then join technician recodes and denied-after-dispatch claims back to the call ID. Each confirmed mismatch becomes a regression test for the next release.

What is a pre-existing condition in a home warranty?

A pre-existing condition is a breakdown that existed before coverage started. One American Home Shield sample agreement excludes breakdowns that existed before the start date and "were either known by you or were reasonably detectable by you." A claims agent should capture onset and history but leave the call on pre-existing status to diagnosis.

The bottom line

Home warranty claims AI breaks downstream, at the work order and the denial letter, not in the transcript. Test every plan contract against misheard items, wrong tiers and coverage promises, then score every live coverage statement against contract truth.