Is Your Real Estate AI Mis-Qualifying Buyers and Sellers?

Your team runs an AI agent that answers buyer and seller calls for hundreds of agents. It asks the usual questions. Buying or selling? When? What price range? Pre-approved? Which area? Then it writes a stage to the CRM and routes the lead. The agent who gets the lead trusts that stage. Nobody listens to the call.

This post is for the team behind that flow. It covers how the stage goes wrong, why speech causes it, and how to test it. For the broader program, start with our real estate leads testing playbook. For which metrics to put on a dashboard, see metrics for a lead qualification voice agent. This post does not repeat them. It goes one level down, to the fields and the errors inside them.

What mis-qualification looks like in a real estate AI

Mis-qualification: a lead whose stored stage differs from the stage a careful human would assign after hearing the same call. It has two directions, and they cost very different amounts.

A false hot lead is a curious caller pushed to "hot." An agent drops everything to call back. The caller was "just looking at what's out there." The agent loses half an hour. Repeat that enough and agents stop trusting the hot tag at all.

A false cold lead is a ready buyer marked "nurture." They are pre-approved and want to tour this weekend. Instead they get a drip email. A competing agent who called back picks them up. You never see this loss in a dashboard, because the lead looks like any other nurture contact.

In classifier terms, a false hot is a false positive and a false cold is a false negative. Teams often tune real estate lead scoring AI as if both cost the same. They do not.

Two quieter failures sit beside these. A misrouted lead has the right stage but the wrong agent. A corrupted record has the right stage but a wrong field, like a price off by 10x. Both pass a "was it qualified?" check.

The fields that decide a lead's stage

Most AI ISA lead qualification flows ask about the same seven things. Each one has a spoken form that a transcript-based check will happily accept.

Two fields need extra care. Pre-approval is a caller claim, not a verified fact. Store the claim, not a checked status. Seller motivation separates a listing appointment from a valuation request. "We move to Denver in August" and "just curious" can both end in "What's my house worth?" Seller lead qualification must capture the reason, not just the request.

Buyer lead qualification questions should also allow "both." Many sellers are buyers too. A single intent field forces a choice. The losing side never gets a call.

Why speech breaks these fields

The voice call is the front door to a workflow: capture, normalize, score, route, write back. An error at the front flows through every later step. Nothing raises an exception, because each step receives a valid-looking value.

Numbers pass through inverse text normalization

Speech recognition first produces spoken-form words. Inverse text normalization (ITN) then turns them into written form, such as "$450,000." The authors of the open-source NeMo ITN library note the task has "extremely low tolerance to unrecoverable errors." That is why many production systems still use hand-written grammars for it.

Real estate shorthand is hard for ITN. "Four-fifty" has no unit. It could be a price, a time or a street number. Deepgram's smart format docs show "eight thirty seven pm" becoming "8:37 PM." That helps a scheduling call. In a price answer, a bare "four fifty" may come back as "4:50" or "450." Your extraction code must then scale it. For streaming, the docs say formatting may wait for an entity or finalize after 3 seconds of silence. With `no_delay=true` it "often skips formatting." One config flag changes the text you parse.

Research on ITN shows the numeric classes are the weak spot. Pandey et al. (2022) trained ITN with synthetic spoken-written pairs. They report an overall accuracy gain of 14.44% across cardinal, currency and fraction forms versus in-domain data alone. The production lesson: numeric forms improve with targeted data. Do not assume a general model knows your market's shorthand. Antonova et al. (2022) reframed ITN as tagging partly because sequence-to-sequence models can hallucinate digits. Either way, a hallucinated digit in a price field looks like any other number.

Intent and entities degrade together

The SLURP benchmark (Bastianelli et al., EMNLP 2020) measured spoken language understanding on about 58 hours of audio. On gold transcripts, the best model hit 86.99% intent accuracy. Fed by the best adapted ASR system (16.2% WER), the same model reached 81.42%. SLURP also added a dedicated entity metric, SLU-F1, for finer error analysis. Real estate qualification is mostly entities: prices, dates, names, addresses. Expect the gap to be at least as wide on your calls. Measure it rather than assume it.

Proper nouns and homophones route leads

Area names are proper nouns your recognizer may never have seen. Subdivisions, school districts and lender names fail in the same way. A split word ("Lake Wood" for "Lakewood") or a near-homophone can match a different territory in your routing table. Our guide to evaluating STT for names and entities covers keyword boosting and entity error rates in depth. For this flow, the practical rule is short. Never route on a fuzzy match without reading the matched area back to the caller.

Hedges get coerced into dates

Timeline answers are hedged. "Probably spring, depends on the job." An LLM asked to fill `timeline_days` will pick a number. That number then drives the stage. Let the extractor say "unclear." Store the caller's words beside the parsed value. We cover this pattern in tool argument accuracy. The model fills a required field with a confident guess.

A qualification confusion matrix with costs

Accuracy hides direction. A "92% accurate" flow can still lose the deals that matter. Score each cell of a confusion matrix and weigh it by cost.

Use four stages: hot (call now), warm (call this week), nurture (drip), and not a lead (renter, agent, wrong number). Label 200 or more real calls by hand. Then count where each true stage landed.

Here is a worked example. Every number is an illustrative assumption, not a benchmark. Swap in your own.

• A false hot costs 30 minutes of agent time at $60 an hour: $30.
• A false cold on a true hot buyer loses the deal some fraction of the time. Assume a $450,000 home, a 2.5% buyer-side commission ($11,250) and a 10% chance the lead would have closed: $1,125.
• The cost ratio is 1,125 / 30, or about 37 to 1.

Now run 1,000 qualified calls. Suppose 120 are truly hot and 880 are not.

The false hot errors are six times more common. The false cold errors cost over six times more. A team that only hears agent complaints will tune away false hots and make the expensive error worse.

Two design rules follow. First, set separate thresholds per direction. A hot-to-nurture error rate of 10% should block a release. A small rise in false hots may not. Second, add a "hot-uncertain" route. When the flow is unsure between hot and nurture, a human calls back fast. That trade costs minutes and protects deals.

Research backs a human at the boundary. A review of AI lead generation by Kaplan et al. (2025) cites work finding that human insight helps avoid over-confident false positives. Wu et al. (2024) reviewed 44 lead scoring studies and found 14 metrics used to measure their effect on sales. Your test suite needs one more cut: error direction.

Parsing spoken price ranges into numbers

Price errors are the easiest to catch in code. Parse the caller's words independently and compare them to the CRM field. The script below is simplified and US English only. It handles shorthand, bands, clock-time ITN output and "under" caps. It flags 10x errors apart from range errors.

import re

UNITS = {"zero": 0, "one": 1, "two": 2, "three": 3, "four": 4, "five": 5,
         "six": 6, "seven": 7, "eight": 8, "nine": 9, "ten": 10,
         "eleven": 11, "twelve": 12, "fifteen": 15, "twenty": 20,
         "thirty": 30, "forty": 40, "fifty": 50, "sixty": 60,
         "seventy": 70, "eighty": 80, "ninety": 90}
PLURAL = {"ones": 1, "twos": 2, "threes": 3, "fours": 4, "fives": 5,
          "sixes": 6, "sevens": 7, "eights": 8, "nines": 9}
BAND = {"low": (0.0, 1/3), "mid": (1/3, 2/3), "high": (2/3, 1.0)}

def words_to_num(tokens):
    """'four fifty' -> 450, 'one point two' -> 1.2, '4:50' -> 450."""
    if len(tokens) == 1 and re.fullmatch(r"\d{1,2}:\d{2}", tokens[0]):
        return float(tokens[0].replace(":", ""))      # ITN turned it into a clock time
    if len(tokens) == 1 and re.fullmatch(r"[\d.,]+k?", tokens[0]):
        t = tokens[0].replace(",", "")
        return float(t[:-1]) * 1000 if t.endswith("k") else float(t)
    if "point" in tokens:
        i = tokens.index("point")
        whole = words_to_num(tokens[:i]) or 0
        frac = "".join(str(UNITS[t]) for t in tokens[i + 1:] if t in UNITS)
        return float(f"{int(whole)}.{frac}")
    vals = [UNITS[t] for t in tokens if t in UNITS]
    if not vals:
        return None
    if len(vals) >= 2 and vals[0] < 10 and vals[1] >= 10:   # "four fifty" = 4 * 100 + 50
        return vals[0] * 100 + sum(vals[1:])
    return float(sum(vals))

def scale(n, mult=None):
    """Put a bare spoken number on the home-price scale."""
    if mult == "million" or (n < 10 and n % 1):   # "one point two"
        return n * 1_000_000
    if mult in ("thousand", "k"):
        return n * 1000
    if n < 10:                 # "five" after "four-fifty to" means 500k
        return n * 100_000
    if n < 1000:               # "four-fifty" means 450k
        return n * 1000
    return n

def parse_price(text):
    """Return (low, high) in dollars, or None. Simplified, US English only."""
    t = text.lower().replace("-", " ").replace("$", "")
    m = re.search(r"\b(low|mid|high)\s+(" + "|".join(PLURAL) + r")\b", t)
    if m:                                         # "low sevens" -> 700k-733k
        base = PLURAL[m.group(2)] * 100_000
        lo, hi = BAND[m.group(1)]
        return (round(base + lo * 100_000), round(base + hi * 100_000))
    parts = re.split(r"\b(?:to|and)\b|–", t)
    out = []
    for p in parts[:2]:
        toks = re.findall(r"[a-z]+|\d[\d.,:]*k?", p)
        mult = next((x for x in toks if x in ("million", "thousand", "k")), None)
        toks = [x for x in toks if x in UNITS or x == "point" or x[0].isdigit()]
        n = words_to_num(toks) if toks else None
        if n is not None:
            out.append(round(scale(n, mult)))
    if not out:
        return None
    if re.search(r"\b(under|below|max|up to)\b", t):
        return (None, out[0])
    return (out[0], out[-1])

def check_crm(spoken, crm_min, crm_max, tol=0.05):
    got = parse_price(spoken)
    if got is None:
        return "UNPARSED: route to human review"
    lo, hi = got
    ref = hi or lo
    crm_ref = crm_max or crm_min
    ratio = crm_ref / ref
    if 8 <= ratio <= 12 or 1/12 <= ratio <= 1/8:
        return f"FAIL 10x: heard {got}, CRM {crm_min}-{crm_max}"
    ok_lo = lo is None or crm_min is None or abs(crm_min - lo) <= tol * lo
    ok_hi = hi is None or crm_max is None or abs(crm_max - hi) <= tol * hi
    return "PASS" if ok_lo and ok_hi else f"FAIL range: heard {got}, CRM {crm_min}-{crm_max}"

if __name__ == "__main__":
    cases = [
        ("four-fifty to five", 450_000, 500_000),
        ("low sevens", 700_000, 733_000),
        ("mid sixes", 633_000, 666_000),
        ("one point two million", 1_200_000, 1_200_000),
        ("under six hundred", None, 600_000),
        ("4:50", 45_000, 45_000),            # CRM wrote 45k: a 10x miss
        ("around 525k", 525_000, 525_000),
        ("high fours", 400_000, 500_000),    # CRM band too wide
    ]
    for spoken, lo, hi in cases:
        print(f"{spoken!r:24} -> {parse_price(spoken)!s:20} {check_crm(spoken, lo, hi)}")

Output from a real run (Python 3):

'four-fifty to five'     -> (450000, 500000)     PASS
'low sevens'             -> (700000, 733333)     PASS
'mid sixes'              -> (633333, 666667)     PASS
'one point two million'  -> (1200000, 1200000)   PASS
'under six hundred'      -> (None, 600000)       PASS
'4:50'                   -> (450000, 450000)     FAIL 10x: heard (450000, 450000), CRM 45000-45000
'around 525k'            -> (525000, 525000)     PASS
'high fours'             -> (466667, 500000)     FAIL range: heard (466667, 500000), CRM 400000-500000

Three gotchas surfaced while writing this.

1. Scale depends on market. "Under 1,000 means thousands" fails for land and rentals. Make scale a per-market setting.

2. "Low sevens" is a band, not a point. Store a range. A CRM with only `max_price` drops the floor.

3. Parse the caller channel, not the agent's summary. Parsing the agent's readback tests the agent against itself. Our post on transcript vs audio evaluation explains why the judge must not share the agent's ears.

The same pattern works for timelines. Map "spring" to a date window, allow "unclear," and assert the follow-up date falls inside it.

Fair Housing questions the flow must handle

Qualification calls drift. A buyer asks about price. Then they ask, "Is it a safe neighborhood?" or "Are there many families like us?" Your agent answers live, for thousands of callers. Consistency is both your risk and your best defense.

This section is not legal advice. Have fair housing counsel review your agent's scripts.

The statute. The Fair Housing Act at 42 U.S.C. 3604(c) bars any "notice, statement, or advertisement" about a dwelling sale that "indicates any preference, limitation, or discrimination." The protected classes are race, color, religion, sex, handicap, familial status and national origin. HUD's rule at 24 CFR 100.75 says these prohibitions "shall apply to all written or oral notices or statements." The rule does not carve out a speaker that is software.

Steering. 24 CFR 100.70(c) lists steering conduct. One example is "discouraging any person from inspecting, purchasing or renting a dwelling" on a protected basis. Another is "communicating to any prospective purchaser that he or she would not be comfortable or compatible with existing residents."

Guidance changed in 2026. NAR's 2023 broker guidance told agents to act as a "source of sources." It advised clarifying vague words like "safe" and avoiding school opinions. On April 24, 2026, HUD's fair housing office issued a Dear Colleague letter. Assistant Secretary Craig Trainor signed it. It says agents "do not violate the Fair Housing Act merely by discussing with prospective homebuyers or renters the prevalence of crime." It criticizes refusals to share such information. It still states that "it is unlawful for realtors to treat their clients differently based on race." As of October 2026, the letter does not mention AI.

For an AI agent, both documents point to one rule. Give every caller the same objective answer, and never describe who lives somewhere.

The last column is the key. The strongest test is a paired call. Run the same scenario twice with one detail changed, such as accent, name or a stated family size. The stage, the routing and the substance of the answers must match. Any difference is a finding. Protected-class details must never enter the scoring model or the routing table. A paired test is the only way to show that they did not.

For more on writing these as executable checks, see testing policy adherence and regulated voice agent policy test cases.

Designing the test set: caller archetypes times ambiguity

A test set for real estate AI lead qualification needs two axes. One is who is calling. The other is how hard the answer is to hear.

Archetypes carry a known true stage:

• Ready buyer: pre-approved, touring this week (true hot)
• Curious browser: "just seeing what's out there" (true nurture)
• Must-sell seller: relocation, fixed date (true hot)
• Curious seller: wants a value, no plan to move (true nurture)
• Buy-and-sell: needs both sides, timing linked (hot on one side)
• Renter or investor: not a fit for the default buyer flow (route elsewhere)
• Fair Housing prober: any archetype plus a protected-class question

Ambiguity conditions stress the speech layer:

• Clean speech, explicit numbers (control)
• Price shorthand: "four-fifty," "low sevens," "one-two"
• Vague timeline: "spring," "after the kids finish school"
• Homophone or rare area name, school district or lender
• Self-correction: "five hundred... no, five-fifty"
• Noisy or fast speech, or a strong accent

Seven archetypes by six conditions gives 42 cells. Cover every archetype with the control and three hard conditions. Run each cell three times or more, since LLM agents vary. Add paired calls and you reach a few hundred calls per release. That is only practical with simulated callers. Our guide to synthetic callers covers how to build them.

Evalgent generates scenarios like these from the agent's prompt and flows. Caller profiles set accent, pace, interruptions, noise and behavior. Calls go over phone, SIP or WebRTC, so the speech layer stays in the loop. That matters, because most mis-qualification starts in the audio.

How to test real estate AI lead qualification before each prompt change

1. Write the field contract. List every CRM field the call writes, its type, allowed values and an "unclear" option. Include intent, timeline window, price range, pre-approval claim, lender, area and seller motivation.

2. Label a golden set. Hand-label 200 or more real calls with true stage and true field values. Our post on golden datasets covers sampling and labeling.

3. Build the archetype matrix. Cover each archetype under the control and three hard conditions. Add paired Fair Housing calls for every archetype.

4. Run each scenario three times or more over a real phone path. Record the audio, the transcript, the CRM payload and the routing decision.

5. Score the confusion matrix by direction. Report false cold and false hot rates separately. Block release if the false cold rate rises versus the last version.

6. Diff every field against the CRM record. Use an independent parser like the one above for prices and timelines. Flag 10x price errors and area mismatches as critical.

7. Check paired calls for equal treatment. Any difference in stage, routing or answer substance is a blocking finding.

8. Gate the release and monitor after. Ship behind a flag. Our guide to shipping prompt changes safely covers canaries and rollback.

After launch, keep scoring live calls. A transcript can look fine while the CRM write-back is wrong, which is a classic silent tool failure. Evalgent's production monitoring takes pushed call recordings, maps each call to a scenario and judges it. It flags policy failures and tool-call failures with fix recommendations. Human review of verdicts keeps the labels honest at the hot and nurture boundary.

Buying instead of building? The same tests apply to a real estate lead voice agent vendor. Calling these leads back? Check consent first in real estate speed-to-lead and TCPA.

Frequently asked questions

How does AI qualify real estate leads?

AI qualifies real estate leads by asking about intent, timeline, price range, financing, area and motivation. It extracts those answers into CRM fields, scores a stage such as hot, warm or nurture, and routes the lead to an agent. Each step depends on the speech layer hearing numbers and names correctly, so errors often start in the audio.

What questions should an AI ask to qualify a home buyer?

An AI should ask a home buyer whether they are buying, selling or both, and when they want to move. It should ask their price range, whether they have talked to a lender or have a pre-approval, and which areas they want. It should read back price and area before saving, and accept "not sure" as an answer.

How do you qualify a seller lead over the phone?

Qualifying a seller lead over the phone means capturing the property address, the reason for selling, the timeline and whether they also need to buy. The reason matters most. "We relocate in August" is a listing appointment. "Curious what it's worth" is a valuation request. Read the address back digit by digit before writing it.

Can an AI ISA replace a human ISA?

An AI ISA can handle first-touch qualification at any hour and at high volume. It struggles at the boundary between hot and nurture, where hedged answers need judgment. Many teams route uncertain leads to a human for a fast callback. That hybrid keeps coverage high while protecting the costly false cold leads.

Can real estate AI tell a buyer if a neighborhood is safe?

Real estate AI can tell a buyer about neighborhood safety only by giving every caller the same objective sources, never personal opinions or resident descriptions. HUD's April 2026 letter says discussing crime is not automatically a violation, but unequal treatment is. Test with paired calls. This is not legal advice; have counsel review scripts.

How accurate is AI lead scoring for real estate?

AI lead scoring accuracy for real estate depends on the speech layer, the extraction prompt and your stage definitions. We know of no public benchmark for spoken real estate calls. Measure your own by labeling 200 or more calls and building a confusion matrix. Report false cold and false hot rates separately, because one overall accuracy number hides the costly direction.

How do you test an AI lead qualification agent?

Test an AI lead qualification agent with simulated calls that cross caller archetypes with hard speech conditions. Run each scenario several times over a real phone path. Compare every CRM field against an independent parse of the caller's words. Score errors by direction, and add paired calls that check Fair Housing equal treatment.

What happens when AI mishears a price range?

When AI mishears a price range, the CRM stores a wrong budget. "Four-fifty" can become $45,000 or a clock time. The buyer then gets the wrong listings, may land with the wrong agent tier, and may be scored cold. Parse the caller's words independently and flag any ratio near 10x as a critical error.

The bottom line

Real estate AI lead qualification fails in two directions, and the false cold direction costs far more. Test every field against the CRM record, weigh errors by direction, and prove equal treatment with paired calls before each change ships.