AI for Real Estate Leads: Is Your Agent Qualifying Right?

Your product answers real estate leads by phone. Maybe it is a real estate lead response tool inside a brokerage platform, a CRM add-on or a real estate AI phone agent for agent teams. The voice agent runs for hundreds of agents and teams. Each has its own territory, calendar and scripts. A prompt change on Tuesday reaches every team's new leads that night.

The question you get from a team leader is simple. Is it qualifying right? This post helps you answer with numbers. This is a real estate AI agent accuracy audit. It covers AI for real estate leads in 12 checks, with a scoring rubric of suggested thresholds, a one-week protocol for live calls, and sample-size math. It stays at audit level across all sales use cases. For the full test build, read our real estate lead testing playbook.

What real estate AI agent accuracy means when a call ends in a CRM record

A lead call is not a chat. It is the front door to a record. The caller speaks. The agent captures fields, answers a few questions and offers a showing. Then it writes a lead into the CRM, tags a source and routes it to an agent. A text or email goes out.

Every step can sound fine while one field is wrong. That is why "the call sounded good" is a weak signal. The caller heard a pleasant voice. Nobody heard the record.

Three traits of this workflow hide errors:

• Silent corruption. A wrong budget is a valid number. The CRM stores it without complaint.
• Delayed discovery. The agent finds out at the first call-back, or never. A mis-scored lead just goes cold.
• Per-team variation. "Our downtown team" maps to different territories, calendars and routing rules for each customer.

The diagram shows where a spoken lead drifts, field by field. It pairs what the caller says with a common failure and the value that gets stored. These are failure patterns to test for, not measured rates.

Why transcripts alone miss the errors that matter

Most teams audit by reading transcripts. That works for tone. It fails for numbers, addresses and names.

Take a caller who says "four fifty, maybe four seventy-five." The speech model may hear both numbers. The language model may store the second as the budget ceiling. The transcript reads fine. The audio shows the caller hedged. A reader of text never hears the hedge.

Speech models also add words nobody said. Koenecke and colleagues found that roughly 1 percent of Whisper transcriptions contained entire hallucinated phrases. Of those hallucinations, 38 percent included explicit harms. The study used one model family. Treat the figures as a warning, not a rate for your stack.

Phone audio adds its own problems. Speakerphones, car noise and 8 kHz telephony hurt street names and digit strings. Our guides on word error rate and accent robustness cover the mechanism. Judge the stored record against the audio.

Evalgent's thesis matches this. Most evaluation reads transcripts, so the referee is deaf. Real failures live in the audio. Evalgent judges the audio itself and attaches evidence to each verdict.

The 12 accuracy checks

Each check is binary per call, so you can count errors and compute a rate. Critical checks change who calls whom, or create legal exposure. Major checks create rework. The weights are a judgment call, not a standard.

Checks 1 and 2: lead type and listing facts

Lead type drives everything after it. A caller who says "I need to sell before I can buy" is both. Many agents record one. The seller half of that lead is the more valuable half, and it vanishes. Test mixed intents and callers who phone a sales line about a rental.

Listing facts are the hallucination risk. If the agent states a price, an HOA fee or "it's still available," that fact needs a source. Check each spoken fact against the listing feed at call time. A cached listing can show a home as active after it went under contract.

Checks 3 to 6: contact, timeline, budget and financing

Names and emails fail first. Test names with common variants and names your speech model rarely hears. The read-back is the control, and the corrected value, not the first guess, must reach the record.

Timeline has a date-math trap. "By the end of summer," "before school starts" and "within 90 days" are different fields. Store the caller's words next to any normalized date.

Budget is where magnitude errors live. "Four fifty" is $450,000. "Four-fifty a month" is rent. A stored $45,000 passes every format check and wrecks lead scoring. Scripts should ask for a range and read it back as a full number.

Financing questions need guardrails. Asking "are you pre-approved?" is fine. Quoting rates or promising approval is not the agent's job. For mortgage-side qualification, see our mortgage lead qualification guide and the TCPA and handoff checks for mortgage AI calls.

Check 7: agent representation

This check changed meaning in 2024. NAR states that as of August 17, 2024, an MLS Participant working with a buyer must have a written agreement before touring, in person or virtual. The agreement must state compensation in an objectively ascertainable way.

For your agent, the question "are you working with an agent?" now has a harder follow-up. Does the caller have a signed agreement? A caller who says "sort of" is a trap. The agent should record the words, avoid advice, and route to a person. Rules vary by market and by brokerage. This is not legal advice, so confirm the script with counsel.

Score two failures separately. The agent never asked, or it asked and stored the wrong answer. Both put an agent in an awkward call with someone else's client.

Check 8: search criteria

Criteria errors are quiet. "Three bedrooms, near Lincoln" becomes a two-bedroom search. Neighbourhood names collide across cities, so check that the agent asked which state.

Check 9: the showing slot

A showing is a promise with a clock on it. Check four values: date, time, time zone and property. Compare the slot spoken back with the calendar entry. Time zones matter when the caller is out of area, and "tomorrow" depends on the call time.

Our post on speed-to-lead and TCPA checks covers the outbound side of the first contact.

Check 10: routing, write-back and handoff

This check separates a demo from a product. After the call, does exactly one lead exist? Is it owned by the right agent under the team's rules? Does it carry the source, the notes and the confirmed fields? Count four failures separately: no record, duplicate record, wrong owner, and missing summary.

Handoff belongs here too. Some calls should reach a human fast: a seller with an urgent deadline, a complaint, a caller asking for a person. Score the trigger and the summary. Our guide to escalation accuracy shows how to measure good handoffs.

Check 11: consent and opt-out

Compliance is a record-keeping problem as much as a speech problem. The FCC ruled on February 8, 2024 that AI-generated voices count as "artificial or prerecorded" under the TCPA. Such calls are generally prohibited without prior express consent. Consent rules keep moving. The FCC adopted new revocation rules on September 30, 2026 and released them October 1, 2026. Per Kelley Drye and Nixon Peabody, they narrow "revoke-all" for informational calls and let callers name exclusive opt-out methods. Effective dates depend on Federal Register publication, which had not happened as of October 7, 2026.

Audit the facts you control. Was consent on file before an outbound call? Did a caller say "stop calling"? Was it recorded and honored? This is not legal advice. Our AI cold calling guide goes deeper on outbound rules.

Check 12: fair housing safe answers

The Fair Housing Act covers buying a home, per HUD. It protects race, color, national origin, religion, sex, familial status and disability. A sales line is not exempt. Callers ask "is this a good neighbourhood for families?" An agent that answers with its own judgment risks steering.

HUD's May 2, 2024 release said housing providers and advertisers are covered when AI performs these functions. That release addressed screening and digital advertising, not phone agents specifically. The audit logic still applies. Test neighbourhood questions, school questions and "who lives there" questions. The safe answer points to objective sources and offers a human agent. See AI leasing agent fair housing mistakes for the rental side.

A scoring rubric you can copy

Score each call per check as pass, fail or not applicable. An inbound call with no showing has no check 9. Then compute the error rate per check.

These thresholds are suggestions, not an industry standard:

Set your own lines by what an error costs. Slice every number by team. An average hides the one team whose territory map is broken. Our post on multi-tenant voice agent testing explains per-customer slices.

One headline number still helps. The share of calls with zero critical failures resembles task success rate. Report it beside the per-check rates, never instead of them. For lead-specific metrics, see metrics for a lead qualification voice agent.

How to run a one-week accuracy audit on your live calls

You need call recordings, the CRM records they created, and one reviewer for a few hours a day. The steps fit one team or a small group.

1. Day 1: define the unit and the frame. One unit is one completed lead call that ended in a record, a showing or a handoff. Exclude voicemail, wrong numbers and spam. Pull every eligible call from the last 7 days per team.

2. Day 1: stratify. Split by call type (inbound buyer, inbound seller, outbound follow-up), by time of day and by any language flag. Sample at least 100 calls per team, at random within each stratum. Keep every call that ended in a handoff or an error flag.

3. Day 2: join calls to records. For each sampled call, fetch the CRM lead and the calendar entry, or note their absence. Build one row with confirmed values from the audio and stored values.

4. Days 2 to 3: label. A reviewer listens to the field-bearing segments, not the whole call. Mark each check pass, fail or not applicable. Record the wrong value and the turn where it went wrong.

5. Day 4: double-label a slice. A second reviewer labels 20 percent of the calls. Compute agreement per check. Where two people disagree often, the pass rule is vague. Rewrite it.

6. Day 4: compute rates and intervals. Use the code below. Report each rate with a confidence interval, per team and overall.

7. Day 5: classify root causes. Tag each failure as speech capture, language-model reasoning, tool call, stale data or prompt policy. The fix depends on the layer.

8. Days 6 to 7: fix, then replay. Change one thing at a time. Replay the failing scenarios before the new version ships, then re-sample next week.

For sampling design beyond this week, see how to sample live calls.

A simplified scoring script follows. It assumes one CSV row per call with `team` and one `1/0/NA` column per check, where `1` is pass. It also flags budget magnitude errors.

import csv, math
from collections import defaultdict

CHECKS = ["lead_type","listing","contact","timeline","budget","financing",
          "representation","criteria","showing","writeback","consent","fair_housing"]

def wilson(k, n, z=1.96):
    if n == 0:
        return (0.0, 0.0)
    p = k / n
    d = 1 + z * z / n
    c = (p + z * z / (2 * n)) / d
    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return (max(0.0, c - h), min(1.0, c + h))

def magnitude_off(spoken, stored):
    # flags a 10x slip such as 450000 stored as 45000
    return spoken > 0 and stored > 0 and max(spoken, stored) / min(spoken, stored) >= 5

fails, seen = defaultdict(lambda: defaultdict(int)), defaultdict(lambda: defaultdict(int))
with open("audit_rows.csv", newline="") as f:
    for row in csv.DictReader(f):
        for c in CHECKS:
            if row[c] == "NA":
                continue
            seen[row["team"]][c] += 1
            fails[row["team"]][c] += (row[c] == "0")

for team in seen:
    for c in CHECKS:
        n, k = seen[team][c], fails[team][c]
        if n:
            lo, hi = wilson(k, n)
            print(f"{team:10} {c:14} {k}/{n} {k/n:5.1%} CI {lo:5.1%}-{hi:5.1%}")

Sample size: how many calls prove a rate

A rate from 40 calls is a rumor. The interval is the honest part. The table is a worked example, not data from any team. Assume a 2 percent observed error rate on a critical check. The 95 percent Wilson interval at each sample size follows.

At 100 calls you cannot tell a 1 percent problem from a 7 percent one. At 400 calls you can say the rate is probably above 1 percent and below 4. If your red line is 3 percent, even 800 calls leave the upper end at 3.2 percent, so a 2 percent rate is still not proven safe. That is why scoring every call beats a weekly sample for rare, costly errors. Our post on scoring every call covers the economics.

Zero-failure claims need the rule of three. Zero failures in n calls gives a 95 percent upper bound near 3 divided by n. Zero consent findings in 100 calls still leaves an upper bound near 3 percent. So consent needs a query across all calls, not a sample.

Illustrative example: 600 lead calls a week at a 3 percent write-back error rate means 18 bad or missing records a week.

Testing a change before it reaches every team

An audit finds errors that already happened. A pre-release test stops the next one. Write scenarios that force each failure. A caller who corrects the budget mid-call. A seller who is also a buyer. A caller who says "my cousin is my agent." A neighbourhood question about families. A showing request for a home that went pending an hour ago.

Evalgent generates test scenarios from the agent's prompt and flows, then runs them with caller profiles that vary accent, pace, interruptions, response delay and background noise. Calls reach your agent over phone, SIP or WebRTC. A custom metric fits this audit well. "The stored budget matches the spoken range" can run on every call, with audio evidence behind each verdict and human review of the results.

Our guides to shipping prompt changes safely and the go-live checklist give you one scenario set for every change.

For production, Evalgent's call monitoring takes your recordings, transcribes them, maps them to scenarios and judges them. It flags tool-call failures, PII leakage, latency spikes and policy failures with fix suggestions. Failed CRM writes show up as tool-call failures on every call, not only in a sample.

Where to go deeper: the real estate cluster

This hub stays at audit level. Each neighbour goes deep on one slice. Pick the next read by the failure you found.

For the metrics layer, see lead qualification metrics and task success rate.

Frequently asked questions

How accurate are AI voice agents at qualifying real estate leads?

Real estate AI agent accuracy depends on the team, the agent and the call mix. No public benchmark covers it. Measure your own by scoring a sample of calls against the CRM records. Treat any critical check above about 1 percent error as a fix-first item, and slice results per team.

How do I audit my real estate AI agent's lead records?

Pull a random sample of completed lead calls, fetch the matching CRM records and compare confirmed values with stored ones. Score the 12 checks as pass or fail. Compute error rates with confidence intervals per team, tag root causes, fix one thing at a time and re-sample next week.

What errors do real estate AI agents make most often?

No audited public data ranks them. The recurring risks are wrong budgets and timelines, missed seller intent, wrong showing slots, duplicate or misrouted leads, and unsupported statements about listings. Your audit shows which of these hit your teams, and which layer causes them.

How many calls do I need to sample?

For a 2 percent observed error rate, 100 calls give an interval of about 0.6 to 7.0 percent. 400 calls narrow it to about 1.0 to 3.9 percent. Sample at least 100 per team for a first pass, and score every call for rare, costly errors.

Does the TCPA apply to a real estate AI voice agent?

The FCC ruled on February 8, 2024 that AI-generated voices are "artificial or prerecorded" under the TCPA, so consent rules apply. New revocation rules were adopted on September 30, 2026. Rules and effective dates move, so check current text with counsel. This is not legal advice.

Can a real estate AI agent answer neighbourhood questions?

It should be careful. The Fair Housing Act covers buying a home, and opinions about who lives where risk steering. Safe answers point to objective sources, such as listing data or official maps, and offer a human agent. Test these questions before launch, and confirm the script with counsel.

How do I test a real estate AI agent before changing the prompt?

Write scenarios that force each of the 12 failures: budget corrections, mixed buyer and seller intent, represented callers, pending listings and opt-out requests. Run them against the old and new versions with varied caller profiles. Ship only if no critical check regresses.

When should a real estate AI agent hand off to a human?

Hand off when a caller asks for a person, has an urgent deadline, raises a complaint, mentions an agreement with another agent, or asks fair housing or legal questions. Pass a summary so the caller does not repeat themselves. Score both the trigger and the summary.

The bottom line

Real estate AI agent accuracy means the CRM lead, the showing and the consent record match what the caller said, so audit the record, not the conversation. Score the 12 checks per team, sample enough calls to trust the interval, and fix critical errors before the next prompt change ships.