Motivated Seller AI Calls: Measure What It Gets Wrong

Last verified October 16, 2026, against 47 CFR 64.1200, FCC 24-17 and the FCC's January 2026 revocation waiver order (DA 26-12).

This post is for the product team inside a real estate investor CRM. Your agent handles seller lead qualification for many investors and wholesalers. It captures the property, scores motivation, writes the record and sets the follow-up cadence. Investor CRM vendors now describe this openly. REsimpli, for example, says its built-in AI is answering inbound seller calls, qualifying on motivation, timeline and condition, scoring leads and updating the record after calls.

The hard part is knowing how often the record is wrong. We already cover agent-side buyer and seller qualification and speed-to-lead TCPA checks. This post covers wholesaling AI calls to sellers, and how to put numbers on their errors.

Why motivated seller AI calls fail without an error

A seller call is the front door to a workflow. Speech becomes fields. Fields become a score. The score sets a pipeline stage and a follow-up cadence. Every step after the call trusts the step before it.

Take one inbound call. The seller says: "Mom passed in June. The roof leaks. We owe maybe forty on it. I want it gone before winter." A human acquisitions rep hears a hot lead. An inherited house, needs work, high equity, a deadline.

Now look at what the agent stored. Speech-to-text heard "one forty." The mortgage balance field says $140,000. The extractor could not map "before winter" to a date bucket, so timeline says "12+ months." Motivation and condition are right. Two fields are wrong, and nothing threw an error.

The score drops from Hot to Cold. The lead moves to a slow nurture cadence. A competitor makes the offer that week. Your dashboard shows a completed call and a full record. The miss never appears anywhere.

The weights below are an illustrative model, not any vendor's formula.

Two field errors moved the score 55 points. That is why a transcript-level "the call went fine" grade tells you almost nothing. You have to grade the record.

The seller record: what to capture and how each field breaks

Each field breaks in a different way, so each needs its own match rule. Use this table as the schema for your golden labels.

Asking price, mortgage balance and liens feed the equity estimate, usually the heaviest score input. Money fields deserve their own test set. Our guide to STT accuracy on names and entities covers why numbers and street names fail more often than ordinary words. Our post on tool-argument accuracy from speech covers what happens when those values reach an API call.

Field-level accuracy for motivated seller AI calls

"Accuracy" hides five different outcomes. Count them separately for each field on each labelled call.

Captured-correct (C): the seller gave a value, and the stored value matches under the field's rule.

Captured-wrong (W): the seller gave a value, and a different value was stored.

Missed (M): the seller gave a value, and the field is empty.

Invented (I): the seller never gave a value, but the field has one.

Correct-empty (E): the seller never gave a value, and the field is empty.

From those counts:

• Field precision = C / (C + W + I). Of the values you stored, how many are right?
• Field recall = C / (C + W + M). Of the values the seller gave, how many did you capture right?
• Field-level accuracy = (C + E) / all calls.

Invented values deserve their own alarm. An agent that writes "no liens" when nobody asked is worse than one that leaves the field blank. A blank field prompts a human to ask. A confident wrong value does not.

The golden label records what the seller said, not the truth about the property. If the seller says $40,000 and title shows $52,000, the agent was right.

Speech makes slot extraction harder than text. The SLURP benchmark paper shows this on 58 hours of spoken commands. Its best entity tagger scored an F1 of 78.19 on gold transcripts. Run over ASR output, the same tagger's standard F1 fell to 62.69. Your seller agent faces the same gap on noisier phone audio.

Here is a simplified scorer. It assumes your golden file and CRM export share a call ID.

# Simplified field scorer for seller-call records
from collections import Counter

MONEY = {"asking_price", "mortgage_balance"}

def match(field, gold, got):
    if field in MONEY:
        tol = max(5000, 0.05 * gold)
        return abs(got - gold) <= tol
    if field in {"liens", "decision_makers"}:
        return set(gold) == set(got)
    return str(gold).strip().lower() == str(got).strip().lower()

def score_field(field, pairs):
    """pairs: list of (gold_value, stored_value); None = not given/empty"""
    c = Counter()
    for gold, got in pairs:
        if gold is None and got is None:
            c["E"] += 1
        elif gold is None:
            c["I"] += 1
        elif got is None:
            c["M"] += 1
        elif match(field, gold, got):
            c["C"] += 1
        else:
            c["W"] += 1
    stored = c["C"] + c["W"] + c["I"]
    given = c["C"] + c["W"] + c["M"]
    return {
        "precision": c["C"] / stored if stored else None,
        "recall": c["C"] / given if given else None,
        "accuracy": (c["C"] + c["E"]) / sum(c.values()),
        "invented": c["I"],
    }

Run it per field and per caller profile. One overall number averages away the field that matters most.

Motivated seller lead scoring: precision and recall on hot sellers

Treat "hot" as a binary label. A human reviewer listens and decides whether this seller deserved same-day attention. Then compare the agent's label.

Accuracy is the wrong headline here. Hot sellers are rare. Take an illustrative 1,000 calls where 100 sellers are truly hot. The agent flags 70 as hot. Of those, 55 are right.

Accuracy is 94%. That looks fine. Precision is 55 / 70 = 0.79. Recall is 55 / 100 = 0.55. The agent misses almost half of the sellers who matter.

This is a known trap with rare positives. A 2015 PLOS ONE study by Saito and Rehmsmeier found that precision-recall plots predict real performance on imbalanced data better than ROC plots. Report precision and recall, never accuracy alone. Our lead qualification metrics guide lists the companion metrics.

False-cold vs false-hot: price the two errors

The two errors do not cost the same. The figures below are illustrative assumptions. Replace them with your customers' numbers.

False hot. A rep calls back a seller who was never serious. Assume 20 minutes of acquisitions time plus overhead: about $40.

False cold. A hot seller lands in a 30-day nurture cadence. Assume $12,000 gross per closed deal and an 8% close rate on worked hot leads. The lead is worth $960. Assume a slow cadence loses half of that to faster buyers: $480.

Back to the matrix. The 45 false colds cost 45 × $480 = $21,600. The 15 false hots cost 15 × $40 = $600. The "94% accurate" agent leaks $21,600 per 1,000 calls. Almost all of it hides in the cell nobody reviews.

The cost ratio also tells you where to set the threshold. With a calibrated hot probability p, route to a human when:

p ≥ C_false_hot / (C_false_hot + C_false_cold) = 40 / (40 + 480) ≈ 0.077

So an agent should escalate any seller with roughly an 8% chance of being hot. That is far lower than most default thresholds. In practice, set a recall floor for hot sellers, say 0.90. Then accept whatever precision your customers' rep capacity can absorb.

A CRM serving many investors has one more twist. Each investor has a different buy box. One buys in Detroit with an equity floor of 40%. Another buys in Phoenix and ignores condition. A single prompt change ships to all of them at once. So compute precision and recall per tenant configuration, not just across the platform.

Building a golden dataset from labelled seller calls

A golden dataset is a set of real calls with human labels for every field and for the hot flag. It is the ruler for every metric above. Our golden dataset guide covers the general method. Here is what changes for seller calls.

Stratify by lead source. Direct mail, PPC, outbound cold calls and SMS callbacks sound different. Outbound sellers are guarded; inbound sellers ramble. Sample each source separately.

Oversample hot sellers. Pull calls whose leads later went under contract. Those are your confirmed positives.

Label from audio. Labellers must listen, not read the transcript. If the transcript says "one forty," a transcript-only labeller writes $140,000 and the error disappears from your test set.

Double-label and adjudicate. Two labellers per call, a third to break ties. Track agreement per field. If humans disagree on "condition," your rubric is vague, not your agent. Our post on human-in-the-loop evaluation has a review workflow.

How many calls? Use the standard sample-size formula for a proportion: n = z² × p(1 − p) / e².

• Hot-seller recall near 0.85, ±5 points at 95% confidence: n = 1.96² × 0.85 × 0.15 / 0.05² ≈ 196 hot calls.
• One field's accuracy near 0.90, ±3 points: n = 1.96² × 0.90 × 0.10 / 0.03² ≈ 385 calls where that field was given.

At a 10% hot rate, 196 hot calls means about 1,960 random calls. That is why oversampling confirmed deals matters. Keep a held-out slice you never tune prompts against.

Caller profiles that break seller calls

Motivated seller AI calls skew toward stressful moments: probate, divorce, foreclosure, tired landlords. Test each caller profile on purpose.

The research supports the profile split. Koenecke et al. tested five commercial ASR systems on 19.8 hours of interviews. Average word error rate was 0.35 for Black speakers and 0.19 for white speakers. Our accent robustness guide covers how to test for that gap.

For older speakers, newer models can do well. A 2025 Leiden study found Whisper large-v3 reached a WER of 0.06 on Dutch adults aged 65 and over. But it also found hallucinations about as frequent as mishearings in its error analysis. A hallucinated number in a seller call is an invented field value. Our emotional tone evaluation post covers grading empathy with distressed callers.

Building these profiles by hand is slow. Evalgent generates test scenarios from your agent's prompt and flows, then runs caller profiles with set accent, pace, interruption level, response delay, behaviour and background noise. Its call simulator reaches the agent over PSTN, SIP or WebRTC, so outbound and inbound flows get tested on the same phone path real sellers use.

Compliance checks for AI cold calling for real estate investors

Outbound seller calls carry TCPA exposure that inbound calls mostly do not. Here is what the rules say as of October 2026. This section is not legal advice. Talk to counsel about your specific campaigns.

AI voices are artificial voices. In FCC 24-17, the FCC ruled that the TCPA's limits "encompass current AI technologies that generate human voices." Under 47 CFR 64.1200(a)(1), a call to a wireless number using an artificial voice needs the called party's prior express consent, absent an emergency. That holds whatever the call is about.

Telemarketing raises the bar. If a call includes an advertisement or telemarketing, (a)(2) and (a)(3) require prior express written consent. The rule defines telephone solicitation as encouraging "the purchase or rental of, or investment in, property, goods, or services." Whether an offer to buy a house fits that definition is a question to put to your counsel.

Do-not-call. Section 64.1200(c)(2) restricts solicitations to numbers on the national registry. Section (d)(3) requires honoring a company-specific do-not-call request within a reasonable time, not to exceed 10 business days. Calls must stay within 8 a.m. to 9 p.m., the called party's local time.

Revocation. Consent revoked by "any reasonable method" must be honored within ten business days under (a)(10). The broader "revoke-all" piece is not in force yet. On January 6, 2026, the FCC extended its waiver to January 31, 2027. That piece would make a revocation apply to unrelated future robocalls and robotexts.

Identification. Artificial voice messages must state the responsible business at the start and give a callback number, per 64.1200(b).

State rules. Many states add their own telemarketing registration or calling rules. Check the states your investors dial into.

Vulnerable sellers. Distressed homeowners are a target for scams. The FTC's April 2026 alert on mortgage relief scams lists the patterns: upfront payment demands, pressure to stop talking to the lender, and pressure to transfer the deed. An honest acquisition agent must never sound like that. Write hard constraints. The agent never promises to stop a foreclosure, never discourages contact with a lender, and never states an offer it was not given. Test each one with adversarial callers.

For the dialing-side checks before an outbound launch, see our outbound voice agent testing plan.

How to measure what motivated seller AI calls get wrong

1. Freeze the schema. List every field, its type and its match rule, using the table above. Add the do-not-call flag as a field.

2. Pull and stratify calls. Sample by lead source, tenant and caller profile. Oversample sellers who later went under contract.

3. Label from audio. Two labellers per call, adjudicated. Label what was said, the hot flag and any compliance event.

4. Score fields. Compute precision, recall, accuracy and invented-value count per field, per profile, per tenant.

5. Score the hot flag. Report precision and recall, plus dollar cost per 1,000 calls using your false-cold and false-hot estimates.

6. Set gates. For example: hot recall ≥ 0.90, money-field accuracy ≥ 0.95, zero invented liens, 100% DNC write-back. Tune the numbers to your customers.

7. Replay before every change. Run the golden set and synthetic caller profiles against each prompt, model or flow change. Block the release if any gate fails for any tenant.

8. Monitor production. Score live calls for the same fields and flags. Send low-confidence and disputed calls to human review, and add them to the golden set.

Step 8 is where Evalgent's production monitoring fits. You push call recordings. Calls are transcribed, mapped to scenarios and judged against your metrics, including custom ones like field accuracy. It flags tool-call failures, policy failures and PII leakage with fix recommendations. Because it evaluates from the audio, not only the transcript, it can catch the "forty" heard as "one forty" that a transcript-reading judge would mark correct.

For the wider picture, start with our real estate AI leads testing playbook.

Frequently asked questions

Can AI cold call motivated sellers?

AI can place outbound calls to sellers, but the FCC treats AI-generated voices as artificial voices under the TCPA (FCC 24-17). Calls to wireless numbers need prior express consent, and telemarketing calls need written consent. National and company do-not-call rules and the 8 a.m. to 9 p.m. window also apply. Confirm your campaign design with counsel.

How do you score motivated seller leads?

Motivated seller lead scoring usually weights four inputs: motivation (reason for selling), timeline, property condition and equity. Equity comes from asking price, mortgage balance and liens. Score each input from the captured fields, sum the weighted parts and set a hot threshold. Then check the score against human-labelled calls, measuring precision and recall on hot sellers.

How accurate are AI seller lead calls?

No public benchmark reports accuracy for AI seller lead calls. You have to measure your own. Label real calls from audio, then compute field-level precision and recall for each captured field. Add precision and recall on the hot flag. Break results down by caller profile, because published ASR studies show large error gaps between speaker groups.

What questions qualify a motivated seller?

A seller qualification call typically covers the property address, condition, occupancy (owner, tenant or vacant), timeline, reason for selling, asking price, mortgage balance, liens and who else is on title. The agent should also capture any do-not-call request. Each answer maps to a CRM field that needs its own match rule for testing.

How to test an AI acquisition agent?

Testing an AI acquisition agent starts with a labelled golden dataset of real seller calls. Add synthetic callers covering distressed, elderly, angry and non-native English profiles. Replay both sets against every prompt or model change. Score field accuracy, hot-seller recall and compliance events, and block releases that miss a gate for any investor tenant.

How many labelled calls do you need?

The sample-size formula n = z² × p(1 − p) / e² sets the count. To estimate hot-seller recall near 0.85 within ±5 points at 95% confidence, you need about 196 truly hot calls. One field's accuracy near 0.90 within ±3 points needs about 385 calls where that field was given. Oversample confirmed deals to reach the hot count faster.

Do AI voice calls need consent under the TCPA?

AI voice calls generally need consent under the TCPA. FCC 24-17 holds that AI-generated voices are artificial voices. Under 47 CFR 64.1200(a)(1), artificial-voice calls to wireless numbers need prior express consent, absent an emergency. Calls with telemarketing or advertising need prior express written consent. This is a summary, not legal advice.

What is a false-cold seller lead?

A false-cold seller lead is a seller who was truly ready to sell but whom the agent scored as cold. Usually a captured field was wrong, such as a misheard mortgage balance or an unresolved timeline. The lead drops into a slow follow-up cadence. False colds typically cost far more than false hots, so set a recall floor for hot sellers.

The bottom line

Motivated seller AI calls rarely fail loudly; they fail by storing a wrong number that silently turns a hot seller cold. Measure every field against labelled audio, report hot-seller precision and recall weighted by what each error costs, and gate every release on those numbers per investor.