Property Management AI Phone Answering: Can You Trust It?

A resident calls at 9 p.m. "Water is coming through my ceiling, unit three-oh-four." The agent says "got it, maintenance will be out soon." The work order lands on unit 403, flagged routine, with no note about the dog. The call sounded perfect. The record is wrong.

This post is an audit, not a build guide. It gives you 12 checks for property management AI phone accuracy, a scoring rubric, sample-size math and a one-week plan for live calls. It is written for teams that built and run an AI receptionist for property managers and an AI answering service for property management themselves, often inside software used by many management companies. Each customer has its own properties, rules and emergency lists. One average number hides the failures.

The build playbook lives in Property Management AI Answering: The QA Playbook. This post tells you whether your property management AI phone answering is accurate today, and how to prove it to a customer who asks.

What accuracy means on a property call

A property call is the front door to a workflow. The resident speaks, and the agent captures fields. It then writes to the property management software: a work order, a tour, a lease note, a page to the on-call technician. Staff trust that record later.

So accuracy has three layers, and each can fail on its own.

1. Heard. Did speech recognition get the property, unit number, name, phone number and date right?

2. Decided. Did the agent classify the request, set the right priority, and say only what it is allowed to say?

3. Written. Did the right values land on the right resident, unit and work order, once?

A transcript audit sees only the first layer. A record audit sees the third but not why it broke. This audit checks all three, from call audio to system record. Evalgent's premise is that the referee should not be deaf: it evaluates from the audio, not only the transcript, because real failures sit between what was said and what was stored.

A portfolio adds a fourth problem. The same unit number exists in dozens of properties, so "unit 304" means nothing until the property is pinned down. The agent resolves the property first, from the caller's number, the number dialed or a spoken address. An error there poisons everything after it. For a general method, see Voice Agent Accuracy Audit: What It Measures.

The 12 accuracy checks

Here is the scorecard. Each check has a source of truth you can compare against. If you have no source of truth for a check, that gap is itself a finding.

The tier decides how you treat a miss. Gate checks carry legal, safety or money risk: one confirmed failure with real impact blocks the next release. Core checks have a pass-rate target. Watch checks track drift. Tiers and targets are suggestions; set yours with your compliance owner.

Identity: checks 1, 2 and 3

Check 1 is the foundation. Compare the stored property and unit to what the caller said, then to the lease. Unit numbers are short strings with no redundancy. "Three-oh-four" and "four-oh-three" share digits. "Fifteen" and "fifty" differ by one syllable. Letters in 2B and 2D blur on a phone line.

Check 2 asks who is calling. Households have co-tenants, guarantors and occupants, and vendors call too. The agent must not read ledger details or entry codes to anyone off the lease or authorized-contact list. It is a gate because a wrong answer discloses another person's account. For names, see Evaluating STT for Names and Entities.

Check 3 is routing. A tenant who says "the portal charged me twice" has not made a maintenance request. Compare the agent's classification to what staff finally did with the call.

Safety and work orders: checks 4, 5 and 6

Emergency triage matters most. Managers define emergencies in writing. Buildium's guide lists no heat in freezing weather, major leaks, gas odor, broken exterior locks and fire, and says fire calls follow a 911 call. It puts single lockouts and burned-out bulbs in a next-business-day group. Replay your list in callers' own words: "my ceiling is dripping brown water" is a leak and a possible sewage problem. See After-Hours Maintenance Answering: Can AI Spot Emergencies?.

Check 5 covers the work order body in AI maintenance request calls. A good one has the right unit, a location in the unit, a plain description, a permission to enter, a pet note and a callback number. Missing one costs a second truck roll. Wrong Work Orders From AI Maintenance Calls? goes deep on these errors.

Check 6 asks whether priority and vendor dispatch followed the rules. A plumber should not be paged for a dripping faucet at 2 a.m. Test it against each property's rules.

Money and leasing: checks 7, 8 and 9

Ledger statements are a gate because the agent is quoting money. The failure is a stale number stated with confidence, or a late-fee promise that policy does not allow. If you collect on past-due balances by phone, see Auditing AI Payment Arrangement Calls for FDCPA Risk and Auditing AI Answers on HOA Balances and Board Questions.

Leasing facts are rent, deposit, fees, pet rules, dates and application steps. Compare each spoken statement to the live listing, since AI leasing agent accuracy starts there. For follow-up, see Are Your AI Leasing Follow-Ups Converting?.

Check 9 is fair housing handling, in two parts. The agent must not steer or screen on a protected characteristic. It must also recognize a disability-related request, such as an assistance animal, and route it instead of answering "our policy is no pets." Rules are in the next section, and AI Leasing Agent Mistakes That Break Fair Housing Rules shows the failure modes.

Policy, handoff and write-back: checks 10, 11 and 12

Policy answers come from per-property settings such as office hours, quiet hours and move-out notice. They drift when a manager edits settings and the agent's context does not follow. One Voice Agent, 500 Customer Setups explains how to test this at scale.

Handoff checks that a transfer keeps the reason and callback number. Escalation Accuracy: Measuring Good Handoffs covers the measurement.

Write-back catches what the others missed. List every field the call should have set, then diff against the record. A silent failure is a duplicate profile: a new contact instead of the resident, so history splits. Evaluating Tool-Argument Accuracy from Speech explains how to test the arguments.

Why errors start at the audio layer

Phone audio is thin. Twilio's Media Streams documentation says the encoding is always `audio/x-mulaw` at 8000 Hz. That strips the detail separating "F" from "S" and "B" from "D." Add a TV, a baby or a running tap, and the model hears less still.

Short alphanumeric strings suffer most. The Auto Review paper studied 9,456 calls in which an AI system phoned insurers to verify healthcare benefits. First-stage extraction errors were 10.8% for agent name, 12.9% for reference number and 9.8% for group number. The authors trace them to homophones, noise, similar-sounding names and dropped or added digits. The domain differs, but the mechanism hits unit numbers and phone numbers too. Measure your own rate.

Agents are also inconsistent. The tau-bench paper (Yao et al.) defines pass^k, the chance of succeeding on all k repeats of a task. Its abstract reports under 50% task success and pass^8 below 25% in retail. One clean test call proves little, so replay key scenarios several times.

Here is a small guard you can add. It checks the spoken unit against the property directory and the caller's lease. It is simplified, and it runs as written.

def unit_check(heard, property_id, directory, lease_unit=None):
    """Return a label for a captured unit number."""
    units = directory.get(property_id, set())
    if heard not in units:
        return "no_such_unit"
    if lease_unit and heard != lease_unit:
        if sorted(heard) == sorted(lease_unit):
            return "digit_swap_suspect"
        return "differs_from_lease"
    return "ok"

directory = {"maple-court": {"304", "403", "310"}}
print(unit_check("403", "maple-court", directory, "304"))  # digit_swap_suspect
print(unit_check("340", "maple-court", directory))         # no_such_unit

A `digit_swap_suspect` should trigger a read-back before any work order is written.

Where the audit touches the rules

Some checks are quality checks. Others sit next to law. This section is a map, not legal advice. Ask counsel how each rule applies to your customers and states.

Fair housing. Under 42 U.S.C. 3604(c), it is unlawful to make or publish any statement about renting a dwelling that shows a preference, limitation or discrimination based on a protected characteristic. A spoken answer is a statement. The same section, at 3604(f)(3)(B), treats refusing a reasonable accommodation as discrimination.

Accommodation requests. HUD's regulation at 24 CFR 100.204 bars refusing reasonable accommodations in rules, policies, practices or services. The HUD and DOJ joint statement on reasonable accommodations says a request can be made orally or in writing and need not use the words "reasonable accommodation." So "my service dog can't use the stairs, can I get ground-floor parking?" is a request. Your agent must recognize it and hand it to staff.

AI in housing. HUD's May 2, 2024 news release 24-098 covers applicant screening and digital advertising. It says the Fair Housing Act applies when AI performs those functions. It does not mention phone agents, so treat it as direction, not a voice rule.

Outbound calls. If your agent also dials residents for reminders, consent rules apply. The FCC's Declaratory Ruling FCC 24-17 was adopted February 2, 2024 and released February 8, 2024. It holds that AI-generated voices are an "artificial or prerecorded voice" under the TCPA, which generally calls for the called party's prior express consent. A newer order, FCC 26-67, was adopted September 30, 2026 and released October 1, 2026, per Kelley Drye and Nixon Peabody. It changes consent revocation and takes effect 30 days after Federal Register publication, which had not happened as of October 7, 2026. For prospects, see Real Estate Speed-to-Lead AI: TCPA Checks Before You Dial.

The scoring rubric

This is the part that turns property management AI phone accuracy into numbers you can compare week to week.

Score every sampled call on every check it exercises. Mark each pass, fail or not applicable. Do not average across checks. Report per check.

Each check then gets a color from its error interval, not its raw rate.

Targets are yours to set. A starting point, which is an assumption and not a standard: gate 1% error, core 5%, watch 10%. One confirmed gate failure with real impact is a release blocker, whatever the interval says. A missed gas-leak call is not a statistic.

Worked math: how many calls is enough

The interval is a confidence interval, not a verdict. Sixty calls with zero errors do not prove a zero rate. The 95% Wilson upper bound is about 6.0%, and 11.4% at 30 calls. This is an illustrative calculation with the standard Wilson score interval.

Most checks do not apply to every call. If 24 of 100 calls are emergencies, check 4 has 24 observations. Zero failures on 24 calls still leaves an interval up to 13.8%, so the check is amber against a 1% target. The honest answer is "unknown," not "safe."

Plan the sample by check, with at least 30 observations on each one you care about. Over-sample rare, high-risk intents with synthetic calls.

Here is the scoring helper. It is simplified and runs as written.

from math import sqrt

def wilson(k, n, z=1.96):
    """95% Wilson interval for k errors in n observations."""
    p = k / n
    d = 1 + z * z / n
    c = (p + z * z / (2 * n)) / d
    h = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return max(0.0, c - h), min(1.0, c + h)

def color(k, n, target):
    lo, hi = wilson(k, n)
    if lo > target:
        return "red"
    if hi < target:
        return "green"
    return "amber"

print(color(9, 41, 0.05))   # red
print(color(0, 24, 0.01))   # amber: need more calls

How to run a one-week accuracy audit on your live calls

This plan assumes you record calls and can export records from the property system. Time figures are planning assumptions, not benchmarks.

1. Define truth for each check (Monday). For each of the 12 checks, write the source of truth from the table above. Write the target error rate and tier. If a check has no source of truth, add it to a gap list.

2. Pull and stratify the sample (Monday). Export last week's calls with their system records. Group them by intent: maintenance, emergency, leasing, ledger question, policy question, vendor or other, transfer. Pick at least 60 calls, with every intent represented and every property type you serve.

3. Add the hard cases (Tuesday). Include noisy calls, accents, interruptions, units with letters, and callers who are not the leaseholder. Tag them to report separately.

4. Label from the audio (Tuesday and Wednesday). A person listens to each recording, not just the transcript. They write what the caller said per field and mark each check pass, fail or N/A. Budget about eight minutes per call. Have a second person re-label 10 calls.

5. Diff against the record (Wednesday). For each call, compare the labelled truth to the stored fields. This is check 12. Record the first point of failure: heard, decided or written.

6. Score and color (Thursday). Count errors per check, compute the interval, and assign green, amber or red. Amber means unknown, and that is useful.

7. Turn failures into tests (Friday). Every confirmed failure becomes a replayable scenario: words, accent, noise and expected record. Replay each at least three times.

8. Set the next trigger (Friday). Re-run the scorecard after any prompt, model, speech or customer-config change. Spot-check 30 calls weekly.

Evalgent generates test scenarios from your agent's prompt and flows. Its call simulator reaches the agent over phone, SIP or WebRTC with caller profiles for accent, pace, interruption and background noise. It judges from the audio with evidence and human review, including custom metrics such as unit match. Evalgent works with teams to set up evals for a first agent; it is sales-led, not self-serve.

For live traffic, you push call recordings. They are transcribed, mapped to scenarios and judged, with flags for PII leakage, tool-call failures and policy failures. See also How to Monitor Voice Agents in Production.

Reading the results: three patterns

The diff table earlier shows the three patterns.

Heard wrong. Speech recognition produced the wrong entity and the agent trusted it. Add a read-back and rule-based validation, such as the unit check above.

Heard right, decided wrong. The transcript is correct but the priority or statement is wrong. Test the prompt, the emergency list and the customer's config first. A stale list looks like an AI error.

Decided right, written wrong. The agent said the right thing, but the record disagrees: a dropped note, a duplicate contact. This is a tool-call or integration bug.

Sorting failures this way shows who owns the fix.

Where to go deeper

Use this map after your first scorecard.

Frequently asked questions

How accurate is an AI phone agent for property management?

There is no single number. Accuracy varies by check, property and caller. Unit match, emergency triage and write-back usually decide trust. Measure each of the 12 checks against the property system record, then report an interval. With 60 scored calls and zero errors, the true error rate could still be near 6%.

How do I audit my property management AI phone agent?

Name a source of truth for each of the 12 checks. Sample at least 60 live calls across intents. Label each from the audio, diff it against the stored record, and score each check with an error interval. Turn each failure into a replayable test, and repeat after every change.

How many calls should I review to audit an AI answering service?

Review at least 60 calls overall and at least 30 observations per check you care about. Fewer leaves wide intervals. Zero errors in 30 observations still allows a true rate near 11%. Sample by intent so each check appears, and add synthetic emergency calls.

What mistakes does AI make most on maintenance calls?

Expect wrong unit numbers, a routine priority on an urgent leak, missing permission-to-enter or pet notes, and duplicate resident contacts. The quiet ones sit in the record, not the call, so diff the work order against the audio.

Can an AI phone agent handle emergency maintenance calls safely?

It can, if it follows a written emergency list and your tests prove it. Replay each emergency in callers' own words, with noise and interruptions. Treat a failed triage as a release blocker. Fire and gas calls should get 911 guidance first, per the customer's policy.

Does AI mishear unit numbers and resident names?

Yes, especially on 8 kHz phone audio with noise. One study of insurer phone calls found first-stage error rates near 10% to 13% on short alphanumeric fields. Validate every captured unit against the property directory and the lease. Read back unit, name and callback number before writing the work order.

Can a property management AI quote rent, fees and balances?

It can if it reads live listing and ledger data and policy allows it. Treat money statements as a gate check. The risks are stale data, a confident guess when data is missing, and promises about late fees. Compare each spoken figure to the system of record.

Is a property management AI leasing agent subject to fair housing rules?

Fair housing rules apply to statements about renting a dwelling, including spoken ones, and to refusing reasonable accommodations. See 42 U.S.C. 3604 and 24 CFR 100.204. Test for steering language and for recognizing oral accommodation requests. This is not legal advice; ask counsel how the rules apply to you.

The bottom line

Property management AI phone accuracy is a set of per-check error rates measured against the property record, with honest intervals. Run the 12-check audit on 60 or more live calls this week, and turn every failure into a test that must pass before the next release.