Are Your AI Leasing Follow-Ups Converting? How to Measure

This post is for the product team behind a leasing CRM or leasing contact-center platform. Your outbound agent calls and texts prospects after an inquiry or a tour. It re-engages them, answers questions, books a tour or helps them rebook one, nudges applications and logs objections. It writes all of that back to the guest card. It runs across hundreds of properties. A few engineers own it. Nobody owns measurement.

The category is shipping now. In September 2026, ShowMojo described an outbound AI voice agent built for speed to lead. It calls and texts renters about five minutes after an inquiry with no tour booked. It answers property questions, books showings and syncs outcomes back. Anyone Home describes a leasing assistant that can "automatically set up follow-up messages" and take tour bookings.

So the question your customers ask is fair. Are leasing follow-up AI calls converting? Most dashboards can't answer it. They count what the agent said happened. They mix voicemails with conversations. They credit the AI for tours that would have happened anyway. This post shows how to measure it properly.

What a leasing follow-up call has to do, and write

A follow-up call is a small agentic workflow. The voice is only the front door. A trigger fires from the leasing CRM. The dialer places the call. Answering machine detection decides human or machine. The agent talks or leaves a message. Then it acts: book a tour, send an application link, record an objection, or honor an opt-out. Finally it writes the result to the guest card.

Follow-up outcome record: the fields one attempt writes to the leasing CRM. Answered by (human, machine, unknown), outcome, tour ID, application link sent, objection code, opt-out flag, property ID and units mentioned.

Every metric you report is computed from that record. If the record is wrong, the metric is wrong, and nobody hears it.

Our property management AI answering playbook covers inbound resident and prospect calls. This post covers the outbound side, where the agent starts the call and the metric is conversion.

The funnel, defined by its denominators

Conversion rates fight over denominators. Write them down once, per tenant, and freeze them.

Two rules make the table work. First, the unit is the lead, not the call. One prospect may get four attempts. Second, outcomes come from the system of record. A transcript that says "You're all set for Saturday at 11" is a claim. The calendar entry is the fact.

Apartment List's guide to measuring AI leasing performance makes a related point: a conversion rate and a conversion lift are different claims. A rate describes the funnel. A lift needs a comparison group. Most vendor numbers in this category are rates or before-and-after comparisons.

How voicemail detection errors bend the numbers

Most follow-up attempts reach voicemail. That makes answering machine detection, or AMD, the first classifier in your funnel. Its errors move every rate below it.

Twilio's AMD documentation shows the shape of the problem. With `MachineDetection=Enable`, `AnsweredBy` returns `human`, `machine_start`, `fax` or `unknown`. `DetectMessageEnd` waits for the greeting to end and returns `machine_end_beep`, `machine_end_silence` or `machine_end_other`. The default `MachineDetectionTimeout` is 30 seconds, and a timeout returns `unknown`. Twilio states plainly that "it's possible that AMD will not always return the right answer." It publishes no accuracy figure.

Research numbers give a rough sense of scale. Altwlkany and colleagues trained a YAMNet plus GRU detector on about 4,200 Infobip call clips. It reached 96.67% test accuracy, and 98.10% with silence detection. The test set was balanced and clean. Your traffic is neither, so treat that as a ceiling, not a spec.

Two errors, two different biases

There are two ways to be wrong, and they push metrics in different directions.

• False human. A voicemail greeting is classed as a person. The agent pitches a tour to a recording. Your logs show a "conversation" that never converts. Conversation-to-tour drops.
• False machine. A real prospect is classed as a machine. The agent waits for a beep, then leaves a message to a live person. Many hang up. You lose a real conversation, and contact rate drops.

Then there is the logging bug that does the most damage. Some flows write `contacted` on any completed attempt, voicemail included. Contact rate jumps toward 100%. Worse, a human leasing agent sees "contacted" on the guest card and skips the call back.

Worked math: 1,000 leads

These are illustrative assumptions, not benchmarks. Take 1,000 eligible leads. Say 350 answer as humans and 650 reach voicemail. AMD is wrong 4% of the time in each direction. Real conversations convert to tours at 25%.

• False humans: 650 × 0.04 = 26 voicemails logged as conversations.
• False machines: 350 × 0.04 = 14 real prospects treated as voicemail.
• Conversations logged: 336 + 26 = 362.
• Tours booked: 336 × 0.25 = 84.
• Measured conversation-to-tour: 84 ÷ 362 = 23.2%, against a true 25%.
• Tours lost to false machines: 14 × 0.25 = 3.5.

Now add the `contacted`-on-voicemail bug. The CRM shows 362 + 624 = 986 contacts, a 98.6% contact rate. The true human contact rate was 35%. Each of those 624 voicemails now looks "handled."

The fix is a labeled sample. Pull a few hundred attempts per release, stratified by carrier and time of day. Have someone listen to the first ten seconds and label human, machine or unknown. Compare against `AnsweredBy`. Our deep dive on answering machine detection in LiveKit and Pipecat covers tuning. The outbound voice agent testing plan covers the rest of the dialer.

CRM write-back failures: the silent second error

The second error happens after the call goes well. The agent books the tour. The tool call to the calendar times out. The agent's prompt says "confirm the booking," so it confirms anyway. The prospect hears Saturday at 11. The calendar has nothing.

That breaks measurement in both directions.

• Count from transcripts, and you overcount tours that never existed.
• Count from the CRM, and you undercount tours that exist on the calendar but never reached the guest card.
• Retries can create duplicate tours, which inflates tours and then shows up as no-shows.

Tool reliability is not a solved problem. In τ-bench, even strong function-calling agents completed under 50% of tasks. Their pass^8 consistency in the retail domain was under 25%. A leasing follow-up is simpler than those tasks. It still has the same failure shape: the agent says one thing and the system records another. Our guide to tool call accuracy breaks down how to score it.

Reconcile three records per lead: the call log, the property calendar and the CRM. Here is a simplified daily audit.

# Simplified: reconcile follow-up attempts against calendar and CRM.
from collections import Counter

def classify(attempt, calendar, crm):
    lead = attempt["lead_id"]
    card = crm.get(lead, {})
    on_calendar = calendar.get(lead, [])           # tour slots for this lead
    heard = attempt["answered_by_label"]           # human-labeled, not AMD
    if heard != "human" and card.get("status") == "contacted":
        return "voicemail_logged_as_contact"
    if attempt["agent_claimed_tour"] and not on_calendar:
        return "claimed_tour_missing"              # tool failed, agent confirmed anyway
    if on_calendar and card.get("status") != "tour_scheduled":
        return "tour_not_in_crm"                   # write-back failed
    if len(on_calendar) > 1:
        return "duplicate_tour"                    # retry without idempotency key
    return "consistent"

def audit(attempts, calendar, crm):
    return Counter(classify(a, calendar, crm) for a in attempts)

# Example output for one tenant-day:
# Counter({'consistent': 412, 'voicemail_logged_as_contact': 9,
#          'tour_not_in_crm': 3, 'claimed_tour_missing': 2, 'duplicate_tour': 1})

The example output is illustrative. The labels are the point. Each one maps to a different bug, and each one moves a different rate.

Attribution: did the AI call cause the tour?

A prospect who inquired about a two-bedroom may tour whether or not the AI calls. Before-and-after comparisons can't separate the call's effect from season, rent changes, concessions or a new listing site. Spring leads convert differently from November leads.

Holdout group: a random share of eligible leads that gets business-as-usual follow-up instead of the AI. This control group shows the AI's incremental effect.

Randomize at the lead level, at the moment the lead becomes eligible. Hash the lead ID so the assignment is stable across retries. Then analyze by intent-to-treat. Every lead assigned to the AI arm counts, even if the dialer never reached them. Dropping unreached leads makes the AI arm look better, because reachable people convert more.

Why the samples get big

Conversion effects are small next to the noise. Lewis and Rao studied 25 large advertising field experiments with major US retailers and brokerages. The median confidence interval on ROI was over 100 percentage points wide. Informative tests could need more than 10 million person-weeks. Leasing funnels are less noisy than retail ads, but the lesson holds. Lift is easy to claim and hard to prove.

Here is the math for a lead-to-tour test. These are illustrative inputs. Baseline is 20%, and you want to detect a lift to 23%. Use a two-sided alpha of 0.05 and 80% power.

from math import sqrt
from statistics import NormalDist

def n_per_arm(p1, p2, alpha=0.05, power=0.80):
    za = NormalDist().inv_cdf(1 - alpha / 2)
    zb = NormalDist().inv_cdf(power)
    pbar = (p1 + p2) / 2
    num = (za * sqrt(2 * pbar * (1 - pbar))
           + zb * sqrt(p1 * (1 - p1) + p2 * (1 - p2))) ** 2
    return num / (p1 - p2) ** 2

print(round(n_per_arm(0.20, 0.23)))  # 2943 leads per arm
print(round(n_per_arm(0.20, 0.25)))  # 1094 leads per arm

A 50/50 split needs about 5,900 leads. A 90/10 split, which keeps most leads on the AI, needs about 16,350 leads in total. One property with 80 leads a month will never get there. Pool across properties, and stratify the randomization by property so each one contributes to both arms. This is where a multi-tenant platform has an edge over a single operator.

The tour-to-application rate is harder still. The denominator is completed tours, which is a fraction of leads. Measure lead-to-tour conversion as the primary metric. Treat applications as a secondary readout.

Variance reduction helps. CUPED, from Deng, Xu, Kohavi and Walker, uses pre-experiment data as a covariate. It cut variance by about 50% at Bing. In leasing, lead source, bedroom count and days since inquiry are natural covariates.

Our A/B testing guide for voice agent prompts covers live splits in more depth. The shift confirmation measurement post uses the same pattern: define the label, then audit the label before trusting the rate.

Quality metrics that sit beside conversion

A follow-up agent can lift tours while doing damage. Track these next to conversion on every release.

Wrong-property and wrong-unit mentions

On a multi-tenant platform, one prompt template serves hundreds of properties. A config bug can make the agent at Property A quote Property B's special. Score every call for property names, unit numbers, prices and specials. Check each against that lead's property record. Our multi-tenant testing guide shows how to catch config drift.

Stale availability

The agent says "Unit 2B is still open." Unit 2B was leased yesterday. Compare every unit the agent quoted against a unit availability snapshot taken at the call timestamp. The reverse error matters too. Saying a unit is gone when it isn't loses a lead. Under 42 U.S.C. § 3604(d), telling someone a dwelling is "not available for inspection, sale, or rental when such dwelling is in fact so available" is unlawful when done because of a protected trait. Availability accuracy is a compliance metric, not only a conversion one.

Fair Housing consistency

Follow-up is where differential treatment hides. Does every prospect get the same number of attempts? The same specials? The same units offered? HUD's 2012 paired-testing study found Black renters were told about 11.4% fewer available units than equally qualified white renters. Hispanic renters were told about 12.5% fewer. That is the pattern to test your agent against.

Don't infer protected traits from production data. Run paired synthetic tests instead. Hold the script constant and vary the caller's name, voice, accent or language. Then compare cadence, units offered, specials and tone. The Fair Housing guide for AI leasing agents covers the rules and the scripts in detail.

TCPA consent and opt-outs

A leasing follow-up that encourages someone to tour or apply is likely marketing. That matters for consent. The FCC ruled in FCC 24-17 (February 2024) that AI-generated voices count as "artificial or prerecorded voice" under the TCPA. Under 47 CFR 64.1200, telemarketing calls with an artificial voice to a wireless number need prior express written consent. Solicitation calls are barred before 8 a.m. or after 9 p.m. in the called party's local time. Revocations made "in any reasonable manner" must be honored within 10 business days.

The rules are moving. As of October 2026, the FCC's Report and Order FCC 26-67 lets callers scope an opt-out to one category of informational calls. It keeps "revoke all" for telemarketing, and it lets callers name an exclusive opt-out method if clearly disclosed. It takes effect 30 days after Federal Register publication, which had not occurred when Squire Patton Boggs reported on it. Separately, the Eleventh Circuit vacated the one-to-one consent rule in January 2025. Our speed-to-lead TCPA checklist has the pre-dial checks.

For measurement, three numbers matter. Track opt-out rate per 1,000 attempts. Track the share of spoken opt-outs ("stop calling me") that reached the consent ledger. Track calls placed outside quiet hours. This is not legal advice. Have counsel review your consent flows.

How to measure AI leasing follow-up conversion

Use this sequence for every tenant rollout of leasing follow-up AI calls and every prompt or flow change.

1. Freeze the definitions. Write numerator, denominator and source of truth for each rate in the table above. Version them. A changed definition is a new metric.

2. Give every attempt an ID. Carry one attempt ID from the CRM trigger through the dialer, the call, the tool calls and the write-back. Use it as the idempotency key on tour creation.

3. Label a voicemail sample. Each release, label a few hundred attempts by ear. Report false-human and false-machine rates separately. Fix any flow that writes `contacted` on a machine.

4. Reconcile daily. Run the three-way join of call log, calendar and CRM. Alert on `claimed_tour_missing` and `tour_not_in_crm` above zero.

5. Randomize a holdout. Hash lead IDs into arms at eligibility, stratified by property. Keep the holdout on business-as-usual follow-up. Analyze by intent-to-treat.

6. Test before the change ships. Run scripted prospects against the new flow: rebooks, price objections, "is 2B still open," opt-outs mid-call, wrong-number answers and voicemail greetings. Evalgent generates these scenarios from the agent's prompt and flows. It calls the agent over PSTN, SIP or WebRTC with caller profiles that vary accent, pace, interruptions and background noise.

7. Run paired Fair Housing tests. Same scenario, varied caller profile. Compare offers and cadence across the pair.

8. Monitor production. Score every call against the guardrails, not a sample. Evalgent's production monitoring maps pushed recordings to scenarios and judges them from the audio. It flags tool-call failures and policy failures with fix recommendations.

9. Read results with intervals. Report lift with a confidence interval. If the interval crosses zero, say so.

Our AI tour booking testing guide covers the booking step in depth: ID checks, wrong units and no-shows.

What a credible conversion claim looks like

Say your customer asks whether the AI is worth it. A credible answer follows this template:

"Across N properties from [start] to [end], 90% of eligible leads got AI follow-up and 10% got the existing process. Lead-to-tour was X% versus Y%, a lift of Z points (95% CI A to B). Voicemail labeling error was under 3%. CRM reconciliation mismatches were under 1%. No opt-out was missed, and no call was placed outside quiet hours. Paired tests showed no difference in offers."

Fill in your own numbers. The structure is what counts. It names the comparison, the interval and the guardrails. A claim missing any of those three is a rate, not a lift.

This is also how you prove quality to a property group's compliance team. They don't need your prompt. They need evidence that the agent treats prospects the same way, honors opt-outs and writes true records.

Frequently asked questions

Do AI follow-up calls increase apartment tours?

AI follow-up calls increase apartment tours only if a randomized holdout shows it. Before-and-after numbers mix the AI's effect with season, pricing and lead mix. Hold back a random share of leads on the existing process. Compare lead-to-tour by intent-to-treat, with tours counted from the property calendar. Report the lift with a confidence interval.

How do you measure AI leasing follow-up conversion?

Measure AI leasing follow-up conversion per lead, not per call. Define each rate's numerator, denominator and source of truth. Count tours from the calendar and applications from the application system. Label a voicemail sample, reconcile CRM write-backs daily, and compare against a randomized holdout group to get lift rather than a raw rate.

What is a holdout group for AI follow-up?

A holdout group for AI follow-up is a random share of eligible leads that gets your existing follow-up instead of the AI. Assign leads by hashing the lead ID at eligibility, stratified by property. The difference in tours and applications between the two arms is the AI's incremental effect.

How many leads do you need to measure conversion lift?

Detecting a lead-to-tour lift from 20% to 23% needs about 2,943 leads per arm at 95% confidence and 80% power. A 90/10 split needs about 16,350 leads in total. Single properties rarely reach that, so pool across properties and stratify randomization by property. These inputs are illustrative; plug in your own baseline.

How does voicemail detection affect conversion metrics?

Voicemail detection errors shift conversion metrics in both directions. False-human errors log greetings as conversations, which deflates conversation-to-tour. False-machine errors drop real prospects, which lowers contact rate. Logging voicemails as "contacted" inflates contact rate and stops human callbacks. Label a sample by ear each release and compare it with the detector's output.

Do leasing follow-up calls need TCPA consent?

Leasing follow-up calls that use an AI voice and encourage a tour or application likely need prior express written consent when placed to wireless numbers. The FCC treats AI voices as artificial voices. Calls must also respect 8 a.m. to 9 p.m. local time and honor revocations. This is not legal advice; check with counsel.

Can AI leasing follow-ups violate Fair Housing?

AI leasing follow-ups can violate Fair Housing rules if prospects get different cadence, units, specials or availability answers based on a protected trait. Falsely saying a unit is unavailable because of a protected trait is unlawful under 42 U.S.C. § 3604(d). Run paired tests that vary only the caller's name, voice or language.

What should an AI leasing follow-up write to the CRM?

An AI leasing follow-up should write who answered (human, machine or unknown), the outcome, any tour ID, whether an application link was delivered, an objection code, an opt-out flag, and the property and units mentioned. Every field should come from a confirmed system response, never from what the agent said.

The bottom line

Leasing follow-up AI calls are converting only when a randomized holdout, counted from the calendar and CRM, shows a lift with a confidence interval. Fix voicemail labeling and write-back reconciliation first, because both quietly distort every rate you report.