Evalgent
Back to Blog
Voice AI Testing

Hotel AI Reservation Errors: Rate Codes, Dates, Room Types

Deepesh Jayal
15 min read
Hotel AI Reservation Errors: Rate Codes, Dates, Room Types
On this page

This post is for the product team behind hotel guest-communication software. Your voice agent books rooms for dozens or hundreds of hotels. The category is crowded with builders. Canary Technologies launched AI Voice with a central reservations assistant in February 2025. Akia launched AI Voice in August 2026, pushing bookings back into the PMS. Each prompt change reaches every property at once.

Our hotel voice AI QA guide maps the whole trade. This post goes deeper on one call type, the new reservation. It covers the eight hotel AI reservation errors that matter most, why each happens, a per-property test matrix, a Python checker and a monitoring protocol.

35.2%
GPT-4o pass^1 on airline booking tasks with policy rules (tau-bench, 2024)
39%
Average LLM performance drop in multi-turn vs single-turn tasks (Laban et al., 2025)
$113
FY 2027 federal standard lodging per diem, from Oct 1, 2026 (GSA)
May 12, 2025
FTC total-price rule for short-term lodging took effect (16 CFR 464)

The reservation is a contract with ten fields

Reservation contract: the set of fields a booking call must write correctly for the stay, the bill and the guest's rights to match what was agreed on the phone.

A reservation call ends in a write. The agent checks availability, books, then reads a confirmation number back. The property trusts only the record.

That record has more structure than the conversation suggests. Take Apaleo's documented booking request example. It books `arrival` 2026-03-15 and `departure` 2026-03-17. It carries two `timeSlices`, one per night. Each slice holds `ratePlanId` "MUC-NONREF-SGL". That one code encodes the property, a non-refundable policy and a single room. A wrong rate plan ID is not a pricing typo. It changes the refund rights and the room in one field.

FieldWhat the caller saysWhat gets storedSilent failure
Arrival"the 30th"2026-10-30Wrong month or year
Departure"through the 2nd"2026-11-02Night added or lost
Nightsrarely said3, or one slice per nightDisagrees with dates
Room type"a king"KNG, or a run-of-house codeNearest-sounding code
Occupancy"me, my wife and two kids"adults 2, children 2, agesOver max occupancy
Rate code"the AAA rate"rate plan IDIneligible or wrong plan
Policy and guaranteerarely saidcancellation rule, depositNon-refundable, never disclosed
Total price"how much is it?"quoted total with feesResort fee left out
Loyalty and requests"high floor, number is…"profile ID, request notesRequest booked as a guarantee
Confirmation"what's my number?"PMS or CRS confirmationReadback differs from record

Guests rarely say nights, policy or guarantee type. Those come from the rate the agent chose. So the transcript can be perfect while the stored rate plan is wrong.

Grid of eight hotel AI reservation errors sorted by where each one breaks: hearing, reasoning, policy and write-back, with the record field each one corrupts

Why reservation agents fail in ways that sound fine

Three findings from tool-use research explain the pattern.

First, policy-bound booking is hard even for strong models. In tau-bench (Yao et al., 2024), GPT-4o solved 35.2% of airline tasks on the first try. Those tasks follow rules such as "Basic economy flights cannot be modified." Removing the policy text cut airline success from 33.2% to 10.8% in the authors' ablation. Rate eligibility and cancellation rules are the hotel version of that policy text. Retail consistency also fell, with pass^8 under 25%. The same booking can pass once and fail on a rerun.

Second, the arguments are where tool calls break. ComplexFuncBench (2025) tested multi-step function calls across hotel, flight and car rental APIs. The best model, Claude 3.5 Sonnet, reached 61.0% overall success. For Qwen2.5-72B, parameter value errors made up 78.8% of failures. The function was right. The value inside it was wrong.

Third, changes mid-call hurt. LLMs Get Lost in Multi-Turn Conversation (Laban et al., 2025) found an average 39% drop when task details arrive across turns. Guests do exactly that. "Actually, make it a king. And can we add Saturday?" Each change is a chance to keep a stale field.

None of these studies used phone calls or hotel data, and the models are older. Re-measure on yours. But all three point at the value written, not the words heard. Our post on tool-argument accuracy from speech covers the general scoring method.

Eight hotel AI reservation errors and their mechanisms

1. Off-by-one night

The pillar guide covers "the 3rd through the 5th." The less obvious trap is a disagreement between two representations. Some booking tools accept both a departure date and a night count. If the model fills both, one of them can be wrong. Which one wins depends on the tool, not the guest.

Apaleo's example shows the safer shape. Dates define the stay, and one time slice exists per night. Your tool layer should compute nights from dates and reject any request where the two disagree. Never let the model do the subtraction.

2. Dates across a month or year boundary

"The 30th through the 2nd" is three nights: Friday, October 30 to Monday, November 2, 2026. An agent that assumes one month can store a departure of October 2. That date is before the arrival. A good API rejects it, and the agent then improvises. A worse one guesses December 2 and books 33 nights.

Two more boundaries matter. "December 30 for three nights" checks out on January 2, 2027, and the year must roll. "February 27, 2028, three nights" checks out on March 1, because 2028 is a leap year. An agent that assumes 28 days in February stores March 2 and adds a fourth night.

Calendar strips showing a correct three-night stay from October 30 to November 2, a wrong departure of October 2 before arrival, and a leap-year stay ending March 1, 2028

3. Wrong room type

Guests describe beds, not codes. "A double" may mean a double bed, two beds or two people. "Two queens" might map to QQ, DQ or "Double Queen." When the PMS offers a near match, the agent often books it and reads back the guest's words. The hero image above shows the result: the guest asked for a king, and the record says Double Queen.

An accessible room raises the stakes. The ADA rule at 28 CFR 36.302(e) covers reservations "by telephone." Properties must let guests with disabilities book accessible rooms the same way as others. They must describe accessible features in enough detail. On request, they must reserve the room and remove it from all reservation systems. An agent that books "accessible king" as a standard king, or drops "roll-in shower," fails that duty.

4. Occupancy over the room's limit

"Three adults and a baby in a king" is a real request. Each room type has a maximum occupancy and often an extra-person charge, and child ages can change both. The agent must capture adults and children separately and check the room's limit. If it books the room anyway, the guest learns at the desk. If it adds an extra-person charge without saying so, the quoted total was wrong.

5. Wrong or ineligible rate code

Rate codes carry eligibility rules. The model wants to help, so it applies discounts the guest has no right to.

  • AAA and AARP. AAA's own hotel page says a valid membership card "is typically required at check-in." The Choice Hotels terms there read: "Guest must show proof of valid AAA Membership at check-in." An agent that applies AAA because a caller said "do you have discounts?" sets up a rate change at the desk.
  • Government per diem. GSA's FY 2027 notice sets the standard lodging rate at $113 per night from October 1, 2026. Many localities have higher rates. A government rate is a property's own plan, often requiring ID. The agent must not invent the cap.
  • Corporate codes. "The Acme rate" must match a code the property actually loaded. A fuzzy match to "ACM" or a guess creates a rate the company never negotiated.
  • Packages. A package rate bundles breakfast or parking. Booking it changes the total and what the guest believes is included.
  • BAR. The best available rate is the safe default. Fall back to it when eligibility is unclear, and say so.

The fix is a property-level eligibility table that the booking tool enforces, not only the prompt. Each restricted code needs a condition the agent must capture or disclose first.

6. Non-refundable rate booked without disclosure

Non-refundable rates are often the cheapest, so a price-sorting agent offers them first. The guest hears "$159 a night" and says yes. The terms live in the rate plan, and the PMS stores them silently.

Test the order of events, not just their presence. The cancellation policy and any deposit must be spoken before the guest's "yes." A disclosure after the confirmation number is a disclosure after commitment. Our vacation rental direct booking post applies the same test to house rules and minimum stays.

7. Resort and mandatory fees not disclosed

The FTC's Rule on Unfair or Deceptive Fees covers short-term lodging from May 12, 2025. The pillar post covers its audible-disclosure standard. One point from the FTC's business FAQ matters most for reservation calls. Its example is a $199 nightly rate with a required $39 daily resort fee. The fee belongs in the total price. The FAQ also says that if a guest picks a feature that raises the rate, the total must be updated.

On a call, that happens constantly. The guest switches to a king, adds a night or drops the AAA rate. Each change needs a new total with mandatory fees. Agents often re-quote only the nightly rate. Assert a fresh total after every change. Neither this section nor the ADA note above is legal advice; have counsel confirm how each rule applies to your properties.

8. Double booking

Duplicates have two causes. A change call creates a new booking instead of modifying the old one. Or the booking tool times out, the agent retries, and both requests land.

Apaleo's channel booking guide says `externalId` "should contain the provider confirmation ID." Use your own call-scoped ID the same way. Search for it before any retry. Then test it by forcing timeouts in staging. Our guide to detecting silent tool failures covers fault injection.

Card capture and loyalty: the two side fields

Guarantees often need a card. PCI DSS v4.0.1 is current; v4.0 retired on December 31, 2024, and future-dated requirements took effect March 31, 2025. Requirement 3.3.1 bars storing sensitive authentication data after authorization. The PCI Council's FAQ 1210 applies that to call audio. Card security codes in recordings after authorization violate it, even if encrypted. A voice agent copies spoken digits into recordings, transcripts, LLM context and logs. Use masked keypad entry or a texted payment link instead. Test leaks with the method in our PII leakage detection post. Your QSA decides scope.

Loyalty numbers and special requests fail quietly. A misheard member number credits the wrong account. A request for a high floor gets read back as "I've guaranteed a high floor." Requests are notes, not guarantees, unless the property's config says otherwise. Our entity accuracy guide covers alphanumeric capture.

The per-property rate-and-room matrix

Every property sells a different mix. That makes a single global test suite misleading. Build a matrix per property, or per archetype of similar properties.

Start with four factors:

FactorExample levelsWhat to assert
Rate codeBAR, AAA, AARP, government, corporate, package, non-refundable, advance purchaseEligibility captured or disclosed; policy spoken before "yes"
Room typeKing, double queen, suite, accessible king, accessible roll-in, run of houseExact code; accessible features described and held
Occupancy1 adult, 2 adults + 2 children, 3 adultsWithin max; extra-person charge in total
Date shapeSame month, month boundary, year boundary, leap dayNights equal date difference; departure after arrival

The worked math

This is an illustrative worked example, not customer data.

Say one property sells 8 rate codes and 6 room types. Add 4 occupancy levels and 3 date shapes. The full matrix is 8 × 6 × 4 × 3 = 576 bookings. That is too many per property per release.

Pairwise coverage tests every pair of factor levels at least once. The floor is the product of the two largest factors: 8 × 6 = 48 bookings. Generators usually land a little above that. So about 50 synthetic bookings cover every rate-and-room, rate-and-date and room-and-occupancy pair for one property.

Add negative cases: an ineligible AAA claim, a fake corporate code, a room over capacity, and mid-call room and rate switches. Group properties by config so you test archetypes, not 300 copies. Every config edit to one property should re-run that property's matrix. A typo in a fee table is a release. Our guide to shipping prompt changes safely covers gating.

A checker for policy-bound reservations

The pillar post's validator checks dates, names and fee totals. The sketch below covers the rest: eligibility, disclosure order, occupancy, nights and duplicates. It is simplified and uses only the standard library.

# Simplified: policy checks on a booking written by a hotel voice agent.
from datetime import date

RESTRICTED = {  # per-property config: code -> condition the call must satisfy
    "AAA": "member_id_or_id_notice",
    "GOV": "id_notice",
    "NONREF": "policy_before_consent",
}

def check(rec, turns, room_max, existing):
    """rec: PMS record. turns: [(speaker, text, kind)]. existing: other PMS records."""
    errs = []
    arr, dep = date.fromisoformat(rec["arrival"]), date.fromisoformat(rec["departure"])
    nights = (dep - arr).days                     # month, year and leap-day safe
    if nights <= 0:
        errs.append("departure not after arrival")
    if len(rec.get("time_slices", [])) not in (0, nights):
        errs.append("nights and time slices disagree")
    if rec["adults"] + rec.get("children", 0) > room_max[rec["room_type"]]:
        errs.append("occupancy over room maximum")

    kinds = [k for _, _, k in turns]
    consent = kinds.index("consent") if "consent" in kinds else None
    for tag in rec["rate_tags"]:                  # e.g. ["AAA"] or ["NONREF"]
        rule = RESTRICTED.get(tag)
        if rule == "policy_before_consent":
            spoken = "policy" in kinds and consent is not None and kinds.index("policy") < consent
            if not spoken:
                errs.append("non-refundable terms not spoken before consent")
        elif rule and not (rec.get("member_id") or "id_notice" in kinds):
            errs.append(f"{tag} rate without eligibility evidence")

    for other in existing:                        # duplicate guard
        if (other["last_name"].casefold() == rec["last_name"].casefold()
                and other["arrival"] == rec["arrival"] and other["id"] != rec["id"]):
            errs.append(f"possible duplicate of {other['id']}")
    return errs

The `turns` list needs a label per agent turn: policy, consent, fee total or ID notice. An audio-aware judge can produce those labels. A keyword match on the transcript is a weak stand-in. A disclosure that a barge-in cut off appears in the transcript but was never heard. Our post on transcript versus audio evaluation explains that gap.

How to test hotel AI reservations before every release

1. Export each property's rate and room config. Pull rate codes, eligibility rules, room types, maximum occupancy, mandatory fees and cancellation policies. These are your expected values.

2. Write the eligibility table. For each restricted code, record the condition: membership number, ID at check-in, or a loaded corporate code. Enforce it in the booking tool, not only the prompt.

3. Generate the pairwise matrix. Cover rate × room × occupancy × date shape per property archetype. Add the negative cases and mid-call changes. Evalgent generates scenarios from an agent's prompt and flows, including policies, hard constraints and coverage gaps. That fills in the branches one engineer tends to miss.

4. Pin the call date. Run the date shapes with a fixed "today" in the property's time zone. Include a month boundary, a year boundary and February 2028.

5. Vary the callers. Evalgent's caller profiles vary accent, pace, interruption level, background noise and language. Its simulator calls over PSTN, SIP or WebRTC, so audio takes the same path as a guest's.

6. Inject tool faults. Force booking timeouts and slow availability calls in staging. Assert one reservation per call afterwards.

7. Score the record and the order of speech. Run the checker on every written booking. Confirm the policy and fee total came before consent, and that a fresh total followed every change.

8. Gate the release per archetype. Block it on any duplicate, any ineligible rate, any undisclosed non-refundable booking or any PCI leak. Use our go-live checklist when onboarding a new hotel.

Monitor the confirmation against the PMS record

Pre-release tests cover the cases you thought of. Production brings everything else. The most useful signal is a simple diff. Compare what the agent confirmed out loud with what the PMS holds.

Reconciliation scorecard comparing a spoken confirmation with the PMS record field by field, flagging mismatched room type and rate code, with weekly alert thresholds per property

Run it as a protocol:

  • Extract the spoken confirmation. Pull dates, nights, room, rate, total and confirmation number from the agent's final readback.
  • Fetch the record by confirmation number. Read it from the PMS or CRS after write-back settles.
  • Diff field by field. Count a booking as mismatched if any contract field differs. Log which field.
  • Alert per property. Rates differ by property, so thresholds should too.

Here is the alert math as an illustrative example. A property takes 300 AI bookings a week with a 1% baseline mismatch rate. You expect 3 mismatches. Under a Poisson model, 8 or more happens by chance about 1.2% of the time. Nine or more happens about 0.4% of the time. An alert at 9 fires rarely by accident and still catches a tripled rate within a week.

Also watch duplicates per caller, rate code mix and booking tool errors. A jump in AAA bookings after a prompt change is a policy regression until proven otherwise. Evalgent's production monitoring takes pushed call recordings, maps them to scenarios and judges them. It flags tool-call failures, PII leakage, latency spikes and policy failures with fix recommendations. Whatever you use, score every call rather than a sample. A per-property error hides inside a global average.

Frequently asked questions

What are the most common hotel AI reservation errors?

The most common hotel AI reservation errors are an extra or missing night, dates across a month boundary, the wrong room type, an ineligible rate code, an undisclosed non-refundable rate or resort fee, and duplicate bookings. Most sound fine on the call. They show up only when the PMS record is compared with what the guest agreed to.

How do you stop a hotel AI from booking the wrong number of nights?

To stop a hotel AI booking the wrong number of nights, compute nights in code from arrival and check-out dates. Never let the model subtract. Reject requests where nights and dates disagree. Read back weekday names and the night count. Test month boundaries, year boundaries and leap days with a pinned call date.

Can a hotel AI apply an AAA or government rate?

A hotel AI can apply an AAA or government rate only when the property's eligibility rule is met or disclosed. AAA says a valid membership card is typically required at check-in. Government rates usually need ID. Enforce each condition in the booking tool. Fall back to the best available rate when eligibility is unclear.

Does a hotel AI have to disclose a non-refundable rate?

A hotel AI should disclose a non-refundable rate's cancellation terms and any deposit before the guest agrees to book. The policy lives in the rate plan, so the PMS stores it silently. Test the order of speech: the terms must come before consent, not after the confirmation number. Check local consumer rules with counsel.

Does a hotel AI have to quote resort fees?

A hotel AI quoting prices to US guests falls under the FTC fee rule for short-term lodging, effective May 12, 2025. Mandatory fees such as resort fees belong in the total price. The FTC's FAQ says the total must update when a guest picks a feature that raises the rate, such as a room upgrade.

How do you prevent duplicate reservations from a voice agent?

To prevent duplicate reservations from a voice agent, give each call a unique ID and store it on the booking as an external reference. Search for it before any retry. Route change requests to a modify tool, not a create tool. In staging, force booking timeouts and assert that each call leaves exactly one reservation.

Is it PCI compliant for a voice agent to hear card numbers?

A voice agent that hears card numbers brings recordings, transcripts and logs into PCI DSS scope. Requirement 3.3.1 bars storing card security codes after authorization. PCI Council FAQ 1210 applies that to audio recordings. Safer designs use masked keypad entry or a texted payment link. Your QSA decides the final scope.

How do you check AI bookings against the PMS?

To check AI bookings against the PMS, extract the agent's spoken confirmation and fetch the record by confirmation number. Compare dates, nights, room type, rate code, occupancy and total field by field. Track the mismatch rate per property, with alert thresholds sized to each property's weekly booking volume.

The bottom line

Hotel AI reservation errors live in the record, not the conversation. Test every rate, room and date shape per property before release, then diff each spoken confirmation against the PMS after it.

Related Articles