AI Pizza Phone Ordering: Why Half-and-Half Orders Break

A pizza order is not a list of items. It is a small tree. The order has pizzas. Each pizza has a size, crust, sauce, bake and cheese. Each portion of each pizza has its own toppings. A deal may then bind several pizzas into one price.

This post is for the product and engineering team behind a pizza POS or online-ordering platform. Your AI phone ordering for pizzerias now runs across hundreds of stores. Every prompt or menu change to that AI phone ordering pizza flow reaches every one of them. Our restaurant AI phone ordering QA playbook covers generic menu modifiers. Here we cover what is specific to pizza, and how each error turns into a wrong kitchen ticket that looks fine.

What the research says about pizza ordering AI

Four public sources each add one fact a QA plan needs.

Real-world accuracy. Intouch Insight placed 660 mystery-shop orders across 12 pizza brands in fall 2024. Each shopper ordered a small specialty pizza and removed one topping. Phone orders handled fully by automation were 88% accurate. Employee-handled phone orders were 97%. Calls that started automated and moved to an employee scored 92% (Intouch Insight 2025 pizza study).

Flat slots are not enough. Classic slot filling stores one value per slot. Amazon's PIZZA benchmark treats the problem as semantic parsing of pizza and drink orders into trees (Arkoudas et al., 2022). It trains on about 2.46 million synthetic utterances and tests on 1,357 human-written ones. The best model reached 78.56% exact match on test. A grammar-based parser reached 68.02%. The paper names a case every pizza team should test: "four pizzas, half with ham and half with peppers." That sentence has two valid readings. It can mean two ham pizzas and two pepper pizzas. It can also mean four half-and-half pizzas.

Real callers order several pizzas and point back at them. Google's Taskmaster-1 has 5,507 spoken two-person dialogs, including pizza ordering (Byrne et al., 2019). Callers spoke through a microphone. A trained human played the assistant. A simple keyword count over the 970 pizza dialogs in the public files is revealing. About 68% of them (657) ask for two or more pizzas. About 22% (218) use a reference like "the second pizza" or "the other one." Twenty contain "half." The crowdsourcing scenario shaped these rates, so treat them as a floor on variety, not a real call mix.

The schema loses the portion. Taskmaster's pizza ontology has `name.pizza`, `size.pizza`, `type.topping` and `type.crust`. It has no field for portion or pizza index. One dialog asks for two pizzas: half ham and pineapple, half sausage, then half green pepper, half pepperoni. Its labels are a flat list of topping spans like "half ham" and "sausage." Nothing records which half or which pizza.

Speech recognition widens the gap. Meta's STOP dataset has 218 hours of spoken task requests (Tomasello et al., 2022). A cascaded parser scored 85.25% exact match on gold transcripts. On ASR transcripts it fell to 72.36%. The paper's best ASR model scored a 4.26% word error rate on test. A low word error rate still costs a lot of parse accuracy, because one wrong word can move a whole branch of the tree.

How a pizza order maps to the POS ticket

Your agent does not send English to the kitchen. It sends a structured order to the pizza POS. The way that POS encodes portions decides which errors are even possible.

Toast is a clear example. Its orders API lets a menu item have portions, with the example names "half one" and "half two." You add each portion as a modifier of the pizza with `selectionType` set to `PORTION`. Then you put that portion's toppings inside its own `modifiers` array (Toast developer guide). Two more rules from that page matter for pizza:

• Default modifiers are not added for you. Omitting a default removes it. A specialty pizza on one half must carry that half's default toppings explicitly.
• Modifier quantity must match the item quantity. Two pizzas with different portions need separate selections. "Two larges, one half pepperoni" cannot be one line with quantity 2.

Not every POS has a portion object. In a 2023 Square seller community thread, pizza sellers described building separate half-topping modifiers for every size. A Square moderator said native partial toppings were not planned at that time (Square community thread). Check the current state before you rely on it.

So one spoken order has several valid encodings. "Pepperoni on half, mushrooms on the whole thing" can be one whole mushroom line plus a half pepperoni line. It can also be two halves, each with mushrooms, one with pepperoni. The kitchen makes the same pie. The price can differ if the store prices half toppings differently. Your test oracle has to know which encoding is the contract.

Seven ways a pizza call becomes a wrong ticket

Each failure below ends in a ticket the POS accepts. This is the core risk in AI phone ordering pizza work. No API error fires, so the error survives until a customer opens the box.

Portions: halves, quarters and "the whole thing"

The word "half" carries two meanings. It can be a portion ("half pepperoni") or a quantity ("half of them pepperoni"). The PIZZA example above shows the trap. A second trap is scope. In "pepperoni on half, mushrooms on the whole thing," the mushrooms belong to the pizza, not to the other half. An LLM that fills a two-slot `half1` and `half2` template puts mushrooms on half two only.

Quarters are rarer but real at larger pies. Removals stack on top: "half Supreme, no onions on that side." That needs a specialty pizza on a portion, its defaults spelled out, and one removed. This is one of the questions a pizza order accuracy test must ask: why does the AI put toppings on the wrong half? Usually the schema gave it no place to say "whole" while halves exist.

Size, crust, sauce and bake

These are pizza-level attributes, but callers attach them anywhere. "Both of them thin crust" scopes over two pizzas. "Make it well done" scopes over the last one mentioned. Constraints bite here too. A gluten-free crust may exist only in one size. If the agent hits that rule, it must ask. A silent fix to a valid size gives the wrong pizza.

Sauce words are easy to mishear on a phone line. "Light sauce," "no sauce" and "white sauce" are close on narrowband audio. Each one is a different ticket line. Test them in the phone audio conditions your callers actually use.

Multiple pizzas and "make the second one large"

"The second one" points to a position in the caller's head. Your cart has its own order. After a removal or a correction, those orders can drift apart. Then "make the second one large" upsizes the wrong pizza. Taskmaster callers used references like this in about one dialog in five.

Test references after every kind of cart edit: add, remove, reorder and correct. Assert on the cart after each turn, not just at checkout. Our guide to testing memory and context covers the general pattern.

Deals and coupon rules

A deal is a bundle with rules. "Two large one-toppings" might cap toppings per pizza. It might count a half topping as a full one. It might exclude specialty pizzas. When the caller adds a second topping after the deal is applied, three things can happen. The deal drops off, an upcharge applies, or the agent ignores the rule. The ticket is valid in all three cases. Only the total differs from what the agent said aloud.

So assert on price, not only items. Compare the total the agent spoke against the total on the ticket. Also check that the deal links to the right pizzas.

Delivery address and delivery zone

House numbers are the classic speech error: "fifteen" and "fifty" sound alike on a bad line. Street names have near neighbors too. Address validation can then make the wrong address look right. Google's Address Validation API returns flags such as `hasInferredComponents`, `hasReplacedComponents` and `hasSpellCorrectedComponents`. Its docs say `possibleNextAction` "should not be considered as a guarantee of address accuracy or deliverability" (Google API reference).

A corrected address passes the delivery zone check for the wrong house. If any of those flags are set, require a digit-by-digit read-back of the house number. Include out-of-zone addresses in your tests and assert the agent offers carryout. Our post on names and entities in STT explains why numbers and street names fail more than common words.

Out of stock and substitution

Toppings run out mid-shift. When a store marks one out of stock, the agent has to say so and ask. Three silent failures are common. It drops the topping. It swaps in a "close" topping. Or it applies the substitution to the whole pizza when only one half had it. Each is an out of stock path your test set needs, on a half, a whole and a specialty pizza.

Read-back that does not match the ticket

A read-back is only a check if it is built from the ticket. Many agents read back from the model's own summary of the conversation. That summary can be right while the tool call was wrong. The caller hears "half pepperoni, mushrooms on the whole pie" and says yes. The ticket has mushrooms on one half.

Render the read-back from the POS payload you are about to submit. Then a correct read-back means a correct ticket.

What accuracy numbers really measure

Pizza order accuracy numbers come from different methods. They are not comparable until you know the method.

Donatos reported 99.9% accuracy across 301,388 calls handled by its AI phone ordering system from June to August 2025. The article defines that as roughly 1 in 1,000 orders needing correction (PMQ Pizza Magazine, September 2025). The Intouch Insight figure instead compares a known scripted order against what was made. One counts corrections that someone reported. The other checks every order against a gold order. Both can be right about what they measure.

For your own platform, define four numbers and report them separately:

• Pizza exact match = pizzas whose size, crust, sauce, bake and portion layout all match gold ÷ gold pizzas.
• Order exact match = orders where every pizza, side and drink matches ÷ orders.
• Portion layout accuracy = pizzas with the correct toppings on the correct portions ÷ pizzas that had any portion request.
• Spoken total match = orders where the total the agent said equals the ticket total ÷ orders.

Report portion layout accuracy on its own for every AI phone ordering pizza release. Half-and-half orders are a small share of calls. A blended number hides them. Our post on vendor-reported voice AI metrics explains why the definition matters more than the number.

The pizza order test matrix

Build the suite as portion patterns crossed with conditions. Happy-path calls will not find these bugs.

Cross those eight rows with five conditions: clean phone line, kitchen or street noise, a mid-order correction, a deal applied, and an out-of-stock topping. That gives 40 cells. Mark the cells where a miss reaches the kitchen silently as critical. In the worked example below, 16 cells are critical.

Worked sizing (illustrative). Write three phrasings per cell and run each three times, because LLM agents vary run to run. A full pass is 40 × 3 × 3 = 360 calls. The release gate runs only the critical cells: 16 × 3 × 3 = 144 calls.

What the numbers can tell you. At 360 calls and 90% order exact match, the 95% confidence interval is about ±3.1 points (1.96 × √(0.9 × 0.1 ÷ 360) ≈ 0.031). For the gate, use the rule of three. Zero failures in 144 runs puts the 95% upper bound on the failure rate near 3 ÷ 144 ≈ 2.1%. One failure in a critical cell is a signal to stop, not noise to average away.

Add two cross-cutting sets on top. First, ten delivery addresses, including near-homophone house numbers and two out-of-zone streets. Second, every active deal, with a rule-breaking add-on for each. Keep the gold orders versioned with the menu, as in our guide to building a golden dataset.

A portion-aware order diff in Python

The scorer in our food ordering playbook compares modifier tuples. Pizza needs one more idea. Two tickets can differ in their modifiers and still describe the same pie. A pie can also be mirrored, so "left" and "right" labels are arbitrary.

The scorer below is simplified. It expands each topping to the quarters it covers. It compares layouts under rotation and mirroring. It then matches pizzas across the order, because the ticket order does not matter. Finally it separates real errors from encoding differences that only affect price.

from itertools import permutations

QUARTERS = ("q1", "q2", "q3", "q4")          # clockwise from 12 o'clock
PORTION = {"whole": QUARTERS, "left": ("q3", "q4"), "right": ("q1", "q2"),
           "q1": ("q1",), "q2": ("q2",), "q3": ("q3",), "q4": ("q4",)}
ROTATIONS = [QUARTERS[i:] + QUARTERS[:i] for i in range(4)]

def coverage(toppings):
    """[('pepperoni','left'), ('mushroom','whole')] -> {quarter: frozenset}."""
    cov = {q: set() for q in QUARTERS}
    for name, portion in toppings:
        for q in PORTION[portion]:
            cov[q].add(name)
    return {q: frozenset(v) for q, v in cov.items()}

def same_layout(a, b):
    """Kitchen-equal if b matches a under any rotation or mirror of the pie."""
    for rot in ROTATIONS:
        for order in (rot, rot[::-1]):
            if all(a[q] == b[o] for q, o in zip(QUARTERS, order)):
                return True
    return False

def shape(toppings):
    """How each topping was keyed: two halves is not the same line as whole."""
    return sorted((name, len(PORTION[p])) for name, p in toppings)

def pizza_errors(gold, pred):
    errs = []
    for attr in ("size", "crust", "sauce", "bake"):
        if gold.get(attr) != pred.get(attr):
            errs.append(f"{attr}: {gold.get(attr)} -> {pred.get(attr)}")
    g, p = coverage(gold["toppings"]), coverage(pred["toppings"])
    if not same_layout(g, p):
        g_all = set().union(*g.values())
        p_all = set().union(*p.values())
        if g_all == p_all:
            errs.append("portion: right toppings, wrong layout")
        else:
            errs.append(f"toppings: missing {sorted(g_all - p_all)}, "
                        f"extra {sorted(p_all - g_all)}")
    elif shape(gold["toppings"]) != shape(pred["toppings"]):
        errs.append("encoding: same pie, different modifiers (check price)")
    return errs

def score_order(gold, pred):
    """Match pizzas across the order, then diff each pair. Small orders only."""
    best, warns = None, []
    n = max(len(gold), len(pred))
    g = gold + [None] * (n - len(gold))
    for perm in permutations(pred + [None] * (n - len(pred))):
        errs, w = [], []
        for i, (a, b) in enumerate(zip(g, perm)):
            if a is None:
                errs.append(f"pizza {i+1}: extra pizza on ticket")
            elif b is None:
                errs.append(f"pizza {i+1}: missing from ticket")
            else:
                for e in pizza_errors(a, b):
                    tag = f"pizza {i+1}: {e}"
                    (w if e.startswith("encoding") else errs).append(tag)
        if best is None or len(errs) < len(best):
            best, warns = errs, w
    return {"order_exact": not best, "errors": best, "warnings": warns}

# Caller: "Two large thin crust. First one pepperoni on half, mushrooms on
# the whole thing. Make the second one plain cheese, well done."
gold = [
    {"size": "L", "crust": "thin", "sauce": "red", "bake": "normal",
     "toppings": [("pepperoni", "left"), ("mushroom", "whole")]},
    {"size": "L", "crust": "thin", "sauce": "red", "bake": "well_done",
     "toppings": []},
]
pred = [
    {"size": "L", "crust": "thin", "sauce": "red", "bake": "well_done",
     "toppings": []},
    {"size": "L", "crust": "thin", "sauce": "red", "bake": "normal",
     "toppings": [("pepperoni", "left"), ("mushroom", "right")]},
]
print(score_order(gold, pred))
mirror = [dict(gold[0], toppings=[("pepperoni", "right"), ("mushroom", "whole")]),
          gold[1]]
print(score_order(gold, mirror))
both = [dict(gold[0], toppings=[("pepperoni", "left"), ("mushroom", "left"),
                                ("mushroom", "right")]), gold[1]]
print(score_order(gold, both))

Output:

{'order_exact': False, 'errors': ['pizza 1: portion: right toppings, wrong layout'], 'warnings': []}
{'order_exact': True, 'errors': [], 'warnings': []}
{'order_exact': True, 'errors': [], 'warnings': ['pizza 1: encoding: same pie, different modifiers (check price)']}

The first ticket has every topping and both pizzas, so an item-level check passes it. Only the layout check sees the error. The second ticket swaps left and right, which the kitchen does not care about. The third makes the same pie through two half lines, which may change the price.

Three extensions before you rely on it. Write an adapter from your POS payload, Toast portions or per-size half modifiers, into this shape. Expand specialty pizzas into their default toppings, so a removal shows up. Add deals and the spoken total as a separate check.

How to test an AI phone ordering pizza agent before a release

This is the release loop for a team shipping voice AI pizza ordering to many stores.

1. Pin the menu. Export pizzas, sizes, crusts, sauces, portions, defaults, deals and stock flags from the POS. Version the snapshot so tests and stores agree on what exists.

2. Write gold orders first. For each test call, write the expected ticket as data, then the caller script that should produce it. Include the expected total.

3. Fill the matrix. Cover the eight portion patterns across the five conditions. Add the address set and the deal set.

4. Call the agent the way customers do. Dial in over the phone network with varied accents, pace, noise and interruptions. Evalgent's call simulator reaches agents over PSTN, SIP or WebRTC with caller profiles for these traits, so the test audio matches a real pizzeria line.

5. Capture every layer. Log the audio, transcript, each tool call, the cart after each turn, the read-back and the final POS request.

6. Score the ticket, then diagnose. Run the portion-aware diff on the submitted payload. Use the logs to place each failure in speech, parsing, cart logic, read-back or POS. Judging from the audio rather than the transcript matters here, since "fifteen" and "fifty" look fine in a bad transcript.

7. Gate the release. Block on any new failure in a critical cell. Then roll out to a few stores first, as in our guide to shipping prompt changes safely.

8. Keep watching live calls. Score production calls against the same checks. Evalgent's production monitoring takes pushed call recordings, maps them to scenarios, and flags tool-call failures and policy failures. A failed POS submit that the agent reports as success is covered in detecting silent tool failures.

For a platform that serves many brands, run the matrix per menu template, not per store. Stores that share a template share most bugs. Our drive-thru QA guide for 500 stores covers how to sample stores for the rest.

Frequently asked questions

Can AI take pizza phone orders accurately?

AI can take simple pizza phone orders well, but accuracy drops on structure. Intouch Insight's 2025 study found fully automated pizza phone orders 88% accurate, against 97% for employees. Their test order was a specialty pizza with one topping removed. Half-and-half, multi-pizza and deal orders need their own measured accuracy before you trust a blended number.

How does AI handle half and half pizza orders?

AI handles half-and-half pizza orders by filling a portion structure: toppings per half, plus toppings for the whole pie. It fails when the schema has only two half slots, so "mushrooms on the whole thing" lands on one half. A safe design has explicit whole, half and quarter portions and validates them before the ticket is sent.

How do you test an AI pizza ordering agent?

Test an AI pizza ordering agent with gold orders written as structured tickets. Cross portion patterns, such as half-and-half or specialty on a half, with conditions like noise, corrections, deals and out-of-stock toppings. Place real phone calls, then diff the submitted POS ticket portion by portion against gold. Check the spoken total against the ticket total.

Why does the AI put toppings on the wrong half?

An AI pizza agent puts toppings on the wrong half for three common reasons. Its schema may have no "whole" option once halves exist. It may resolve "that side" or "the other half" against the wrong portion. Or it may read "half" as a quantity instead of a portion. Read-backs built from the model's summary then hide the error.

How should half and half pizzas be sent to the POS?

Half-and-half pizzas should be sent in the POS's own portion structure. In Toast, each portion is a modifier with `selectionType` set to `PORTION`, holding its own toppings. POS systems without portions often use per-size half-topping modifiers. Pick one canonical encoding per POS, test against it, and check that the price matches the store's half-topping rules.

How do you test a menu change before it ships to every store?

Test a menu change by rerunning the critical cells of your pizza test matrix against the new menu snapshot before release. Add cases for whatever changed, such as a new deal, crust or specialty pizza. Block on any new failure, then release to a small group of stores and score their live calls before the full rollout.

Does reading back the order prevent wrong pizzas?

Reading back the order prevents wrong pizzas only if the read-back is generated from the ticket being submitted. Many agents read back from the model's own summary, which can be right while the tool call is wrong. Render the read-back from the POS payload, and test that the spoken read-back and the ticket match.

How do you measure pizza order accuracy?

Measure pizza order accuracy by comparing each submitted ticket to a gold order. Report pizza exact match, order exact match, portion layout accuracy and spoken total match separately. Treat a mirrored half as correct, since the kitchen does not care which side is left. Report half-and-half results on their own so a blended rate does not hide them.

The bottom line

Pizza phone orders fail on structure: portions, pizza references, deals and addresses, not on single words. Score the kitchen ticket portion by portion against a gold order, under real phone conditions, before every change reaches every store.