Load Tracking AI Calls: Testing What Shippers Hear

A shipper calls a broker and asks, "Where's my load?" A freight broker AI voice agent can answer in seconds. The risk is that it answers from a record that is wrong, old or about a different load. The shipper then plans a dock crew or a customer promise around a sentence your agent invented. Load tracking AI calls need their own test plan.

This post covers the inbound side: AI shipper status calls from shippers and consignees to a broker or 3PL. Outbound driver calls are covered in monitoring AI check calls. Rates and lanes are in AI carrier calls for brokers. Here the question is narrower: what does the shipper hear, and does it match the system of record?

What a load tracking AI call is and where it breaks

A where is my load AI agent answers an inbound status call by reading your TMS and shipment visibility feed. The caller is usually a shipper's logistics coordinator, a consignee's receiving desk or a customer-service rep. They want one of six things: location, ETA, appointment time, detention status, a delivery document, or a person. These automated load status calls are a small agentic workflow.

The agent captures an identifier, looks up a record, picks fields to read, says something, and sometimes writes a note or opens a ticket. Every step can fail without an error. A wrong load number returns a perfectly valid status for the wrong shipment. Stale tracking data returns a confident position that is two hours old.

That is why transcripts alone are a weak judge here. The transcript shows "load 48213." It does not show that the caller said "48218" over forklift noise. Evalgent's approach is to judge from the audio and the call outcome, not only the text. That matters most on this call type, because the caller is often standing on a dock.

Five numbers a shipper calls "the load number"

Shippers rarely say your load number. They say whatever their own system prints on the screen. Your agent has to work out which identifier it heard before it looks anything up.

Two facts make this harder. First, the bill of lading is a core record. Under 49 CFR 373.101, a motor carrier issues a receipt or bill of lading naming the consignor and consignee, origin and destination, and the freight. A broker's own record must also carry the bill of lading or freight bill number under 49 CFR 371.3(a). So callers legitimately read you a BOL number and call it "the load."

Second, spoken alphanumeric strings are hard for speech recognition. The SNuC corpus paper (LREC 2022) describes itself as the first published corpus of spoken alphanumeric identifiers. It has more than 13,000 sequences from over 50 speakers. Adapting a neural recognizer to this data improves accuracy. The lesson: identifiers need their own test set.

Worked example: identifier capture math

These numbers are illustrative assumptions, not measurements. Suppose a recognizer gets each digit right 98% of the time. For an 8-digit load number, the chance that all digits are right is 0.98^8, about 85%. That means about 15 calls in 100 start with a wrong number.

Add a read-back step. Assume the caller catches 80% of read-back errors. The residual error is 15% x 20%, about 3%. That is still 3 wrong-load answers per 100 calls.

The worst case is a wrong number that matches a real load. Load numbers are issued in sequence. If loads 48213 and 48218 both exist, a one-digit slip returns a clean record for the wrong shipment. A no-match is a safe failure. A collision is not.

So test collisions on purpose. Build scenarios where the spoken number is one digit from another open load. Also scope every lookup to the caller's account, and read back a second field (origin city or pickup date) before giving status.

Stale data: how old is the position the agent reads out?

Agents read "last known location" fields as if they were live. The field has an age, and the age changes the answer.

The ELD rule sets a floor. Under 49 CFR 395 Appendix A, section 4.5.1.2, an intermediate log is recorded only while the vehicle is in motion, and only if no other log event was recorded in the previous hour. Location is recorded with those events. So if your only source is the ELD log, the newest position can be up to an hour old before any upload delay. Providers may report more often. Check what your feed actually delivers, and log the timestamp with every position.

Worked example: position drift

Again, these are illustrative numbers. A truck holds 55 mph. Its last log is at 11:02. The shipper calls at 11:47. That is 45 minutes of silence, or about 41 miles of drift.

If the TMS says "62 miles from the consignee" at 11:02, the truck is probably near 21 miles out at 11:47. An agent that says "about 60 miles out" is off by 40 miles. The shipper hears "an hour away" and the truck is 25 minutes away.

The fix is not a smarter model. It is a rule: every spoken position carries its age. "As of 11:02, about 62 miles out" is true. "About 62 miles out" is not. When the age passes your limit, the agent should say it, offer a callback, and open a tracking task.

Replay a feed that returns old positions on purpose. The agent must state the age, skip a fresh ETA from stale miles, and log the timestamp it used. ETA arithmetic and hours-of-service limits are in the outbound check-call post.

ETA is not the appointment

Shippers ask two different questions and use one word. "When will it get there?" asks for an ETA. "When is it scheduled?" asks for the appointment. An agent that blends them says "on time" while the ETA sits after the appointment.

Check three things on every time answer.

1. Which time is it? Pickup appointment, delivery appointment, ETA and actual arrival are four separate fields. The agent should name which one it is reading.

2. Whose clock? A delivery appointment is set in the facility's local time. The shipper may call from another zone. The X12 214 status message carries the date, the time and a time code. The ArcBest 214 specification shows an AT7 segment with date, time and a two-letter time code (the sample uses LT). Whatever your feed sends, your TMS may flatten it into one zone. Test that the spoken time carries the right zone.

3. Window or point? "Between 8 and 10" and "8 a.m. sharp" are different promises. First-come-first-served and appointment stops differ again. The agent must not turn a window into a time.

A good answer has a fixed shape: field, value, zone, age. "Your delivery appointment is 2:00 p.m. Central. The latest estimate is 3:30 p.m., as of 11:02. That is after the appointment." The agent should say the last part out loud. A shipper would rather hear "late" than learn it at the dock.

Delivered has three meanings

"Has it been delivered?" has at least three honest answers. The truck arrived. Unloading is complete. The signed proof of delivery (POD) is in your system. They happen at different times, and a shipper's billing often waits on the last one.

The ArcBest sample uses distinct status codes for these steps: X1 for arrived at delivery location and D1 for completed unloading. Treat the POD image as a third event with its own field. The agent should climb the ladder one rung at a time.

The rule is to say only the rung you can prove. If the TMS says "delivered" but no POD is attached, the agent says the document is not in yet and opens a request. It never says "I'll email it right now" unless the send actually happens and writes a note back.

The bill of lading rule in 49 CFR 373.101 does not mention proof of delivery. So define what a POD must contain in your own policy, and test against that.

Detention talk: capture the claim, do not rule on it

Dwell time and detention are where status calls turn into money disputes. A consignee says the truck has waited three hours. A shipper asks if the carrier will charge. The agent has no business ruling on either.

FMCSA's detention research page cites a 2014 study with an average dwell of 3.4 hours, of which 1.4 hours was detention beyond a 2-hour standard. It also reports detention on roughly 1 in 10 stops. These are research averages. Your free time is whatever the rate confirmation says.

A safe flow has three moves. Capture the caller, load, facility and claim. Read back only timestamps your system holds, such as the arrival event. Then say a person will review accessorial charges, and open a ticket with the claim and the call reference.

Test both directions. The agent must not say "no detention applies," and it must not say "we'll cover it." Also test an angry caller who pushes for a yes. Evalgent generates scenarios from the agent's prompt, including policies and hard constraints, and simulates callers with different pace, interruption levels and behaviour. A pushy caller who talks over the agent is a profile worth running.

Who is calling, and what must not be said

A broker's record is open to the parties. Under 49 CFR 371.3(c), each party to a brokered transaction may review it. That does not mean a voice agent should read the record to anyone who calls. A caller claiming to be a shipper may be a competitor, a carrier hunting for a margin, or a fraudster looking for a pickup.

Set three rules and test them.

• Verify before status. Match the caller's number to a contact on the account, then confirm one fact only the real contact knows, such as a PO. Do not accept "I'm with the shipper."
• Scope what is said. Location and appointment status are fair. Carrier pay, broker margin and carrier contact details are not for the phone.
• Hand off the sensitive asks. Rate questions, record requests and claims go to a person.

Test with a caller who asks, "What did you pay the carrier?" The right answer is a handoff, not a lookup.

This is not legal advice. Check your own obligations with counsel.

The promise ledger: a framework for status calls

Here is the framework we suggest. A status call is a list of promises. Every sentence the agent says is one of four types, and each type needs a backing field and a maximum age.

• Fact: a value the record holds, such as the appointment time. Needs a field and a timestamp.
• Estimate: a derived value, such as an ETA. Needs the inputs, the basis and the age.
• Commitment: something the agent says will happen, such as a callback or an emailed POD. Needs a write-back.
• Refusal: something it will not say, such as detention rulings or rates. Needs a handoff.

After each call, diff the spoken claims against the TMS snapshot from the moment of the call. Snapshot at call start, because the record moves while the call runs. Here is a simplified checker. It assumes you already extract claims from the transcript.

from dataclasses import dataclass
from datetime import datetime, timedelta

MAX_AGE = {"position": timedelta(minutes=60), "eta": timedelta(minutes=60)}

@dataclass
class Claim:
    kind: str            # position | eta | appointment | stage | commitment
    value: str           # what the agent said, normalized
    stated_age: bool     # did the agent say "as of ..."?

def check(claims, snap, notes, now):
    """snap: TMS snapshot taken at call start. notes: write-backs made."""
    issues = []
    for c in claims:
        field = snap.get(c.kind)
        if c.kind in ("position", "eta"):
            age = now - field["updated_at"]
            if age > MAX_AGE[c.kind] and not c.stated_age:
                issues.append(f"{c.kind}: {age} old, age not stated")
        if c.kind in ("position", "eta", "appointment", "stage"):
            if field is None or field["value"] != c.value:
                issues.append(f"{c.kind}: said {c.value}, TMS {field}")
        if c.kind == "commitment" and c.value not in notes:
            issues.append(f"commitment without write-back: {c.value}")
    return issues

Set the age limit from your own feed. Run the checker on simulated calls before a change ships and on recorded calls after. A fact that disagrees with the snapshot is a hard fail. So is a commitment with no note, because someone will wait for a callback nobody owes.

Model behaviour varies between runs. The tau-bench paper simulates user conversations against domain tools and policies. It reports that gpt-4o agents succeed on fewer than half of tasks, and that pass^8 on retail is below 25%. That is the share of tasks solved on all eight repeats. A call that passes once may not pass again.

Test matrix for load tracking AI calls

For 3PL voice agent testing, cross scenarios with conditions. Thresholds below are suggested starting targets, not industry standards. Tune them to your volume and risk.

Each row needs several runs per condition. Include the quiet case too: a calm caller who only wants a status. Needless handoffs flood your team.

Related reading: weight and lane capture in freight quote AI calls, the wider AI voice agents for freight playbook, and parcel cases in delivery exception AI calls. If one agent serves many brokerage customers, see multi-tenant voice agent testing, since each tenant has its own identifier formats.

How to test a load tracking voice agent before a change ships

Follow these steps for each prompt, flow or data-feed change to your load tracking AI calls.

1. Write the claim inventory. List every fact, estimate, commitment and refusal the agent can say, and name the TMS field behind each.

2. Build the identifier set. Collect real formats for load, BOL, PO, PRO and shipper references. Add near-collision pairs from your own open loads.

3. Freeze three feed states. Fresh, stale and missing. Replay the same call against each.

4. Generate scenarios from the prompt and flows. Cover states, branches, policies and hard constraints, such as no detention rulings and no rate disclosure. Generating them from the prompt itself exposes coverage gaps before callers find them.

5. Add caller profiles. Vary accent, pace, interruption, response delay and background noise. A dock, a warehouse office and a moving truck cab sound different. Add Spanish-language callers if your receiving desks use them.

6. Judge from audio and outcome. Score the identifier heard, the spoken age and zone, the stage named, and the TMS write-back. Evalgent evaluates from the audio, not only the transcript, and supports custom metrics that run on every call, such as "age stated when data is stale."

7. Run repeats and compare to baseline. Run each scenario several times. Treat any hard-constraint failure as a block, and compare soft metrics against the last release. For release hygiene, see shipping prompt changes safely and the go-live checklist.

What to monitor after launch

Load tracking AI calls drift after launch, as feeds and customers change. Testing covers what you thought of. Live calls cover the rest. Watch four places.

• Lookup failures: a number the caller rejected, or a fallback to a human.
• Stale answers: positions read out with no age.
• Tool-call failures: a note or ticket that never saved.
• Policy failures: a detention ruling, a rate disclosure, or status for an unverified caller.

Evalgent's production call monitoring takes customer-pushed recordings, transcribes them, maps them to scenarios and judges them. It flags PII leakage, tool-call failures, latency spikes and policy failures, with fix recommendations. Human review of verdicts keeps the judge honest. For a freight desk, the most useful early signal is a commitment with no write-back, since that one becomes a missed callback.

Segment by customer account and tracking source. A feed fresh for one carrier and stale for another hides inside averages. The voice agent testing checklist gives a general pre-launch gate.

Frequently asked questions

How do AI voice agents answer where's my load calls?

The agent captures an identifier, confirms the caller, looks up the load in the TMS or tracking feed, and reads back fields such as location, ETA and appointment. Good agents state each field's age, say when an ETA falls after the appointment, and open a ticket for anything they cannot answer.

What is the difference between a load number, BOL number and PRO number?

A load number is usually your TMS identifier. A bill of lading number comes from the shipper or carrier and names the shipment's parties and freight. A PRO number is typically the carrier's own tracking reference. Callers mix them, so an agent should work out which one it heard before looking anything up.

How often does an ELD update a truck's location?

Under the federal ELD specification, the device records an intermediate log while the vehicle moves if no other event was recorded in the previous hour. So the floor is one location record per hour. Telematics providers may send positions more often. Check your own feed's timestamps.

Can an AI agent discuss detention with a shipper?

It can capture the claim, read back arrival timestamps from the record and say a person will review it. It should not confirm or deny detention charges. Free time and rates come from the rate confirmation, and disputes need a human, so the agent opens a ticket instead of ruling.

How should an AI agent handle proof of delivery requests?

It should check whether a signed POD is attached before saying anything. If it is, it sends the document to a verified contact and writes a note. If not, it says the POD is not yet in the system and opens a request. It never says "delivered" when only arrival is recorded.

How do you test a load tracking voice agent?

Build scenarios from the agent's prompt and flows, then vary identifiers, caller accents, noise, feed freshness and time zones. Compare every spoken claim with the TMS snapshot taken at call start. Run each scenario several times, since one pass does not prove consistency, and block releases on hard-constraint failures.

How do you verify a caller is the shipper before sharing load status?

Match the caller's phone number to a contact on the account, then ask for one detail only the real contact would know, such as a PO number. Share only location and appointment status. Route rate, carrier contact and record requests to a person. This is not legal advice.

What ETA should an agent give when tracking data is stale?

It should say when the position was last updated, avoid computing a new ETA from old miles, and offer a callback after a fresh ping. If the last ETA falls after the appointment, it should say so plainly. Hiding the age is the failure, not the lack of an exact time.

The bottom line

A load tracking AI calls program is only as honest as the age, zone and stage it speaks aloud. Test every inbound status answer against the TMS snapshot, and treat any unbacked claim or missing write-back as a release blocker.