Staffing Agency AI Screens: Testing Skills and Pay Rates

This post is for the product team behind voice AI for staffing software. Your agent runs for dozens of agencies. Each has its own job orders, pay rules, states and ATS fields. One prompt change ships to all of them at once. Below: where staffing agency AI screening corrupts the record, which pay errors create legal exposure, and a copyable test matrix.

The scale is real. The American Staffing Association counts about 2.2 million temporary and contract workers on the job in an average week in 2024. Agencies hired 12.7 million of them in 2023. Industrial work is 36% of staffing employment. Office and clerical is 24%. Health care is 8%. Those are the three call types this post tests, plus IT contract.

What a staffing screen actually writes to the record

A staffing call is not a conversation that ends. It is the front door to a workflow. The agent captures skills and availability. It matches the candidate to a job order. It states a pay rate. It records a yes or no. Then it writes structured fields back to the ATS, and sometimes to the client's VMS.

The category is public. Sense, a talent engagement platform, announced Conversational Voice AI in December 2024. The release says it runs pre-screening calls and schedules interviews. It names staffing firm TSR Consulting Services as a pilot customer. If you build a product like that, every error below lands in your customers' databases.

A mishearing at stage one becomes a match score, then a submission, then a placement.

The hiring pillar on AI phone screening and interview QA covers general screening flows. The sibling post on AI phone screeners for hourly workers covers knockout questions that reject good candidates. This post covers AI phone screening for staffing agencies specifically: job orders, rates and many agencies on one agent.

Skills inflation and deflation from misheard answers

The most common silent error is a number. A candidate says "fifteen years on a forklift." Speech-to-text writes "fifty." Or it writes "five." The LLM accepts the value and fills the slot.

Teen and -ty numbers differ mostly in final stress and a short syllable. On an 8 kHz phone line with warehouse noise, that cue is easy to lose. Credentials have the same problem. "CNA" and "RN" are short, and "LPN" sits between them. "Sit-down" and "stand-up" forklift are separate certifications at many sites. "Seconds" can mean second shift or a time unit.

Four research findings matter here:

1. WER hides entity errors. VoiceCodeBench tested 19 ASR systems on 1,482 audited entities. Ranking by word error rate barely matched ranking by exact entity match (Spearman ρ = -0.28). The best system kept every critical value in only 68.7% of segments.

2. Context prompts may not save you. ProfASR-Bench gave Whisper and audio LLMs domain and speaker prompts. Even oracle prompts produced "little to no change" in average WER. Do not assume a "this is a nursing screen" hint fixes "RN."

3. Errors are not evenly spread. A PNAS study of five commercial ASR systems found average WER of 0.35 for Black speakers and 0.19 for white speakers. In staffing, that gap becomes a skills-capture gap. See accent robustness testing and AI interviewer bias testing for fairness methods.

4. LLMs can partly repair noisy transcripts. A study of LLM slot filling on ASR output gained 8.3% absolute SLU-F1 with fine-tuning and knowledge injection. Repair helps, but it can also make a wrong number sound confident.

Skills inflation: a captured qualification that is higher than what the candidate said, such as "RN" for "CNA" or "50 years" for "15." Skills deflation is the reverse, and it quietly drops good candidates from matches.

The fix is not a better STT model alone. Add three checks around every skill slot. First, read back high-stakes values in a different form: "That's one-five, fifteen years?" Second, compare against the ATS profile and flag big gaps. Third, never mark a license as verified from speech. For nurses, the NCSBN's Nursys QuickConfirm is a free service for license verification. Speech can capture a claim. It cannot verify one.

A simplified plausibility check for slot output before write-back:

# Simplified: flag skill slots that need readback or human review.
TEEN_TY = {13: 30, 14: 40, 15: 50, 16: 60, 17: 70, 18: 80, 19: 90}
CONFUSABLE_CREDS = [{"CNA", "RN", "LPN"}, {"sit-down forklift", "stand-up forklift"}]

def check_experience(spoken_years: int, ats_years: int | None) -> list[str]:
    flags = []
    pair = {**TEEN_TY, **{v: k for k, v in TEEN_TY.items()}}
    if spoken_years in pair:
        flags.append(f"TEEN_TY_CONFUSABLE: {spoken_years} vs {pair[spoken_years]}")
    if ats_years is not None and abs(spoken_years - ats_years) >= 5:
        flags.append(f"ATS_MISMATCH: spoken {spoken_years}, profile {ats_years}")
    return flags

def check_credential(captured: str, required: str) -> list[str]:
    for group in CONFUSABLE_CREDS:
        if captured in group and required in group and captured != required:
            return [f"CRED_CONFUSABLE: captured {captured}, order needs {required}"]
    return []

print(check_experience(50, 14))
# ['TEEN_TY_CONFUSABLE: 50 vs 15', 'ATS_MISMATCH: spoken 50, profile 14']

Flags should force a readback turn, not a silent correction. For deeper method, see entity accuracy for STT and tool-argument accuracy from speech.

Pay-rate errors: bill rate vs pay rate, units and promises

Pay is where a staffing agent creates the most risk per word. The job order holds several money fields, and saying the wrong one is easy.

Bill rate vs pay rate: the pay rate is what the worker earns. The bill rate is what the client pays the agency. The difference is the markup, which covers payroll taxes, insurance, overhead and margin.

Here is a worked example with illustrative numbers. Pay rate is $18.00 per hour. Markup is 45%. Bill rate is $18.00 × 1.45 = $26.10. If the prompt or tool result exposes both fields, the agent may say "$26.10 an hour." The candidate now expects 45% more than the order allows.

Units fail the same way. Travel nursing pay is often quoted as a weekly package with a per diem stipend for housing and meals. Light industrial is hourly. IT contract roles may be hourly on a W-2 or a C2C rate. A quote with no unit is a defect.

Overtime is the subtle one. Under the FLSA, non-exempt workers get one and one-half times the regular rate after 40 hours in a workweek. Expense reimbursements that reasonably match the expense are excluded from the regular rate under 29 CFR 778.217. Any excess is included. So an agent that blends stipends into an "hourly rate" will also get overtime wrong.

Worked example, illustrative only:

• Taxable hourly: $30.00 × 40 hours = $1,200
• Housing stipend: $1,400 per week
• Meals and incidentals stipend: $500 per week
• Weekly package: $3,100

A careless agent divides $3,100 by 40 and says "$77.50 an hour." Asked about overtime, it says "time and a half, so about $116 an hour." If the stipends reasonably match expenses, overtime is 1.5 × $30.00 = $45.00. That gap is a wage dispute waiting to happen. Whether a stipend qualifies is a payroll decision, not something the agent should assert.

The last failure is the promise. "I'm sure we can get you $25" on a $19–$21 order sounds helpful. It isn't. The agent should only speak values from the job order's pay fields. It should name the unit every time. It should route negotiation to a recruiter.

A simplified checker for spoken amounts. Run it on the agent's output text and on a transcript of the audio:

import re
from dataclasses import dataclass

@dataclass
class JobOrder:
    pay_min: float
    pay_max: float
    bill_rate: float

MONEY = re.compile(r"\$(\d{1,5}(?:\.\d{2})?)\s*(an hour|per hour|/hr|a week|per week|a day|per day)?", re.I)

def check_rates(turn_text: str, jo: JobOrder) -> list[str]:
    out = []
    for m in MONEY.finditer(turn_text.replace(",", "")):
        amt, unit = float(m.group(1)), (m.group(2) or "").lower()
        if abs(amt - jo.bill_rate) < 0.01:
            out.append(f"BILL_RATE_SPOKEN {amt}")
        elif not unit:
            out.append(f"NO_UNIT {amt}")
        elif "hour" in unit or "/hr" in unit:
            if not jo.pay_min <= amt <= jo.pay_max:
                out.append(f"OUTSIDE_ORDER {amt}")
    return out

print(check_rates("It's $26.10 an hour, or $25 if you push.", JobOrder(19, 21, 26.10)))
# ['BILL_RATE_SPOKEN 26.1', 'NO_UNIT 25.0']

Check both sources. The LLM text can be correct while TTS mispronounces "$19.50." That is why audio-based evaluation catches what transcripts miss.

Pay transparency laws that shape what the agent says

Several states now regulate pay disclosure in hiring. Most rules target written postings. A few reach what an agent says on a call. Here is what we could verify from primary sources as of October 2026. This is not legal advice. Have counsel map each agency's footprint.

• California. Labor Code 432.3 requires employers with 15 or more employees to include the pay scale in any job posting. An employer that uses a third party to post must give it the pay scale, and "the third party shall include the pay scale." Employers must provide the pay scale to an applicant "upon reasonable request." SB 642 (2025) defines pay scale as "a good faith estimate" of the range expected "upon hire."
• California salary history. The same section bars employers from seeking salary history "orally or in writing, personally or through an agent." A voice screen that asks "what were you making at your last job?" is an obvious test case.
• Illinois. The Equal Pay Act requires employers with 15 or more employees to include pay scale and benefits in postings. A third party that posts is liable for missing pay information unless the employer did not provide it. Employers must also disclose pay and benefits at the applicant's request when no posting was made available.
• Washington. L&I's Administrative Policy ES.E.2, issued December 3, 2025, covers recruitment done "indirectly through a third party." Postings must show the wage scale or salary range and describe benefits. Open-ended "up to" phrasing with no minimum is not enough.
• Colorado. CDLE's INFO #9 requires the rate or range and a general benefits description. It covers posts an employer makes or "has another party make." It defines a posting as an electronic or hard copy communication.
• New York. The state DOL applies its rule to employers with four or more employees. It says the law "does not apply to temporary help firms" recruiting for other organizations. The client employer's ads still need a range, and a range "cannot be open-ended."

What does this mean for testing? The agent's spoken range should match the posted range for that order. "Up to $24" fails where a minimum is required. A "what does it pay?" question should never get "the recruiter will discuss that." And the agent should never ask about past pay in California calls.

Wrong job-order matches, availability errors and an ambiguous yes

Candidate redeployment calls bring their own failures, mostly in job order matching. An agency may have three open orders at one client: days, seconds and weekends. The agent pitches order A and writes order B. The candidate shows up for the wrong shift. This is a silent tool failure: the tool call succeeded, with the wrong ID.

Availability is relative language turned into a date. "I can start Monday" on a Friday call means three days. On a Sunday call it means one. "Weekends except Sunday" is Saturday. Time zones matter for travel nurses calling from home before a contract in another state.

Then comes the yes. Candidates say "yeah, sounds good" to mean "I heard you." They say "yeah, I'll think about it." They say "sure" and then ask about parking. A model that maps any "yeah" to `accepted` creates phantom placements.

Assignment confirmation: a structured readback of order ID, client or site, shift, start date and pay rate with unit, followed by an explicit yes to that summary. Only then should the agent write `accepted`.

Treat acceptance as three outcomes, not two: accepted, declined and unclear. Unclear goes to a recruiter with the audio timestamp. Store the confirmation clip next to the field for disputes.

A test matrix per agency configuration

Your agent is one codebase with many configurations. Each agency has its own verticals, states, pay fields, ATS mapping and scripts. A prompt change that is safe for a clerical agency in Texas can break a travel nursing agency in California. Test by configuration, not by prompt.

Build the matrix in three axes:

1. Scenarios. Skills capture with confusable numbers and credentials. Pay question. Overtime question. Bill-rate bait ("what does the client pay you?"). Salary history bait. Multi-order redeployment. Relative start dates. Ambiguous acceptance.

2. Caller conditions. Accents, fast speech, interruptions, warehouse or car noise, long pauses and Spanish.

3. Agency configurations. Group agencies into tiers by risk: states with pay rules, licensed verticals and VMS write-back.

Now the sample-size math. If you run n calls with zero failures, the 95% upper bound on the true failure rate is about 3/n. This is the rule of three. To claim pay misstatements stay under 1%, you need about 300 clean runs of the pay scenarios. Spread across 40 agencies, that is impossible per agency per release. So tier it. Run 300 pay-scenario calls on the highest-risk tier. Run 30 per configuration elsewhere as a smoke test. Zero failures in 30 only bounds the rate at about 10%. Be honest about what each tier proves.

Evalgent generates test scenarios from the agent's prompt and flows, including branches and hard constraints like "never state the bill rate." It runs them with caller profiles that vary accent, pace, interruptions, noise and language, over PSTN, SIP or WebRTC. That makes per-configuration matrices practical without hand-dialing.

How to test a staffing agency AI screening agent before each change

1. Export each agency's job-order schema. List every money field, unit, range and shift field the agent can see. Mark bill rate and markup as never-speak.

2. Write hard constraints as assertions. Examples: spoken rate within pay_min and pay_max, unit present, no salary history question, acceptance only after readback.

3. Seed confusable skills. Script candidates with 13 to 19 years, CNA vs RN vs LPN and forklift types. Check the written slot, not the reply.

4. Add bait turns. Ask "what's the bill rate?", "can you do $25?" and "is the stipend tax-free?" Expect a recruiter handoff, not an answer.

5. Run the matrix per tier. Use real phone audio and caller profiles. Score from audio, so mispronounced rates count.

6. Diff the write-back. Compare ATS and VMS fields after each call against expected values. A pass on the conversation with a wrong field is a fail.

7. Gate the release. Block the change if any must-pass cell fails. Compare failure rates against the last release for the rest.

8. Ship gradually. Roll to the lowest-risk tier first, then the licensed and pay-regulated tiers.

For launch-level checks on each new agency, the go-live checklist for every new customer's agent is the companion piece. If you buy rather than build, use the recruiting voice agent vendor evaluation guide.

Monitoring staffing screens in production

Pre-release tests catch regressions you can imagine. Production finds the ones you can't. Monitor calls against the job order, not just the transcript.

Thresholds are illustrative starting points. Tune them per tier. Recruiter edits are your best free label. Every field a recruiter fixes is a test case you didn't have.

Evalgent's production monitoring takes call recordings you push, transcribes them, maps them to scenarios and judges them. It flags policy failures, tool-call failures, PII leakage and latency spikes with fix recommendations. For staffing, policy checks like "rate within order range" can run as custom metrics on every call. Feed the failures back into the matrix, as described in turning production failures into tests. For the general setup, see how to monitor voice agents in production.

Frequently asked questions

How accurate are AI phone screens for staffing agencies?

Accuracy of AI phone screens for staffing agencies depends on the field. Free-text answers survive small errors. Numbers, credentials and IDs do not. VoiceCodeBench found the best of 19 ASR systems kept every critical value in only 68.7% of segments. Measure slot accuracy per field from audio, not overall word error rate.

Can an AI recruiter tell candidates the pay rate?

An AI recruiter can state the pay rate if it reads it from the job order's pay fields. It should give the range with a unit, such as per hour or per week. It should never speak the bill rate or markup. Negotiation above the order's maximum should go to a human recruiter.

What is the difference between bill rate and pay rate?

The pay rate is what the worker earns. The bill rate is what the client pays the staffing agency. The difference is the markup, which covers payroll taxes, insurance, overhead and margin. For example, an $18.00 pay rate with a 45% markup gives a $26.10 bill rate. A screening agent should only ever say the pay rate.

Do pay transparency laws apply to staffing agencies?

Pay transparency laws reach staffing agencies in several states, mostly through third-party posting rules. California and Illinois require third parties to include the pay scale employers give them. Washington covers recruitment through a third party. New York exempts temporary help firms recruiting for clients, but not their clients. This is not legal advice.

Can an AI screener ask about salary history?

An AI screener should not ask about salary history on California calls. Labor Code 432.3 bars employers from seeking it "orally or in writing, personally or through an agent." The safest design asks about desired pay, not past pay, and tests that rule on every release.

How do you test an AI screening agent for multiple staffing agencies?

Test an AI screening agent for multiple staffing agencies by configuration. Group agencies into risk tiers by state rules, licensed roles and VMS write-back. Run a scenario matrix per tier with varied caller audio. Run about 300 clean pay-scenario calls on the top tier to bound failures near 1%.

How should an AI agent confirm assignment acceptance?

An AI agent should confirm assignment acceptance with a full readback. It states the order, site, shift, start date and pay rate with unit. Then it asks for an explicit yes to that summary. Anything else is marked unclear and routed to a recruiter. Store the confirmation clip next to the record.

How do you monitor AI screening calls in production?

Monitor AI screening calls in production by comparing each call to its job order. Check spoken amounts against pay fields, flag confusable skill slots and track unclear acceptances. Recruiter edits to ATS fields are free labels. Turn every confirmed failure into a regression test for the next release.

The bottom line

Staffing agency AI screening fails in numbers, credentials and yeses, not in tone. Test each spoken value against the job order, per agency configuration, from the audio, before every change ships.