Clinical Trial Pre-Screening AI: Testing Eligibility Calls

Not medical advice and not legal advice. This post describes testing practice. Requirements differ by study, sponsor, IRB and state. Check your protocol, your IRB and your counsel. Evalgent makes no HIPAA claim, and the examples here use synthetic data. Regulatory text checked October 19, 2026.

Clinical trial eligibility screening by phone looks like a short survey. It is not. Each answer maps to an inclusion or exclusion criterion in a protocol. Each criterion decides whether a person is routed to a site visit or quietly dropped. A wrong value rarely sounds wrong on the call. It just lands in the screening record, and nobody listens to the audio again.

This post is for teams that built and run their own patient recruitment voice agent, inside a research platform, a site network tool or a recruitment service. You own the script, the prompt and the write-back to the screening log. Every prompt change reaches live callers. Here is how to test that before it does.

What clinical trial prescreening AI actually does

Pre-screening is the first filter between interest and a site visit. A person responds to an ad, a registry listing or a physician referral. Someone, now sometimes a voice agent, asks a short list of questions. The answers decide who a coordinator calls back.

FDA's recruitment guidance is explicit that this step matters. It calls direct advertising "the start of the informed consent and subject selection process." It also notes that the first contact is often a receptionist following "a script to determine basic eligibility," and that the IRB should make sure those procedures protect prospective subjects. See FDA's Recruiting Study Subjects page.

That has three consequences for clinical trial prescreening AI.

1. The script is a regulated artifact. Treat it like a protocol document, with versions and approvals.

2. The agent is doing subject selection. Fairness and accuracy are ethics questions, not only quality metrics.

3. What the agent collects is sensitive. FDA says an IRB should understand how pre-consent personal information is handled, including whether non-eligible names are kept.

Our healthcare AI receptionist QA guide covers the general front-desk case. This post covers the narrower case where the call outcome is an eligibility signal.

Pre-screening is not consent

Keep two things apart in the design and in the tests.

Pre-screening: a brief, scripted exchange that checks basic eligibility and routes the person. Informed consent: the process in which a prospective subject gets the information needed to decide, with time to decide and no coercion.

The consent rule at 21 CFR 50.20 says the investigator must obtain "legally effective informed consent." It says the process must give time to decide and minimise coercion or undue influence. It requires language the subject can understand. It also bars exculpatory language, "whether oral or written."

For research under the HHS rule, 45 CFR 46.116 adds that consent "must begin with a concise and focused presentation of the key information." Whether that rule governs your study depends on funding and institution. Ask your IRB.

FDA's screening guidance draws a second line. Research-only screening procedures need consent first. FDA's Screening Tests Prior to Study Enrollment says investigators "may discuss availability of studies and the possibility of entry" before consent. It also says clinical screening for research eligibility is part of subject selection and needs IRB oversight.

The practical rule: your agent may ask what the IRB-approved script lets it ask. It may not drift into explaining risks, comparing the study with the caller's current treatment, or collecting extra health detail "to be helpful." Your tests should try to push it there.

Where eligibility calls go wrong

Most failures are not dramatic. Eligibility criteria capture goes wrong in small ways, with large downstream effects. Here are the categories we would test first, using a fictional study throughout.

Fictional study for examples: adults 18 to 75 with type 2 diabetes. Inclusion: HbA1c 7.0% to 10.5%, on a stable metformin dose for at least 90 days. Exclusion: pregnancy, current insulin use, a heart attack in the last 6 months. This is not a real protocol.

Unit and format errors

Callers say "my A1c is fifty-three." That is 53 mmol/mol, which is about 7.0%. A script that expects a percentage may store "53." The record then shows a value far outside the 7.0 to 10.5 window. The person gets a screen failure on a unit error. Nobody ever knows.

The same goes for dates. "March of last year" is not a date. "Two months ago" depends on the call date. Weight in pounds versus kilograms is another classic.

Negation and compound questions

Exclusion criteria are phrased as "any history of." A caller answering "no" to "Have you ever had a heart attack?" is clean. A caller answering "no" to "Do you have no history of heart attack?" is a coin flip for the model. Compound questions are worse: "Are you pregnant or planning to become pregnant, or breastfeeding?" gets one "no" that covers three criteria, or one "yes" that covers none.

Stability windows and medication names

"Stable dose for 90 days" fails in two ways. The agent records "takes metformin: yes" and drops the duration. Or the caller says "I changed it a while back" and the agent records a date. Brand and generic names add a layer. See our post on drug name accuracy in refill calls for the mechanism.

Who is actually speaking

Caregivers answer for parents. A spouse picks up. A teenager calls about a study with an adult-only age range. The agent has to know whose answers it is capturing. If the caller is a caregiver, the record must say so. It must not merge the caregiver's age with the patient's condition.

Script-version drift

The IRB approves script v3. The prompt in production still has a v2 question. Or an engineer rewords a question to "sound friendlier" and changes what it asks. Because every prompt change ships to live callers, version control and regression tests are the only controls. Our prompt-change shipping guide covers the mechanics.

Scope creep into advice and promises

"Would this study help my diabetes?" "Will I get the new drug?" "Is it safe?" FDA's recruiting guidance says ads should not claim, explicitly or implicitly, that an investigational product is safe or effective. A voice agent improvising an answer would break that rule in speech. The right behaviour is a fixed deflection and a handoff to staff.

Why field-level accuracy is the number that matters

Call-level metrics hide the problem. "The call completed" says nothing about whether twelve fields were all right. Fields multiply.

If a script captures 12 criteria and each is captured correctly with probability p, the chance all 12 are right is p to the 12th power. This is an illustrative calculation, not a measured result.

A 98% field accuracy sounds excellent in a vendor demo. At 1,000 completed pre-screens it means about 215 records carry at least one wrong value. Some errors send an ineligible person to a site, which wastes coordinator time. The costlier errors go the other way. An eligible person is screen-failed on a bad value and never called back. That loss is silent.

Track errors by direction. A false screen-fail and a false pass have different costs and different review paths. Report them separately.

For more on how intake metrics should be defined, see metrics for a healthcare intake voice agent.

The boundary: capture, route, never decide

A workable design splits the agent's job into three lanes. The lane labels are our suggestion, not a regulatory term.

• Capture. Ask the approved questions. Read back the values that gate eligibility. Store each answer with the raw phrase, the normalised value and a confidence flag.
• Route. Apply the approved logic to produce one of three states: possible match, not a match for this study, or unclear. "Unclear" goes to a person. The agent hands off to site staff to book a consent conversation.
• Never. No consent. No risk or benefit statements. No advice on stopping or changing medication. No promise of a spot. No payment or stipend claims unless the approved script includes them verbatim.

The middle state matters most. Many silent failures come from forcing a binary. When the agent cannot resolve "stable dose," the safe output is "unclear, route to coordinator," not a guess.

This is also where fairness enters. 21 CFR 56.111(a)(3) requires that selection of subjects be equitable, and it asks IRBs to consider vulnerable groups. An agent that mishears accented speech more often will screen-fail some people more often. That is a selection effect. Our accent robustness post shows how to measure it.

Test design for clinical trial prescreening AI: a matrix, not a script

Clinical trial prescreening AI needs a test matrix with two axes. The first axis is criteria, grouped by type. The second is conditions.

Criteria types: numeric lab values, dates and windows, medication and dose, negated history, pregnancy or lactation, age and identity of the speaker.

Conditions: accent and speech pace, background noise, interruption, long pauses, mid-call corrections ("no wait, it was two years ago"), caregiver speaking, a caller who refuses a question, a caller who asks for medical advice.

Each cell gets several test callers with known ground truth. For each call you know the true answer, the correct stored value and the correct routing state. That is what lets you score fields.

What to score on every call

A transcript-only check can pass most of these. It cannot hear the pause before "I think so," or the half-spoken correction under noise. Evalgent's approach is to judge from the audio as well as the transcript, so those cases surface as evidence on the call instead of a clean-looking text log.

How to test a clinical trial pre-screening agent

Follow these steps before any script version reaches live callers.

1. Freeze the approved script. Store the IRB-approved version ID next to the prompt version. Fail the build if they disagree.

2. Turn each criterion into a test case. Write the ground-truth answer, the correct stored value with units, and the correct route. Include boundary values: HbA1c exactly 7.0 and 10.5, one day short of the stability window, age 17 and 18.

3. Add the traps. Unit swaps, negated questions, compound answers, mid-call corrections, caregiver callers, refusals and out-of-scope questions.

4. Build caller profiles. Vary accent, pace, interruption level, response delay, background noise and language. Evalgent generates scenarios from an agent's prompt and flows and applies caller profiles like these.

5. Run synthetic calls over the real path. Use phone, SIP or WebRTC, the way live callers arrive. Use only synthetic identities and invented health details.

6. Score fields and routes. Compare each stored field with ground truth. Review every failure by listening, not only by reading.

7. Check the forbidden list. Ask the agent whether the study is safe, whether to stop a medication, and whether the caller will get the drug. Pass only if it deflects and hands off.

8. Set release gates. For example: zero scope violations, zero unit errors on gating fields, and a field-accuracy floor your team and IRB accept. These thresholds are suggestions, not standards.

9. Re-run on every change. A reworded question, a model update or a TTS swap can shift capture. Our LLM update regression post covers the update case.

10. Monitor live calls. Sample recordings weekly, mapped to scenarios, and review flagged failures with a human.

How many test calls

Use the "rule of three" as a rough guide. If a scenario runs n times with zero failures, the 95% upper bound on its true failure rate is about 3/n. That is illustrative math for planning, not a guarantee.

Not every scenario needs 299 runs. Put the heavy volume on the gating criteria and the scope-violation cases, where one failure matters most. Give lighter criteria fewer.

Compliance questions for clinical trial prescreening AI

This section is not legal advice. It names the rules that usually come up, so you can ask better questions of counsel and your IRB.

IRB review of the script and the agent. FDA's page says IRBs should review recruitment methods and materials. Later recruitment materials may be handled as amendments. Ask whether the IRB wants the voice, the wording, the deflection lines and the handoff language reviewed. Ask what a prompt edit counts as.

Health information before consent. For covered entities, 45 CFR 164.512(i)(1)(ii) allows reviews preparatory to research only with representations that the use is solely to prepare a protocol, that no protected health information is removed, and that it is necessary for the research. Callers phoning in about an ad are a different flow from a site mining its own records. Ask counsel which path applies.

Outbound calls. If your agent calls people, 47 CFR 64.1200 matters. The FCC ruled on February 8, 2024 that AI-generated voices are "artificial" under the TCPA, as summarised by Akin Gump. Calls to cell phones using an artificial voice generally need prior express consent, and advertising or telemarketing needs prior express written consent. The healthcare exemption in paragraph (a)(9)(iv) is narrow and excludes solicitation. Whether recruitment fits is a question for counsel. On revocation, the FCC adopted new rules on September 30, 2026 and released them October 1, 2026. They had not taken effect as of October 7, per Kelley Drye.

Inbound versus outbound. Many of these concerns apply differently when the person calls you after seeing an ad. The testing needs do not change.

For the wider healthcare rule set, see voice agent testing for healthcare and policy and guardrail test cases for regulated voice agents.

Monitoring after launch

For clinical trial prescreening AI, pre-launch tests cover what you thought of. Live callers cover the rest. They give answers like "I was told by my cardiologist it's fine," or they switch to Spanish halfway through.

For AI screening calls, Evalgent's production monitoring takes call recordings your team pushes in. It transcribes them, maps them to test scenarios and judges them. It flags PII leakage, tool-call failures, latency spikes and policy failures, with fix recommendations. For pre-screening, that means a review queue of calls where the stored value and the audio disagree, plus custom metrics such as "gating field read back" or "scope violation" run on every call.

Close the loop. Every live failure becomes a new test case in the matrix. Version it with the script.

If you run many sites, treat each site as a tenant with its own script variants and criteria. Our multi-tenant testing post covers that. Before launch, run through the AI phone agent go-live checklist. To compare vendors on these points, use the healthcare voice agent vendor scorecard.

Frequently asked questions

Can an AI voice agent pre-screen patients for a clinical trial?

Yes, clinical trial prescreening AI is workable if the IRB approves the script and the agent stays inside it. FDA's recruitment guidance accepts scripted first contact for basic eligibility. The agent should capture answers and route people. It should not consent, advise or promise enrolment. Confirm your specific design with your IRB and sponsor.

Does an AI pre-screening call count as informed consent?

No. Consent under 21 CFR 50.20 requires information in understandable language, time to decide and minimal coercion. A short eligibility call does not do that. Treat pre-screening as routing. The consent conversation belongs to qualified study staff, who handle it under the approved process.

Does an IRB need to approve an AI pre-screening script?

FDA says IRBs should review the methods and materials used to recruit subjects, including receptionist scripts. A voice agent is a recruitment method. Ask your IRB what it wants to review: wording, voice, deflection lines, and how prompt edits are handled afterward.

How do you test an AI agent on inclusion and exclusion criteria?

Turn every criterion into a test case with known ground truth. Include boundary values, unit swaps, negated questions and mid-call corrections. Score each stored field, the routing state and the scope. Vary accents and noise. Re-run the whole set on every prompt, model or voice change.

What errors do AI eligibility calls make most often?

The common ones are lost units, mishandled negation, dropped stability windows, merged caregiver and patient answers, and compound questions answered with one word. Script-version drift is another. They are silent: the call sounds fine while the stored record is wrong.

Can the AI tell a caller they qualify for a study?

It should not. Eligibility is confirmed by study staff after consent and protocol procedures. Have the agent say a coordinator will follow up. Use the "possible match" state internally, and test that the agent never says "you qualify" or hints at a spot.

How many test calls do you need before going live?

It depends on the risk. With zero failures, the rule of three gives a rough 95% upper bound of 3 divided by the run count. About 59 clean runs bound a scenario near 5%, and about 299 near 1%. Spend more runs on gating fields and scope violations.

What happens to caller health data collected during pre-screening?

That depends on your design, your role and the law. FDA says IRBs should understand how pre-consent information is handled, including whether non-eligible names are kept and how records are disposed of. Decide retention up front, test that the agent does not repeat details, and ask counsel.

The bottom line

Clinical trial prescreening AI performs a subject-selection step, so the agent must capture approved answers accurately and hand everything else to people. Test field by field with synthetic callers, because the errors that cost patients a place in a study are the ones nobody hears.