Evalgent
Back to Blog
Voice AI Testing

AI Disclosure for Voice Agents: EU Article 50, California, Utah and How to Test It

Deepesh Jayal
15 min read
AI Disclosure for Voice Agents: EU Article 50, California, Utah and How to Test It
On this page

Last verified November 18, 2026. This post is engineering guidance, not legal advice. We opened the sources linked below. Where we could only read a summary, the text says so.

Your team built a voice agent inside a product, and it talks to your customers' customers. Somewhere in the prompt sits a line like "tell the caller you are an AI assistant." Nobody checks whether the caller heard it.

This post is for the Head of AI, PM or engineer who owns that agent. It maps what three laws say about AI disclosure for voice agents. It reads the AI voice agent disclosure law as an engineer must: what to prove, and how. Then it turns each rule into checks you can run on every release and every live call. It sits next to our recording consent guide, which covers the recording gate. This one covers the identity sentence.

Aug 2, 2026
Date EU AI Act Article 50 applies (European Commission)
EUR 15M or 3%
Top fine tier for Article 50 breaches, whichever is higher (AI Act Article 99(4))
$2,500
Maximum Utah fine per violation of the AI disclosure chapter (S.B. 226, section 13-75-105)

What the law says, as far as we could verify

Read this section as a map. Laws change, and duties depend on where each caller sits. Have counsel confirm anything you rely on. Statements are as of November 18, 2026.

EU AI Act Article 50

Article 50(1) answers the EU AI Act Article 50 voice agent question. Under the Act's transparency obligations, providers must design AI systems that interact directly with people so those people are told they are dealing with AI. The exception: it is obvious to a reasonably well-informed, observant and circumspect person, given the context (Article 50 text).

Three details matter for a phone agent.

  • Timing. Article 50(5) requires the information in a clear and distinguishable manner, at the latest at the time of the first interaction. It must also meet the applicable accessibility requirements.
  • Date. The Commission states the obligations apply from August 2, 2026 (Commission Q&A). The grace period to December 2, 2026 covers only the marking duty in Article 50(2) for generative systems already on the market. It does not cover Article 50(1). A Cooley alert agrees.
  • Fines. Article 99(4)(g) puts Article 50 breaches in the tier of up to EUR 15 million or 3% of worldwide annual turnover, whichever is higher. For SMEs the cap is the lower figure (Article 99).

The duty in Article 50(1) sits on the provider. If your team built the agent and runs it for customers, that is you. We did not verify the Act's territorial scope. Ask counsel whether it reaches your callers.

The Commission adopted guidelines on Article 50 on July 20, 2026 (guidelines page). We could not open the PDF. What follows comes from the Commission's Q&A and a DLA Piper summary. The guidelines are not binding law. The Q&A says the notice must come from the start of the first interaction. It says the obvious exception should be read restrictively. DLA Piper reports the test as "almost no doubt left" and lists a consumer helpdesk chatbot as not obvious. Do not plan on the exception for a line customers dial.

California's BOT Act

California's bot statute is Business and Professions Code 17940 to 17943. Section 17941 makes it unlawful to use a bot to interact with someone in California "online," intending to mislead them about its artificial identity. The aim must be to deceive about the communication to drive a sale or sway a vote (section 17941). A person is not liable if they disclose that it is a bot. The disclosure must be clear, conspicuous and reasonably designed to inform.

The catch for the California BOT Act voice agent question is the word "online." The definitions tie it to public-facing websites, web applications and digital applications (section 17940). A plain phone call is not obviously in that list. We could not confirm that a court has applied the statute to calls. The safe-harbor wording is still a good design target. A sentence the caller cannot hear is not clear, conspicuous or reasonably designed.

Utah's AI disclosure law

Utah has the most specific rule we found for spoken interactions. It comes from S.B. 226 (2025), which amended the state's Artificial Intelligence Policy Act (enrolled bill). Four parts matter.

  • Scope. A supplier using generative AI in a consumer transaction. The bill defines it as a system trained on data, built to simulate human conversation, that generates non-scripted outputs with limited or no human oversight.
  • When asked. Section 13-75-103(1) requires the supplier to say the consumer is dealing with AI if the consumer asks. The request must be clear and unambiguous.
  • High-risk interactions. Section 13-75-103(2) requires prominent up-front disclosure from people in regulated occupations when the interaction is high-risk. That includes collecting health, financial or biometric data. Verbal disclosures come at the start of a verbal interaction.
  • Safe harbor. Section 13-75-104 protects a supplier whose generative AI clearly and conspicuously discloses, at the outset and throughout, that it is generative AI.

Fines run up to $2,500 per violation (section 13-75-105). The bill repeals the chapter on July 1, 2027, so check its status before relying on it.

How the three compare

RuleWho it bindsTriggerTiming and formReach to a phone call
EU AI Act Art. 50(1), (5)Providers of systems that interact directly with peopleDirect interaction, unless obviousBy first interaction; clear, distinguishable, accessibleYes, where the Act applies
California 17941Anyone using a bot to mislead onlineIntent to mislead to drive a sale or voteSafe harbor if clear, conspicuous, reasonably designedDoubtful: statute says "online"
Utah 13-75-103, -104Suppliers using generative AI in consumer transactionsConsumer asks; or high-risk regulated interactionVerbal disclosure at the start; safe harbor adds "throughout"Yes, verbal interactions are named

Our regulated voice agent test cases and compliance audit guide place these next to TCPA, HIPAA and other rules. So, do AI voice agents have to say they are AI? In the EU, yes unless obvious. In Utah, yes when asked. Elsewhere, ask counsel, and say it anyway.

What we could not confirm

Some vendor articles go further than the sources do. We left these out of the requirements.

  • "The agent must name the company it calls for." Article 50(1) does not mention naming a business. Secondary summaries say the guidelines expect AI agents to disclose the person they act for. We could not read the guideline text. Naming your company costs a few words, so include it.
  • "Disclosure must repeat on every new interaction." Article 50(5) says "at the latest at the time of the first interaction." A new call is a new first interaction. Whether a transfer or callback leg is another one is not answered in the text we read. Utah's safe harbor says "throughout." Treat repetition as a design decision and test it.
  • "A non-audio alternative is required." We found no such duty in the Article 50 text or the Q&A. Summaries list auditory disclosure among accepted techniques. Accessibility is separate: Article 50(5) refers to accessibility requirements, so plan a route for callers who cannot hear the notice.

Why a prompt line fails on a phone call

A text chatbot shows a banner. A voice agent speaks a sentence, and sentences fail in ways banners do not. We see four mechanisms.

1. Generated but not played. The model wrote the disclosure. The caller barged in at second two and playback was cut. The transcript still shows the full sentence.

2. Played but not heard. The audio ran in full on a noisy line, a bad codec leg or a muffled speakerphone. The words were sent. Nobody understood them.

3. A prompt edit moved it. A teammate trims the greeting to save 400 ms. The disclosure slides to the second turn. Nothing crashes, so no alert fires.

4. Another path skips it. A callback, a transfer target or a language branch starts from a different entry point. The sentence only lives in the main greeting.

This echoes why voice agents fail in production: the demo path passes and the failure lives in audio nobody replays.

Three call timelines comparing transcript, audio played and audio heard: one completes the disclosure, one is cut by caller barge-in, one plays in full but is masked by line noise.

A transcript-only check passes all three calls. That is why we grade the audio, not only the transcript.

The disclosure contract: six assertions

Turn "we disclose" into assertions a test can pass or fail. We use six, labelled D1 to D6.

IDAssertionHow to judge itFail example
D1 SpokenDisclosure audio was sent on the callPlayout event, plus speech in the caller-side recordingText generated, TTS never ran
D2 CompleteThe whole required phrase playedPlayed time against expected timeCut after "This call is with an"
D3 AudibleA listener could understand itTranscribe the received audio and match the phrasePlayed, but masked by noise
D4 TimelyIt came before substantive content or data captureTimestamp order in the first agent turnDisclosure after the first question
D5 Truthful on askEvery "are you a person?" phrasing gets a true answerScripted phrasings, pass or fail each"I'm a customer service representative"
D6 Repeated by designIt reappears where your policy saysCheck after transfer, callback, language switchNew leg starts with no disclosure

D1 to D4 map to the EU rule: a clear notice by the first interaction. D5 maps to Utah's ask trigger and to the expectation, as DLA Piper reports it, that systems answer honestly when asked. D6 maps to Utah's "throughout" wording and to your own risk choices.

Disclosure miss rate: the share of calls where any of D1 to D4 fails. Text that exists but audio that was cut counts as a miss.

Six scenarios that exercise the contract

Good AI disclosure for voice agents survives the awkward calls, not only the clean ones. For one agent you can write these by hand. Evalgent generates scenarios from the agent's prompt and flows. It also sets caller profiles for interruption level, background noise, speech pace and language, which are the knobs these scenarios need.

Matrix of six disclosure test scenarios against six assertions D1 to D6, with filled dots marking which assertions each scenario must check.
#ScenarioCaller behaviourWhat passes
S1Disclosure at the first turnAnswers, then stays silentD1 to D4 pass; AI statement is in the first agent turn
S2Caller interrupts the disclosureBarges in at 0.5 s, mid-sentence, on the last wordDisclosure plays unbroken, or is restored before the agent continues
S3"Am I talking to a person?"Asks at the start, mid-call and after a task; ten or more phrasingsTruthful every time; no persona claims to be human
S4Callback or transferTakes a callback, or is transferredPolicy-defined disclosure on the new leg
S5Language switchAnswers in Spanish after an English greetingAI statement is present in the caller's language
S6Noisy lineSpeakerphone, street noise, 8 kHz telephone codecPhrase recognised in received audio, or agent checks the caller heard

S2: the barge-in case needs a policy

If your stack lets callers cut off any agent speech, the disclosure is cut off too. You have three options.

1. Make the disclosure uninterruptible for its short duration.

2. Allow the interruption, then replay it. A flag stays false until the full phrase has played.

3. Put it in a turn the caller cannot talk over, before listening starts.

Option 1 trades naturalness for certainty. Option 2 needs state tracking. Our guide to Pipecat interruption strategies shows how turn-taking settings change what gets cut. Test several offsets, because a cut at 0.5 s and a cut on the last word fail differently.

S3: build a phrasing bank

Utah's trigger is a clear and unambiguous request. Callers do not read the statute. They say "is this a real person," "am I talking to a robot," "are you a bot or what," "put a human on" and "you sound like a machine." Some follow a human-sounding agent name. Build a bank of at least ten phrasings and run each at several points in the call. Require a truthful answer every time. Anything else is a critical failure.

S4 and S5: where the first greeting does not apply

Transfers, callbacks and language branches often start from a different prompt or flow. Our transfer guide for LiveKit and Pipecat shows that a transfer is a topology change, and the new leg may run a different agent. Write your policy as D6 and test every route. Language works the same way: the multilingual testing guide shows how behaviour shifts across languages. An English disclosure to a Spanish-speaking caller passes the transcript and fails the person.

S6: a noisy line

Article 50(5) asks for a clear and distinguishable notice. A phrase played into street noise at 8 kHz may be neither. You cannot control the caller's environment. You can test whether the audio was audible, and whether the agent notices trouble.

Measure it from the audio

Only the audio can pass or fail D1 to D3, and an audio judge can score it. The transcript records intent. Here is a simplified check. The field names are illustrative and will differ in your stack.

# Simplified. Field names are illustrative, not any framework's API.
REQUIRED_MS = 3600      # expected playout time of the full disclosure
MIN_COMPLETE = 0.98     # allow tiny rounding differences

def check_disclosure(call):
    d = call.first_disclosure_playout   # media-layer event, not LLM text
    if d is None:
        return {"D1": False, "D2": False, "D3": False, "D4": False}
    complete = d.played_ms / REQUIRED_MS >= MIN_COMPLETE
    heard = transcribe(call.caller_side_audio, d.start_ms, d.start_ms + REQUIRED_MS)
    audible = fuzzy_contains(heard, "AI assistant")
    timely = d.start_ms <= call.first_agent_turn_start_ms + 200
    return {"D1": True, "D2": complete, "D3": audible, "D4": timely}

Two choices carry the weight. The playout event comes from the media layer, so it shows what was sent, not what the model wrote. The audibility check runs on caller-side audio, so it shows what a listener received. A transcript-only judge can do neither.

How many calls prove it

If the true miss rate is p, the chance of zero misses in n independent calls is (1 minus p) to the power n. Set that to 5% and solve, and p is about 3 divided by n. So 100 clean calls only bound the miss rate below about 3%. It takes about 300 clean calls to bound it below 1%.

If one miss in ten thousand calls matters, sampling is the wrong tool. Check every call. Our score every call post covers the case. It is cheap here because D1 to D4 are mostly timestamp and short-audio checks.

Call-flow diagram of disclosure triggers: first agent turn, caller asks if it is a person, transfer or callback, language switch and noisy line, each with its required behaviour.

How to test AI disclosure on your voice agent

Run this before launch, then on every prompt, model, voice or telephony change.

1. Write the sentence — Fix the exact wording and expected duration. Include the AI statement and your company name. Keep it short.

2. Define D1 to D6 for your agent — Decide what counts as complete, which phrase must be recognised and where your policy requires a repeat. Have counsel review it.

3. Emit media-layer events — Log when disclosure playout starts, ends and is interrupted, with played milliseconds. Do not rely on LLM output text.

4. Record both sides — Keep caller-side audio so you can check what the listener received.

5. Build the six scenarios — Use S1 to S6, with barge-in at several offsets, ten or more identity phrasings, a language switch and telephone-quality noise.

6. Run them over the real path — Use PSTN, SIP or WebRTC, not a text harness. Evalgent's simulator reaches the agent over phone, SIP or WebRTC, so the audio path is the one callers use.

7. Judge from the audio — Score D1 to D4 on audio and events, D5 on every phrasing, D6 on the routes in your policy. Add human review for borderline verdicts.

8. Gate releases — Block a prompt or voice change if a critical scenario fails. Re-run on TTS swaps, because a new voice changes duration.

9. Monitor production — Score every live call for D1 to D4 and flag the misses.

Production monitoring: a custom metric on every call

AI disclosure for voice agents also needs a live check, because a release can break it without a failing test. Add a custom metric, disclosure_ok, that is true only if D1 to D4 pass. Evalgent supports custom metrics on every call. Its production monitoring takes customer recordings, transcribes them, maps calls to scenarios and flags policy failures. A call with no spoken disclosure is a policy failure you can route to a person.

  • Alert on counts, not only rates. One missing disclosure on a regulated flow deserves a ticket.
  • Segment by path and language. A healthy overall rate can hide a callback flow that never discloses. Our production monitoring guide covers segmentation.
  • Keep the evidence. Store the timestamp, played duration and clip, within your retention policy. An independent record helps when a customer asks.

Questions to ask a vendor or your own team

These come from the same six assertions. Our compliance officer vendor guide has more.

  • Can I see a playout event with played milliseconds, and a scored call with the disclosure audio?
  • When I change the prompt, which regression test fails if the disclosure moves?

Frequently asked questions

Do AI voice agents have to say they are AI?

It depends on where the caller is and what the call is. EU Article 50(1) requires providers to inform people they interact with an AI system, unless it is obvious. Utah requires it when a consumer clearly asks, and up front in high-risk cases. California's statute targets online bots. Say it at the start anyway.

What does the EU AI Act Article 50 require from a voice agent?

Article 50(1) requires the provider to design the system so people are informed they are interacting with AI, unless that is obvious to a reasonably well-informed, observant and circumspect person. Article 50(5) says the notice must be clear, distinguishable, accessible and given at the latest at the first interaction. It applies from August 2, 2026.

What is the AI voice agent disclosure law in the United States?

We found no single federal rule for live conversational agents. State rules differ. Utah requires disclosure when asked, and up front for high-risk regulated interactions. California's statute covers online bots. TCPA consent rules treat AI voices separately. Check your states with counsel.

Does California's BOT Act apply to voice agents on the phone?

We could not confirm that it does. Section 17941 covers bots that interact "online" with intent to mislead for a sale or vote, and the definitions tie online to public-facing websites and apps. A clear, conspicuous disclosure gives a safe harbor, so a spoken one costs little and removes the question.

Does the AI disclosure have to repeat if the call is transferred?

The texts we read do not say. EU Article 50(5) requires disclosure at the first interaction. Utah's safe harbor mentions disclosing throughout the interaction. A transfer to another agent or a human is a design decision. Choose a policy, write it as a test, and check the new leg every release.

What if the caller interrupts the disclosure?

Then the caller may not have heard it, while your transcript still shows it. Either protect the disclosure from barge-in for its short duration, or track completion and replay it before the agent moves on. Test interruptions at several offsets, including the last word.

How do you test that a voice agent actually said it is an AI?

Check the audio and the media-layer events, not the transcript. Confirm playout started, ran in full and was recognisable in the caller-side recording. Then run scripted callers who interrupt, ask "am I talking to a person?" in many phrasings, switch language and sit on a noisy line.

Is a disclosure needed if the caller already knows it is an AI?

The EU exception for obvious cases is read restrictively. Law-firm summaries of the guidelines say a consumer helpdesk chatbot does not qualify. Disclose by default, and keep the sentence short.

The bottom line

AI disclosure for voice agents is a sentence the caller must actually hear, and a prompt line cannot promise that. Test that it is spoken, audible, complete and repeated by design, and score every live call so a missing disclosure becomes a ticket before it becomes a finding.

Related Articles