Evalgent
Back to Blog
Voice AI Evaluation

Jev Use Cases for AI Voice Agents, and How to Use Jev

Deepesh Jayal
Updated
14 min read
Jev Use Cases for AI Voice Agents, and How to Use Jev
On this page

Jev is a new kind of model, and it is easy to misfile. It does not write sentences. It makes decisions. You hand it a "state" — a transcript, a message, some structured data — plus one or more questions, and it returns typed, probabilistic answers, each with a confidence number attached.

That shape fits voice agents unusually well. A single turn hides a dozen small decisions an LLM handles slowly and expensively: which intent, which slot, is this safe, is the caller angry, did the tool result help. Jev answers those in tens of milliseconds, at a fraction of the cost, and cannot invent an option that does not exist.

This post is the granular roundup. It breaks the live call into micro-decisions grouped into clusters. Then it shows the real payloads that answer many of them in one request. For the model background, read our pillar on Jev for voice agents. For the single highest-value runtime pattern, see routing and escalation with Jev.

What Jev is, in one line

Jev: TypeSafe AI's first "System One Model," launched September 15, 2026. It takes a state plus questions and returns typed probabilistic decisions with calibrated confidence. It does not generate text.

Jev answers three question types. A Choice picks one of up to 255 options and returns a probability per option plus confidence. A Score maps to ordered levels and returns a continuous score, a distribution, and confidence. A Noul returns a yes or no probability. TypeSafe reports it cannot hallucinate and posts a 0% type-error rate, because the schema constrains every output.

The other headline numbers, per TypeSafe's launch post: 70–500ms per call, roughly 40–200x faster than an LLM. Input costs $0.042 per million tokens and output is free, which TypeSafe frames as about 400x cheaper. You can ask many questions in one call, and adding questions barely moves latency or cost. Confidence is calibrated, so a reported 0.9 means what it says. Jev is trained with RLCD.

Two facts shape every use case below. Jev decides and scores; it does not produce language. And it reads text or structured state, not raw audio. Keep both in mind as you read.

How to use Jev, in one call

Using Jev comes down to one request with two parts: a state and a set of questions.

1. Get access. Install the SDK with `pip install typesafe-sdk` and set `TYPESAFE_API_KEY`, or call `POST https://api.typesafe.ai/v1/systemone` with model `jev-latest`. Jev is also listed on OpenRouter as `typesafe/jev-1.13`.

2. Build the state. Pass the text Jev should judge: the current turn, a transcript, or a JSON object. Keep it to what the decision needs, because accuracy drops as irrelevant state grows, per TypeSafe's jaggedness notes.

3. Ask typed questions. Add a Choice for "which one," a Score for "how much," and a Noul for "yes or no." Mix several in the same call.

4. Read the answers. Each answer comes back typed, with probabilities and confidence. Act on confident answers. Route low-confidence ones to an LLM or a human.

Where to use Jev first: offline scoring of finished calls, then live decisions once its answers match your human labels. The full payload and Python SDK example are below.

Two places Jev fits: runtime and evaluation

Group A is runtime — decisions made while the caller is on the line, where latency is the constraint. Group B is evaluation and QA — scoring calls after the fact, where consistency and cost per judgment matter most. Group B is Evalgent's home turf, and we will be specific about the line between using Jev as a scorer and running an independent evaluation.

Jev runtime use cases, as micro-decisions

Stop thinking of runtime Jev as a few big jobs. Think of it as a swarm of narrow questions, each a "smart if-statement" with a fixed set of outcomes. Below they are grouped into clusters. Each micro use-case is one narrow Jev question; the catalog table carries the compact list.

A map of micro use-cases Jev handles in a voice agent, grouped into understanding the turn, driving the next action, auth and safety, sentiment and escalation, outbound, and post-call labeling

Cluster 1: understanding the turn

  • Detect intent from the first utterance (Choice). Which intent is this: billing, cancellation, tech support, sales? A fixed option set means Jev cannot route to an intent outside your menu.
  • "Answering my question or changing topic?" (Choice). After the agent asks something, classify the reply as on-topic, a topic switch, or a clarification request. This keeps dialog state honest.
  • "Did the caller give the info I asked for?" — slot-fill confirmation (Noul). A yes/no on whether the last turn actually supplied the requested slot, before you try to parse it.
  • Resolve an ambiguous yes/no (Noul). "Yeah, I guess" or "sure, whatever" maps to a clean boolean with confidence, so downstream logic is not guessing.
  • Detect a self-correction (Noul). "No, the other card" — flag that the caller just retracted the prior value, so you overwrite the slot instead of appending.
  • "Is the caller confused by my last reply?" (Noul). Catch confusion early and rephrase, rather than repeating the same line louder.
  • Which slot are they filling? (Choice). When a caller volunteers data out of order, pick which field it belongs to: date, amount, account, name.

Cluster 2: driving the next action

  • Pick the next best question to ask (Choice). Given filled and empty slots, choose which one to collect next for the smoothest path to resolution.
  • Choose the matching FAQ/KB article to ground the answer (Choice). Select the right knowledge-base entry from a fixed list, so the reply is grounded before generation.
  • "Is this request in scope for the bot?" (Noul). A hard boundary check: if the ask falls outside supported tasks, hand off rather than improvise.
  • "Should I confirm before this irreversible action?" (Noul). Gate refunds, cancellations, and transfers behind an explicit confirmation decision.
  • "Did the tool result actually answer the need?" — post-tool check (Noul). After an API call returns, verify the result addresses the caller's request before reading it back.
  • Pick confirmation phrasing / reply tone (Choice). Select among pre-authored phrasings or a tone bucket that matches the moment.

Cluster 3: auth, risk, and safety

  • "Already authenticated?" (Noul). Before any account action, confirm the caller cleared verification earlier in the call, so you do not re-ask or leak.
  • Choose verification method by risk (Choice). Route to knowledge-based, OTP, or step-up verification depending on the sensitivity of the request.
  • Jailbreak / prompt-injection per turn (Noul). Score whether the current turn tries to steer the agent off-policy. Cheap enough to run on every turn, not just suspicious ones.
  • Tool-call risk gating (Noul). Ask whether an about-to-fire tool call is safe given the transcript and policy. Low confidence or "no" holds it for confirmation.

Cluster 4: sentiment and escalation per turn

  • Sentiment now (Score). Rate caller frustration on an ordered scale, turn by turn, so the agent can adapt before the caller asks.
  • "Had to repeat themselves?" (Noul). Detect that the caller is restating something already said — a strong early frustration signal.
  • Repeated failed attempts → escalate (Noul). Flag when the same task has failed more than once this call and should route to a human.
  • Confidence-gated escalation (Choice/Score confidence). Escalate not on keywords but on Jev's own low confidence — the model itself is unsure.
  • "Wants a callback vs hold?" (Noul). When escalation is due, classify the caller's stated preference so the handoff matches it.

Cluster 5: outbound-specific

  • "Human or voicemail?" — AMD from transcript cues (Noul). Read answering-machine detection off transcript signals like "leave a message after the tone," not just audio energy.
  • "Right moment to offer the upsell?" (Noul). Time the offer to a receptive turn instead of interrupting a complaint.

Cluster 6: post-call micro-labels

  • Auto wrap-up / disposition code (Choice). Assign a disposition — sale, no-answer, callback, do-not-call — the instant the call ends.
  • "Was a promise/commitment made?" (Noul). Flag calls where the agent committed to something, for follow-up and compliance.
  • "Does this need a follow-up?" (Noul). A binary that feeds your task queue without a human reading the transcript.

Every one of these is narrow, typed, and fast. The economics work because you do not pay per question the way an LLM prompt would. That is the point of the next section.

One call, many micro-decisions: the real payload

Here is the move that changes the math. You do not fire one Jev request per micro-question. You bundle several into a single call that shares one input-token cost. The state is the transcript snippet; the questions carry the micro-decisions.

One Jev call, many micro-decisions: a single call transcript plus several typed questions returns intent, frustration, needs-human, in-scope and self-correction answers in one parallel request, sharing one input-token cost

The endpoint is `POST https://api.typesafe.ai/v1/systemone` with model `jev-latest`. Note the field shapes: a Choice carries `criteria` as an object of options, a Score carries `criteria` as an ordered array, and a Noul carries only `instructions`.

POST https://api.typesafe.ai/v1/systemone

{
  "state": "Agent: I can help move that payment. Which card should I use?\nCaller: Use the... no, the other card, the Visa. And honestly this is the third time I've explained this.",
  "model": "jev-latest",
  "questions": {
    "intent": {
      "type": "choice",
      "instructions": "What does the caller want on this turn",
      "criteria": {
        "billing": "A dispute or question about a charge",
        "scheduling": "Move, reschedule, or change a payment date",
        "account_update": "Change a stored detail such as a card"
      }
    },
    "self_correction": {
      "type": "noul",
      "instructions": "The caller just retracted a value and gave a different one"
    },
    "in_scope": {
      "type": "noul",
      "instructions": "This request is something the payments bot is allowed to do"
    },
    "needs_human": {
      "type": "noul",
      "instructions": "The caller should be routed to a human agent"
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated the caller sounds",
      "criteria": ["Calm", "Frustrated but civil", "Very angry"]
    }
  }
}

The response returns one answer per question, each typed to its question. Choice answers carry the winning label, a confidence, and a probability per option. Score answers add a legend mapping levels to values. Noul answers return a boolean plus confidence. A top-level `usage` block reports token counts.

{
  "answers": {
    "intent": {
      "choice": "account_update",
      "confidence": 0.91,
      "probabilities": {
        "billing": 0.06,
        "scheduling": 0.18,
        "account_update": 0.76
      }
    },
    "self_correction": {
      "noul": true,
      "confidence": 0.95,
      "probabilities": { "true": 0.95, "false": 0.05 }
    },
    "in_scope": {
      "noul": true,
      "confidence": 0.89,
      "probabilities": { "true": 0.89, "false": 0.11 }
    },
    "needs_human": {
      "noul": false,
      "confidence": 0.62,
      "probabilities": { "true": 0.38, "false": 0.62 }
    },
    "frustration": {
      "score": 1.71,
      "confidence": 0.86,
      "legend": { "0": "Calm", "1": "Frustrated but civil", "2": "Very angry" },
      "probabilities": { "0": 0.03, "1": 0.23, "2": 0.74 }
    }
  },
  "usage": {
    "input_tokens": 174,
    "output_tokens": 18
  }
}

Read the economics off that `usage` block. Five micro-decisions came back from one call. They shared a single input-token cost of 174 tokens, and the output tokens are free. Firing five separate LLM prompts would multiply the input, add five round trips of latency, and cost five times as much. With Jev, the marginal cost of the sixth or tenth question is a rounding error.

The same call from the Python SDK

pip install typesafe-sdk
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient()  # reads TYPESAFE_API_KEY, defaults to jev-latest

state = (
    "Agent: I can help move that payment. Which card should I use?\n"
    "Caller: Use the... no, the other card, the Visa. "
    "And honestly this is the third time I've explained this."
)

result = client.system_one(
    state=state,
    questions={
        "intent": Choice(
            instructions="What does the caller want on this turn",
            criteria={
                "billing": "A dispute or question about a charge",
                "scheduling": "Move, reschedule, or change a payment date",
                "account_update": "Change a stored detail such as a card",
            },
        ),
        "self_correction": Noul(
            instructions="The caller just retracted a value and gave a different one",
        ),
        "in_scope": Noul(
            instructions="This request is something the payments bot is allowed to do",
        ),
        "needs_human": Noul(
            instructions="The caller should be routed to a human agent",
        ),
        "frustration": Score(
            instructions="How frustrated the caller sounds",
            criteria=["Calm", "Frustrated but civil", "Very angry"],
        ),
    },
)

if result.answers["self_correction"].noul:
    overwrite_slot("card", parse_card(state))

if result.answers["frustration"].score >= 1.5:
    soften_tone()

The client reads `TYPESAFE_API_KEY` from the environment and defaults to `jev-latest`. Jev is also reachable through LangChain: the `langchain-typesafe` package exposes a `TypeSafeClassifier` for teams building their harness there.

Jev use cases in evaluation and QA

This is where an independent evaluator lives. The same properties that make Jev good at runtime decisions — typed output, calibrated confidence, near-zero cost per judgment — make it a strong scoring engine for voice agent evaluation and testing.

Calibrated scoring as an LLM-judge alternative

Teams grade transcripts with an LLM-as-judge. It works, but judges are inconsistent and slow. LangChain's testing, reported in "Can Jev Be a Better Agent Evaluator?", found Jev's quality-score variance was 92–913x lower than GPT-5.6 Luna, Terra, and Claude Sonnet 4.6 as judges. It averaged 0.44 seconds and $0.00035 per call — $0.34 total against $28.17 for the Claude run — and it matched the human labels on all 500 decisions. More consistent, faster, cheaper. A Score question replaces the judge prompt for anything that maps to an ordered rubric. We go deeper in Jev as a judge for voice agents.

Scoring every call, not a sample

Here the economics flip. Most teams grade a small sample because judging is expensive. When judging is nearly free and fast, the sample stops making sense. This is a Jevons paradox effect: make judgment cheap enough and you consume far more of it. You can score 100% of calls on every rubric instead of 2% on one. Full coverage changes what you can catch — rare failures stop hiding in the unsampled 98%. See score every call, not a sample.

Policy and disclosure adherence scoring

Regulated calls require specific disclosures and forbidden phrases. A Noul question per rule asks: was the recording disclosure spoken, was the mini-Miranda read, did the agent avoid a prohibited promise? You run every rule on every call in one Jev request. This complements deterministic checks and verifiable rewards, which handle the parts a simple assertion can prove.

Hallucination and grounding detection

A Noul question asks whether a claim in the transcript is grounded in the retrieved context or the knowledge base. When the agent states a policy detail or a price, Jev flags whether the source supports it. Because it is cheap, you can check every factual assertion, not a spot sample. This gives your grounding metric a per-call denominator instead of an estimate.

Inferred CSAT and sentiment

Post-call satisfaction surveys have single-digit response rates. A Score question infers a satisfaction level from the transcript for every call. You still validate against real survey responses where you have them, but you get a dense signal in between. The distribution Jev returns tells you how confident the inference is, so you can weight it.

Auto-labeling outcomes

Containment and resolution labeling is slow, manual work. A Choice question labels each call: contained, escalated, resolved, abandoned, callback. A Noul question can settle a binary like "was the issue resolved." This auto-labels your whole call log, turning raw transcripts into an analyzable dataset without a human touching each one. It pairs with runtime disposition coding covered in Jev call monitoring.

Red-team result scoring

After a red-team run, someone has to grade thousands of adversarial transcripts: did the agent hold the line or break? A Noul question scores each attempt as a pass or a break, with confidence. Jev turns a manual grading backlog into a batch job that finishes in seconds and costs cents.

Routing low-confidence items to human review

Jev reports calibrated confidence on every score, which gives you a clean triage rule. High-confidence judgments post automatically. Low-confidence ones route to a human reviewer. This is human-in-the-loop evaluation done right: humans spend their time only where the model is genuinely unsure, not re-checking obvious passes.

An evaluation-side payload

The same one-call-many-questions pattern works on a completed transcript. Here a rubric bundles several Score and Noul questions for a finished call.

POST https://api.typesafe.ai/v1/systemone

{
  "state": "<full call transcript>",
  "model": "jev-latest",
  "questions": {
    "quality": {
      "type": "score",
      "instructions": "Overall handling quality of the call",
      "criteria": ["Poor", "Adequate", "Good", "Excellent"]
    },
    "disclosure_read": {
      "type": "noul",
      "instructions": "The recording disclosure was spoken before any account detail"
    },
    "resolved": {
      "type": "noul",
      "instructions": "The caller's issue was resolved on this call"
    },
    "inferred_csat": {
      "type": "score",
      "instructions": "Likely caller satisfaction at hang-up",
      "criteria": ["Dissatisfied", "Neutral", "Satisfied"]
    }
  }
}

Four rubric judgments, one call, one shared input cost — run across 100% of your call log instead of a thin sample. For the metric-by-metric breakdown of what to score, see the Jev voice agent metric catalog.

Micro use-case catalog

The table maps each standout micro use-case to its cluster, its Jev question type, and what it replaces or complements. Choice picks one option, Score rates an ordered level, Noul answers yes or no.

Micro use-caseClusterJev question typeReplaces / complements
Detect intent from first utteranceRuntimeChoiceSmall LLM classifier or regex router
"Answering me or changing topic?"RuntimeChoiceLLM dialog-state reasoning
Slot-fill confirmationRuntimeNoulBrittle parse-and-hope logic
Resolve ambiguous yes/noRuntimeNoulKeyword matching
Detect self-correctionRuntimeNoulManual retraction handling
Pick next best questionRuntimeChoiceHard-coded dialog tree
Choose FAQ/KB article to groundRuntimeChoiceLLM retrieval-choice step
"Is this in scope for the bot?"RuntimeNoulLoose prompt guardrail
Confirm before irreversible actionRuntimeNoulHand-written safety rules
Post-tool "did it answer the need?"RuntimeNoulUnchecked tool output
Already authenticated?RuntimeNoulSession-flag bookkeeping
Choose verification method by riskRuntimeChoiceStatic auth rules
Jailbreak / injection per turnRuntimeNoulSeparate moderation model
Tool-call risk gatingRuntimeNoulHand-written safety rules
Sentiment nowRuntimeScoreStandalone sentiment model
Repeated failures → escalateRuntimeNoulKeyword escalation triggers
Confidence-gated escalationRuntimeChoice / Score confidenceKeyword escalation triggers
Human or voicemail (AMD cues)RuntimeNoulAudio-only AMD
Right moment to upsellRuntimeNoulFixed script timing
Auto wrap-up / disposition codeRuntime → post-callChoiceManual call labeling
Promise / commitment made?Runtime → post-callNoulManual QA review
LLM-judge scoringEvalScoreLLM-as-judge prompt
Policy / disclosure adherenceEvalNoulManual QA review
Hallucination / groundingEvalNoulLLM grounding check
Inferred CSATEvalScorePost-call surveys
Red-team result scoringEvalNoulManual transcript grading

How to choose your first Jev use case in a voice stack

Do not try to adopt every micro use-case at once. Pick one, prove it, then expand. Here is the order we recommend.

1. Start in evaluation, not runtime. Scoring is offline and low-risk. A bad score costs you nothing on a live call. Runtime decisions touch real callers, so earn confidence first.

2. Pick a decision you already make with an LLM. Look for an existing LLM-as-judge prompt or a classification step. That is your baseline to beat on cost, speed, and consistency.

3. Choose the question type by output shape. One of a fixed set is a Choice. An ordered rating is a Score. A yes-or-no is a Noul. Match the type to the decision, not the other way around.

4. Bundle related micro-questions into one call. Once you have one working, add more questions to the same request. They share the input cost, so coverage compounds almost for free.

5. Run Jev and your baseline side by side. Score the same calls with both. Compare variance, latency, and cost per judgment. Keep your existing labels as ground truth.

6. Check calibration on your data. Confirm a reported 0.9 confidence maps to roughly 90% accuracy on your calls. TypeSafe reports calibrated confidence); verify it holds before you trust the number.

7. Set a confidence threshold for human review, then promote. Decide where low-confidence items route to a person. Once a use case is proven offline, move it to runtime, starting advisory before it gates actions.

Where Evalgent fits

Jev is a powerful scorer. It is not an evaluation program. You can use Jev as a calibrated judge, and it will be faster and more consistent than an LLM. But the vendor's own scorer grading the vendor's own agent is not independent evidence.

That is the gap we fill. Evalgent runs independent evaluation on your own calls and your own test cases, with methods you control and results you can defend. We use calibrated scorers where they help, alongside deterministic verifiable rewards and human review, and we keep the latency) and cost math honest. If Jev is the engine, an independent audit is the dyno. The full model background lives in our Jev for voice agents pillar, and the technical docs are here. Jev is in early access, so treat every number as a starting point to verify, not a settled benchmark.

Jev can't abstain: always include an escape option

Jev always returns one of the options you define. It has no built-in "I don't know." When a call transcript holds no evidence for any option, such as a two-word turn, a dropped call or a caller talking about something else, Jev still picks the least-wrong option, and that label looks just like a confident answer. TypeSafe's own Choice docs recommend adding an "other" or "none of the above" option when the list may not cover every input.

Every micro use case above needs this. A disposition, a sentiment band or a compliance tag computed on a 20-second dropped call is a guess with a label on it unless the option set lets Jev say the evidence isn't there.

Four rules keep this from hurting you on live calls:

  • Add an explicit escape option such as `not_enough_information`, `not_applicable` or `none_of_the_above` to every Choice question where evidence can be missing. Describe it as concretely as the real options, because Jev reads only the descriptions, not your option keys.
  • Ask whether the evidence exists first. A Noul like "does the caller state a reason for calling?" before the Choice question separates "no evidence" from "evidence for option B".
  • Check the distribution, not just the label. In one independent edge-case test, a forced choice with no valid option still returned a pick at 0.52 probability but only 0.04 confidence (jev-exploration). The numbers flagged the problem; the label did not. Set your human-review cutoff from your own labeled calls, because an independent study on data unlike Jev's training found its confidence can run high (jev-ood-calibration).
  • Track the escape-option rate. If it climbs, your transcripts, STT or option set have drifted away from what callers actually say.

Frequently asked questions

What are the use cases of Jev for voice agents?

Jev use cases split into two groups. At runtime it answers narrow micro-decisions: intent, slot-fill confirmation, self-correction, in-scope checks, auth, jailbreak detection, risk gating, sentiment, escalation, and disposition coding. In evaluation: LLM-judge-style scoring, policy adherence, hallucination detection, inferred CSAT, outcome labeling, red-team scoring, and low-confidence triage.

How do you use Jev for AI voice agents?

To use Jev, send one request with a state (a transcript, message, or JSON) plus one or more questions, via the `typesafe-sdk` Python client or the `/v1/systemone` endpoint. It returns typed answers with calibrated confidence. Pick Choice, Score, or Noul per decision and bundle them in one call. Start in offline evaluation, then promote proven decisions to live calls.

Can Jev replace an LLM judge for voice agents?

Jev can replace an LLM judge for anything that maps to a Choice, Score, or Noul. LangChain's study found Jev's quality-score variance was 92–913x lower than GPT-5.6 and Claude judges, at 0.44s and $0.00035 per call, matching human labels on all 500 decisions. It cannot write freeform critique, so keep an LLM for narrative feedback.

What Jev question type do I use for routing?

Use a Choice question for routing. It picks one of up to 255 options and returns a probability for each plus a confidence score. Because the option set is fixed to your intents or tools, Jev cannot route to something outside your menu. Use the confidence value to trigger escalation when the model is unsure.

How do you use Jev to score every voice agent call?

Because Jev output is free and input is roughly 400x cheaper than an LLM, per-call judgment cost drops far enough to grade 100% of calls. Send each transcript with your rubric questions bundled in one call. This is a Jevons paradox effect: cheap judgment means you consume far more of it, replacing thin samples with full coverage.

Can Jev detect jailbreaks in voice agents?

Jev can score jailbreak and prompt-injection risk with a Noul question that returns a yes-or-no probability per turn. Because it runs in 70–500ms and adds little cost, you can check every turn, not just suspicious ones. A high probability triggers a refusal, a safe fallback, or an escalation to a human agent.

How does Jev pick a reply with no LLM in the loop?

In an emerging architecture, Deepgram Flux feeds Jev, which feeds text-to-speech, with no LLM generating text. Jev uses a Choice question to pick a reply from a pre-written response bank in about 300ms and reports confidence. This works only for bounded replies. Open-ended conversation still needs an LLM or a hybrid design.

What can Jev not do for voice agents?

Jev does not generate language — it decides and scores, so freeform replies need a response bank or an LLM. It reads text and structured state, not raw audio, so it works on transcripts rather than the waveform. It is in early access. Treat published speed, cost, and quality figures as starting points to verify on your data.

The bottom line

Seen up close, Jev's use cases for voice agents are not a short list of big jobs. They are dozens of narrow micro-decisions per turn, bundled into one typed call that shares a single input cost, plus cheap and consistent evaluation scoring. Use it as an engine for both, but run your production evaluation independently, on your own calls, so the results hold up under scrutiny.

Want to see where Jev fits in your stack and how independent evaluation catches what a self-graded scorer misses? Book a demo and we will score a sample of your real calls.

Related Articles