Jev for Real-Time Routing and Escalation in Voice Agents

Routing and escalation are two live-call jobs that must be fast and correct. An LLM can do both, but it is slow and its stated confidence rarely matches its accuracy.

Jev is built for this narrow job. It reads the call state and returns a typed decision with a confidence number. Below you will find a real request and response, Python for a three-band escalation policy, a threshold-tuning method, an audit log, and a warm handoff payload. Then we cover how Evalgent measures whether it was right.

What Jev is, and what it is not

Jev: TypeSafe AI's first System One model, launched September 15, 2026, in early access. It reads a state and a set of questions, then returns typed probabilistic decisions with calibrated confidence.

Jev is not a language model. It does not generate text. TypeSafe's summary is "unstructured state in, typed probabilistic decisions out." Founder Diogo Almeida previously worked at OpenAI on instruction-following research. Jev is trained with RLCD, Reinforcement Learning for Calibrated Decisions, not RLHF. A parallel sampler returns every answer in one query, not token by token.

TypeSafe documents three question types:

• Choice picks one of up to 255 options, with a probability per option plus confidence.
• Score takes ordered levels and returns a continuous score, distribution, legend, and confidence.
• Noul returns a yes or no probability.

The zero type-error figure holds by construction, not by testing. A route label outside your list cannot come back. TypeSafe reports 70 to 500ms end to end, versus 3 to 329 seconds for frontier LLMs. On its workflow evals, TypeSafe reports 193.6 times faster and 444.6 times cheaper than a reference average, and says that is likely the high end of real-world gains. See the launch note and docs.

Use an LLM for generation. Use Jev for fast structured decisions. For the wider picture, see Jev for voice agents and Jev vs LLMs for voice agents.

Routing with Jev: intent, model, and tool selection

Routing is a probabilistic classification task, and it maps directly to a Choice question.

Intent routing

A caller says, "I keep getting charged for the premium plan and I don't want it anymore." Is that billing or cancellation? A good router should hear both.

Here is a real Jev request for that turn. It asks a five-way Choice for the route and, in the same call, a Noul: does this caller need a human now? The option descriptions are your routing policy in plain language.

{
  "model": "jev-latest",
  "state": "Agent: Thanks for calling Northwind Mobile. How can I help?\nCaller: I keep getting charged for the premium plan and I don't want it anymore. This is the third month.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Pick the flow that best serves the caller's primary need on this turn.",
      "criteria": {
        "billing": "Questions or disputes about charges, refunds, invoices, or payment methods.",
        "cancellation": "Caller wants to end, downgrade, or stop renewing a plan or service.",
        "scheduling": "Booking, moving, or canceling an appointment or technician visit.",
        "support": "Something is broken: device, network, app, or login problems.",
        "general": "Anything else, including store hours, coverage, or new sales."
      }
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Does this caller need a human agent now? Yes if they ask for a person, show strong distress, raise a legal or safety issue, or the request is outside what an automated flow can resolve."
    }
  }
}

The endpoint is `POST https://api.typesafe.ai/v1/systemone`. The response follows the documented shape. The values are illustrative.

{
  "model": "jev-1.13.0",
  "answers": {
    "route": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.61,
      "probabilities": {
        "billing": 0.58,
        "cancellation": 0.34,
        "scheduling": 0.02,
        "support": 0.04,
        "general": 0.02
      }
    },
    "needs_human": {
      "type": "noul",
      "noul": 0.22
    }
  },
  "usage": {
    "input_tokens": 214,
    "output_tokens": 12
  }
}

Read three signals, not one. The top choice is billing, but confidence is only 0.61. Cancellation holds 0.34, close behind billing's 0.58, so the top-two margin is 0.24. Needs-human is 0.22. A router that only reads `choice` sends this caller to billing, which may be wrong. The distribution flags the ambiguity before the misroute.

Top-two margin: the gap between the highest and second-highest route probabilities. A small margin means the model is split between two specific routes, a strong cue for a clarifying question.

For more patterns, see Jev routing use cases.

Model routing

A "yes" or a ZIP code does not need a frontier LLM. The `langchain-typesafe` package ships a `ModelRouterMiddleware` that asks Jev a Choice over your models before each turn. Probabilities and confidence stay in the agent state for audit.

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import ModelChoice, ModelRouterMiddleware

router = ModelRouterMiddleware(
    choices={
        "fast": ModelChoice(
            model="openai:luna",
            criteria="Direct lookups, extraction, and localized changes.",
        ),
        "powerful": ModelChoice(
            model="openai:sol",
            criteria="Architecture and high-stakes decisions.",
        ),
    },
    instructions="Choose the least costly model that can complete the task.",
)

agent = create_agent("openai:gpt-5.6-luna", middleware=[router])

See the LangChain harness post. Jev input is $0.042 per million tokens and output is free. Frontier LLM input runs roughly $0.20 to $10 per million, with output around five times input. More in Jev model routing.

Tool and slot selection

Which tool runs this turn, and which plan did the caller mean? Both are Choice questions, and typed answers rule out malformed tool names. `AutoModeMiddleware` uses Jev to check tool calls for risky decisions and blocks them before the tool executes.

from langchain_typesafe.experimental.middleware import AutoModeMiddleware

guardrail = AutoModeMiddleware(tools=["bash"])
agent = create_agent("openai:gpt-5.6-luna", middleware=[guardrail])

In a voice agent, the gated tools would be things like `issue_refund` or `cancel_plan`. See Jev for tool calling in voice agents and our tool-calling guide.

Confidence-based escalation with Jev

Should the agent hand this call to a human now? One Noul question already beats a keyword rule that fires on "representative." Combining signals is better. This is the core idea of a confidence-based escalation voice agent.

Confidence-based escalation: the agent escalates to a human when its own decision confidence falls below a threshold, rather than only when a rule fires.

A single threshold wastes human time on calls a short question could fix. A three-band policy is better:

1. Route automatically when confidence is high and the top-two margin is wide.

2. Ask a confirming question in the middle band, or when the margin is narrow.

3. Escalate to a human when confidence is below the low threshold, or when needs-human is high.

The needs-human check runs first and overrides everything. A caller who says "get me a person" should not be routed anywhere else.

The escalation policy in Python

Install the SDK with `pip install typesafe-sdk`. The client reads `TYPESAFE_API_KEY` and defaults to `jev-latest`.

from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient()

ROUTES = {
    "billing": "Questions or disputes about charges, refunds, invoices, or payment methods.",
    "cancellation": "Caller wants to end, downgrade, or stop renewing a plan or service.",
    "scheduling": "Booking, moving, or canceling an appointment or technician visit.",
    "support": "Something is broken: device, network, app, or login problems.",
    "general": "Anything else, including store hours, coverage, or new sales.",
}

# Illustrative starting values. Tune them on your own labeled calls.
THRESHOLDS = {
    "low":  {"auto": 0.85, "floor": 0.55, "margin": 0.15, "human": 0.60},
    "high": {"auto": 0.95, "floor": 0.70, "margin": 0.30, "human": 0.40},
}
ROUTE_RISK = {
    "billing": "high", "cancellation": "high",
    "scheduling": "low", "support": "low", "general": "low",
}


def decide(transcript: str) -> dict:
    response = client.system_one(
        state=transcript,
        questions={
            "route": Choice(
                instructions="Pick the flow that best serves the caller's primary need on this turn.",
                criteria=ROUTES,
            ),
            "needs_human": Noul(
                instructions="Does this caller need a human agent now?",
            ),
        },
    )
    route = response.answers["route"]
    needs_human = response.answers["needs_human"].noul

    ranked = sorted(route.probabilities.items(), key=lambda kv: kv[1], reverse=True)
    (top, p1), (runner_up, p2) = ranked[0], ranked[1]
    margin = p1 - p2
    t = THRESHOLDS[ROUTE_RISK[top]]

    if needs_human >= t["human"]:
        action, reason = "escalate", "needs_human"
    elif route.confidence < t["floor"]:
        action, reason = "escalate", "low_confidence"
    elif route.confidence >= t["auto"] and margin >= t["margin"]:
        action, reason = "route", "confident"
    else:
        action, reason = "confirm", "ambiguous"

    return {
        "action": action,
        "reason": reason,
        "route": top,
        "runner_up": runner_up,
        "confidence": route.confidence,
        "margin": round(margin, 4),
        "needs_human": needs_human,
        "probabilities": route.probabilities,
    }

Run the earlier transcript through it. Billing is tagged high risk. Needs-human is 0.22, below 0.40. Confidence is 0.61, below the 0.70 high-risk floor, so the illustrative policy escalates. Under the low-risk bands, the same call would get a confirming question. Thresholds encode your risk appetite per route, not a universal constant.

Here is how four illustrative turns move through the same policy:

The second row shows why margin matters. Confidence of 0.72 clears the low-risk floor, but a 0.12 margin means Jev is torn between two specific flows. The fourth row shows why order matters. The router is fairly sure, yet the caller has asked for a person, so the override wins.

Turning the middle band into a question

Skip the generic "Can you repeat that?" Use the top two routes to build a targeted question:

CONFIRM_PROMPTS = {
    ("billing", "cancellation"): "Just to check: do you want help with a charge, or do you want to cancel the plan?",
    ("support", "scheduling"): "Should I help fix this now, or book a technician visit?",
}

def confirm_prompt(top: str, runner_up: str) -> str:
    key = (top, runner_up) if (top, runner_up) in CONFIRM_PROMPTS else (runner_up, top)
    return CONFIRM_PROMPTS.get(key, "Could you tell me a bit more about what you need today?")

The prompts are pre-written; Jev only picks one. Re-run `decide` after the caller answers. If the second pass still lands in the middle band, escalate.

Stack the signals, take the stricter one

Many teams add a frustration Score to the same call. TypeSafe's docs example uses three levels, from calm to very angry. Take the strictest signal: a correctly routed but furious caller may still belong with a person. At 70 to 500ms, the gate adds little latency). See our escalation guide for handoff mechanics.

Why calibrated confidence beats an LLM's overconfidence

Language models are frequently overconfident. Their stated certainty does not track their accuracy. TypeSafe trained Jev with RLCD and reports that its confidence is calibrated: higher confidence means higher accuracy.

Calibration: a model is calibrated when its confidence matches its real accuracy. If it says 0.9 on many decisions, about 90% of them are correct.

See calibration (statistics)) for the formal definition. TypeSafe's core argument: a model right 95% of the time cannot automate a task if it cannot say which 5% are wrong. Calibration points at that 5%. That is why "auto-route above 0.95" means something. With an uncalibrated score, the same line is a guess.

LangChain's study "Can Jev Be a Better Agent Evaluator?" replayed five weather-agent runs 100 times each. Jev matched human pass or fail labels on all 500 decisions. GPT-5.6 Terra reached 99.8%, Luna 96.4%, and Claude Sonnet 4.6 80.0%. The others showed 92 to 913 times higher score variance. See the LangChain blog. That study shows high consistency, not routing accuracy on your callers.

Mapping decisions to Jev question types

The table below maps each routing and escalation decision to its Jev question type. It also shows what the confidence number controls in each case.

Every row can be one question inside a single batched call. The confidence value is the control knob for each gate.

How to set and tune thresholds on labeled calls

The numbers in `THRESHOLDS` are starting points. You must fit them to your traffic. Here is the method we use.

Step 1: label a set of real calls

Pull a few hundred real routed turns. Record Jev's answer plus two human labels: the correct route, and whether the caller truly needed a person. Include mixed intents, noisy lines, and angry callers on purpose. Easy calls make any threshold look good. See human-in-the-loop evaluation.

Step 2: treat escalation as a classifier

The escalation decision has precision and recall like any classifier.

• Escalation precision. Of escalated calls, how many truly needed a human? Low means wasted agent time.
• Escalation recall. Of calls that needed a human or would have been misrouted, how many escalated? Low means stuck callers.
• Auto-route accuracy. Of calls routed with no human and no confirm, how many were right?

Step 3: sweep the thresholds

def sweep(labeled_calls, floors, autos):
    """labeled_calls: list of dicts with Jev outputs plus human labels."""
    results = []
    for floor in floors:
        for auto in autos:
            if auto <= floor:
                continue
            tp = fp = fn = auto_ok = auto_n = 0
            for c in labeled_calls:
                escalated = c["needs_human"] >= 0.5 or c["confidence"] < floor
                auto_routed = not escalated and c["confidence"] >= auto
                needed = c["should_escalate"] or c["jev_route"] != c["true_route"]
                tp += escalated and needed
                fp += escalated and not needed
                fn += (not escalated) and needed
                if auto_routed:
                    auto_n += 1
                    auto_ok += c["jev_route"] == c["true_route"]
            results.append({
                "floor": floor,
                "auto": auto,
                "esc_precision": tp / (tp + fp) if tp + fp else None,
                "esc_recall": tp / (tp + fn) if tp + fn else None,
                "auto_accuracy": auto_ok / auto_n if auto_n else None,
                "auto_coverage": auto_n / len(labeled_calls),
            })
    return results

A would-be misroute counts as "needed" on purpose. Escalating it is a good outcome.

Step 4: pick thresholds per action risk

Raising the floor buys recall and costs precision. Raising the auto line buys accuracy and cuts coverage. Decide by the cost of each error, route by route:

A misrouted "what are your hours" call costs seconds. A wrongly automated cancellation costs a customer.

Step 5: check calibration on your own data

Bucket labeled calls by Jev confidence, such as 0.6 to 0.7. In each bucket, compare average confidence with the share routed correctly. If a bucket is off, move the threshold for that route. Calibrated on TypeSafe's data is not the same as calibrated on your callers.

Logging every decision for audit

A threshold you cannot audit is one you cannot defend. Log one record per decision.

{
  "call_id": "c_7f3a91",
  "turn": 2,
  "timestamp": "2026-09-28T14:03:11Z",
  "model": "jev-1.13.0",
  "policy_version": "routing-thresholds-v4",
  "state_hash": "sha256:9b1e...",
  "answers": {
    "route": {
      "choice": "billing",
      "confidence": 0.61,
      "probabilities": {"billing": 0.58, "cancellation": 0.34, "scheduling": 0.02, "support": 0.04, "general": 0.02}
    },
    "needs_human": {"noul": 0.22}
  },
  "derived": {"margin": 0.24, "risk": "high"},
  "action": "escalate",
  "reason": "low_confidence"
}

Three fields matter most. `model` pins the Jev version, since early-access versions change. `policy_version` separates a model change from a threshold change. `probabilities` keeps the full distribution, so an evaluator can replay the policy offline. Store a hash or redacted state; see our PII handling guide.

Warm vs cold handoff: passing context to the human

A cold transfer drops the caller into a queue with nothing attached. A warm transfer sends context first.

Jev's typed answers make a warm handoff cheap. Send them to the agent desktop with the call:

{
  "call_id": "c_7f3a91",
  "escalation_reason": "low_confidence",
  "summary_fields": {
    "likely_route": "billing",
    "runner_up_route": "cancellation",
    "route_confidence": 0.61,
    "needs_human": 0.22
  },
  "suggested_opening": "I can see you've been charged for the premium plan. Do you want a refund, a cancellation, or both?",
  "transcript": "Agent: Thanks for calling Northwind Mobile. How can I help?\nCaller: I keep getting charged for the premium plan and I don't want it anymore. This is the third month."
}

The runner-up route tells the human exactly what the bot was unsure about. Jev returns no rationale. The reason code is your policy's label, and the opening line is your template.

How to wire Jev routing and confidence-gated escalation into a voice agent

1. Define routes as a Choice schema. Up to 255 options, each with a one-line description.

2. Capture the call state. Send the transcript and structured context. Jev reads text, not raw audio.

3. Ask routing and needs-human together. One call, with an optional frustration Score.

4. Compute the top-two margin.

5. Apply the three-band policy. Needs-human first, then the floor, then the auto line plus margin.

6. Tag routes by risk. Stricter thresholds where money or accounts change.

7. Add a model router and tool gate. Use `ModelRouterMiddleware` and `AutoModeMiddleware`.

8. Wire a warm handoff. Send transcript, Jev answers, runner-up, and reason.

9. Log every decision. Include Jev version, policy version, and full probabilities.

10. Tune on labeled calls, then evaluate independently.

Measuring whether the routing and escalation were right

Jev does not tell you whether a decision was correct. A mis-decision is silent unless something evaluates it. Evalgent is an independent, third-party evaluator for AI voice agents. We do not build agents or sell models. We measure three failure types on your own calls:

Metrics are defined in escalation accuracy and handoffs. We also replay your logged probabilities at other thresholds, such as a 0.60 floor instead of 0.70, without redeploying. We check calibration by bucket and flag metric drift after model or prompt changes. Escalation errors cluster in rare calls, so we score every call, not a sample. See Jev for voice agent evaluation and call monitoring.

A vendor grading its own routing is not a check. See independent voice AI evaluation, voice agent evaluation, and voice agent testing.

Honest limits of Jev for routing and escalation

It decides; it does not explain. Jev returns choices, scores, and probabilities, with no free-text rationale.

It does not generate language. A pipeline with no LLM in the turn loop works only with scripted replies.

It reads text, not audio. Speech-to-text quality shapes its input. A garbled "cancel" is a speech problem, not a routing problem.

Calibrated is not correct on your data. Calibration is an average, not a per-call guarantee. Validate first.

It is early access. Pin the model version, pilot it, and keep a fallback path.

Consistency is high, not absolute. Treat Jev as highly consistent, not as a deterministic rule engine.

For timing, see our latency guide and turn-taking evaluation. For the wider stack, see orchestration for voice agents in 2026, Jev guardrails, and Jev use cases.

Jev can't abstain: always include an escape option

Jev always returns one of the options you define. It has no built-in "I don't know." When a call transcript holds no evidence for any option, such as a two-word turn, a dropped call or a caller talking about something else, Jev still picks the least-wrong option, and that label looks just like a confident answer. TypeSafe's own Choice docs recommend adding an "other" or "none of the above" option when the list may not cover every input.

For routing, the escape option is usually the escalation path itself. If none of billing, technical or cancellation fits the caller's words, `unclear_needs_clarifying_question` or `route_to_human` should be a real option, not something Jev can only reach by picking a wrong queue.

Four rules keep this from hurting you on live calls:

• Add an explicit escape option such as `not_enough_information`, `not_applicable` or `none_of_the_above` to every Choice question where evidence can be missing. Describe it as concretely as the real options, because Jev reads only the descriptions, not your option keys.
• Ask whether the evidence exists first. A Noul like "does the caller state a reason for calling?" before the Choice question separates "no evidence" from "evidence for option B".
• Check the distribution, not just the label. In one independent edge-case test, a forced choice with no valid option still returned a pick at 0.52 probability but only 0.04 confidence (jev-exploration). The numbers flagged the problem; the label did not. Set your human-review cutoff from your own labeled calls, because an independent study on data unlike Jev's training found its confidence can run high (jev-ood-calibration).
• Track the escape-option rate. If it climbs, your transcripts, STT or option set have drifted away from what callers actually say.

Frequently asked questions

How does Jev route voice agent calls?

Jev routes by answering a Choice question. You send the transcript as the state and list your flows as options, up to 255, each with a description. Jev returns a probability for every option plus a confidence value. The agent routes to the top option only when confidence and the top-two margin clear your thresholds.

Can Jev decide when to escalate to a human?

Yes. Ask a Noul question such as "does this caller need a human now?" in the same call as the routing question. Jev returns a probability. Check it first and let it override routing. Also escalate when routing confidence falls below your low threshold, so the agent hands off instead of guessing.

What is confidence-based escalation in a voice agent?

It means the agent escalates when its own decision confidence is low, not only when a keyword rule fires. A three-band version works best. High confidence routes automatically. A middle band asks a targeted confirming question. Low confidence, or a high needs-human score, sends the call to a person with context attached.

How fast is Jev for routing decisions?

TypeSafe reports Jev latency of 70 to 500ms end to end per call. Frontier LLMs take 3 to 329 seconds on comparable tasks, so Jev is roughly 40 to 200 times faster. All questions in a request, such as route and needs-human, return together in one round trip, which keeps the gate inside a conversational turn.

Does Jev route between models?

Yes. The langchain-typesafe package ships a ModelRouterMiddleware. You define model choices with plain-language criteria, such as fast for direct lookups and powerful for high-stakes decisions. Jev picks one per turn. The probabilities and confidence stay in the agent state, so you can log, audit, and evaluate each routing decision later.

Why does calibrated confidence matter for routing?

Calibrated confidence means the number tracks real accuracy. TypeSafe trains Jev with RLCD for this property. It is what makes a threshold meaningful: auto-routing above 0.95 tells you roughly how accurate those calls are. With an overconfident model, the same line is a guess. Still, verify calibration on your own labeled calls.

How do you measure Jev routing accuracy?

Log every decision with its full probabilities, then score it against labeled outcomes. Evalgent measures per-route precision and recall and a confusion matrix. For escalation, we measure misroutes, late escalations, and unnecessary escalations. We also replay your logged probabilities at other thresholds and check calibration by confidence bucket on your traffic.

Can Jev replace the LLM in a voice agent?

Not fully. Jev decides and scores, but it does not generate language or give rationales. A pipeline with no LLM in the turn loop works only with pre-written responses. For open-ended conversation you still need an LLM, with Jev handling the fast routing, gating, escalation, and scoring decisions around it.

The bottom line

Jev turns routing and escalation into typed, calibrated decisions that return in 70 to 500ms, with a full probability distribution you can gate on. Those thresholds are only as good as their validation on your own calls, which is why an independent evaluator should measure every misroute and every handoff.

Evalgent measures whether your Jev-driven routing and escalation actually served callers. Book a demo to audit your router and confidence thresholds on your own calls.