Jev Model Routing: Use Jev as a Model Router for Voice Agents

Not every turn in a call needs your best model. A ZIP code confirmation does not need a frontier LLM. A tangled billing dispute might. Model routing sends each turn to the cheapest model that can still get it right.

The catch is the router itself. It must be near-instant and near-free. If it is slow or costly, it eats the savings it was built to create. And if it is wrong, it fails quietly.

This guide is hands-on. You will get the real LangChain code, a voice version with three tiers, and a direct API call. You will also get Python gating logic, worked cost math, latency tactics, and an evaluation plan. Every number we invent is labeled illustrative.

Model routing vs intent routing

These two terms sound alike. They are different jobs. Mixing them up leads to the wrong design.

Model routing: deciding which model tier should handle a single turn. The output is a tier, such as canned, fast, or powerful.

Intent routing: deciding where the caller's request should go. The output is a flow, a department, or a human.

This post covers model routing only: turn to model. Intent routing is covered in our post on voice agent routing and escalation. One picks a model. The other picks a destination.

Both can use Jev, and both are classification. But they fail differently. An intent misroute is loud, because the caller lands in the wrong flow. A model misroute is quiet, because the agent still answers, just worse.

What Jev is, and what it is not

Jev: TypeSafe AI's first System One model, launched September 15, 2026, in early access. It reads a state plus one or more questions and returns typed decisions with calibrated confidence.

Jev is not a language model. It does not generate text. You pass it a state, such as a transcript or message list. You pass it questions. It returns typed answers, not sentences.

TypeSafe documents three question types:

• Choice picks one option from a set. It returns a probability per option plus confidence.
• Score returns an ordered level or continuous score, plus confidence.
• Noul returns a yes or no probability.

TypeSafe reports latency of 70 to 500ms. Input costs $0.042 per million tokens, and output is free. For contrast, frontier LLMs charge roughly $0.20 to $10 per million input tokens. Their output runs about five times the input rate. Their latency ranges from about 3 to 329 seconds on comparable tasks. See the launch note and the TypeSafe docs.

The mental model is simple. Use an LLM to talk. Use Jev to decide fast. Model routing is one of those decisions. For a wider comparison, read Jev vs an LLM for voice agents.

Why jev model routing fits a turn-by-turn decision

Picking a model per turn is a classification problem. A turn arrives. The router labels it. The label decides which model runs next.

That maps cleanly to a Jev Choice question. The options are your tiers. Jev returns a probability for each tier and a confidence value.

Three properties make Jev a good model router:

• It is fast. A 70 to 500ms decision can hide inside a live turn.
• It is cheap. Output is free, so a routing call costs a fraction of a cent.
• It is typed. The answer is always one of your tiers. There is no free text to parse.

Compare that to routing with an LLM. An LLM router adds seconds and real token cost. Its stated confidence may not track its accuracy. That makes it a weak gate for escalation.

The real LangChain ModelRouterMiddleware code

LangChain published a harness built on Jev. It includes a model-routing middleware. In LangChain's words: "A simple lookup doesn't need the same model as a difficult debugging task."

Here is the code from the LangChain post, unchanged:

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
    ModelChoice,
    ModelRouterMiddleware,
)

router = ModelRouterMiddleware(
    choices={
        "fast": ModelChoice(
            model="openai:luna",
            criteria="Direct lookups, extraction, and localized changes.",
        ),
        "powerful": ModelChoice(
            model="openai:sol",
            criteria="Architecture and high-stakes decisions.",
        ),
    },
    instructions="Choose the least costly model that can complete the task.",
)

agent = create_agent("openai:gpt-5.6-luna", middleware=[router])

Read it closely. Each tier is a model plus plain-language criteria. The instructions tell Jev to prefer the cheapest tier that can do the job.

LangChain notes that the router picks a model from the latest user message. It then uses that model for the whole run. The probabilities and confidence stay available in agent state.

That last detail matters. You can log the confidence, gate on it, and evaluate it later. Keep it.

Adapting the router to a voice agent with three tiers

The LangChain example targets a coding harness. A voice agent needs different tiers and different criteria. Voice turns are short, spoken, and time-sensitive.

A practical voice setup uses three tiers:

• Canned. A templated reply with slots filled from state. No LLM runs at all.
• Fast. A small LLM for short factual answers and single-step tasks.
• Powerful. A frontier LLM for ambiguity, multi-step reasoning, and high-stakes turns.

Here is an adapted version. The middleware pattern is LangChain's. The tier names, criteria, and the canned-reply model identifier are our own examples.

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
    ModelChoice,
    ModelRouterMiddleware,
)

# Illustrative voice tiers. Swap in your own model identifiers.
voice_router = ModelRouterMiddleware(
    choices={
        "canned": ModelChoice(
            model="openai:luna",  # placeholder; canned turns bypass generation downstream
            criteria=(
                "Greetings, confirmations, yes or no answers, 'repeat that', "
                "'hold on', and reading back a value already in state."
            ),
        ),
        "fast": ModelChoice(
            model="openai:luna",
            criteria=(
                "Single factual questions, one-step lookups, collecting one field, "
                "and short answers about hours, status, or balance."
            ),
        ),
        "powerful": ModelChoice(
            model="openai:sol",
            criteria=(
                "Complaints, disputes, refunds, cancellations, multi-part questions, "
                "policy exceptions, confused or upset callers, and anything involving "
                "money, health, legal terms, or account security."
            ),
        ),
    },
    instructions=(
        "Choose the least costly tier that can fully resolve this spoken turn. "
        "If the caller sounds confused or the request has several parts, "
        "prefer the stronger tier."
    ),
)

agent = create_agent("openai:gpt-5.6-luna", middleware=[voice_router])

A note on the canned tier. The middleware selects a model. It does not know about templates. So treat "canned" as a label your turn handler reads. When Jev picks it, your code skips generation and plays a template.

Write criteria in the caller's words, not the model's. "Give me my balance" is a lookup. "Why were my three refunds reversed?" is not. Clear criteria make cleaner routes.

The general idea is old. Routing means choosing a path by cost and quality. A model routing voice agent applies it per turn, in real time.

Calling Jev directly: Choice, Score, and Noul in one request

You do not need LangChain. You can call Jev directly at `POST https://api.typesafe.ai/v1/systemone`. The Python SDK exposes `Choice`, `Noul`, `Score`, and `TypeSafeClient`.

A direct call gives you more control. You can ask several questions in one round trip:

• A Choice over tiers: canned, fast, or powerful.
• A Score for turn difficulty.
• A Noul for whether the turn is high stakes.

Include the last few turns in state, not just the latest message. Routing on one message can miss context. "Yes, do that" is trivial alone. After "Should I close your account?" it is not.

The request follows the documented schema in the TypeSafe docs. The values are illustrative.

curl -X POST https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "AGENT: I see two charges from the 14th. Which one looks wrong?\nCALLER: The second one. And I already asked for a refund last week.\nAGENT: Thanks. Let me check that refund request.\nCALLER: It was approved and then reversed. I want to know why, and cancel if this keeps happening.",
    "model": "jev-latest",
    "questions": {
      "tier": {"type": "choice",
        "instructions": "Pick the least costly tier that can fully resolve the latest caller turn.",
        "criteria": {"canned": "A pre-written reply fully answers it", "fast": "A short lookup or single-step answer", "powerful": "Multi-step reasoning, disputes, or high stakes"}},
      "difficulty": {"type": "score",
        "instructions": "How hard is the latest caller turn to resolve correctly?",
        "criteria": ["Trivial", "Routine", "Tricky", "Hard"]},
      "high_stakes": {"type": "noul",
        "instructions": "Does the latest caller turn involve money, cancellation, legal, health, or security risk?"}
    }
  }'

All three answers follow the documented response shapes. The values are illustrative.

{
  "answers": {
    "tier": {
      "type": "choice",
      "choice": "powerful",
      "confidence": 0.91,
      "probabilities": {"canned": 0.01, "fast": 0.08, "powerful": 0.91}
    },
    "difficulty": {
      "type": "score",
      "score": 2.6,
      "confidence": 0.88,
      "legend": {"0": "Trivial", "1": "Routine", "2": "Tricky", "3": "Hard"},
      "probabilities": {"0": 0.0, "1": 0.05, "2": 0.30, "3": 0.65}
    },
    "high_stakes": {
      "type": "noul",
      "noul": 0.94
    }
  },
  "usage": {"input_tokens": 212, "output_tokens": 24}
}

This turn has a reversed refund and a threatened cancellation. All three signals agree: route it to the powerful tier.

Here is the same idea with the Python SDK. The class names come from the SDK. The constructor arguments are illustrative.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient()  # reads TYPESAFE_API_KEY from the environment

def route_turn(recent_turns: list[dict]):
    state = "\n".join(f"{t['role'].upper()}: {t['content']}" for t in recent_turns[-6:])  # last few turns
    return client.system_one(
        state=state,
        questions={
            "tier": Choice(
                instructions="Least costly tier that can fully resolve the latest caller turn.",
                criteria={"canned": "A pre-written reply fully answers it",
                          "fast": "A short lookup or single-step answer",
                          "powerful": "Multi-step reasoning, disputes, or high stakes"},
            ),
            "difficulty": Score(
                instructions="How hard is the latest caller turn to resolve correctly?",
                criteria=["Trivial", "Routine", "Tricky", "Hard"],
            ),
            "high_stakes": Noul(
                instructions="Does the turn involve money, cancellation, legal, health, or security risk?",
            ),
        },
    )

Confidence-gated escalation to the stronger model

Calibrated confidence is what makes cheap routing safe. A calibrated model's confidence tracks its accuracy over many decisions. If it says 0.9 across many turns, about 90% should be right.

Confidence-gated escalation: the router accepts a cheaper tier only when Jev is confident. Otherwise the turn moves up to a stronger tier.

Calibration is a population property, not a per-call promise. It also holds on the data Jev was calibrated on. It may not hold on your callers, your accents, or your product. Calibrated does not mean correct on your data. Measure it.

Python routing logic with gating and a no-downgrade rule

Here is a full routing function. It adds three safety rules on top of Jev's answer:

1. Low confidence escalates. If Jev is unsure, move up one tier.

2. High stakes forces powerful. A likely high-stakes turn never runs cheap.

3. No downgrade mid-call. Once a call reaches a tier, later turns do not drop below it.

The third rule deserves a word. Callers notice when an agent suddenly gets dumber. A call that needed the powerful model at turn four often needs it at turn six. Stay up once you go up.

All thresholds below are illustrative starting points. Tune them on your own evaluated calls.

from dataclasses import dataclass, field

TIERS = ["canned", "fast", "powerful"]  # ordered cheapest to strongest

# Illustrative thresholds. Tune on your own evaluated calls.
MIN_CONFIDENCE = {"canned": 0.90, "fast": 0.80}
HIGH_STAKES_CUTOFF = 0.50
HARD_TURN_CUTOFF = 0.70

@dataclass
class CallState:
    call_id: str
    floor_tier: str = "canned"          # lowest tier allowed for the rest of the call
    route_log: list = field(default_factory=list)

def step_up(tier: str) -> str:
    i = TIERS.index(tier)
    return TIERS[min(i + 1, len(TIERS) - 1)]

def max_tier(a: str, b: str) -> str:
    return a if TIERS.index(a) >= TIERS.index(b) else b

def choose_tier(call: CallState, answers: dict) -> str:
    tier = answers["tier"]["choice"]
    confidence = answers["tier"]["confidence"]
    difficulty = answers["difficulty"]["score"]
    high_stakes = answers["high_stakes"]["noul"]
    reason = "jev_choice"

    # Rule 1: low confidence on a cheap tier escalates one step.
    if tier in MIN_CONFIDENCE and confidence < MIN_CONFIDENCE[tier]:
        tier, reason = step_up(tier), "low_confidence"

    # Rule 2: high stakes or a hard turn always gets the powerful tier.
    if high_stakes >= HIGH_STAKES_CUTOFF or difficulty >= HARD_TURN_CUTOFF:
        tier, reason = "powerful", "high_stakes_or_hard"

    # Rule 3: never downgrade below the call's floor.
    if TIERS.index(tier) < TIERS.index(call.floor_tier):
        tier, reason = call.floor_tier, "no_downgrade"

    # Raise the floor to the tier just used. Canned turns never raise it.
    if tier == "powerful":
        call.floor_tier = "powerful"
    elif tier == "fast":
        call.floor_tier = max_tier(call.floor_tier, "fast")

    call.route_log.append({
        "tier": tier,
        "reason": reason,
        "jev_choice": answers["tier"]["choice"],
        "confidence": confidence,
        "difficulty": difficulty,
        "high_stakes": high_stakes,
    })
    return tier

Log every decision, including the reason. That log is the raw material for evaluation later. Without it, you cannot tell a good route from a lucky one.

One tradeoff to watch. The no-downgrade rule raises cost on long calls. That is usually worth it. But measure how many calls get pinned to powerful by one early spike.

Worked cost math for one million voice turns

Now the money. Everything in this section is illustrative. We define example per-turn costs so the arithmetic is easy to follow. Replace them with your own.

Illustrative per-turn LLM costs:

• Canned: $0.00001 per turn. No LLM, just template and state lookup.
• Fast: $0.0004 per turn.
• Powerful: $0.006 per turn.

Illustrative routed blend, after gating: 50% canned, 35% fast, 15% powerful.

Jev's own cost uses TypeSafe's real price of $0.042 per million input tokens. Assume an illustrative 600 input tokens per routing call, since we send recent turns. That is 600 million tokens for one million turns. The cost is 600 × $0.042, or about $25.20. Output is free.

Notice where the savings come from. Jev is about 2% of the routed total. The real driver is the 85% of turns that avoid the powerful model.

Now the uncomfortable part. Savings only count if quality holds. Say a failed resolution costs an illustrative $5 in human follow-up. Assume an illustrative ten turns per call, so one million turns is 100,000 calls. The $4,929.80 saving covers about 986 extra failed calls. That is roughly 1% of calls.

So if routing quietly pushes failures up by more than about one point, you lose money. That is a thin margin. It is also why you must measure. For the full metric, see voice agent cost per resolution. For the wider frame, see total cost of ownership.

Latency: first-token savings and where Jev fits

Money is one win. Speed is the other. On a phone line, silence feels broken. The time to first spoken word matters more than in chat.

Easy turns benefit most from routing. The numbers below are illustrative, not measured:

Here is the tension. Jev itself adds 70 to 500ms. If you run it in series before every turn, you add that delay to every turn. On a canned turn, the router could be slower than the answer.

Two tactics fix this.

Run Jev in parallel with STT endpointing. The speech-to-text layer waits for silence before it ends a turn. Send the partial transcript to Jev during that window. By the time endpointing fires, the route is often ready. The routing delay hides inside time you were already spending.

Only route turns that need an LLM. Some turns are obvious from rules. A DTMF keypress or an exact "yes" to a confirmation needs no router. Skip Jev there and save the round trip.

A third, smaller tactic: speculative start. Begin the fast model while Jev decides. If Jev picks powerful, cancel and switch. This costs a few wasted fast tokens but trims the tail. For the broader budget, see our latency guide for voice agents and the general idea of latency).

Extending routing to STT and TTS tiers

The LLM is not the only tiered component. Speech-to-text and text-to-speech come in tiers too. The same router can pick them.

Illustrative examples:

• Premium TTS voice only for high-value calls. Use a Noul like "is this a high-value account?" at call start. Pay for the premium voice only when it is yes.
• Stronger STT when recognition is struggling. If the transcript shows repeated "sorry, say that again," move to a stronger STT tier.
• Specialist STT for dense data. Switch tiers when the caller reads account numbers or addresses.

One caution. Jev reads transcript and text state only. It does not hear audio. So it can route on what the transcript says, not on how the caller sounds. Audio-quality signals need a different source.

Also keep TTS voice stable within a call. Switching voices mid-call sounds broken. Decide the voice tier once, early, and hold it.

The silent-quality-drop risk of a mis-route

Model routing has one dangerous failure mode. A mis-route to an underpowered model is a silent quality drop.

Walk through it. A hard turn gets labeled easy. The fast model handles it. It does not crash or throw an error. It just answers worse. The call keeps going. No alarm fires.

The damage shows up later. The caller calls back. The refund reason was wrong. The cancellation offer was missed. None of it traces back to the router unless you look.

Common causes of silent mis-routes:

• Latest-message-only routing. "Yes, go ahead" looks trivial without the prior turn.
• Criteria drift. New products or policies appear that your criteria never named.
• Threshold creep. Someone lowers the gate to save money and never re-checks quality.
• Population shift. Calibration holds on one caller mix but not on a new one.
• Early access changes. Jev is in early access, so behavior may change between versions.

Turn types, routes, and Jev signals

The table maps common voice turn types to a route, the Jev signal behind it, and the risk if the route is wrong.

How to add Jev model routing to a voice agent

Follow these steps to route each turn to the right model with Jev.

1. Define your tiers. Start with canned, fast, and powerful. Keep the set small.

2. Write voice-specific criteria. Describe each tier in caller terms, with examples.

3. Send recent turns to Jev. Pass the last few turns as state, plus a Choice, a Score, and a Noul.

4. Apply safety rules. Gate on confidence, force powerful on high stakes, and block mid-call downgrades.

5. Hide the latency. Run Jev in parallel with STT endpointing, and skip it on rule-obvious turns.

6. Log every decision. Keep tier, reason, confidence, difficulty, and high-stakes probability.

7. Evaluate on real calls. Measure routing accuracy, downgrade regret, and resolution by tier.

Steps six and seven close the loop. Without them, you are routing blind.

Evaluating routing accuracy on real calls

A router makes a claim on every turn: this was easy enough for the cheap tier. You have to test that claim. A mis-route will not surface on its own.

Four measurements matter most.

Routing accuracy: the share of turns sent to a tier that could handle them. You need a labeled sample to measure it.

Downgrade regret: the share of cheap-tier turns that failed where the powerful tier would have succeeded. This is the metric that tracks the silent quality drop.

Over-escalation rate: the share of powerful-tier turns that a cheaper tier would have handled. This is wasted money, not lost quality.

Resolution by tier mix: whether calls with more cheap turns resolve less often, controlling for call type.

The strongest method is shadow testing. Take a random sample of routed turns, say an illustrative 1 to 2%. Replay each one on both the chosen tier and the powerful tier. Grade both answers. Where powerful wins and cheap loses, you have found downgrade regret.

import random

SHADOW_RATE = 0.02  # illustrative sample rate

def maybe_shadow(turn, chosen_tier, run_tier, grade):
    """Replay a sample of cheap-tier turns on the powerful tier and compare."""
    if chosen_tier == "powerful" or random.random() > SHADOW_RATE:
        return None
    cheap_answer = run_tier(chosen_tier, turn)
    strong_answer = run_tier("powerful", turn)
    cheap_ok = grade(turn, cheap_answer)
    strong_ok = grade(turn, strong_answer)
    return {
        "turn_id": turn["id"],
        "tier": chosen_tier,
        "regret": (not cheap_ok) and strong_ok,  # cheap failed, powerful would not have
        "both_failed": (not cheap_ok) and (not strong_ok),
    }

Shadow runs happen offline, so callers never hear the second answer. The cost is small, since you only replay a sample.

Also check calibration on your own data. Bucket decisions by Jev confidence. Compare each bucket's confidence to its measured accuracy. If the 0.9 bucket is right only 75% of the time, your thresholds are too loose.

Where Evalgent fits

This is the Evalgent job. We are an independent, third-party evaluator for AI voice agents. We do not sell a router or a model. We measure whether yours works.

The core question is simple. Is the router saving money without losing quality? The team that built the router is not the best judge of that. A neutral evaluator is.

Evalgent scores routing on your real calls, not vendor demos. We measure routing accuracy, downgrade regret, and resolution by tier mix. We check whether Jev's confidence holds up on your caller population. Then we tie the findings to your cost per resolution.

We surface the mis-routes you cannot see. A threshold set too loose. A no-downgrade rule pinning too many calls. A call type that quietly degrades on the fast tier. For the neutral case, read independent voice AI evaluation. For methods, see our voice agent evaluation guide.

The result is a router you can trust because it is checked, not assumed. Book a demo to see how Evalgent evaluates your model routing on real calls: book a demo.

Jev can't abstain: always include an escape option

Jev always returns one of the options you define. It has no built-in "I don't know." When a call transcript holds no evidence for any option, such as a two-word turn, a dropped call or a caller talking about something else, Jev still picks the least-wrong option, and that label looks just like a confident answer. TypeSafe's own Choice docs recommend adding an "other" or "none of the above" option when the list may not cover every input.

For model routing, give Jev a `default_tier` or `use_safe_default` option so a turn with no clear complexity signal falls back to your standard model instead of being pushed to the cheapest or the most expensive tier.

Four rules keep this from hurting you on live calls:

• Add an explicit escape option such as `not_enough_information`, `not_applicable` or `none_of_the_above` to every Choice question where evidence can be missing. Describe it as concretely as the real options, because Jev reads only the descriptions, not your option keys.
• Ask whether the evidence exists first. A Noul like "does the caller state a reason for calling?" before the Choice question separates "no evidence" from "evidence for option B".
• Check the distribution, not just the label. In one independent edge-case test, a forced choice with no valid option still returned a pick at 0.52 probability but only 0.04 confidence (jev-exploration). The numbers flagged the problem; the label did not. Set your human-review cutoff from your own labeled calls, because an independent study on data unlike Jev's training found its confidence can run high (jev-ood-calibration).
• Track the escape-option rate. If it climbs, your transcripts, STT or option set have drifted away from what callers actually say.

Frequently asked questions

What is jev model routing in a voice agent?

Jev model routing uses Jev to choose which model tier handles each turn. Jev reads recent transcript turns and answers a Choice over tiers, often with a difficulty Score and a high-stakes Noul. Easy turns get a canned reply or a fast LLM. Hard turns get a powerful LLM. TypeSafe reports 70 to 500ms per decision.

How do I use Jev as an LLM router in a voice agent?

To use Jev as an LLM router, define tiers, write criteria in caller terms, and send recent turns to Jev with a Choice over tiers. Then apply rules. Escalate when confidence is low. Force the powerful tier on high-stakes turns. Block downgrades mid-call. Log each decision so you can evaluate routing accuracy later.

What is the difference between model routing and intent routing?

Model routing picks which model answers a turn: canned, fast, or powerful. Intent routing picks where the caller goes: a flow, a queue, or a human. Model routing trades cost and latency against quality. Intent routing decides the destination. Model misroutes are quiet. Intent misroutes are loud. Jev can drive both.

How does confidence-gated escalation to a stronger model work?

Jev returns a tier and a calibrated confidence value. If confidence for a cheap tier clears your threshold, the cheap tier runs. If not, the turn moves up a tier. High-stakes turns skip straight to powerful. Calibration holds across many decisions, not every call, so verify thresholds on your own data.

How fast is jev for a model routing decision?

TypeSafe reports Jev latency of 70 to 500ms. That is fast, but not free on a voice line. Run Jev in parallel with STT endpointing so the decision hides inside the silence window. Skip it on turns that rules already settle, like keypad input or a plain confirmation.

How much does jev model routing cost per call?

TypeSafe prices Jev input at $0.042 per million tokens, with output free. In our illustrative example, 600 tokens per turn across one million turns costs about $25.20. That was about 2% of the routed total. The real savings come from keeping most turns off the powerful model.

What happens if jev mis-routes a turn to a weak model?

A mis-route to an underpowered model is a silent quality drop. The cheap model still answers without an error. It just answers worse, and the call continues. The damage shows up later as repeat calls or wrong outcomes. Measure downgrade regret with shadow testing to catch it.

How do you evaluate model routing accuracy on real calls?

Label a sample of real turns and check whether each went to a capable tier. Shadow-test cheap-tier turns on the powerful tier to measure downgrade regret. Compare Jev confidence to measured accuracy. Evalgent, an independent third-party evaluator, runs this on your own calls and ties results to cost per resolution.

The bottom line

Jev model routing can cut voice agent model spend sharply by sending easy turns to canned or fast tiers. The savings are real only when an independent evaluation shows that routing accuracy and resolution held up.