Jev Language Support: English First, CJK Usable — How to Verify It on Your Own Content

TL;DR: The models page gives language support one paragraph, and it is not a marketing paragraph: English is the primary training language, everything else “is handled but not equally well”, and the docs tell you to test your own content and lean on Confidence when routing. This guide turns that instruction into four steps — build a per-language acceptance test, read the confidence distribution rather than a single accuracy number, set thresholds per language, and decide the fallback ladder before the first low-confidence CJK turn arrives.

What the official documentation actually says

Two pages state the position, and they agree with each other.

The Models page carries a section headed Language support:

Jev accepts natural-language text. English is the primary training language and where accuracy is currently best. Other languages, including CJK scripts, are handled but not equally well; test on your own content before relying on Jev for a non-English workload, and pay close attention to Confidence when routing.

The State page, in the warning that describes what state may contain, repeats the limit:

Jev’s primary training language is English; other languages, including CJK scripts, are accepted but currently have lower accuracy.

Three consequences follow, and none of them are optional reading if your traffic is not English.

1. Language is a routing input, not a setup detail. The docs do not say “supported” and stop; they say lower accuracy and tell you which knob to compensate with — confidence.
2. There is no published per-language accuracy table. The models page gives one paragraph and stops. Any site that hands you a ranked list of “supported languages” with percentages is supplying numbers the vendor never published.
3. The remedy the docs prescribe is procedural. “Test on your own content” is an instruction to build a test, not to read a compatibility list.

Where this guide cannot help is important to state: it cannot tell you how well Jev performs on your language, because that depends on your state, your question wording, and your definition of a correct answer. What it can do is make your own measurement cheap and repeatable.

“Accepted” is not “equally accurate”

Send a Chinese support message as state and ask a Choice question with four routing options. You get back the selected option, a probability for each option, and a confidence between 0 and 1 — exactly the contract described on the Choice and Confidence pages. Nothing in the response announces that the input was not English.

That symmetry is the trap. The response shape is language-independent; the response quality is not. Three separate quantities move when you change the input language, and they move in different directions:

The third row is the one teams skip. A model that is 12% less accurate but reports it through a visibly flatter probability distribution is far easier to live with than one that is 12% less accurate at the same face-value confidence. Jev is built to expose that shape — the whole point of returning probabilities next to the answer — so the honest response to a weaker language is not “don’t use it”, it is “gate it”.

Why the docs send you to Confidence for non-English work

confidence is not a separate model call or an extra field you compute. It is derived from the probability distribution the answer already contains: 1 when all the probability sits on one outcome, 0 when it is spread evenly across them. A Choice carries a distribution over your options; a Score carries one over your ordered levels. Noul is the exception — it has no separate confidence, because its single probability p already is the uncertainty, with a value near 0.5 meaning the model is unsure.

That mechanism is why the docs can hand you confidence as the compensation for a language gap. Because it is computed from the distribution rather than from the wording, it keeps working in every language that returns a distribution at all.

The three-band pattern from the confidence page is the smallest useful shape:

• High confidence — act automatically.
• Medium confidence — proceed with caution: confirm, flag for review, or gather more information.
• Low confidence — do not act: route to a human, ask for clarification, or fall back to a different system.

The confidence-gated routing pattern makes the point that a threshold is not one number: riskier actions get higher bars, and its banking example routes anything below its floor straight to a human. For a non-English workload the rule generalises one step further — the bar is per language as well as per action, because the distribution you are thresholding has shifted.

The jagged edges that bite hardest in a second language

Jev 1.13’s own jaggedness page, last reviewed 2026-10-02, lists nine known failure modes. None of them are language-specific, but several of them interact badly with a language the model handles less well, and those are the ones worth designing around first.

The two rows that deserve their own paragraph are literal reading and contradictory instructions, because they are the two you control completely.

On literal reading: the fix the doc gives is to “state the exact condition in the instructions” and to put boundary cases in the criteria — and to treat the explanation you find yourself writing when a wrong answer arrives as “the missing half of the instruction”. In a bilingual pipeline that advice has a sharper edge. If your instruction is written in English and the state is Chinese, you have inserted a translation step into every judgement. Writing the instruction in the same language as the content you are judging removes it, and costs nothing.

On contradictory criteria: the doc’s example is a Noul whose true maps to no. Rather than reference the boolean, write the criterion so the mapping is obvious, and align the criterion with the instruction until they read as one sentence. This is worth doing per language, because a criterion that reads precisely in English can read diffusely in a translation.

Step 1 — Build a per-language acceptance test on your own content

This is the step the docs ask for and the step almost nobody does. Here is a shape that fits in an afternoon.

Sample, per language, from real traffic. Stratify rather than grabbing the first 200 rows: cover the label distribution you actually expect, not the one you hope for. If complaint messages are 8% of your queue, make sure they are represented.

Label with your own domain experts. The ground truth for “is this a refund request” is your refunds team, not a public benchmark. If you already route these items manually today, your historical decisions are the cheapest useful label set — with the caveat that they carry the biases of whoever made them.

Hold a slice back. Tune thresholds on one slice, report numbers from another. Otherwise every threshold you set is fitted to the sample you measured.

Run identical questions in each language. Same instructions, same criteria, same options, translated — not rewritten. Use Choice for the label itself, Score when the answer is graded, and Noul for a binary screen.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient(model="jev-1.13")

# One state, several questions, one request. The label question decides,
# the confidence scales tell you how much of that decision to trust.
sentiment = client.system_one(
    {"ticket": ticket_text, "language": "zh"},
    {
        "intent": Choice(
            "Which queue should this ticket go to?",
            refund="Requests money back for a completed charge",
            technical="Reports the product not working",
            billing="Asks about an invoice or a plan change",
        ),
        "urgent": Noul("Does this ticket describe an outage, an outage risk, or a safety issue?"),
        "quality": Score(
            "How complete is the reported information?",
            levels=["No detail", "Minimal", "Enough to act", "Reproducible"],
        ),
    },
)

for name, answer in sentiment.choices.items():
    print(name, answer.choice, round(answer.confidence, 3))

Compare two things, not one. Per language, record the agreement rate against your labels and the confidence distribution. The pair is what tells you whether you can keep the automation you already run in English.

If you want a size to aim at rather than a vibe, the honest version is: enough items that a 5-point difference between two languages would not be noise, and enough per label class that a failure in a rare class would show up at all. Work that out for your own traffic volume, and write it down before you look at the results.

Step 2 — Read the confidence distribution, not just the accuracy number

Plot, per language, the histogram of confidence on your labelled sample. Two shapes carry most of the information.

Concentrated on the high end, with agreement to match — the language behaves like English on your content. You can hold the same thresholds.

Spread into the middle — the model still answers, but it is telling you the options were not clearly separated. The confidence page is explicit that a flat distribution on a Choice usually means no option is a clear winner, and on a Score that the levels are “ambiguous, multi-dimensional, or the state doesn’t contain enough to go on”. That is a routing finding, not automatically a model failure: the right response may be to shrink the option set, add the missing context to state, or widen the review band.

The failure mode this step prevents is subtle and expensive. A team measures accuracy in German, sees a number it can live with, keeps English thresholds, and never notices that the middle of the distribution has moved — so the fraction of turns that should have gone to a human quietly fell out of the review queue. Compare the histograms before you compare the headline rates.

Step 3 — Set thresholds per language, not one global number

The confidence page’s closing advice is to “start with conservative thresholds, test with your own data, and adjust as you observe results”. Applied across languages, that means a threshold table with a language column.

The shape below is a starting shape to argue with, not a measurement from this site — every number in it is a policy decision you make from your own Step 1 results:

Two rules make the table robust. First, never lower a second-language floor below its English counterpart during a rollout — that is how a language gap becomes an incident. Second, log the threshold version next to the decision, so a later audit can tell whether a wrong answer came from the model or from a number your team changed.

Step 4 — Decide the fallback ladder before you need it

A confidence band is only useful if there is somewhere for the turn to go. Decide the destinations in advance, and pick them per language:

The option worth naming explicitly is the second-chance request. Because a request can carry several questions about one state and they are evaluated independently, you can re-ask with a narrowed option set for the same call budget you already budgeted, rather than building a second pipeline.

Practical constraints: text-only input, mixed languages, and the context budget

Three engineering facts from the models page shape any multilingual deployment, and all three are easy to forget while you are tuning thresholds.

• Text only. state must be a string, JSON object, or array of text values. Images, audio and video are not accepted as input, so pre-process them into text or structured fields before they reach the request. A scanned Chinese invoice is a text-extraction problem first and a judgement problem second.
• One request per state, questions evaluated in parallel. Every question sees the same state and they are evaluated independently, and a single request may mix Choice, Score and Noul. A bilingual ticket does not need one call per language.
• The budget is per request. 64k tokens per request, with 32k available for state plus the longest question. Non-English passages are not automatically larger or smaller — measure your own token usage instead of assuming a ratio, and filter state in code before sending it, since the jaggedness page warns that unrelated detail degrades accuracy.
• Rate limits and price. Jev is limited to 100K tokens per second and 80 requests per second, with 429 Too Many Requests over either, and it is charged per input token at $42 per billion tokens ($0.042 per million). Output tokens are free. A wider review band is therefore cheap; a fatter state is what you pay for.

A pre-ship checklist for a non-English workload

• The language of instructions matches the language of the judged content, with no translation step in between.
• A labelled sample exists per language, with a held-out slice that no threshold was tuned on.
• Agreement rate and confidence histogram are both recorded per language.
• Thresholds are per language and never below their English counterpart.
• Every confidence band has a named destination: clarify, review, or fallback.
• Counting, date arithmetic and numeric interpolation live in code, not in questions.
• state is filtered before sending, and its token size is measured rather than estimated.
• The threshold version is logged beside each decision.

Where this fits with the rest of your pipeline

Language support is one dimension of the same discipline the rest of this site describes: Jev makes the judgement, your code owns the decision. The pieces that pair with it are confidence and fallback for the mechanics behind the bands above, designing state and questions for structuring bilingual content so a single request can judge it, and the three primitives for choosing between Choice, Score and Noul in a test harness.

If you want to see a multilingual decision already worked end to end, the multilingual review moderation case runs a request and reads its response parameter by parameter, and the content moderation scenario looks at the same shape at product scale. When a threshold table starts producing odd results rather than a stable split, troubleshooting is the first place to look, and if the review band turns out to be the expensive part of your pipeline, cost and latency covers what actually drives the bill.