Jev Model's Three Primitives: Judgment, Choice, Scoring

Everything the Jev model does reduces to three question primitives: judgment, choice and scoring. TypeSafe AI built the model around them, and once you internalize the mapping from problem to primitive, most integration decisions make themselves. This guide shows the request shape and response shape for each primitive, the boundary conditions and error patterns you need to know before shipping, how to pick between them, their cost and latency profiles, and how they compose in real pipelines. All examples run through OpenRouter’s OpenAI-compatible endpoint.

TL;DR: Judgment questions return yes/no/unclear; choice questions return one of the options you enumerate; scoring questions return a number on a scale you define. Every response carries a confidence score and a rationale. Pick the primitive whose answer space matches your decision, and chain them when a workflow needs more than one decision.

The three primitives side by side

One rule of thumb covers most cases: if the answer is a yes/no, use judgment; if it is a which, use choice; if it is a how much, use scoring.

Primitive 1: judgment

Judgment is a bounded yes/no with an escape hatch — unclear is a legitimate answer when the input does not give enough information.

# Confirm the exact model slug on the OpenRouter model page
curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe/jev-1.13",
    "messages": [
      {
        "role": "user",
        "content": "Judgment question: is this refund request covered by the 30-day policy? Request: \"...\""
      }
    ]
  }'

A standard chat completions request; the judgment semantics are carried by phrasing the content as a judgment question.

Response — example fixture, confirm exact field names in the official documentation:

{
  "answer": "yes",
  "confidence": 0.97,
  "rationale": "The purchase date is within 30 days and the item is unopened."
}

Use judgment for moderation flags, policy checks, duplicate detection and any gate that either opens or does not.

When judgment fits — and when it doesn’t. Judgment works best when the decision is genuinely binary at the business level. If you find yourself wanting to distinguish three states — “yes”, “no” and “maybe escalate” — that is usually a choice question with three options, not a judgment question. Similarly, if you need a degree of truth rather than a binary gate, scoring is a better fit.

Handling unclear in production. Always write an explicit branch for unclear. A common mistake is to treat it as “no” and reject the item silently. The better default is to enqueue it for human review with the rationale field attached. See the confidence and fallback guide for threshold patterns.

Primitive 2: choice

Choice requires you to enumerate the options. That constraint is the feature: the answer is guaranteed to be one of them, so your switch statement is safe.

import json, os, requests

prompt = (
    "Choice question: which team should own this email? "
    "Options: sales, billing, support. Email: \"...\""
)
resp = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
    json={
        # Confirm the exact model slug on the OpenRouter model page
        "model": "typesafe/jev-1.13",
        "messages": [{"role": "user", "content": prompt}],
    },
    timeout=30,
)
route = json.loads(resp.json()["choices"][0]["message"]["content"])

The request enumerates the options inside the question text; confirm the exact model slug on the OpenRouter model page before shipping.

Response — example fixture, confirm exact field names in the official documentation:

{
  "answer": "billing",
  "confidence": 0.94,
  "rationale": "The email disputes a charge amount rather than asking about a product feature."
}

Use choice for ticket routing, intent classification and any “pick one from this list” decision. The email routing case shows a three-team setup end to end.

Option label design matters. Labels that overlap in meaning — for example “returns” and “refunds” as separate options — push confidence down and make rationale hard to act on. Prefer mutually exclusive, collectively exhaustive labels. If you can’t guarantee mutual exclusivity, add an “other” option as a catch-all and route it to a human.

Distinguishing choice from judgment. A judgment question is a special case of a two-option choice question, but the Jev API treats them differently: judgment always produces unclear as a third path, while a two-option choice question forces a definitive answer between the two you listed. Use judgment when you want the explicit “not enough information” escape hatch; use a two-option choice when the answer must be one of your two labels even under uncertainty.

Primitive 3: scoring

Scoring returns a number on a scale you define, plus the scale echoed back so it is never ambiguous.

import json, os, requests

prompt = "Scoring question (1-10): how well does this FAQ entry answer the user question? Entry: \"...\""
resp = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
    json={
        # Confirm the exact model slug on the OpenRouter model page
        "model": "typesafe/jev-1.13",
        "messages": [{"role": "user", "content": prompt}],
    },
    timeout=30,
)
score = json.loads(resp.json()["choices"][0]["message"]["content"])

The scale lives in the question phrasing; keep it consistent across calls so scores are comparable.

Response — example fixture, confirm exact field names in the official documentation:

{
  "answer": 8,
  "scale": [1, 10],
  "confidence": 0.86,
  "rationale": "The entry answers the core question but skips the edge case mentioned in the question."
}

Use scoring for quality bars, reranking candidates and prioritizing queues.

Scale design. A 1–10 integer scale is the most common choice and gives enough resolution for ranking without over-specifying. Avoid scales wider than 1–100: they produce false precision and make the rationale harder to verify. If you need to compare scores across different pipeline stages, lock the scale in a shared constant rather than re-specifying it per call.

Using confidence alongside the score. The confidence field tells you how sure the model is that the score is accurate, not how high the score is. A score of 9 with confidence 0.55 should be treated differently from a score of 9 with confidence 0.92. In practice, set a confidence floor (e.g. 0.75) below which you hold the item for human review regardless of the score value. The cost and latency guide shows how to budget for those review queues.

Composing the primitives

Real workflows chain them. A support pipeline might judge “is this actionable by a bot?”, then choose the owning team, then score the drafted reply before sending. A useful way to think about composition:

1. Judgment as the gate. Cheap yes/no first; unclear or “no” exits early to a human.
2. Choice as the fork. Once past the gate, route to the right owner.
3. Scoring as the quality bar. At the output end, score the result and hold anything under threshold.

Keep each call to one primitive — mixing a choice and a score in one question muddies the answer space and makes confidence harder to interpret. Question wording, option enumeration and few-shot examples are covered in depth in the state and questions guide.

Boundary conditions and error patterns

Understanding where each primitive breaks down saves debugging time in production.

Judgment edge cases.

• Input is too short to judge. A single-word input (“refund”) almost always produces unclear. Add a minimum length gate before calling Jev, or use a first API call pattern that requests clarification from the user first.
• Question contains two independent yes/no claims. “Is this both urgent and a billing issue?” conflates two questions. Split them into two calls; otherwise rationale will address only the clause the model found more prominent.
• Negation in the question. “Is this NOT a duplicate?” inverts the natural polarity. Rephrase to the positive form and flip the answer downstream.

Choice edge cases.

• Option count over ~10. Latency increases and confidence drops. Use a two-tier routing: coarse choice first, then a refined choice among the top candidates.
• Input matches none of the options. If you have no “other” catch-all, the model is forced to return the closest option. Always include an “other” or “escalate” option for inputs that genuinely don’t fit your taxonomy.
• Options listed in different languages than the input. Keep option labels and input in the same language. Cross-language option matching degrades reliability. See the troubleshooting guide for locale-related issues.

Scoring edge cases.

• Scale not stated in the question. Without an explicit scale, the model may default to an internal heuristic. Always include the scale, e.g. “(1-5)” or “(0.0 to 1.0)”, in the question text.
• Comparing scores from different scales. A 7/10 and a 3.5/5 are mathematically equivalent but programmatically different. Normalise before storing if you need cross-scale comparison.
• Using scoring where a threshold binary would be faster. If you only care whether the score clears a threshold — “is quality above 7?” — consider a judgment question instead. It is cheaper and returns faster.

Cost and latency trade-offs across the three jev api primitives

The three primitives are not equivalent in cost. Understanding the relative profiles helps you build pipelines that stay within budget. The cost and latency guide covers absolute numbers; the relative patterns here hold across model versions.

Practical implications for pipeline design.

Put judgment gates early in pipelines: they are the cheapest way to drop items that don’t need further processing. Reserve scoring for the final quality check where the cost is amortised over the value of a high-quality output. Choice sits in the middle — it is more expensive than judgment but you usually need it only once per pipeline branch.

For latency-sensitive paths (real-time chat, sub-second SLA), favour judgment and choice; defer scoring to an asynchronous quality-monitoring pass that does not block the response. The coding agent skill guide shows an async scoring pattern used in code review pipelines.

If you are uncertain which primitive fits, try judgment first. You can always add a scoring pass downstream once the pipeline is stable and the judgment results tell you which items warrant deeper evaluation.

Common misconceptions about Jev AI primitives

“I should use scoring to express confidence in my data.” No — the confidence field in every response already captures model certainty. Scoring is for evaluating the content of an item on a dimension you define, not for re-expressing the model’s own uncertainty about its answer.

“Choice is just a list of judgment calls.” Choice and judgment behave differently in the API. Judgment always has unclear as a third path; choice forces a selection from your enumerated list. A three-option choice question is not the same as three sequential judgment questions — it is a single, mutually exclusive classification.

“A higher confidence score means the answer is correct.” Confidence reflects calibration against the model’s training distribution, not ground truth. A confidence of 0.99 on a judgment call can still be wrong if the input is adversarial or out-of-distribution. Use confidence as one signal among several; pair it with human review queues for high-stakes decisions.

“I can reuse the same scoring question across different semantic domains.” A 1–10 quality score on a customer email and a 1–10 urgency score on a support ticket are semantically different scales even though they share a range. Never compare raw scores across different question phrasings. If you need cross-domain comparability, normalise by converting each score to a percentile relative to your historical distribution for that specific question.

“Unclear means the model failed.” Unclear is a designed, first-class answer. It is the model correctly identifying insufficient signal. A pipeline that never returns unclear may be suppressing legitimate uncertainty rather than surfacing it. Monitor the rate of unclear responses as a health signal: a sudden spike often indicates an input distribution shift worth investigating. See the what is Jev AI guide for more on the model’s design philosophy.

Frequently asked questions about Jev AI three primitives

Can I use a judgment question to gate an expensive scoring call? Yes, and this is the recommended pattern. Judge whether an item is worth scoring before spending the tokens. The gate filters noise early and keeps your average cost per pipeline run low.

What happens if my choice options contain spaces or special characters? Option labels are matched by the model semantically, not by exact string comparison. Spaces and common punctuation are fine. Avoid labels that are themselves long sentences — keep them to one to three words each so the answer field maps cleanly to a downstream variable name.

How do I version my primitive questions? Store the exact question template in your codebase as a constant, versioned alongside your model slug. When you update the question phrasing, treat it as a new version: re-run calibration on a held-out sample before deploying. The get API key guide covers environment separation useful for staging this kind of migration.

Is there a way to force a minimum confidence threshold in the API call? Not directly — the confidence threshold is enforced in your application layer. Read the confidence field from the response and branch accordingly. The confidence and fallback guide has copy-paste threshold patterns.

How does Jev differ from using a general-purpose LLM for the same decisions? Jev is trained to output structured decisions with calibrated confidence, not free-form text. A general LLM asked a judgment question may produce a paragraph where Jev produces a one-line JSON object. The Jev vs LLMs guide covers the trade-offs in detail.

This guide applies to Jev 1.13.