Jev AI Patterns: Speculative Fan-Out, Noul Screens and Companion Nouls
Jev evaluates every question in a request in parallel and in isolation against the same state. That single design decision — one state, many questions, one response — is what makes a category of architectural patterns possible, and TypeSafe documents six of them. They are not prompt tricks. They are decisions about how you shape the request and where the branching lives, and every one of them ends the same way: Jev returns probabilities, your code decides.
This page walks the six patterns with the worked examples, thresholds and recorded answer values from the official documentation, then covers the batch-size evidence and the arithmetic you need before shipping.
Provenance: every pattern name, threshold, code shape and answer value below comes from TypeSafe’s documentation at
docs.typesafe.ai/patternsand its linked cookbooks, read on 2026-11-09. Where the docs give a recordedjev-1.14.0answer, the number on this page is that number. Nothing here is a made-up illustration.
The six patterns at a glance
| Pattern | What it does | What it buys you |
|---|---|---|
| Speculative fan-out | Send many questions at once, including ones that only matter for some inputs | Cost, speed |
| Screen with a Noul first | One cheap yes/no question gates an expensive stage | Cost, speed |
| Companion Noul | A second question asking whether any option fits at all | Reliability |
| Routing by confidence | Send unsettled Score answers to a person | Reliability, safety |
| Routing by probability | Higher probability thresholds for riskier actions | Reliability, safety |
| Composite scoring | Several atomic scores combined with weights in your code | Cost, reliability, speed |
The shared premise: a model that answers one narrow question reliably is worth more than a model you asked to reason about six things at once. So you decompose the judgment, and you keep the composition — the weights, the thresholds, the branches — in code where you can review it.
Pattern 1 — Speculative fan-out
The documented case is support-ticket triage. You need the ticket’s category, and if it is a bug report you also need its severity and whether the report has reproducible steps, and if it is a billing message you also want to know whether a refund was requested. Frustration is useful regardless.
The naive build makes two calls: classify, then ask follow-ups. Jev’s version sends all five questions in one request — a choice for category, a score for bug severity, a noul for reproducible steps, a noul for refund requested, a score for frustration — and lets code discard what does not apply. The docs call the conditional ones speculative questions and give the reason plainly: additional questions usually have little effect on response time, so you pay almost nothing for asking questions you may ignore.
The branching is ordinary Python:
category = response.answers["category"]
bug_severity = response.answers["bug_severity"]
bug_repro = response.answers["has_reproducible_steps"]
refund = response.answers["refund_requested"]
frustration = response.answers["frustration"]
if category.choice == "bug_report":
if bug_severity.score > 1.5 and bug_repro.noul > 0.6:
escalate_to_engineering(ticket_id, severity="high")
else:
add_to_bug_backlog(ticket_id)
elif category.choice == "billing":
if refund.noul > 0.7:
route_to_billing_with_flag(ticket_id, refund_likely=True)
else:
route_to_billing(ticket_id)
elif category.choice == "feature_request":
log_feature_request(ticket_id)
if frustration.score > 1.5:
flag_for_priority_response(ticket_id)
Two things are worth noticing. First, the whole decision tree — all three branches plus the unconditional frustration flag — came from one call. Second, frustration sits outside the branch structure entirely, which is the pattern’s real payoff: when each question is evaluated independently against the state, a question can be useful to more than one code path and you never have to decide that in advance.
Pattern 2 — Screen with a Noul first
Fan-out is the right default. The exception is when the next step costs something the first question does not — and the docs name exactly three cases: a bigger state (the customer record, the full thread, an attachment you have to fetch), a slower model (an LLM call), or a person. Then a cheap screen pays for itself.
The documented example is a shared inbox. Everything arrives in one stream: order questions, newsletters, payment receipts, thank-you notes, and the occasional “just so you know”. Full triage means fetching the sender’s account and order history and asking several questions. The screen asks one thing the message alone can settle: is the sender waiting for a reply? — with a false criterion that lists the kinds of message that have nothing to answer.
The recorded answers for five messages make the value obvious:
| Message | expects_reply |
|---|---|
| “I ordered a standing desk two weeks ago and the tracking hasn’t updated… I’m moving on the 30th and need it before then.” | 0.98 |
| “Just so you know, the desk arrived with one of the cross-bar screws missing. I had a spare so it’s built now.” | 0.44 |
| “Thanks, the desk arrived yesterday and it’s great.” | 0.10 |
| “20% off every desk and chair until Sunday. Unsubscribe | View in browser” | 0.02 |
| “Your payment of $129.00 was successful… replies to this address are not monitored.” | 0.02 |
Two thresholds split that column into three outcomes, and the middle outcome is the whole point: above 0.7 is a yes, below 0.3 is a no, and anything between them gets its own path instead of being rounded to one side. The 0.44 missing-screw note is a real message from a real person that mentions a problem but asks for nothing — filed away it would be a lost customer signal, triaged immediately it would burn a full fetch for nothing. It goes to a human for a glance.
The screen call itself cost input_tokens: 414 and output_tokens: 21 in the docs’ recorded response. Only the 0.98 message triggered the full triage request (input_tokens: 803, output_tokens: 84), and in that request repeat_contact — a question answered from the fetched customer record — read 0.99 with previous contacts on file and 0.05 with none. Same message, different state, different answer: that is why the fetch had to happen after the screen rather than inside it.
Pattern 3 — Companion Noul
A choice question always picks an option. Ask which of three products a message is about, and a message about none of them still gets an answer, possibly with high probability — the model is answering the question it was asked and picks the closest option.
In the documented example a desk lamp message came back desk_pro at 0.96, with confidence 0.94. Nothing in that answer says the message is not about your desk. Of the three options, a desk lamp is closest to a desk, and the distribution is confident about a wrong question.
The fix is a second question in the same request: a Noul that asks whether any of the options fits at all. Its recorded value for that message was 0.02. Four messages side by side:
| Message | Choice | Top probability | Companion Noul |
|---|---|---|---|
| “The motor on my standing desk has started making a grinding noise… the 140 cm one” | desk_pro | 1.00 | 0.98 |
| “The chair arrived but the monitor arm didn’t. Can you check where the arm is?” | monitor_arm | 1.00 | 0.99 |
| “My desk lamp stopped working after two weeks…” | desk_pro | 0.96 | 0.02 |
| “I’m a recruiter working with several furniture retailers and wondered if your head of operations is open to a quick call” | desk_pro | 0.81 | 0.02 |
Read the Noul first. Below 0.3 the Choice is noise — send the message to the general inbox and do not look at the pick. Above 0.7, the message is about one of your products but the Choice still has to be strong enough (probabilities[choice] > 0.7 in the docs’ code) to route straight to that team; otherwise a general agent decides, with the top option passed along as a hint.
The docs are explicit that the obvious alternative fails: adding other: "Something else" to the Choice looks like less work, but other competes with the real options for probability, so a message that is partly about a product and partly about something else splits its probability between them and neither number tells your code what to do. A separate Noul with concrete true and false criteria usually works better in TypeSafe’s own tests.
box has the same failure shape — it always returns a location in the image — so the docs apply the same remedy: pair it with a Noul such as “Is there a coffee mug in photo?” so the code knows whether the box marks a mug at all. (Box questions require Jev 1.14.)
Pattern 4 — Routing by confidence
Now the answers are settled but the decision has stakes. A score answer gives you a position on the scale and a confidence from 0 to 1 that says how far the probability sits from the most likely level.
The documented case is bug-report triage on a five-level severity Score, where a cosmetic issue waits in the backlog, a broken feature enters the sprint, and anything that stops users completing a core task pages the on-call engineer. One report — “Notes gone after the update, or maybe deleted” — came back with score 3.19 and confidence 0.34, because the probability split between level 4 (0.59, the update lost the notes) and level 1 (0.14, the user deleted them). A score of 3.19 on its own pages a human at night for a problem that may not exist. Confidence first:
severity = response.answers["severity"]
if severity.confidence < 0.5:
send_to_triage(report_id)
elif severity.score < 1.5:
add_to_backlog(report_id)
elif severity.score < 2.5:
add_to_sprint(report_id)
else:
message_on_call(report_id)
Recorded outcomes for four reports: a typo scores 0.0 at confidence 1.0 (backlog), a PDF export failure over 50 pages scores 2.0 at 1.0 (sprint), checkout returning 500 for every customer scores 3.36 at 0.7 (page on-call), and the ambiguous notes report goes to triage. The gate fires on exactly one of the four — which is what a good gate should do.
There is a trap the docs call out. A Score’s confidence treats probability on a neighbouring level as barely any spread, so an answer split between two adjacent levels can pass a confidence check and still land on the wrong side of a threshold you set between them. When the decision depends on one side of a boundary, add up the probabilities of the levels on the side that matters: probabilities[3] + probabilities[4] >= 0.3 is a “real chance users are blocked” test that does not care whether the model was confident.
Pattern 5 — Routing by probability
Same idea, applied to choice, with a different number. Every Jev answer carries probabilities, so for each action your code can take you decide how likely the intent has to be before you take it — and riskier actions get higher bars.
The documented example is voice banking. Below 0.7 on any intent the command did not settle, so it goes to a support agent. check_balance at 0.7 is fine; showing the wrong balance screen is recoverable. But approve_transfer splits into two bands: 0.7 to 0.9 asks the user to clarify, above 0.9 opens the transfer for confirmation.
The note attached to that example is the most useful line in the docs for anyone building on top of these patterns: the thresholds only choose which workflow runs; none of them moves money. A clear transfer request opens a confirmation screen with amount and recipient, and the transfer runs only after the user confirms. That is deliberate, because a hesitant command can still have a clear intent — with jev-1.14.0, “Um, the transfer thing… yeah, go ahead I guess.” scored 0.99 for approve_transfer. The probability answered which action the user wants; it said nothing about whether the user had checked the details. Only your UI can settle that.
The docs also answer the obvious question: why not threshold on confidence here? For a Choice with a fixed option set, confidence is computed from the top probability, so a second threshold on it adds no independent evidence — and probabilities[choice] is easier to read as risk. A rule that acts only above 0.9 says, in plain words, “act when the chance of the wrong option is under 10%”.
Pattern 6 — Composite scoring
Ranking is where single-question judgments break down, because “how good is this candidate” is not one judgment. Composite scoring breaks it into independent dimensions, scores each with its own score question, and combines them with weights in code.
The documented example scores resumes for engineering roles on four dimensions — Python depth, team leadership, system design, generalist breadth — in one request. Each score is normalized to 0–1 (divided by the top level, 4, for a five-level rubric) and then weighted twice, for two different roles:
py = response.answers["python_depth"].score / 4
lead = response.answers["team_leadership"].score / 4
arch = response.answers["system_design"].score / 4
general = response.answers["generalist"].score / 4
# Senior IC
ic_score = (0.40 * py) + (0.10 * lead) + (0.40 * arch) + (0.10 * general)
# Engineering Manager
em_score = (0.15 * py) + (0.40 * lead) + (0.20 * arch) + (0.25 * general)
The stated benefit is not accuracy — it is visibility. When the top-ranked candidates look wrong, you change a coefficient, not a prompt, and you can say out loud which dimension moved them. That is the same argument as the atomic-questions principle in the introduction: instead of “rate this startup pitch”, ask separately about market size, technical feasibility and differentiation, then combine with your own formula.
Does batching change the answers?
It should not, and the docs measure it. The parallel-questions cookbook takes the Wikipedia article on the GDPR (53,777 characters, pinned revision 1363040264) and asks 13 questions about it — 8 Nouls, 2 Choices, 3 Scores — twice over: all 13 in one request, and 13 requests of one question each, five repeats per strategy, comparing run-to-run standard deviation.
Result: batching adds no meaningful noise. 7 of the 13 answers came back identical across all five repeats under both strategies, standard deviation exactly 0.0, and the rest moved by at most 0.015 under either strategy.
Cost and latency do move. The document dominates every request, so 13 single-question calls pay for it 13 times in 13 round trips, while the batched call pays once. In that run the batched version was 12.2x cheaper and 11.8x faster with no change in answers — and the docs note that the bigger the document, the nearer the saving gets to a full Nx.
The arithmetic before you ship
The documented price is $42 per billion input tokens, or $0.042 per million, with output tokens free. Token counts come back in usage.input_tokens, so use your own numbers rather than a guess — but the recorded responses on this page are enough to price a screening design:
| Request | input_tokens | Cost at $0.042/Mtok | 1,000 calls |
|---|---|---|---|
| Noul screen, message only (docs’ recorded value) | 414 | $0.0000174 | $0.0174 |
| Full triage with customer record (docs’ recorded value) | 803 | $0.0000337 | $0.0337 |
| A request that fills the default 64k per-request context | 65,536 | $0.00275 | $2.75 |
That table is the economic argument for screening compressed into two rows: the screen costs about half a cent per thousand messages, so routing 80% of a stream away from the triage path saves more than the screen costs by a wide margin. Anything that keeps you under the default 64k per-request limit is three orders of magnitude cheaper than the LLM call you were considering.
Two limits govern the shapes above: 64k for the whole request and 32k for any single question, both raised to 262k by the TypeSafe-Danger-Zone-Long-Context: true header. The docs are blunt that long context is slower and less accurate, so turn it on for the requests that need it rather than as a default.
Choosing between them
- Nothing to gate, one state → speculative fan-out. Ask everything, ignore what does not apply.
- The next step needs a bigger state, a slower model, or a person → screen with a Noul first, with three outcomes, not two.
- Any
choicewhere “none of these” is a real possibility → companion Noul, and read the Noul before the Choice. - A
scorewhere a wrong route is expensive → check confidence, then route on the score; add the level-probability sum when a boundary matters. - A
choicethat triggers an action with consequences → threshold onprobabilities[choice], one threshold per action, and keep the irreversible step behind a confirmation. - A ranking → composite scoring, atomic scores, weights in code.
All six keep the same discipline: Jev answers, code decides, and every threshold is a named constant someone can review. That last part matters more than the pattern names — the docs make the point that the questions and thresholds in a TypeSafe integration are the parts a human should be reviewing, and they should live in one place so they are easy to find.
Verify your own thresholds. The docs repeat one caveat that is easy to skip: threshold values depend on your domain and on how the model performs on your data. Start conservative, measure against a labelled sample, and adjust. The 0.7 / 0.3 / 0.5 numbers on this page are starting points from the official examples, not universal constants.
FAQ
Do these patterns work on Jev 1.13? Choice, Score and Noul work on both 1.13 and 1.14; box questions need 1.14. Everything on this page except the Box note is expressible with 1.13 primitives.
Can I mix all six in one system? Yes, and the documented examples do: the shared-inbox build screens with a Noul, then sends a triage request that is itself a fan-out, whose answers are routed by probability. Each pattern is a decision about one step, not a system-level choice.
What if two questions in a fan-out disagree? They cannot “disagree” — each is evaluated in isolation against the same state, so each answer is what that question means on its own. Precedence is yours to define in code, which is precisely why the branching lives there.