Jev Troubleshooting: Common Errors and Proven Fixes

Every Jev integration hits the same handful of failures, and almost all of them have short fixes. This guide is the reference: a symptom-to-fix table for HTTP errors, a section on the JSON parsing traps that account for most “the API is broken” reports, and what to do when confidence values look off. Work through it top to bottom and you will resolve the vast majority of issues without filing anything.

TL;DR: 401 means the key; 404 means the model slug; 429 means slow down and back off; parse failures mean the message content was not JSON-loaded; weird confidence usually means a vague question. Check the table below before anything else, and confirm the exact model slug on the OpenRouter model page first — it is the most common culprit.

HTTP errors: symptom, cause, fix

A minimal retry wrapper covers 429 and 5xx behavior in one place:

import json, os, time, requests

def ask_jev_with_retry(prompt: str, max_retries: int = 3) -> dict:
    # Confirm the exact model slug on the OpenRouter model page
    for attempt in range(max_retries):
        resp = requests.post(
            "https://openrouter.ai/api/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
            json={
                "model": "typesafe/jev-1.13",
                "messages": [{"role": "user", "content": prompt}],
            },
            timeout=30,
        )
        if resp.status_code == 200:
            return json.loads(resp.json()["choices"][0]["message"]["content"])
        if resp.status_code in (429, 500, 502, 503):
            time.sleep(2 ** attempt)  # exponential backoff
            continue
        resp.raise_for_status()  # 401/404/400 will raise here with context
    raise RuntimeError("Jev call failed after retries")

The wrapper returns the fixture-shaped answer on success; the response shape shown is an example fixture and official field names should be confirmed in the official documentation. For the official TypeSafe AI channel’s own error codes and retry guidance, follow the official documentation at typesafe.ai.

Parse failures: the usual suspects

These account for most “the API is broken” reports that turn out to be client-side:

1. Treating content as a dict. The message content is a JSON string. resp.json()["choices"][0]["message"]["content"] gives you a string; json.loads it before accessing answer or confidence.
2. Assuming the envelope is the answer. answer lives inside the parsed content, not at the top level of the HTTP response. The HTTP envelope is OpenAI-compatible; the judgment payload is inside it — example fixture, confirm exact field names in the official documentation:

{
  "answer": "billing",
  "confidence": 0.94,
  "rationale": "The message disputes a charge amount rather than reporting a bug."
}

3. No defense against a malformed payload. Rare, but wrap the parse and retry once before paging anyone:

try:
    result = json.loads(resp.json()["choices"][0]["message"]["content"])
except (json.JSONDecodeError, KeyError):
    result = ask_jev_with_retry(prompt)  # one retry, then alert

The retry returns a fresh fixture-shaped answer; the shape shown is an example fixture and official field names should be confirmed in the official documentation.

Confidence anomalies

When confidence looks wrong, work down this list:

• All answers very high. The question probably has no real boundary — everything looks obviously yes or no because the criterion is trivial. Tighten the criterion.
• All answers low. The question is underspecified or the inputs are noisier than the question assumes; add boundaries and few-shot examples per the question-design guide.
• Confidence swinging for similar inputs. Normalize inputs before judging (strip HTML, truncate long text deterministically) and make sure the same scale and option list is used on every call.
• Confidence high but answers wrong on your labels. Re-check the option definitions for overlap; mutually exclusive definitions are covered in the state-and-questions guide.

If none of these move the needle, isolate one input and run it through the first-call tutorial’s minimal request — most residual issues turn out to be state accidentally included in the prompt. The product FAQ scoring case shows a full pipeline where these checks run automatically.

401 and 403 auth errors: a step-by-step triage order

A 401 Unauthorized and a 403 Forbidden look similar but mean different things, and conflating them wastes time. Work through these steps in order before changing anything in your code.

Step 1 — Verify the key is present in the right scope. Print it in the same shell process that fires the request. Environment variables set in one terminal session are not visible to a subprocess spawned from a different shell or a Docker container without explicit passing. echo $OPENROUTER_API_KEY in the shell that runs your script should print a non-empty value.

Step 2 — Confirm the Bearer prefix. The Authorization header must read Bearer sk-or-..., not just the raw key. A missing space or a doubled prefix both produce 401.

Step 3 — Check key scope on OpenRouter. Some keys are scoped to specific models or credit limits. A 403 (rather than 401) usually means the key is valid but lacks permission for the model you requested. Log into OpenRouter and inspect the key’s permissions and remaining balance before regenerating.

Step 4 — Rule out clock skew. Some proxy layers reject tokens with a timestamp drift over 5 minutes. If you are running inside a VM or container that was suspended and resumed, date may be stale. Resync with ntpdate or restart the time daemon.

Step 5 — Rotate and retest in isolation. If the above do not resolve it, create a fresh key and run the minimal request from the first api call guide — no wrappers, no middleware, just a raw curl. If that works, the issue is in your integration layer, not the key.

A 403 that persists after key rotation is almost always a spend cap or model-access restriction, not a code bug. Check the OpenRouter dashboard spend tab first.

429 rate limiting: backoff with jitter

A bare time.sleep(2 ** attempt) works but can amplify the problem when multiple workers hit the same limit simultaneously — every worker wakes at the same second and collides again. Adding jitter breaks the thundering-herd pattern.

The standard approach is “full jitter”: sleep for a random duration between zero and the exponential cap.

import random, time

def backoff_sleep(attempt: int, base: float = 1.0, cap: float = 60.0) -> None:
    ceiling = min(cap, base * (2 ** attempt))
    time.sleep(random.uniform(0, ceiling))

Usage in the retry loop:

if resp.status_code == 429:
    retry_after = int(resp.headers.get("Retry-After", 0))
    wait = max(retry_after, random.uniform(0, min(60, 1.0 * (2 ** attempt))))
    time.sleep(wait)
    continue

The Retry-After header, when present, is authoritative — always respect it and use jitter only as the floor, not a replacement.

Per-key vs. per-account limits. OpenRouter enforces both. A 429 you cannot explain by request volume usually means the key hit a per-key RPM cap even though account-level credits are fine. The response body often says which limit was hit — parse it before assuming you need to reduce total throughput.

For sustained high-volume use, review the latency and cost tradeoffs covered in cost and latency guide — batching or queuing upstream often eliminates 429s without touching retry logic at all.

Stream interruptions and mid-response truncation

Streaming responses ("stream": true) are not atomic. The connection can drop partway through, leaving you with a partial JSON fragment that cannot be decoded. Three failure modes appear most often:

1. Premature connection close. The server closes the SSE stream before a [DONE] sentinel. Your client receives valid JSON for the first N chunks, then EOF. Detection: your stream accumulator never sees [DONE]. Fix: set a generous timeout on the read side (not just the connect side) and treat EOF-without-DONE as a retriable error.

import httpx

with httpx.stream("POST", url, headers=headers, json=body, timeout=120) as r:
    buffer = ""
    for line in r.iter_lines():
        if line.startswith("data: "):
            chunk = line[6:]
            if chunk == "[DONE]":
                break
            buffer += json.loads(chunk)["choices"][0]["delta"].get("content", "")
    # if loop exits without [DONE], buffer is partial — retry

2. Mid-token JSON boundary. When the model’s structured JSON output is streamed, a } or " can land in a different chunk than its opening. Never parse chunks individually — accumulate the full content string, then json.loads the entire thing after [DONE].

3. Gateway timeout at 30 s. Some reverse proxies cut connections at 30 seconds regardless of SSE framing. If you are running behind nginx or a cloud load balancer, increase proxy_read_timeout or the equivalent. The symptom is a clean TCP close exactly at 30 s, not a Jev-side error.

For non-streaming calls that simply time out, a timeout=(5, 90) tuple (connect, read) is more precise than a single value.

JSON parse failures: three root causes beyond the obvious

Beyond the “forgot to json.loads” mistake covered above, three subtler causes account for most parse failures that survive the first fix.

Root cause 1 — Model version incompatibility. Jev api output format is tied to the model slug. If you pinned typesafe/jev-1.13 but a proxy layer rewrote the model field to a different version, the response may return plain prose instead of structured JSON. Log the actual model field in the response (it echoes back the model used) and compare it to what you sent.

Root cause 2 — Proxy content rewriting. Some API aggregators inject a disclaimer or wrap the content in an outer envelope before returning it to the client. The symptom: json.loads raises but resp.text shows valid JSON buried inside a larger string. Test by calling OpenRouter directly (bypassing any proxy) to isolate the layer that is mangling the payload. See api channels guide for how to verify you are on the official channel.

Root cause 3 — Prompt format pollution. If your messages array contains a partially-serialized JSON object as a string (e.g., you did json.dumps({"question": ...}) and then embedded the result in a user message that already has quotes), the model may echo back a double-encoded string. The fix is to send the question text as plain text, not as a pre-serialized JSON blob inside the user message.

# wrong — embeds serialized JSON inside the prompt
messages = [{"role": "user", "content": json.dumps({"question": text})}]

# correct — plain text question, structured output is the model's job
messages = [{"role": "user", "content": text}]

A quick diagnostic: if json.loads(content) raises JSONDecodeError but json.loads(json.loads(content)) succeeds, you are double-encoding.

Context length overflow and silent truncation

When the total token count of your messages array exceeds the model’s context window, the API returns a 400 with a message containing context_length_exceeded or similar. But a more insidious failure is when a proxy silently truncates earlier messages to fit — the call succeeds with a 200, but the model has lost part of the conversation history and starts giving incoherent or off-base answers.

How to detect silent truncation. Check the usage.prompt_tokens field in every response and compare it to your expected count. If prompt_tokens is significantly lower than what you calculated, a layer upstream is trimming your input.

usage = resp.json().get("usage", {})
if usage.get("prompt_tokens", 0) < expected_tokens * 0.9:
    raise RuntimeError(f"Prompt truncated: got {usage['prompt_tokens']} tokens")

Preventing overflow. For multi-turn sessions, count tokens before sending and evict the oldest user/assistant pairs when you approach 80% of the context limit. Keep the system message and the most recent user turn intact — those carry the most signal. The state and questions guide covers conversation state management in detail.

For batch classification pipelines that send long documents, chunk the input and aggregate results rather than cramming everything into one call. A single call with 12,000 tokens of text that gets truncated silently is worse than two clean calls of 6,000 tokens each.

400 errors from mixing model families in the same context

Mixing model families in the same conversation thread is a common source of 400 Bad Request errors that looks mysterious at first because the request body seems fine. The scenario: you started a session using one model, stored the messages array, then switched to a different model for a subsequent call and passed the same history.

Some model families impose different constraints on message structure. A system message at a non-leading position, an empty content field, or a tool_calls field from a model version that the new model does not recognize — any of these can trigger a 400 that the error message only vaguely describes.

Diagnosis checklist:

1. Inspect error.message in the 400 body — it usually names the offending field.
2. Strip the messages array to just the last user turn and resend. If that succeeds, the issue is in historical message structure, not the current prompt.
3. Search for "tool_calls" or "function_call" keys in your stored messages — these are model-version-specific and must be stripped when switching model families.
4. Check that no content field is null or an empty string; some model families reject empty assistant messages that others accept.

def sanitize_messages(messages: list) -> list:
    clean = []
    for m in messages:
        if not m.get("content") and not m.get("tool_calls"):
            continue  # drop empty assistant turns
        entry = {"role": m["role"], "content": m.get("content") or ""}
        clean.append(entry)
    return clean

If you need to switch models mid-session, the safest approach is to summarize the conversation so far into a single system-message recap rather than passing the raw history across model boundaries.

This guide applies to Jev 1.13.