How to Use Jev with Coding Agents (Cursor, Claude Code)

Coding agents are good at producing changes and mediocre at drawing hard lines about them. “Is this acceptable?” is exactly the kind of question where a chat agent waffles — it answers in fluent prose that may or may not be right, gives you no confidence signal to escalate on, and produces nothing machine-checkable. Jev’s three primitives return structured JSON with a confidence score, so they slot neatly into agent workflows as an external referee. The agent writes the code; Jev makes the bounded calls.

This guide covers the full wiring: what a skill package is, how to lay out the files, a complete runnable example that does batch classification via Jev, the rules that keep agents from making mistakes, and how to debug a skill the agent insists it cannot see.

TL;DR: Wrap a Jev call in a script, package it as a skill directory under .claude/skills/ or ~/.claude/skills/, and give the skill a description whose first 57 characters are a self-contained trigger sentence. The agent loads the skill, runs the script when it matches, and reads the structured answer back. Keep secrets in environment variables, batch at most 10 items per call, and add a confidence gate before acting. Confirm the exact model slug on the OpenRouter model page before shipping.

Why give an agent a judgment model

An agent asked “does this change violate our logging policy?” will answer in confident prose that may or may not be right. Two problems follow immediately: the answer is not machine-checkable, and there is no confidence signal to escalate on. A Jev call fixes both. The answer comes back as structured JSON with a confidence score, so the agent’s workflow can branch deterministically. That is the core idea: the agent generates, Jev judges.

There is a second, less obvious benefit. When the agent embeds the judgment in its own reasoning, the reasoning is entangled with the generation — you cannot tell whether the policy check drove the code or the code drove the policy check. Externalising the judgment to a separate, logged call separates the two cleanly. You get an audit trail for every decision, independent of what the agent did with it.

What a skill package is — and what it is not

A skill is a directory the agent runtime loads as a named capability at session start. At minimum it contains a SKILL.md file with a frontmatter header; optionally it adds a scripts/ subdirectory with runnable helpers and a references/ subdirectory with topic-split documentation the agent can read on demand.

Three things it is not:

• Not a prompt template. A template is a string you paste; a skill ships executable code alongside instructions. When the agent needs to call a private API with auth, retry logic and response parsing, a template cannot carry any of that — a skill can.
• Not an MCP server. An MCP server exposes tools over a protocol the runtime discovers and invokes automatically. A skill is a simpler primitive: plain files the agent reads and scripts the agent runs via shell. No server process, no protocol negotiation, no persistent connection. Use MCP when you need the runtime to call typed tool functions; use a skill when you need packaged instructions plus a shell-runnable script.
• Not a plugin. Plugins extend the agent’s own capabilities. A skill extends what the agent knows about your infrastructure — your API keys, your endpoint, your field names — without modifying the agent runtime itself.

The practical test: if you can describe the capability as “run this script with these arguments and parse the JSON back”, a skill is the right packaging. If you need the runtime to register a new tool that can be called mid-generation with structured arguments, reach for MCP instead.

Directory layout and file contract

.claude/
  skills/
    jev-judge/
      SKILL.md          ← required: instructions + frontmatter
      scripts/
        judge.py        ← runnable helper the agent invokes
      references/
        field-names.md  ← topic docs the agent reads on demand

The same structure works under ~/.claude/skills/ for a user-level skill that travels with you across projects, or under .claude/skills/ in a project root for a skill scoped to one repository.

SKILL.md frontmatter fields:

The description constraint is the single most common source of “skill not triggering” bugs. The agent runtime indexes skills by description at load time and matches against the agent’s intent at invocation time. If the first 57 characters are a generic summary (“This skill helps you call the Jev API for various judgment…”), the matcher has nothing specific to latch onto. If they are a trigger sentence (“Run this to judge whether a diff violates the logging policy”), the match is clean.

Two examples:

# Too generic — will not trigger reliably
description: "This skill integrates the Jev AI judgment API into your workflow."

# Self-contained trigger sentence in first 57 characters
description: "Judge a diff or text against a policy rule using the Jev API. Invokes jev-judge/scripts/judge.py with the question and returns a confidence-scored JSON answer."

Loading and reloading: the runtime reads the skill registry once at session start. Adding a new skill or editing SKILL.md requires a session restart to take effect. Edits to scripts inside an already-loaded skill take effect immediately on the next invocation — the script is read from disk at run time, not cached.

Permissions and sandbox: scripts in the scripts/ directory run in the same sandbox as other agent-invoked shell commands. On Claude Code the user approves the first execution; subsequent calls to the same script in the same session run without prompting. Environment variables the agent has access to are available to the script. Write secrets to the environment, not to any file inside the skill directory.

A complete worked example

This example packages a Jev-based batch classifier as a skill. The agent can invoke it to classify up to 10 items at a time against a named policy.

SKILL.md:

---
name: jev-judge
description: "Judge a diff, comment, or text against a policy rule using the Jev API. Pass the question as the first argument to scripts/judge.py."
---

## When to use this skill

Call this skill whenever you need a machine-checkable answer about whether an input satisfies a rule — policy compliance, severity scoring, template matching. Do not use it for open-ended design questions.

## How to invoke

    python .claude/skills/jev-judge/scripts/judge.py "Judgment question: [your question here]"

The script reads OPENROUTER_API_KEY from the environment and writes structured JSON to stdout. Parse the `answer` and `confidence` fields.

## Confidence gate

If `confidence` is below 0.80, do not act automatically. Surface the `rationale` field and ask the user to review before proceeding.

## Batch use

To classify multiple items, call the script once per item in a loop. Keep batches at 10 items or fewer per run to stay within timeout limits.

scripts/judge.py:

import json, os, sys, time, requests

def ask_jev(question: str, retries: int = 2) -> dict:
    url = "https://openrouter.ai/api/v1/chat/completions"
    headers = {
        "Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}",
        "Content-Type": "application/json",
    }
    body = {
        # Confirm the exact model slug on the OpenRouter model page
        "model": os.environ.get("JEV_MODEL_SLUG", "typesafe/jev-1.13"),
        "messages": [{"role": "user", "content": question}],
        "usage": {"include": True},
    }
    for attempt in range(retries + 1):
        resp = requests.post(url, headers=headers, json=body, timeout=30)
        if resp.status_code == 429:
            wait = 2 ** attempt
            time.sleep(wait)
            continue
        resp.raise_for_status()
        envelope = resp.json()
        result = json.loads(envelope["choices"][0]["message"]["content"])
        # Attach token usage for observability — field names are example fixture
        usage = envelope.get("usage", {})
        result["_tokens"] = {
            "prompt": usage.get("prompt_tokens"),
            "completion": usage.get("completion_tokens"),
        }
        return result
    raise RuntimeError(f"Rate limited after {retries} retries")

if __name__ == "__main__":
    if len(sys.argv) < 2:
        print(json.dumps({"error": "Pass the judgment question as the first argument"}))
        sys.exit(1)
    question = sys.argv[1]
    result = ask_jev(question)
    print(json.dumps(result, indent=2))

Invocation by the agent:

python .claude/skills/jev-judge/scripts/judge.py \
  "Judgment question: does this diff add logging of user PII? Diff: \"user_id=${user.id} written to access.log\""

Expected output — example fixture, confirm exact field names in the official documentation:

{
  "answer": "yes",
  "confidence": 0.93,
  "rationale": "The diff writes user_id to an access log. User identifiers are PII under most logging policies.",
  "_tokens": { "prompt": 87, "completion": 41 }
}

The agent reads answer and confidence, applies the 0.80 threshold from the skill instructions, and either proceeds or surfaces the rationale for human review.

Writing instructions agents follow reliably

The script does the API call; the SKILL.md body does the cognitive work. Vague instructions produce inconsistent agent behavior. These six rules cover the patterns that break most often.

Embed the input, not just the question template. Agents copy-paste question templates and forget to substitute the actual input. Write the template with a visible placeholder and an example of the substituted form side by side:

Template:  "Judgment question: does this diff add logging of user PII? Diff: \"{diff_text}\""
Example:   "Judgment question: does this diff add logging of user PII? Diff: \"user_id=42 written to access.log\""

Put the confidence gate in the instructions, not just in your head. State the threshold and what happens below it. “If confidence is below 0.80, do not act automatically — surface the rationale and ask the user” is unambiguous. “Use your judgment about confidence” is not.

Name the environment variable explicitly. OPENROUTER_API_KEY in the instructions matches os.environ['OPENROUTER_API_KEY'] in the script. If the names diverge, the script fails with a KeyError and the agent will try to fix it by guessing — sometimes by hardcoding a value it finds nearby. Prevent this by making the expected variable name a single source of truth across both files.

Set an explicit batch limit. “Classify each item” becomes “classify 200 items in a single loop” unless the instructions say otherwise. Specify the batch size (10 items per run is a safe ceiling for 30-second timeouts), what to do when the list is longer, and whether to pause between batches.

Write the failure path, not just the happy path. The script raises on non-retry-able status codes. Tell the agent what to do when raise_for_status() fires: “If the script exits non-zero, log the raw output, report the error to the user, and do not retry automatically without human approval.” Without this, agents either retry infinitely or silently swallow errors.

Use an idempotency hint for batch work. If the agent is classifying a list of items and the session is interrupted, it will re-run the whole list unless the instructions describe how to detect already-processed items. A simple file-based progress log or a hash of the input in the output file name is enough.

Debugging: five steps when the skill does not trigger

The agent reports it cannot find the skill, or invokes the wrong behavior. Work through these layers in order — each step has a single check and a clear fix.

Step 1: description trigger sentence
   └─ Is the first 57 chars a specific, actionable sentence?
      Fix: rewrite to start with the action, not the product name.

Step 2: directory path
   └─ Is the skill at ~/.claude/skills/name/ or .claude/skills/name/?
      Fix: confirm with ls -la .claude/skills/ from the project root.

Step 3: file permissions
   └─ Can the agent runtime read SKILL.md and execute scripts/?
      Fix: chmod 644 SKILL.md; chmod 755 scripts/judge.py

Step 4: frontmatter syntax
   └─ Is the YAML valid? (trailing spaces, missing quotes, tab vs space)
      Fix: run: python -c "import yaml; yaml.safe_load(open('SKILL.md'))"

Step 5: session cache
   └─ Was the skill added after the session started?
      Fix: restart the session. The registry is read once at launch.

One additional check that catches a different class of failure: the skill triggers but the agent passes a malformed question. Read the rationale field of the actual Jev response — if it describes misunderstanding the question type, the agent copied the template without substituting the input. Add a more explicit example to the SKILL.md body.

The troubleshooting guide has the full HTTP error table for when the script itself fails; the five-step list above is specific to the skill-not-found class of problems.

Jev’s confidence and fallback inside the skill

Jev returns confidence as a 0–1 score. In a skill context, that score drives two distinct decisions.

When to act vs. when to escalate. Set a threshold in the skill instructions — 0.80 is a reasonable starting point for policy work, 0.85 for anything that changes production state. Above the threshold, the agent can act on answer directly. Below it, the skill should output the rationale and hand off to the user. Do not retry the same question with the same phrasing: a low-confidence call means the model has already seen the input and found it genuinely ambiguous. The fix is a richer question — more context, a more specific question type — not a second identical call.

When to fall back entirely. Three situations warrant skipping the Jev call and going straight to human review: the input is shorter than 10 tokens (too little context to judge reliably), the question spans more than one primitive type (a judgment and a scoring question combined), or the call returns an answer of unclear. Wire these as short-circuit conditions in the skill instructions so the agent does not spin on them.

If the input text is fewer than 10 words: skip the skill, ask the user directly.
If confidence < 0.80: surface rationale, route to human review, do not retry.
If answer == "unclear": treat as confidence < 0.80 and route the same way.

The confidence-and-fallback guide covers threshold selection in depth once you have production data to calibrate against.

What agents should delegate to Jev

Each row returns — example fixture — a structured answer such as {"answer":"no","confidence":0.91,"rationale":"..."}. Confirm exact field names in the official documentation.

Six anti-patterns

These are the six patterns that break skill-based Jev integrations most reliably. Each one has a cost proportional to how late you find it.

Stuffing the entire API reference into SKILL.md. A 4,000-word document is not a skill instruction — it is a documentation dump. The agent reads it sequentially and loses the operative rule in the noise. Keep SKILL.md to 200–400 words of actionable instructions and put detailed reference material in references/ files the agent reads on demand.

Hardcoding the API key. api_key = "sk-or-abc123" in scripts/judge.py is a credential leak waiting to happen. The script is in a repository. Repositories are committed. Commits are sometimes public, and git history persists even after you delete the file. Use os.environ['OPENROUTER_API_KEY'] and document the variable name in SKILL.md.

Sending 100 items in a single agent loop without a batch cap. The agent receives a list of 100 issues and calls the script 100 times sequentially in one session. The last call times out, the agent retries from the beginning, and you spend twice the tokens for half the results. Cap each run at 10 items, write the progress to a file, and instruct the agent to resume from the last checkpoint.

No output schema in the instructions. The agent parses the JSON result but has no documented expectation of what fields exist. When a field name changes between Jev versions, the agent silently produces wrong answers or throws a KeyError and retries indefinitely. Document the expected fields in SKILL.md as an example fixture with a note that official field names should be confirmed in the documentation.

No failure handling in the instructions. The script raises on a 500 response. The agent catches the exception, decides to fix the script, and rewrites raise_for_status() with a pass. Write the expected failure behavior explicitly: “If the script exits non-zero, report the raw error output to the user and stop.”

Using a skill where MCP is the right tool. A skill is great for “run this script with these arguments”. It is a poor fit for “expose a callable function the agent can invoke mid-generation with typed parameters”. If you find yourself wrapping the skill in elaborate prompt engineering to simulate a tool call, that is a signal to switch to MCP. See API channels for the endpoint setup that an MCP server would sit in front of.

Where this does not help

• Code generation itself. Jev answers bounded questions about code; it does not write code.
• Open-ended design discussion. “Which architecture is best?” has no enumerable answer space — that is a conversation with the coding agent, not a judgment question.
• Security sign-off as a substitute for review. A confidence score is a fast filter, not an audit. Treat Jev as the first gate before a human review queue, not a replacement for one.

Each agent question should cover one primitive type with the diff or issue embedded. The state-and-questions guide covers how to phrase prompts for stability. The first-api-call guide walks through the full request/response shape if you need to understand what the script is doing under the hood.

This guide applies to Jev 1.13.