How to Use Jev for RAG Reranking (Better Answers, Lower Cost)

TL;DR: Retrieval finds plausible chunks; reranking decides which ones actually answer the question. Use Jev’s scoring primitive to grade each candidate chunk’s relevance on a 1-10 scale, sort, cut at a threshold, and hand only the survivors to your generator. You trade a few cheap scoring calls for fewer, cleaner context tokens — better answers at lower generation cost. Minimal implementation below, under 50 lines.

What reranking means here

• Input: the user query plus N candidate chunks from your first-stage retriever (embeddings, BM25, or hybrid).
• Task type: scoring per chunk — relevance on a fixed scale.
• Output: a structured score per chunk. Example fixture:

{"answer":8,"scale":[1,10],"confidence":0.86,"rationale":"Directly defines the retry policy the query asks about."}

answer is the relevance grade, confidence tells you whether the model is sure about its own grade, and rationale is useful when you debug retrieval quality later.

Flow

User query
   |
   v
First-stage retrieval (top-50 candidates)
   |
   v
Jev scoring call per chunk (relevance, scale 1-10)
   |
   v
Sort by score (descending)
   |
   +--> score >= 7 and confidence acceptable --> keep
   |
   +--> otherwise --> drop
   |
   v
Take top-k survivors as generator context

The heavy lifting stays in your retriever; Jev only judges what was already found.

Minimal implementation

import os, json, requests

URL = "https://openrouter.ai/api/v1/chat/completions"
# Confirm the exact model slug on the OpenRouter model page.
MODEL = "typesafe/jev-1.13"

def score_chunk(query: str, chunk: str) -> dict:
    resp = requests.post(
        URL,
        headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
        json={
            "model": MODEL,
            "messages": [
                {"role": "system", "content":
                 "Score how well this passage answers the query, 1-10. "
                 "1 = irrelevant, 10 = directly and completely answers it."},
                {"role": "user", "content": f"Query: {query}\n\nPassage: {chunk[:2000]}"},
            ],
            "temperature": 0,
        },
        timeout=30,
    )
    resp.raise_for_status()
    # example fixture:
    # {"answer":8,"scale":[1,10],"confidence":0.86,"rationale":"..."}
    return json.loads(resp.json()["choices"][0]["message"]["content"])

def rerank(query, chunks, min_score=7, top_k=5):
    scored = [(score_chunk(query, c), c) for c in chunks]
    scored.sort(key=lambda x: x[0]["answer"], reverse=True)
    return [c for s, c in scored if s["answer"] >= min_score][:top_k]

rerank() returns the cleaned context list; everything else in your RAG stack stays unchanged.

Threshold suggestions

This scenario leans on Jev’s three primitives — scoring is one of the three task types — and the cost-and-latency guide explains why cheap judgment calls in front of an expensive generator usually win on total spend. The case study on RAG chunk rerank batches shows measured results.

Applies to Jev 1.13.

FAQ

• How does Jev fit into a RAG reranking step? After first-stage retrieval, score each candidate chunk’s relevance with a Jev scoring call, sort by the score, keep chunks above threshold, and pass only those to the generator.
• What score threshold should I use? On a 1-10 scale, a score of 7 is a sensible inclusion cutoff, combined with a top-k cap so the generator never sees excess context.
• Does reranking with Jev cost more than embedding-only retrieval? Scoring adds a cheap call per candidate, but it lets you feed fewer, higher-quality chunks to the generator; the batched chunk rerank case study details the trade-off.