Line-by-Line Semantic Search with the Jev API: One Choice Over 218 Line IDs Plus One Noul

Updated Applies to Jev 1.14

A document question-answering pipeline usually fails in a place nobody looks at: the retrieval step hands back a chunk that is on-topic but does not contain the sentence that answers the question. The model then writes a plausible answer from the rest of the chunk, and the citation points at 900 words of text that do not actually say it.

This page is a build guide for the narrower version of that step, using the Jev API. Instead of ranking passages, it ranks individual lines of one document, so the output is a line ID and a quote. It is a companion to the official semantic_find cookbook, and every number below is copied from that cookbook’s published run: GitHub’s Terms of Service as the test document, 218 lines, 43,980 characters, jev-1.14.0. Nothing here is an estimate of what a search “should” score.

Why the unit of retrieval is the line

Retrieval systems are usually built at the passage level because that is the unit an embedding model returns and the unit that fits in a prompt. Both of those reasons are about machinery, not about the question. What a reader wants from a contract search is the clause that says the thing, and a 400-word passage that contains that clause also contains three clauses that do not.

Line-level search puts the citation at the same granularity as the claim:

The cost of the narrower unit is that the candidate set gets bigger for the same document. That is the trade this build has to pay for, and the 255-option ceiling is where it shows up.

Step 1: tag every line with an ID

The test document is GitHub’s Terms of Service, split into 218 clauses so every search result points to one quotable line. The cookbook fetches it from its own examples repository and caches the download:

LINES = fetch_document(DOCUMENT_URL).splitlines()

Then each line gets a short prefix and the lines are joined back into one document:

def line_id(i: int) -> str:
    return f"L{i:03d}"

DOCUMENT = "\n".join(f"{line_id(i)}| {line}" for i, line in enumerate(LINES))

Three things about that one line of code matter later.

The IDs are zero-padded to three digits, so L052 and L0520 cannot be confused as strings, and sorting by ID sorts by document order.

The separator is a pipe and a space, which means the model sees an explicit boundary between the ID and the text it labels. The ID is not part of the sentence it points to.

The document that goes into state is the tagged one, not the original. There is no second copy of the text, no metadata block, and no side table of line numbers, because the IDs are already inside the text the model reads.

The printed shape of the result, as the cookbook shows it:

L052| You own Your Content. If you post Content you did not create, you are responsible for...
L053| You grant us and other Users the licenses in Sections D.4–D.8. These licenses apply...
L054| 4. License Grant to Us

Step 2: ask where the answer is, with a Choice

The ranking is one Choice question whose options are the line IDs:

def where_question(query: str) -> Choice:
    return Choice(
        instructions=f'Which line of the document contains the answer to: "{query}"?',
        criteria={line_id(i): None for i in range(len(LINES))},
    )

The descriptions in criteria are all None. That is not an omission: the document in state already contains the text of every option, so a description per option would repeat 218 lines inside the question itself. The query goes in instructions, and the document stays unchanged between searches.

What comes back is a probability for every option, and the sum of all values is 1. That single fact drives most of the design decisions on this page:

The 255-option ceiling

A Choice question accepts up to 255 options. The cookbook states the consequence plainly: this recipe searches documents of up to 255 lines in one request, and past that you search in two passes, where one Choice question picks a window of lines and a second ranks the lines inside it.

The test document sits under that ceiling at 218 lines, which is why the single-request form works at all. If you point DOCUMENT_URL at something longer, the ceiling becomes the first thing you design around, not an edge case you discover in production.

Step 3: ask whether the answer exists at all, with a Noul

The second question is a Noul, and it asks a different kind of thing:

def exists_question(query: str) -> Noul:
    return Noul(
        instructions=f'Does any line of the document address or answer: "{query}"?',
        criteria=NoulCriteria(
            true="At least one line of the document states or directly implies the answer",
            false="No line of the document addresses this",
        ),
    )

A Noul answer is a single number between 0 and 1: the probability that the answer is yes. There is no separate confidence value, because a yes/no distribution has two outcomes and one number describes it completely.

The two criteria strings are doing real work here. “At least one line states or directly implies the answer” is the pass condition; “no line addresses this” is the fail condition. The gap between them is where partial answers live, and the cookbook handles that gap explicitly in the verdict function rather than leaving it to the reader.

Step 4: both questions in one request

Both questions go into one system_one call, so the document is sent once:

response = client.system_one(
    state=state,
    questions={"where": where, "exists": exists},
    model=model,
)
probabilities = response.answers["where"].probabilities
return {
    "exists": response.answers["exists"].noul,
    "relevance": [probabilities.get(line_id(i), 0.0) for i in range(len(LINES))],
}

Jev ingests the state once and evaluates every question against it in parallel. That is the whole reason the exists check is nearly free at the margin: the document that the ranking needed is the document the check needs, character for character, so batching them costs one ingestion instead of two.

The relevance list keeps one score per line, in document order, filled with 0.0 for any ID the response did not mention. Document order matters for the UI: a top-3 that jumps around the document is harder to read than four adjacent clauses.

Note what this function does not do. It does not re-rank by confidence, it does not drop options that scored zero, and it does not merge lines. It returns the distribution as it came and leaves every decision to the code above it. That is the pattern the rest of this site keeps coming back to: Jev returns probabilities, and thresholds live in your code where they can be tuned and reviewed.

Step 5: turn the exists probability into a verdict

The cookbook collapses exists into three states with two cutoffs:

FOUND, ABSENT = 0.7, 0.35  # `exists` cutoffs for answered and not in this document

def verdict(exists: float) -> str:
    if exists >= FOUND:
        return "answered in this document"
    return "not in this document" if exists < ABSENT else "partially addressed"

Three verdicts instead of two, because a binary gate forces a choice the model has already made unnecessary. A value of 0.65 is not “no”: it is the state where the document talks around the question, which is a different message to the user and a different routing rule for the pipeline.

The cookbook’s own instruction on the numbers is short and worth quoting in spirit: tune these thresholds on your own documents before using them in production. 0.7 and 0.35 are the settings of one published run on one contract, not constants of the model.

Step 6: what the four demo queries actually return

The run asks two questions with direct answers, one with a partial answer, and one with no answer. The published output, verbatim:

218 lines, 43,980 characters

"who owns the code I upload?"
  exists 0.99 -> answered in this document
  L052  0.77  #########     You own Your Content. If you post Content you did not crea
  L046  0.12  #             Short version: You own content you create, but you allow u
  L051  0.06  #             3. Ownership and License Grants
  L053  0.05  #             You grant us and other Users the licenses in Sections D.4–

"can GitHub kick me off the platform without warning?"
  exists 0.98 -> answered in this document
  L168  0.96  ############  GitHub has the right to suspend or terminate your access t
  L167  0.04  #             3. GitHub May Terminate
  L000  0.00  #             Effective date: April 27, 2026 · A. Definitions
  L001  0.00  #             Short version: We use these basic terms throughout the agr

"do I have to take disputes to arbitration?"
  exists 0.65 -> partially addressed
  L205  0.80  ##########    Except to the extent applicable law provides otherwise, th
  L204  0.11  #             S. Miscellaneous · 1. Governing Law

"can I join a class action lawsuit against GitHub?"
  exists 0.05 -> not in this document
  L205  0.23  ###           Except to the extent applicable law provides otherwise, th
  L197  0.19  ##            If you have a dispute with one or more Users, you agree to

Four queries, four different outcomes, and the two failures are the interesting ones.

The arbitration query comes back at 0.65, which lands in “partially addressed”. The Terms never mention arbitration; the line that ranks first at 0.80 is the governing-law clause, which sends disputes to the courts in San Francisco. The ranking is not wrong about where the closest material is. It is answering a slightly different question, and the exists score is what flags that.

The class-action query comes back at 0.05, “not in this document”, while the ranking still puts a line at 0.23. That is the whole argument for the second question in one example: a distribution over 218 options cannot come back empty, so without the Noul gate you would ship the governing-law clause as the answer to a question about class actions.

The two direct-answer queries show what a clean hit looks like. The ownership query puts 0.77 on L052 and 0.12 on the short-version summary line, so the top two are the full clause and its plain-language restatement, which is exactly the pair a reviewer wants. The termination query is sharper still: 0.96 on L168 and 0.04 on the section heading above it.

Reading the two numbers together

The cookbook’s own summary of the rule is one sentence: the ranking tells you where to look, the exists score tells you whether the result answers the question.

In a pipeline that produces both, that becomes two independent decisions:

exists verdictwhat to do with the top linewho reads it
answered in this document (≥ 0.7)cite it as the answerthe end user
partially addressed (0.35 – 0.7)show it as related material, not as the answerthe end user, labelled
not in this document (< 0.35)discard the ranking entirelynobody, unless you log it

The middle row is the one most builds get wrong. Dropping partial answers loses the case where the document does discuss the topic without resolving it, which is exactly the case a support inbox needs to route to a human. Treating them as answers is worse: the user gets a governing-law clause in reply to an arbitration question, with a high-looking 0.80 next to it.

What one search costs

Every search is one request that carries the tagged document once. With the published prices on the models page, $42 per billion input tokens and output tokens free, the cost of a search is the cost of ingesting one document:

cost per search ≈ document tokens × $42 / 1,000,000,000

The cookbook publishes the character count of its document, 43,980, and not its token count, so the arithmetic stops there on purpose. The useful consequence does not depend on the exact token figure: the per-search cost is proportional to document length and independent of how many lines the ranking considered. Searching 218 lines and searching 40 lines cost the same, because the question costs a few tokens per option and the document is what dominates.

That is also where the ceiling on repeated searches sits. Ten questions against the same document are ten ingestions of the same document. If a workload asks many questions of one contract, batching the questions into a single request is the lever, because Jev evaluates every question in a request against one ingested state.

Scaling past 255 lines

The cookbook’s own instruction is the two-pass form: one Choice question picks a window of lines, and a second ranks the lines inside it. The logic is forced by the option ceiling and by how probability mass behaves:

Two other shapes come up in practice, both consistent with the cookbook’s structure rather than documented as recipes: split the document by section and run the whole search per section, or keep the document whole and reduce the candidate set with a cheap filter before the Choice question. Either way the ceiling stays a design input, and the exists gate stays where it is, because it is the part that protects the output.

How this differs from the film-search recipe

This site has a companion page on finding one item among thousands, using one Noul per item and a Choice over the top candidates. The two coexist because the failure modes are different.

The line search ranks options that are already in the model’s hands. Every line is in state, so the question is where the answer lives inside a document the model can read in full. The distinguishing number is the exists probability, and the ceiling is the option count.

The film search ranks items that are not in state at all. Each candidate has to be described in a question, so the cost scales with the number of candidates and the distinguishing design decision is how to shrink the shortlist before the expensive scoring step.

If your candidates already fit inside 255 options and you can afford to send the text, this recipe is the simpler one. If they do not, the shortlist comes first.

Mistakes that show up in first builds

Using the top line’s probability as a relevance score. It is a share of a distribution that always sums to 1. Across a 218-line document, 0.77 on the right line and 0.23 on the right line can both be correct answers to the same question. Compare within one query, never across queries.

Skipping the exists question because the ranking “looks confident”. The 0.23 on the class-action query is the counter-example: a ranking that looks confident while the document has nothing.

Changing the document between searches and expecting the same IDs. The IDs come from enumerate over the split lines, so any edit that adds or removes a line renumbers everything after it. Stored citations become wrong silently. Version the document before you store IDs against it.

Copying the 0.7 / 0.35 cutoffs into a new domain. They were set against one contract in one legal register. The cookbook says to tune them per document, and the tuning set is your own questions with known answers.

Splitting on sentences when the document has clauses. A clause split across physical lines is three candidates here, which is the intended behaviour for a contract: each line is quotable. A prose article would want a different split, and the ID scheme has to follow whatever you split on.

Forgetting that the cache is a cache. The cookbook’s client falls back to the placeholder key cache-only when no API key is set, and a cache file already on disk is never overwritten. That makes the published run reproducible without a key, and it also means a stale cache will happily replay yesterday’s answers if you change the code and forget to delete it.

Wiring it into a retrieval pipeline

The natural place for this is beside a normal embedding retrieval step rather than instead of it. The embedding step decides which document to search, and the line search decides which line to cite:

  1. Retrieve the top documents with whatever you already use.
  2. For each of the top one or two, build the tagged document and ask where plus exists in one request.
  3. Read the exists verdict first. Below the absent cutoff, skip the document and move to the next.
  4. Above the answered cutoff, cite the top line, and keep the second and third as supporting quotes when they are adjacent to it.
  5. Between the cutoffs, route to the human queue with the top line attached as context.

Step 3 is the one that changes the shape of the output, because it makes “no answer” a normal result instead of an exception. A support pipeline that can say “this document does not cover your question” is more useful than one that always cites something.

The API surface involved

Everything above uses two question types from the same request:

Both take state, model, and questions as top-level fields, which is why they can share one request. The document is the state; the query lives inside each question’s instructions, which is what keeps the state unchanged between searches and lets one ingestion serve both questions.

For the field-level detail on each type, this site’s pages on the three primitives and on state and questions cover the request shapes. For the companion check that verifies an LLM’s answer against the paragraphs it was given, the grounding-check guide covers the three-Noul version of the same idea.

What this build does not claim

The cookbook publishes four queries on one document and no accuracy figure across a labelled set. There is no measured recall, no comparison against embedding search on this task, and no published threshold table for line search. This page therefore makes no claim about how often the top line is right.

What the published run does establish is narrower and still useful: on a real contract, the ranking finds the right clause for two directly answerable questions, and the exists question separates a partial answer at 0.65 from an absent one at 0.05. Whether that holds on your documents is a measurement you run yourself, with your own questions and your own review capacity setting the cutoffs.

Frequently asked questions

+ Why return a line instead of a passage?

Because a passage can be true as a whole and still not contain the sentence that answers the question. When each line carries its own ID, the top-ranked ID is the quote you cite, and the reviewer can check it without reading the surrounding chunk. The official cookbook's test document is GitHub's Terms of Service, split into 218 clauses, so every result points to one quotable line.

+ How many options can one Choice question score?

Up to 255. The cookbook says so directly: it searches documents of up to 255 lines in one request, and past that you search in two passes, where one Choice question picks a window of lines and a second ranks the lines inside it.

+ What does the Noul question add that the Choice ranking cannot?

A floor. Choice probabilities always sum to 1, so some line ranks first even when the document says nothing about the query. The Noul probability is not spread over the other lines, so it can fall near zero. In the published run it reads 0.05 for a question the document does not answer, while the ranking still puts a line at 0.23.

+ What are the answered and absent cutoffs?

The script sets FOUND to 0.7 and ABSENT to 0.35 on the exists probability: at or above 0.7 the document is treated as answering the question, below 0.35 as not answering it, and anything between the two as partially addressed. The cookbook's own note is to tune these thresholds on your own documents before using them in production.

+ Does each search re-send the document?

Yes, once per request, which is why both questions go in a single system_one call. Jev ingests the state once and evaluates every question against it in parallel, so the document is sent once for the ranking and the exists check together. Searching a second question means a second request, and a second ingestion of the same lines.

Keep reading