Jev Case Study: Applicant Skill Tag Extraction (Full Code)

TL;DR: This Jev case study builds a resume-tagging pipeline for a hiring platform: a multi-select choice question maps free-text resumes onto a fixed vocabulary of 12 standardized skill tags, and a scoring question attaches a 0–100 proficiency confidence score to each extracted skill. Full request bodies, example-fixture responses, and a runnable Python script are below, plus a comparison against regex/keyword matching.

Scenario

We run the tagging backend for a hiring platform that receives about 4,000 resumes a week. Every resume gets skill tags so recruiters can filter by them, and the tags have to come from a fixed vocabulary — recruiters search for python, not for “experienced with Python-based data pipelines”, and a tag that is one word off from the vocabulary is invisible to every filter.

Before this pipeline, tagging was manual. Two problems compounded: cost and consistency. The cost is obvious at 4,000 resumes a week. Consistency was the worse problem — one contractor tagged “built AWS Lambda functions” as aws, another skipped it because the resume never used the bare word “AWS”. The same resume could get different tags from different people, which quietly poisoned every downstream filter.

We replaced manual tagging with two Jev calls per resume:

1. A multi-select choice question — from a fixed list of 12 tags (python, sql, aws, docker, communication, and seven more), pick every tag the resume supports. Multi-select matters here: a single-choice question would force a false “primary skill” that recruiters then over-weight.
2. A scoring question — for each extracted skill, rate 0–100 how confidently the resume supports real proficiency, not just a name-drop.

Jev is a System One judgment model from TypeSafe AI, released in September 2026. The typed JSON output with a confidence value per answer is what makes the second call possible — a score with no confidence attached would just push the inconsistency problem downstream. The sample resume below deliberately uses synonyms and a project description instead of a tidy skills list, which is exactly where keyword matching falls over.

Request

Call 1 — tag extraction (multi-select choice question):

{
  "model": "typesafe/jev-1.13",
  "messages": [
    {
      "role": "system",
      "content": "Multi-select choice question. From this fixed tag list: python, sql, aws, docker, kubernetes, react, machine-learning, data-analysis, project-management, communication, leadership, customer-service — select every tag the resume clearly supports. Do not infer tags from job titles alone; require concrete evidence in the text. Reply with JSON only: {\"answer\": [<tag>, ...], \"confidence\": <0-1>, \"rationale\": <one sentence>}"
    },
    {
      "role": "user",
      "content": "Resume: Data analyst with 4 years of experience. Built ETL pipelines in Python pulling from a Postgres warehouse, wrote daily reporting queries in SQL. Migrated our team's batch jobs to AWS (Lambda + S3) and containerized local dev with Docker. Led a 3-person analytics pod, presented findings to non-technical stakeholders every sprint. Side project: a small scikit-learn model predicting churn."
    }
  ]
}

Call 2 — proficiency score (scoring question), one call per extracted skill:

{
  "model": "typesafe/jev-1.13",
  "messages": [
    {
      "role": "system",
      "content": "Scoring question. Rate 0-100 how confidently this resume demonstrates working proficiency with the given skill — 0 means no evidence, 50 means mentioned without depth, 100 means sustained professional use with concrete outcomes. Reply with JSON only: {\"answer\": <0-100>, \"confidence\": <0-1>, \"rationale\": <one sentence>}"
    },
    {
      "role": "user",
      "content": "Skill: aws. Resume excerpt: \"Migrated our team's batch jobs to AWS (Lambda + S3).\""
    }
  ]
}

Response

Both responses below are example fixtures — illustrative output, not a live capture.

Call 1:

{
  "answer": ["python", "sql", "aws", "docker", "leadership", "communication", "machine-learning"],
  "confidence": 0.86,
  "rationale": "Each tag has concrete textual evidence: pipelines in Python, SQL reporting queries, Lambda/S3 migration, Docker for dev, a led pod, stakeholder presentations, and a scikit-learn project."
}

Call 2 (for aws):

{
  "answer": 65,
  "confidence": 0.83,
  "rationale": "A real production migration is described, but scope is one team's batch jobs with no scale or outcome details."
}

Reproduce

Save as skill_tags.py, fill in your own BASE_URL and API key, and run python3 skill_tags.py. It uses the OpenAI-compatible SDK, so any OpenAI-compatible endpoint works:

import json, os
from openai import OpenAI

BASE_URL = "https://YOUR-ENDPOINT.example.com/v1"  # replace with your provider
client = OpenAI(base_url=BASE_URL, api_key=os.environ["JEV_API_KEY"])
MODEL = "typesafe/jev-1.13"  # confirm exact slug with your provider

TAGS = ("python, sql, aws, docker, kubernetes, react, machine-learning, "
        "data-analysis, project-management, communication, leadership, "
        "customer-service")

def ask(system, user):
    res = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "system", "content": system},
                  {"role": "user", "content": user}])
    return json.loads(res.choices[0].message.content)

EXTRACT = ('Multi-select choice question. From this fixed tag list: ' + TAGS +
           ' — select every tag the resume clearly supports. Do not infer '
           'tags from job titles alone; require concrete evidence in the '
           'text. Reply with JSON only: {"answer": [<tag>, ...], '
           '"confidence": <0-1>, "rationale": <one sentence>}')

SCORE = ('Scoring question. Rate 0-100 how confidently this resume '
         'demonstrates working proficiency with the given skill — 0 means '
         'no evidence, 50 means mentioned without depth, 100 means '
         'sustained professional use with concrete outcomes. Reply with '
         'JSON only: {"answer": <0-100>, "confidence": <0-1>, '
         '"rationale": <one sentence>}')

resume = ("Data analyst with 4 years of experience. Built ETL pipelines in "
          "Python pulling from a Postgres warehouse, wrote daily reporting "
          "queries in SQL. Migrated our team's batch jobs to AWS (Lambda + "
          "S3) and containerized local dev with Docker. Led a 3-person "
          "analytics pod, presented findings to non-technical stakeholders "
          "every sprint. Side project: a small scikit-learn model "
          "predicting churn.")

tags = ask(EXTRACT, "Resume: " + resume)
print(tags)

tagged = {}
for tag in tags["answer"]:
    scored = ask(SCORE, "Skill: " + tag + ". Resume excerpt: " + resume)
    if scored["answer"] >= 40 and scored["confidence"] >= 0.5:
        tagged[tag] = scored["answer"]
print(json.dumps(tagged, indent=2))

The post-filter at the end (score ≥ 40 and confidence ≥ 0.5) is where the 12-tag vocabulary earns its keep: dropping weak evidence keeps the tag set small enough that recruiters trust it.

Why not regex or keyword matching?

Keyword matching is still the right baseline to keep around: it is free, instant, and catches the literal mentions that would otherwise cost you a model call. We run keywords first and only send resumes (or tag candidates) the keyword pass cannot resolve to a Jev call. The pipeline is not replacing the cheap method — it is catching what the cheap method drops.

Key parameters

Notes

• The responses above are example fixtures; verify field names against official docs before relying on them. Official API details live at typesafe.ai — start with the System One announcement post.
• The raw HTTP response is an OpenAI-compatible chat completion; the JSON shown here is the model’s typed output parsed from the message content.
• One scoring call per extracted tag multiplies cost on tag-heavy resumes. Cap the tags sent to scoring at the top 8 by extraction confidence if budget matters.
• When you change the tag vocabulary, re-tag a fixed sample of old resumes and diff the results before shipping — vocabulary edits change old tags too.
• Applies to Jev 1.13 (released September 2026) via the OpenRouter demonstration channel.