JuL is a Python library that answers typed questions about a piece of text: pick an option (Choice), say yes or no (Noul), give a score (Score). Each answer comes with a probability. It has the same API as the SDK of Jev, TypeSafe's hosted decision model.

The model is stopped one step before its first syllable and the answer is read straight out of its hidden states: no monologue, no reasoning trace, no opinion on the matter. It has nothing to say, and it says it in 55 milliseconds.

It runs on your own machines, from a Mac to a Linux server, so you can keep it running 24/7 on your own infrastructure. Your text never goes to a third party and there is no per-call bill. The model weights are downloaded once from the Hugging Face Hub, or loaded from a local path; after that it runs with no network at all.

Why JuL

• Zero-shot: the default model scores 0.857 on Jev's public benchmark (Jev: 0.753) without one example of its tasks. Its settings were fitted on dev sets the benchmark never touches. On AG News, the one task embedding models haven't trained on, the default gets 0.90, wemm-4b 0.95 and Jev 0.91.
• Close to Jev on typed decisions: jul-decision-wemm-4b, the default model on PyTorch, answers 0.849 of 2,108 Choice, Noul and Score questions over 12 task families, Jev 0.873, at 97 ms per decision on one A10G GPU (per family). On a bench of 300 typed questions written from scratch (sentiment, finance, support, agent routing, moderation) it answers 0.677, and jul-decision-minicpm5-2b (alias fast), half its size, 0.680 at 85 ms.
• Many options: option vectors are computed once and cached, so adding options barely changes the cost of a call. On Banking77's 72 intents the default model gets 0.87, on a task its training data (MTEB) contains.
• Probabilities you can use: ECE 0.084, so you can automate above a confidence threshold and send the rest to a human.
• Small and fast: 2.6 GB and 55 ms per decision on a Mac. f2llm-1.7b (converted locally, not published yet) fits in 1 GB and answers in 24 ms.
• Replaceable model: we measured 18, and jul models add wires in a new one without touching your code, including other open decision models (Kev and Strands Decider checkpoints, the letter-readout models JevK5, plumb-4b, Quyet and spark-s1, each read as its own runtime reads it) and Laya on its own runtime.
• Measured on your own rows: jul bench test.jsonl --models fast,accurate,typesafe answers them with every model you name, local or remote (Jev, Ollama, Clef, any System One URL), and says which one to pick.
• Local first, remote when unsure: jul serve --escalate-to typesafe (or jul.Escalation) answers locally and sends only the questions below a confidence bar to a bigger decider.
• Long inputs are never cut silently: a cut is logged and counted in usage.truncated_tokens, and on_long="error" refuses a text longer than the model reads (input limits).
• Tunable when you have labels: autotune(...) takes the default model from 0.857 to 0.897 with 1000 examples, and keeps the head only if it beats zero-shot.
• Drop-in for the Jev SDK: change the import and the same code runs on your machines. jul serve speaks the Jev HTTP protocol for other languages.

Quickstart

# from typesafe_sdk import TypeSafeClient, Choice, Noul, Score
from jul import TypeSafeClient, Choice, Noul, Score

client = TypeSafeClient()                             # jul-decision-wemm-4b, or model="minicpm5-2b"

response = client.system_one(
    state={"ticket": "I was charged twice for my subscription this month."},
    questions={
        "team": Choice(instructions="Which team should handle this ticket?",
                       criteria={"billing": "payments, invoices, refunds",
                                 "technical": "bugs, errors, crashes",
                                 "sales": "pricing, plans, demos"}),
        "is_bug": Noul(instructions="Does the message report a software bug?"),
        "frustration": Score(instructions="How frustrated is the customer?",
                             criteria=["Calm", "Frustrated but civil", "Very angry"]),
    },
)

response.choices["team"].choice           # "billing"
response.choices["team"].probabilities    # {"billing": 0.88, "technical": 0.11, "sales": 0.01}
response.nouls["is_bug"].noul             # 0.12
response.scores["frustration"].score      # 1.15

The model compares your text with each option's description, so write descriptions a colleague would understand; the key is only the name you get back. noul is the probability of yes, and score is the expected level on your scale, from 0 ("Calm") to 2 ("Very angry"). AsyncTypeSafeClient has the same API. Arguments that only make sense for a remote API (api_key, retry, …) are accepted and ignored, so code written for Jev runs unchanged. From the shell, use jul ask. Non-Python callers can use jul serve, which speaks the Jev HTTP protocol (docs/serve.md).

Showcases

Eight demos from jul-showcases, each an idea from jevable.com running entirely on device, for $0.

• Browser agent: books a train on SNCF Connect from its accessibility tree.
• QA from the ticket: runs a plain-language acceptance test in a real browser.
• Ticket triage at scale: 50,000 real support tickets in 668 s, $0.
• Ticket triage with autotune: a fast model goes from 82.0% to 96.5% with a head trained in 6.2 s.
• Self-branching form: picks the next question from the answers so far.
• Intent re-ranker: sorts a list by a plain-language intent.
• Notification triage: mutes marketing, keeps OTPs and appointments.
• Prompt difficulty: routes a prompt to fast mode or the full model before it is sent.

Install

pip install jul
jul setup          # picks MLX or PyTorch, installs it, downloads the default model, runs one decision

Python ≥ 3.10. The default model weighs 2.6 GB and is downloaded once from the Hugging Face Hub. To pick the backend yourself (jul[mlx], jul[torch], jul[onnx]), see docs/installation.md.

How it works

JuL reads the hidden state the model built for your text and compares it with the vectors of your option descriptions. Your task never reaches the weights, so you can change the options between two calls, or swap the model, without retraining anything. When it is off on your data, client.autotune(...) fits a small head on labeled examples and keeps it only if it beats zero-shot in cross-validation (docs/tuning.md).

Results

Jev's published benchmark, 300 examples, zero-shot for every JuL model, run with scripts/bench_jul.py. Each mean is ±3 points. Banking77 and Emotion are in MTEB, which the wemm-* and f2llm-* models trained on, so AG News is the clean comparison: 0.95 for wemm-4b, 0.91 for Jev, and 0.90 for the default model. The default model's ECE is 0.084 against Jev's 0.156, which is what lets you automate above a confidence threshold. Every setting was fitted on English, and we haven't tested domains far from this kind of text (sensor logs, chemistry and so on). Per-task scores, the 18 models measured and the tuned results are in docs/benchmarks.md.

Decision bench against Jev

The default model, jul-decision-wemm-4b, is WeMM-Embedding-4B with LoRA adapters that read the question and the text together (usejul/jul-decision-wemm-4b). The decision bench asks it 2,108 typed questions over 12 task families, in English and French, with texts written for the bench and never seen in training. PyTorch bf16 on one A10G GPU, one question per call; Jev through its API, network included in its latency.

A family has 60 to 300 questions, so one row moves by ±4 to ±12 points: read the types and the total first. The bench shares its task families and label sets with the training data (not its texts); on the four label sets of jul's dev, never trained on, the adapters take Choice from 0.707 to 0.730. Emotion is where Jev is furthest ahead, and inference is where the model is weakest. On a Mac (MLX, 4-bit) the adapters are not attached yet: the default model reads every question with the vectors, and the Jev benchmark numbers above are its numbers.

The baseline worth remembering

A TF-IDF + linear SVM, trained on 1000 labeled examples with no LLM at all, scores 0.88 on AG News, 0.76 on Banking77 and 0.43 on Emotion: a mean of 0.690 at 0.17 ms per prediction (scripts/bench_tfidf.py). It is 11 points behind the default model on Banking77 and far behind on Emotion, where telling feelings apart takes meaning, not vocabulary. If you have labels and your problem is sorting by topic or intent, try it first. CI re-measures these numbers on every PR (numbers.yml, CLAIMED), so update both together.

Models

¹ Choice questions, the only ones in the Jev benchmark, are read by the vectors of wemm-4b-4bit, which the adapters leave untouched (switched off, the features are the same): its numbers.

² Measured on MLX, where the default model reads with the vectors of wemm-4b-4bit. On PyTorch its Choice reading mixes in the adapters; the Jev benchmark has not been rerun there yet.

To use another model, run jul models add <name> --repo <hf-repo>. It fits the layer, center and temperature on the dev sets; a decision model (a decision.json, a Strands Decider checkpoint, a letter-readout model) brings its own format and is read as is. The 18 models we measured, encoders, decision models, Laya, embeddings APIs and every setting are listed in docs/models.md; how to pick among them, in docs/hub.md.

Decisions inside an AWS Lambda

jul-decision-e5-small runs inside an AWS Lambda function, model included in the package: 17 ms per message and $0.59 per million at 1,769 MB, a 2.4 s cold start, no GPU and no server. Jev is a hosted API and CLM-8B needs a GPU server; to our knowledge no other typed-decision library puts its model in a Lambda. It is not a general-purpose endpoint: it is a small model embedded in a function that decides one precise thing on every event, with the questions fixed at build time, and it should be tuned on labeled examples of that thing (autotune: 0.557 zero-shot, 0.780 tuned on the Jev bench). jul pack ships only the model(s) the questions need. Step by step, with the AWS CLI or CDK: docs/aws-lambda.md.

Documentation

• Installation: backends, devices, batching, jul setup
• The hub: every model family, backend and remote decider, and how to pick
• Models: presets, the 18 models measured, decision models, Laya, input limits, jul models add
• AWS Lambda: a tuned decision in a Lambda function, step by step (CLI or CDK)
• Adapting to your data: Context, autotune(...), hybrid heads, jul synth
• Deployment: jul pack, ONNX bundles, AWS Lambda
• Serving over HTTP: jul serve, escalation to Jev, Clef or any System One server
• Telemetry: opt-in OpenTelemetry metrics and events, named after Claude Code's
• Command line: every command and file format, jul bench on your own data
• Benchmarks: full results and how to reproduce them
• Development and Publishing

Contributing

Issues, measurements on your own data and pull requests are welcome on GitHub. pip install -e ".[dev]", then pytest tests (no model, about 30 s). Tune on the dev datasets and run the benchmark once at the end (how).

License

Apache 2.0, see LICENSE and NOTICE. The decision-model format comes from Kev (Jared Palmer, Apache 2.0) and jul-decision-minicpm5-2b is MiniCPM5-2B (OpenBMB, Apache 2.0) trained with Kev's code. JuL follows TypeSafe's public System One API and uses no TypeSafe or Jev code, weights or outputs.