Getting started with decision models

Guides, run instructions and open experiments for System One decision models: Jev, Liquid d1, GLiDE, GLiNER2.5-Decide, Kev, Laya and 250+ open models on Hugging Face.

This repo is maintained by Baobab Tech. Baobab Tech is not affiliated with any model, vendor, benchmark or dataset listed here. Each model, name, benchmark and dataset belongs to its authors, and each doc links to the original source. What this repo adds is the docs, experiment plans and results.

What a decision model does

You send a piece of text (the state) and typed questions. You get back a probability for every possible answer, from one forward pass with no generated text.

{
  "state": "My payouts have failed for three days, please help today.",
  "questions": {
    "department": {"type": "choice", "instructions": "Which team handles this?",
                   "criteria": {"billing": "Payments", "technical": "Bugs"}},
    "urgent":     {"type": "noul",   "instructions": "The message is time-sensitive"},
    "frustration":{"type": "score",  "instructions": "How frustrated?",
                   "criteria": ["Calm", "Frustrated", "Furious"]}
  }
}

{"answers": {
  "department":  {"choice": "technical", "probabilities": {"billing": 0.07, "technical": 0.93}},
  "urgent":      {"noul": 0.99},
  "frustration": {"score": 1.01, "probabilities": {"0": 0.01, "1": 0.96, "2": 0.03}}
}}

The same request works against Jev, Liquid d1, Kev, Laya (laya-serve), Bonsai-Llama-Jev, Decider 1 and Solar Decide. More in concepts.

Start here

1. Concepts: the format, calibration, and how many-label classification works.
2. Quickstart: run a model on a Mac or through an API, with one client for every compatible endpoint.
3. Model classes: how a 300M BERT encoder, a 4B Qwen head and a 27B model differ.
4. Benchmarks: Decision Index, JevBench, typed-decision-bench, S1Bench.
5. Data governance: who can self-host, fine-tune, or process in the EU.

Models

New releases and download counts as of 2026-10-02: scan. The Hugging Face collection Decision models experiments gathers our dataset, the benchmarks, the judge model and the open models above.

All 255 models in Han Xiao's All about Jev dataset are in landscape, grouped by backbone and size.

Findings so far

From our experiments (excerpt tagging of 600 evaluation-report excerpts; reference: two LLMs that agree at 88.8; story):

• Zero-shot decision models fall short of the LLM range. Jev scores 72.5 and d1 69.0; the open decision models score 38–65 (01).
• Fine-tuned small general encoders score 79.7–81.4, including a 31M-parameter Ettin at 80.4. A fine-tuned decision-model backbone (ModernJEV-Decide) did no better than general encoders (02).
• For a fixed task with LLM labels to train on, fine-tune a small encoder. Without training data, zero-shot decision models remain the option (not tested beyond this task).

From published sources:

• On Decision Index 0.2.1, Liquid AI reports d1 at 58.9 and Jev 1.13 at 57.9 from its own run of the suite (KuCoin/BlockBeats, 2026-09-29). The public board, generated 2026-09-28 before d1's release, lists Jev first at 57.91 and has no d1 row (Space, checked 2026-09-30). Every BERT-family encoder scores below 12 (benchmarks).
• On JevBench v1.5.4, which also scores speed and cost, two open Gemma-4-12B models (Cygnet 73.7, Winnow-12B Q8 73.2) lead Jev 1.13 (72.1) in a statistical tie, and 4B models follow within 0.5 points (live board, checked 2026-09-30).
• Choice is capped at 255 options on Jev and Kev, and at 26 on AnyJev. No vendor publishes accuracy for 100+ options (experiment 01).
• Multi-label tagging means one Noul per label on every Jev-compatible API. Only GLiNER2.5-Decide has a native multi-label mode.
• Jev and d1 process data in the US and cannot be fine-tuned. Open models can run and train on your own hardware (data governance).

Experiments

Each experiment starts as a written plan. Results, code and run metadata are published in its folder (index).

docs/
  README.md                   docs index
  concepts.md                 format, calibration, many-label classification
  quickstart.md               run each model locally or via API
  model-classes.md            models by backbone family and size
  benchmarks.md               benchmarks and cross-benchmark table
  benchmarks-leaderboards.md  full leaderboard tables
  fine-tuning.md              fine-tuning options by backbone
  data-governance.md          governance comparison across models
  landscape.md                all 255 dataset models (generated)
  models/                     one doc per model or family
experiments/
  README.md                   index and status
  common/                     shared datasets, metrics, conventions, code
  <nn>-<slug>/                one folder per experiment
scripts/                      doc generators
data/                         external datasets (raw files gitignored)

Contributing

Issues and pull requests are welcome, especially corrections to the model docs (with a source link) and new open decision models. Contributor and agent conventions are in AGENTS.md.

Credits

docs/landscape.md is generated from All about Jev, a dataset compiled and curated by Han Xiao (announcement).

License

MIT