System One — encode once, decide in parallel

System One is Qwen3.5-4B-Base fine-tuned (LoRA + a scalar head) and temperature-scaled to answer typed questions with a calibrated probability distribution over exactly the options the caller supplies. That is the model's contribution. This Space shows the inference path that makes many of those decisions cheap, next to the instruct model on the same base doing the same job by generating JSON.

The three lanes

Token accounting follows §4.1 of the report: naive is P·B + Σ tails, cached is P + Σ tails, for a P-token state and B branches.

Why branching is exact here

The scorer's training format tokenizes State:\n{state} and \n\nQuestion:\n{q}\n\nOption:\n{o} separately and concatenates them, so the head is a token-exact shared prefix and each tail is a branch. Qwen3.5 is a hybrid: branching copies the KV cache of its full-attention layers and the conv and recurrent state of its gated-delta-net layers (Cache.reorder_cache with a zero index does both). On a random-weight Qwen3.5 in fp32 the branched scores match a full re-encode to ~1e-7; on the real model in bf16 the Space reports the max |Δp| against the naive path on every run.

This is not the letter-logit trick from arm B of the report, where extra options are a free gather from one logits row. The 4B model is a scalar cross-encoder, so every option costs a branch — but only its tail tokens, never the state again.

Honest limitations

• Trained with sequences truncated at 384 tokens and option sets capped at 16. The Space accepts states up to 16,384 tokens — so long inputs such as pull requests can be explored — and flags anything past the training length as out of distribution. That range is an experiment in how far the scorer stretches, not a supported one.
• Questions over 96 tokens and options over 64 are refused rather than truncated (the training format never truncates them). If the instruct prompt would exceed 20,000 tokens that lane is skipped, not truncated; if the naive path would exceed 400,000 tokens the equivalence check is skipped and says so.
• Each branch carries its own copy of the state's cache (~32 KB per token per branch), so long states get fewer branches per forward: 96 at most, 200,000 / state_tokens otherwise. The forward-pass count shown is the real one.
• Wall-clock is one run on a shared ZeroGPU with reference PyTorch kernels for Qwen3.5's linear-attention layers (sdpa attention only). Forward passes and tokens are the hardware-independent numbers; read the milliseconds as indicative.
• The instruct baseline is greedy with thinking disabled and a prompt that lists the allowed options verbatim. It is a fair-effort baseline, not a constrained-decoding one.
• The ticket component of the training data is CC-BY-NC-4.0, so the model is non-commercial.

Pull requests

Measured on 3,219 decisions about 500 real Gradio PRs (100–16k tokens, held-out, temperature from a separate cal split): the scorer gets 0.740 accuracy against a 0.508 majority baseline with ECE 0.047, and the published temperature transfers (refit: 1.867). But the untouched base model, read through letter logits, is as good on the judgement questions and better above 2,000 tokens; the fine-tune's advantage ends where its 384-token training data does. Full write-up: pngwn/open-jev-pr-eval.

Load PR fetches a public GitHub pull request (title, description, file list and per-file patches; lockfile diffs are left out, and so are labels and changeset files, because they state the answers to some of the questions) and uses it as the state, with a set of typed review questions: kind of change, area, risk, review effort, semver bump, tests, API, breaking, security, reviewer. It uses the unauthenticated GitHub API unless a GITHUB_TOKEN secret is set, so it can be rate-limited; pasting the text works the same. The PR · 12 example is gradio-app/gradio#13822, about 4,400 tokens.

Running locally

SYSTEM_ONE_MOCK=1 python app.py serves the frontend with a fake backend (no GPU, no model download beyond the tokenizer). SYSTEM_ONE_ADAPTER and SYSTEM_ONE_TEMPERATURE override the scorer adapter and its temperature.