rsi-jev-v3.0-qwen3.5-2b

A typed-decision model: Qwen/Qwen3.5-2B-Base with its tower fine-tuned and a trained option-scoring head. It answers choice / noul / score questions about a document in one forward pass, and speaks the Jev HTTP API.

This checkpoint is seed 17, the primary seed fixed in advance:

Re-computed from these exact files after reload. Kernel stack fla-0.5.2/torch-2.7.1+cu128; full recipe in meta.json, and the sha256 of every file in MANIFEST.json.

import sys; sys.path.insert(0, 'code')   # after snapshot_download of this repo
from load_release import load_release
model, tok, enc, meta = load_release('.', device='cuda')

Everything below is the version record from the project repository, unchanged.

---

RSI-Jev v3.0

Describes release v3.0 · updated 2026-09-28

One typed-decision model, 2B. Given a document and a set of typed questions — yes/no, pick-one-of-k, rate-on-a-rubric — it returns a calibrated probability for every option in one forward pass, with no generated text.

v3.0 is the first release where reinforcement learning works. The reward that made the difference is a listwise ranking reward for reranking: it scores the order of many separately scored candidates, which no per-item label can express. With it, v3.0 puts the right memory first 60% more often than its own supervised parent (hippo R@1 0.192 → 0.308, p = 2.4 × 10⁻¹³) while the fifteen-benchmark suite stays level. Finding it took four waves and 59 reward-trained arms. Against v2.1 it is ahead on a new held-out set, +0.016 (p = 0.011), and on the suite, 0.736 → 0.756. v3.0 is v2.1's lineage with more training data, a new readout combine, and that RL stage on top. ../docs/rl.md is the full story of what was tried and why only this one won.

§6 lists what this release does not change.

---

1. The task

Unchanged from v1.0: a document (the state) plus one or more typed questions — noul, choice, score — and a probability per option, returned in one forward pass. Top-1 is the fraction of decisions whose highest-probability option matches the label; ECE is expected calibration error, the average gap between the confidence a model reports and how often it is right at that confidence, lower being better.

2. How the model is built and trained

Qwen3.5-2B-Base, with v1.0's option scorer and v2.0's confidence head. Two training stages, then the confidence head is refitted.

Stage 1: supervised, 16,416 steps. The same objective and optimiser as v2.1 (soft cross-entropy, the bottom 8 of 24 tower layers at one tenth the learning rate), with three changes:

1. Wider data (data-scale-2). 266,131 questions from 36 sources against v2.1's corpus: more benchmark train splits at up to 6,000 questions per source, 30,000 tasksource and 10,000 Open-Jev train items, ten coverage sources, and the emotion, SST-5 and ANLI train splits. One question in ten is held out by a hash of its case id and never trained on; it is what the confidence head is fitted on. ../data/v3.0_corpus_manifest.json lists every file with its sha256 and counts.
2. A two-layer MLP combines the option and cross-attention features (xattn_combine: mlp), where v2.1 summed them. Worth +0.002 on the suite on its own.
3. Up to 160 options per question in training and evaluation (80 before), because tasksource has questions with 151.

Batches are length-bucketed (width 64 tokens), which makes each step cheaper.

Stage 2: listwise reranking RL, 1,500 steps, continued from stage 1. For each training query the scorer gives every one of 16 candidates a relevance probability; 16 rankings are sampled from the Plackett-Luce distribution those probabilities define; each ranking earns its NDCG@5 against the relevance labels; the update is REINFORCE with a leave-one-out baseline and a KL penalty (β = 0.1) toward the stage-1 model. Ordinary supervised replay on the stage-1 data is mixed into every step. The RL data is 1,000 queries of 20–40 candidates built from the train splits of MS MARCO, TopiOCQA and FiQA, with BM25 hard negatives. It never touches LongMemEval, which the reranking result below is measured on.

Calibration. The confidence head (oof_head_scorefloor, as in v2.0 and v2.1) is refitted on the RL model, on 9,862 held-out questions of the stage-1 corpus. 534 held-out cases were dropped first because their document text also appears in a trained case. It rescales each question's probabilities without changing which option wins.

Everything needed to run both stages is in this repo: rsijev/fit.py and rsijev/rl2.py, driven by scripts/release_train.py from the spec in meta.json. The builders for what is new in this release's data — scripts/build_ds2_addons.py, scripts/build_hippo_rr.py, scripts/build_rl2_sources.py — and the calibration fit, scripts/fit_release_calibration.py, are here too. See §6 for which of them were re-run to the same bytes.

3. Checkpoints

Both rows are scored on the fifteen-benchmark suite by the same evaluator, in the same job. v2.1's own card reports 0.0556 on its twelve-benchmark suite; 0.059 is the same checkpoint on this one.

2B only, one seed's checkpoint.

Reproducing its training run. Re-scored from disk with this repo's code on an H200, the checkpoint reproduces the training run's per-question predictions at 1.0000 on all three verification targets — typed-decisions in both option orders, and MMLU-Pro (verify.json). On a second GPU architecture (Blackwell, verify_blackwell.json) typed-decisions again agrees at 1.0000 and MMLU-Pro at 0.9990, one question in a thousand, the same order-of-summation effect v2.1 §3 describes; read MMLU-Pro as ±0.001. verify_bf16.json is the bf16 tower (0.9975 / 0.9975 / 0.985), roundtrip_200.json the reload check with calibration applied, and cached_agreement.json the check that the two serving paths — with and without the prefix cache — return identical probabilities (162 questions, 0 flips, largest difference 0.0). MANIFEST.json holds the sha256 of every file.

The checkpoint ships calibration.safetensors and calibration.json; load_release.py applies them automatically. Without them the model loads uncalibrated and its probabilities match no number in this card.

Apache-2.0, following the base model, as every release is.

4. Results

4.1 The held-out comparison — read this first

v2.1's held-out set was spent: those three benchmarks joined the suite. eval_final_v2 is a fresh one, frozen during this release's search and built from sources in no training corpus: LegalBench (11 tasks), MuSR, BIG-Bench Hard (5 tasks), the relevance split of the Berkeley Function-Calling Leaderboard, and ConditionalQA. The suite decided every keep, but this set was read along the way as a consistency check: it backed the mlp readout (the same small suite gain twice, and a held-out gain), and it counted against two arms whose suite gains it did not share. So it is held out from training, not sealed from the search; read the gap below with that in mind.

All four models were scored in one job, on one GPU, with the same evaluator; post-calibration wherever a release ships a calibration. p is an exact two-sided McNemar test on the questions where v3.0 and v2.1 disagree (pooled: 296 won by v3.0, 236 by v2.1). ConditionalQA's 135 questions give it a standard error of about 0.04.

v3.0 is ahead of v2.1 on the fresh held-out set — significantly on LegalBench, BFCL and pooled — and v2.1 was ahead of v1.0 there by a similar margin, so each release has moved the number the search never selected on. Calibration on this set is 0.063 ECE (v2.1: 0.078).

Against its own supervised parent the RL stage is flat on this set (0.649 against 0.651): it gains on four benchmarks and gives back 0.027 on BIG-Bench Hard.

4.2 The fifteen-benchmark suite

tasksource, SemIf and scienthoon were v2.1's held-out benchmarks and are in the suite now; tasksource is in-domain from this release, because its train split is in the corpus. Ten of the fifteen benchmarks contribute training data in some form, so none of these numbers is zero-shot; §4.1 is the comparison that is.

Against v2.1 on the same evaluator the suite moves from 0.736 to 0.756 (v1.0, uncalibrated: 0.622). The largest gains are where the new data landed — Open-Jev +0.187, Jev-Style +0.084, tasksource +0.083, all in-domain now — and the largest losses are scienthoon (−0.070) and MMLU-Pro (−0.019); scienthoon has no train split and was v2.1's best held-out result. The RL stage itself costs 0.0028 on the suite against its parent, inside the −0.003 it was allowed, and 0.009 on MMLU-Pro.

4.3 Reranking, where the RL stage is aimed

Scored per candidate on LongMemEval through hippo-memory's retrieval (500 questions, 40 candidates each, no prompt truncated):

The RL stage lifts R@1 by 0.116 over its parent, +60% (64 questions won, 6 lost, exact McNemar p = 2.4 × 10⁻¹³). For reference, hippo's own retrieval order scores 0.484; that gap is where the next reranking data is aimed. The evidence that RL, and not merely more steps on these rows, is what helps comes from a separate paired run on a different parent and one seed (+0.052 over pointwise SFT on the same rows, z 3.9); ../docs/rl.md has it.

4.4 Calibration and the properties we kept

Suite ECE after the confidence head is 0.066 (0.101 before it) and 0.063 on the held-out set (0.154 before). The suite number hides unevenness: ECE is 0.091 on typed-decisions (v2.1: 0.063), 0.144 on MMLU-Pro and 0.129 on JevBench.

Three properties of v2.1's that external testing found useful were checked on v3.0 with the same scripts:

The first two hold. The third moved: the RL stage sharpens probabilities a little on questions whose answer cannot be known, and it is now checked on every training change.

5. Speed, the wire format, and what confidence means

Inference is unchanged: one tower pass per question, the calibration a division inside the readout, nothing generated. The weights are the same size as v2.1's.

Two serving changes ship with this release (see ../serve/README.md):

• Up to 160 options per question, matching training; 64 before.
• --accept-model NAME answers requests that name another model, such as an app pinned to jev-1.13.0. The answer still names this model.

Two things do not change and are easy to miss. A structured criterion (an object rather than a string) is still rejected with 422: no release has been trained on one. And a document longer than 2,048 tokens is cut from its start, silently, so a request that puts its query first and long candidates after it loses the query.

The API returns a probability and a confidence, and they are different numbers: confidence is the peak statistic Jev documents, (K * p_max - 1) / (K - 1). Route on the probability of the option you intend to act on.

6. Limitations

• The weaknesses external testing found in v2.1 are still here. Run inside open-source apps built on Jev, v2.1 leaned toward the more severe or more expensive option on ordered choices and rubric scores, rarely chose "unknown" when a question could not be answered from the document — picking a stereotyped answer instead on a social-bias benchmark — and answered a question and its negation inconsistently. v3.0 was trained before the data aimed at these existed. Do not use it where an "unknown" option has to be honoured without checking it on your own data.
• Reranking has further to go: +0.116 R@1 over the parent, and hippo's own retrieval order (0.484) is still ahead (§4.3).
• The RL evidence is one seed. The paired RL-versus-SFT result is a single run on a different parent; v3.0 itself has no matched SFT control.
• MMLU-Pro is 0.364, 0.009 below its own parent.
• Its benchmark scores are not zero-shot; ten of fifteen suite benchmarks contribute training data (§4.2).
• Documents over 2,048 tokens lose their beginning (§5).
• Rebuilding the data is partly verified. The data-scale-2 additions, the RL-stage file and the calibration pool were rebuilt with this repo's scripts to identical bytes. The reranking corpus builder (build_hippo_rr.py) was not re-run end to end: it needs the raw MS MARCO, TopiOCQA and FiQA exports and about 96 GB of memory; its post-processing stage was re-run and matched. The older part of the corpus, inherited from earlier releases' builders, is listed file by file in the manifest but its builder chain is not yet all in this repo.
• One model, one size, one seed.

7. How it got here

Rejected along the way, and worth knowing because each closes a direction: training on the items the model got wrong (loses on its own targets), a joint option-set transformer head (−0.011 on the held-out set), a second attempt at MMLU-Pro through more multiple-choice coverage (suite −0.003), a level offset in the confidence head for rubric scores (fails three of five criteria), and the bandit reconstruction of RLCD (no better than supervised training on the same feedback). Counting every arm trained on a reward, this project has now run 59 and kept one recipe. The full record is ../EXPLORE.md; the RL part is ../docs/rl.md.