solvi-ai/solvi-base (preview)

A 150M-parameter ModernBERT-base cross-encoder for solvi typed questions (same format and answer kinds as solvi-large), distilled from solvi-large for CPU / browser use: 50 ms per question on a CPU (ONNX fp16).

Honest summary. On typed questions over JSON states it matches the large model (typed-decisions 54.5%), and its contract evidence is much better than the previous base model (46% vs 23% supporting quotes). On zero-shot choice questions it is not better than the previous base models (Fast Decisions dev 56.3%; L14d 57.3%, GLiNER2.5-Decide 62.9%).

Results (held-out tests; never used for training or checkpoint selection)

Fast Decisions numbers are on the public dev split in our harness (Fastino's official numbers are on a hidden test split).

For reference: GLiNER2.5-Decide 62.9% on Fast Decisions dev and 50.3% on typed-decisions; Laya base 36.0% on typed-decisions.

Speed (CPU, 4 threads): ONNX fp16, one question with 10 options and ~20 words: 50 ms (fp32 46 ms); a question over a long JSON state ~170–270 ms. Block layout (several questions per pass) agrees with one question per pass only 90.2% on typed-decisions, so multi_question.enabled = false; the block ONNX export is included for experiments.

Training

• Initial weights: L14g (our previous base, ModernBERT-base lineage, Apache-2.0).
• Distillation: 65 minutes on one GPU (≈ 9.3k updates, batch 64) on the same mix as solvi-large; every target is 0.5 × the data label + 0.5 × solvi-large's probabilities. Checkpoint chosen on the same development metric as the large model, never on the tests.
• Data and teachers: as solvi-large (see its card): votes of Qwen2.5-32B / 14B-Instruct, Mistral-Small-24B-Instruct-2501 and GLiNER2.5-Decide (all Apache-2.0), a Qwen2.5-32B data agent verified by two other models, ContractNLI train, VitaminC, SQuAD 2.0, BoolQ, synthetic typed states; de-duplicated against every test set.

Training data and attribution

The full manifest with licenses, URLs and counts is LICENSES.json. Policy: permissive, public-domain and share-alike data (CC BY-SA, with attribution here); excluded: non-commercial, research-only, unlicensed, or terms that forbid training.

| VitaminC (includes FEVER-based claims) | CC BY-SA 3.0 | Tal Schuster, Adam Fisch, Regina Barzilay: "Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence", NAACL 2021; Wikipedia contributors | | ContractNLI (train split only) | CC BY 4.0 | Yuta Koreeda, Christopher D. Manning, "ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts", Findings of EMNLP 2021; Hitachi America, Ltd. | | LLM-generated texts and labels (Qwen2.5-32B / 14B-Instruct, Mistral-Small-24B-Instruct-2501) | Apache-2.0 (model licenses) | Qwen team (Alibaba Cloud); Mistral AI |

SQuAD 2.0, BoolQ and VitaminC are share-alike datasets: this card is their attribution. The model weights are released under Apache-2.0; if you redistribute the datasets themselves, their own licenses apply.

Checkpoint selection only (not trained on): DBpedia-14 (CC BY-SA 3.0), HuffPost News Category (CC BY 4.0), zeroshot twitter-financial-news topic / sentiment (MIT), Civil Comments (CC0), poem_sentiment (CC BY 4.0), LexGLUE SCOTUS (CC BY 4.0), Amazon counterfactual (CC BY 4.0).

Independent benchmarks (as released, zero-shot)

Yes / no questions are the weak spot on jabr: the model ranks them reasonably but says "yes" 36% of the time where the gold labels have 50% — calibrate the threshold on your data (act_guard, fit).

Escalation with a guarantee (solvi ≥ 0.5.1)

The act threshold shipped in solvi_decide.json was fitted on development data and does not keep its promise on new real text. Calibrate on a few hundred labelled examples of your own stream instead:

part.act_guard(examples, risk=0.10)   # P(answered alone and wrong) ≤ 10% of all questions, for inputs like the examples

Measured with 300 calibration examples per data set and 200 random splits (the rest of the set is the test). "Answered" is the share decided without a person, "error" the error among those, "risk" the share of all questions answered alone and wrong — the number the guarantee is about:

The guarantee holds on every set; how much can be automated depends on how hard the questions are. It holds for inputs like the calibration examples, not under a shift of domain: recalibrate when your inputs change.

Evaluation only — never trained on: Fast Decisions (fastino, dev split), typed-decisions test, ContractNLI test, Taskmaster-2 held-out domains.

Limitations

• Zero-shot choice questions are not improved over the previous base models (Fast Decisions dev 56.3%; GLiNER2.5-Decide 62.9%). Fit on 30–60 examples of your task (Sc k=32: 60.2%).
• Contract evidence supports the answer less often than the large model (46% vs 62%); act AUROC on typed-decisions 0.60.
• The act / escalate thresholds shipped with the model are indicative only: fitted on development data, they are over-confident on new real text (see solvi-large). For a guaranteed risk, calibrate on your own labelled stream with act_guard (solvi ≥ 0.5.1, table above).
• Several questions per pass disagree with one-question-per-pass in ~10% of answers on real states — disabled by default.
• English only; truncation beyond 512 tokens (1024 in block layout). Not for medical, legal or credit decisions on its own.

Files

model.safetensors (bf16), onnx/model_fp16.onnx (full layout: input_ids, attention_mask → logits [B, L, 6]), onnx/model_block_fp16.onnx (block layout: input_ids, position_ids, full_attention_mask, sliding_attention_mask), onnx/parity.json, config.json, tokenizer.json, tokenizer_config.json, solvi_decide.json (capabilities, temperatures, act calibrator, multi_question.enabled = false), l14g_format.py and l14f_format.py (the exact input / output format), LICENSES.json (training data manifest).

Citation

@software{solvi,
  title  = {solvi: verifiable decision systems from catalogs of functions and checks},
  author = {mxkuzn and solvi contributors},
  year   = {2026},
  url    = {https://github.com/solvi-ai/solvi}
}