Linnaeus

CUDA-optimized small multimodal decision models. Forked from an AMD ROCm research project; the same recipe, retargeted to NVIDIA servers.

The upstream project trains and evaluates on AMD ROCm (RX 7900 XTX). Linnaeus retargets the same recipe to CUDA:

• torch/torchvision pinned to +cu128 builds via a dedicated package index (pyproject.toml); FLA's Triton kernels run unchanged on CUDA
• Triton causal convolution enabled on all backends (upstream gated it to HIP); SDPA automatically dispatches to FlashAttention/cuDNN on NVIDIA
• VRAM allocator cap is configurable: LINNAEUS_VRAM_FRACTION (default 0.8; use 0.95 on headless servers)
• GPU telemetry auto-detects the DRM device and falls back to NVML (uv sync --extra telemetry); override with LINNAEUS_GPU_DEVICE / LINNAEUS_GPU_INDEX
• Linux + CUDA only; macOS is not a supported runtime

Toolchain and commands are managed by mise (mise.toml): mise install provisions Python 3.12 + uv, mise tasks lists all commands.

mise install && mise run install
LINNAEUS_VRAM_FRACTION=0.95 uv run python scripts/run_experiment.py

On hosts without direct Hugging Face access: export HF_ENDPOINT=https://hf-mirror.com.

---

What changed from upstream

Recipe: identical — rank-8 LoRA + decision head, joint RLCD + cross-entropy, same data mixture, same isolation/split rules, same checkpoint-selection protocol. Only the backbone moved (Qwen3.5-0.8B → 2B) and it is now a flag.

Runtime and training ops

• --base-repo / --base-revision swap the pinned backbone; --lora-rank, --workers, --eval-batch-size, --cpu-threads, --dev-cap are recipe-level knobs
• --skip stage groups and --train-only for cheap smoke runs (50-step pipeline check in ~8 min instead of a full day)
• fused AdamW on CUDA, prefetch_factor=4 training loader, per-worker image decode cache (LRU 128), expandable_segments allocator
• restart-safe stages: prepare-data re-verifies its outputs against scratch-disk loss instead of trusting completed.json; per-case resume in the Laya benchmark; curl stall guard (--speed-limit/--speed-time) on all manifest downloads
• disk reclamation in prepare-data: archives deleted right after extraction (~36 GB steady-state instead of ~58 GB)
• China-network support out of the box: HF_ENDPOINT URL rewriting for manifest curl downloads, HF_HUB_DISABLE_XET path, ModelScope fallback for model weights
• PDM → uv + mise.toml toolchain; fla lazy-imported so macOS still runs lint/typecheck/tests without Triton wheels

Results (details in Benchmarks)

• Held-out macro accuracy 80.78% vs upstream 78.21% — 20 wins / 6 ties / 0 losses across 26 groups
• JevBench public 231 tasks 73.16% vs upstream 65.80% — top of the ~2B local-model class
• XNLI 15-language 76.0% — recovers the suite upstream lost to Laya multilingual

---

Linnaeus builds small multimodal models for direct decisions. Give the model a message, a document, or an image, and ask it to choose, judge, or score. It returns probabilities in a single forward pass, with multiple questions sharing the same input. That is our take on System 1.

The Python toolkit covers training, inference, evaluation, and agent integration. The model runs locally on a consumer GPU.

Linnaeus-0.1.0-2B · Model card · Documentation · Benchmarks

One message, several decisions

Route a support request and check whether it asks for a refund in the same call:

from linnaeus.predictor import Predictor

model = Predictor.from_checkpoint("pi-dal/Linnaeus-0.1.0-2B")
result = model.predict(
    {"message": "I was charged twice. Please refund the duplicate."},
    {
        "route": {
            "type": "choice",
            "instructions": "Which team should handle the request?",
            "criteria": ["billing", "technical support", "sales"],
        },
        "refund": {"type": "noul", "instructions": "Is a refund requested?"},
    },
)

print(result["answers"]["route"]["choice"])
print(result["answers"]["refund"]["noul"])

Each answer has its own distribution. Use choice to select a candidate, noul to estimate whether a statement is true, and score to rate something on an ordered scale. Add a PIL image to the state for visual questions.

See installation and inference for setup, checkpoint loading, and response fields. The loader downloads and caches the checkpoint and its pinned base model, then merges LoRA and applies the saved calibration.

Train, evaluate, integrate

One workflow prepares the public-data mixture, trains with joint RLCD and cross-entropy, selects a checkpoint, calibrates it, and runs the evaluations:

uv run python scripts/run_experiment.py

The training guide covers setup and resuming a run. The model card reports quality, latency, and limitations. For agents, Bub integration exposes the same interface as one decision tool through the Bub SDK.

Benchmarks

Held-out macro accuracy 80.78% over 26 task groups (180,031 test examples), temperature-calibrated on an independent partition. Same recipe, same harness, same data as the upstream baseline — only the backbone changed (Qwen/Qwen3.5-0.8B → Qwen/Qwen3.5-2B, revision pinned).

Per-group accuracy (test split, calibrated)

Group	Linnaeus-2B	Dohnuts-0.8B	Δ
scienceqa	92.66%	84.58%	+8.1
esci_us	57.64%	51.00%	+6.6
sharc	73.19%	66.80%	+6.4
clevr_count	89.72%	83.50%	+6.2
aokvqa	83.66%	78.12%	+5.5
vqav2_yesno	85.88%	80.82%	+5.1
boolq	87.71%	82.97%	+4.7
esci_es	60.26%	56.07%	+4.2
contract_nli	85.59%	81.50%	+4.1
xnli_en	87.20%	83.77%	+3.4
esci_jp	63.81%	60.62%	+3.2
xnli_zh	78.20%	75.58%	+2.6
typed_decisions	73.05%	71.75%	+1.3
massive_zh-CN	76.82%	75.56%	+1.3
screenqa_noul	72.35%	71.14%	+1.2
clevr_exist	98.58%	97.40%	+1.2
banking77	73.70%	72.95%	+0.7
ag_news	90.04%	89.49%	+0.6
clevr_attribute	98.92%	98.36%	+0.6
emotion	77.00%	76.45%	+0.6
wikiqa	96.17%	95.94%	+0.2
mail_spam	98.95%	98.83%	+0.1
sms_spam	99.37%	99.37%	−0.0
mail_phishing	98.95%	99.24%	−0.3
massive_en-US	78.89%	79.19%	−0.3
screenqa_choice	22.05%	22.41%	−0.4

Vision and product-intent groups gain the most; nothing regresses beyond noise (screenqa_choice is hard for both models at ~22%).

Latency

Warm single-GPU BF16 predict latency, RTX 4090, merged LoRA (3 warmups, 20 repetitions; load/network excluded):

Upstream-published references on the same harness (RX 7900 XTX, not our measurement — hardware differs): Dohnuts-0.1.0-0.8B text 1q 15.1 ms, text 50q 112.5 ms, image 1q 24.4 ms; Laya multilingual 9.8 ms text 1q, no image path; Laya Vision 75.4 ms image 1q.

JevBench v1.2.2 — 231 public tasks (measured)

Official runner, pinned datasets, rejections count as wrong:

Reference points above us: SemIf (Qwen3.5-4B) 80.95%, Jev 1.13.0 86.58%, GPT-5.6 Luna 97.40%. Among small local models we are at the top of the ~2B class and ahead of every sub-1B classifier.

Head-to-head on identical tasks (per-task outcomes, our 231 vs each system's published outcomes — wins/losses where models disagree):

Per-topic deltas expose one systematic weakness: hard/temporal_numeric (arithmetic on amounts, dates, quotas — 15 tasks) is our worst cell at 2/15 correct while most systems answer 5–12 of them. The model is uncertain rather than confidently wrong there (scattered probabilities); the training mixture is thin on multi-step numeric decisions. Everything else — intent, routing, policy, adversarial — is at or above the 4B tier.

Same backbone (Qwen3.5-2B + rank-8 LoRA + decision head), JevBench v1.4.2 official — note v1.4.2 uses expanded tier sizes (standard 96 / hard 220), so tier numbers are not on the same task set as our v1.2.2 run:

Same-backbone peer without an official JevBench entry: mohit67890/imajev-2b — same pinned Qwen3.5-2B@15852e8c base and PEFT adapter form; reports only its own ImajevBench (279 questions), not comparable to the table above.

Laya protocol suites (measured vs published)

Upstream-published baselines on the same frozen inputs; Linnaeus measured on RTX 4090:

Multilingual suites (measured vs published)

Where upstream lost XNLI to Laya, Linnaeus-2B takes it back.

Provenance

Linnaeus-0.1.0-2B was trained on a rented NVIDIA RTX 4090 with the upstream recipe unchanged: rank-8 LoRA plus decision head, joint RLCD + cross-entropy, 3,600 steps, ~3.4 h of GPU training, checkpoint selected by development macro accuracy.

Acknowledgements

Linnaeus is a CUDA retarget of the upstream Dohnuts project by PsiACE. The training recipe (RLCD + cross-entropy), data mixture, isolation rules, evaluation protocol, and execution design are theirs; the baseline numbers quoted throughout come from their published artifacts.

License

The code is licensed under Apache-2.0. Model weights are released under CC BY-NC-SA 4.0 for non-commercial research. See the model card for checkpoint terms and training-data restrictions.