train-jev-27b-lora

A JEV-style System 1 decision model: Qwen3.8-27B + LoRA + a 24-slot decision head, trained with train-jev. It does not generate text. It returns a calibrated probability for every option in one forward pass, so you can act on confident answers and route the rest to a person. Yes/no, multiple-choice and rating questions; the server speaks TypeSafe's System One API.

Matches the official JEV-27B-VL, better calibrated. Same accuracy on the same questions (Kev transfer-v4 0.851 vs 0.852, Decision Index 54.6 vs 54.1) with a much lower calibration error (ECE 0.053 vs 0.097, NLL 0.411 vs 0.460). Needs an 80 GB GPU; for a smaller, faster model use train-jev-4b-lora.

Quick start

git clone https://github.com/shibing624/train-jev.git && cd train-jev
pip install -r requirements.txt
python -m trainjev.serve --base Qwen/Qwen3.8-27B --adapter shibing624/train-jev-27b-lora --port 8020

curl -s localhost:8020/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
  "questions": {
    "department": {"type": "choice", "instructions": "Which team should handle this?",
                   "criteria": {"returns": null, "shipping": null, "billing": null}},
    "escalate":   {"type": "noul", "instructions": "Does this need urgent human attention?"},
    "frustration": {"type": "score", "instructions": "How frustrated is the customer?",
                    "criteria": ["Calm", "Frustrated", "Very angry"]}
  }}'

This repo holds the LoRA adapter, the decision head (head.pt) and the temperatures (calibration.json); the backbone is downloaded on first start. Load it through train-jev: plain PEFT / Transformers loading applies the adapter but not the head or the temperatures. More: serving guide.

Results

Same questions for every model. ↑ higher is better, ↓ lower is better.

Full protocol and breakdowns: docs/results.md.

Training

Qwen3.8-27B with a rank-16 LoRA on every attention, DeltaNet and MLP projection, and a 24-slot fp32 head (false/true, 0–5, A–P) initialised from the backbone's lm_head. Trained for 5,123 steps (batch 128) on SargeDev/jev-distill-corpus-v3 with KL to the teacher distribution plus 0.5 × RPS on rating questions; per-type temperatures fitted on a held-out calibration split. Recipe: docs/training.md.

Limitations

• Calibrated in distribution; check a probability threshold on your own data before relying on it.
• Only the first 16 choice labels (A–P) are trained; choices with more than 16 options use extra, untrained labels.
• Trained for decisions only, on a single seed.

Citation

@misc{train-jev,
  title={train-jev: Training JEV-style Decision Models},
  author={Ming Xu},
  year={2026},
  howpublished={\url{https://github.com/shibing624/train-jev}}
}

Independent research reproduction, not affiliated with TypeSafe AI or autotrust. Apache-2.0.