OpenJeff — small answers, serious judgment

An experimental LoRA adapter for evidence-conditioned, finite-choice decisions. This repository contains adapter weights, not the foundation model. Download the pinned Gemma foundation separately under its upstream terms. The included Apache-2.0 license covers OpenJeff's original artifacts.

Code and reproducible runs · Optional support · Evaluator handoff

Download

from huggingface_hub import snapshot_download
path = snapshot_download("greyecho/openjeff-pilot-v1")

For reproducible use, pin the Hub commit returned with this release. The files adapter_config.json and adapter_model.safetensors are standard PEFT artifacts. The original adapter config is unchanged so the frozen digest remains valid.

Run the decision model

Use the tested runtime and launch instructions. The GitHub release includes the same adapter plus the complete run directory required by PilotScorer. Generic text generation is not the evaluated task: OpenJeff compiles candidate labels to single-token codes and normalizes their logits with the frozen temperature. The calibrator is bound to the A100 runtime; loading weights on another device does not validate those probabilities.

git clone https://github.com/greyaperez/openjeff.git
cd openjeff
# In the documented PyTorch CUDA container on an A100 SXM 80 GB:
python -m venv --system-site-packages .venv
.venv/bin/python -m pip install -r configs/train-gpu-requirements.txt
.venv/bin/python -m scripts.stage_weights
.venv/bin/python -m openjeff.serve --run runs/pilot-v1 --port 8765

Initial evidence

On the same 231 public JevBench IDs, the initial OpenJeff pilot scored 90.04% versus 86.58% from Jev 1.13.0's published outcomes. Hard-subset accuracy was 80.18% vs. 72.97%. Jev was not rerun here; prompts and runtimes differ. This is not an official leaderboard rank or a full benchmark comparison. The diffusion follow-up remains experimental; this release is AR-only.

Detailed model card

OpenJeff is an experimental adapter for finite-choice, evidence-conditioned judgments. It returns candidate probabilities and supports explicit abstention. The trained adapter, original code, and original synthetic adaptation data are Apache-2.0. Foundation weights and third-party material retain their licenses and notices; see NOTICE and the pinned source snapshots.

Model and intended use

• Foundation: google/gemma-4-12B-it, revision 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7.
• BF16 foundation; float32 LoRA parameters. No quantization.
• Adapter: 21,331,968 trainable parameters; rank 16, alpha 32, dropout 0.05; text attention q/k/v/o projections. Config-and-weight SHA-256: a1811874542eb3b9ca4235d79826dfabb847f0c5ad076bfd25b5e4b2f426ba2b.
• Intended use: reproducible research into structured decisions, policy-rule interpretation, and probability readout. It does not perform actions itself.
• Serving: local serial HTTP prototype, 2–26 candidates and 1,536 input tokens. Confidence threshold 0.9 is an abstention policy, not an accuracy guarantee.

Google's developer/base-weight lineage is accepted for this project. Neither Google's U.S. location nor this adaptation establishes U.S.-only pretraining data. Full foundation pretraining is not reproduced.

Training and selection

8,000 original synthetic examples from 20 rule-grounded families, generated by openjeff/curriculum.py. Labels come from executable reference logic; no external teacher, private record, Jev API, or JevBench task supplies training examples. One epoch: 500 AdamW optimizer steps, microbatch 4, accumulation 4, learning rate 1e-4 with warmup/cosine decay, candidate-only cross entropy, seed in the run recipe.

Development (400 rows) alone selected step 500 and then the adapter over the base. Selection froze at 2026-09-23T04:34:38Z before calibration and final scoring. Separate partitions: calibration-fit 600, calibration-check 400, final 600, adversarial 200, and production-simulation 200. Exact request/group hashes include unique case IDs. Removing decorative IDs reveals 197/600 final states matching training states. On the remaining 403, OpenJeff scores 96.28% versus 75.93% for the base. Calibration-fit states also recur in evaluation. See reports/curriculum-overlap.json. Templates and logical patterns are shared; this is not a natural-language domain holdout or semantic deduplication guarantee.

Evaluation and calibration

Synthetic final accuracy gain is 17.83 percentage points, paired row-bootstrap 95% interval [14.67, 20.83]. This interval concerns the sampled synthetic generator distribution. Related candidate reversals are excluded from the independent-row bootstrap. The rule-grounded tasks are also solvable with deterministic programs; the result validates adaptation, not a need to replace those programs with an LLM.

Each frozen backend has its own positive temperature fitted on the 600-example fit set: base 2.7638, adapter 1.0207. Calibration artifacts bind scorer settings, weights, prompt, package versions, and GPU model. Changing that environment requires new calibration rather than reusing an unsupported probability claim.

JevBench uses only the 231 public tasks, the pinned upstream scoring function, and unchanged temperatures; no checkpoint was altered afterward. Only state, question, and options enter prompts. Gold answers, rationales, author metadata, and gold probabilities are excluded. The diagnostic raises the context guard to 8,192 tokens; longest prompt 4,071. It is not the official 534-item ranking. See complete results for slices and publisher comparisons.

Runtime and reproducibility

Tested on NVIDIA A100 SXM 80 GB, Linux/Python 3.12.3, PyTorch 2.8.0+cu128, Transformers 5.17.0, PEFT 0.21.0, and SDPA. Training/evaluation script elapsed 28.65 minutes with 44.57 GiB peak allocated memory. Public-benchmark local serial adapter scoring: median 72.3 ms, p95 553.3 ms. These exclude cold load, network, queues, and concurrency; they establish no deployment SLA.

The saved adapter passed offline reload and HTTP probability parity checks. Release validation recomputed 2,800 held-out score rows and both temperature fits. Weight/tokenizer hashes, recipe, raw scores, selection, calibration, and runtime are under runs/pilot-v1/. Source and data manifests accompany the release. The container tag was recorded but its OCI digest was not; bitwise retraining reproducibility is not claimed. Usage guide.

Limitations

This pilot does not establish fairness, robustness to natural-language policy ambiguity or prompt injection, employment suitability, medical/legal reliability, privacy compliance, or production throughput. A narrow synthetic curriculum and public benchmark can understate distribution shift. Candidate-order sensitivity was reduced, not eliminated. Hard-task probability calibration remains imperfect. Foundation pretraining may contain public benchmark material; its contamination status is unknown even though OpenJeff adaptation excludes benchmark examples.

No full Jev API comparison, learned hybrid bridge, scalar continuous-output head, production authentication, or validated enterprise deployment is included. At the time of the original pilot archive, the optional diffusion control had failed during startup before model evaluation. A subsequent completed diffusion study tested structured guidance and controls without changing this adapter. It found mixed gains and leaves AR as the default. Its H200 calibrators do not replace this pilot's A100 calibrators.