Papers: Look When Unsure, Check When Sure · DOI 10.5281/zenodo.23146970 · When Does a Consequence Model Make AI Agents Safer? · DOI 10.5281/zenodo.23197341 · Code: github.com/OpenInterpretability/ekbasis · Site: openinterp.org/ekbasis
FP8 build of Ekbasis-27B, an open world model for agents: given the current state and an action, it predicts what the action will do — will this lose work? will it fail? what will X be afterwards? — as calibrated, typed answers, in one forward pass. Not a model that thinks, nor one that judges: a model that foresees.
Weights in FP8 (per channel) with FP8 activations computed per token at run time, made with llm-compressor (FP8_DYNAMIC, compressed-tensors format), with no calibration data. The vision tower, MTP layers, embeddings and LM head stay in bf16. The prompt format, the letter readout and the calibration (calib.json, T = 1) are the same as the bf16
model. See the Ekbasis-27B card for what the model is, how it was
trained, the full evaluation, its limits, and how to let it look at the real state when unsure (predict, observe,
correct) over long sequences.
Context window: 262,144 tokens, as the bf16 model (the architecture's); the serving commands set 16,384, and the evaluation's prompts average 0.7–1.6K tokens per state.
Requires vLLM ≥ 0.30.0. One GPU; the API and the client are the same as for Ekbasis-27B.
hf download caiovicentino1/Ekbasis-27B-FP8 --local-dir Ekbasis-27B-FP8
bash Ekbasis-27B-FP8/serve_vllm.sh Ekbasis-27B-FP8 8001 # vLLM on 127.0.0.1:8001
python Ekbasis-27B-FP8/serve.py --model Ekbasis-27B-FP8 --vllm-url http://127.0.0.1:8001 --port 8000 # the API
pip install "git+https://github.com/OpenInterpretability/ekbasis" # the client
EKBASIS_URL=http://127.0.0.1:8000 ekbasis git-check -- "git reset --hard"
The same sets and scripts as the release evaluation, the build served the same way (vLLM 0.30, one RTX PRO 6000).
| Ekbasis-27B (bf16) | Ekbasis-27B-FP8 | |
|---|---|---|
| Size | 55 GB | 30.4 GB |
| Git, all 3,105 questions (seen + never-seen command types), % | 96.6 | 96.3 |
| "Will work be lost?" AUROC — seen / never-seen types | 0.998 / 0.989 | 0.997 / 0.987 |
| Work-losing flagged at P ≥ 0.2 — seen / never-seen, % | 98.0 / 95.2 | 98.0 / 95.2 |
| False alarms at P ≥ 0.2 — seen / never-seen, % | 0.0 / 2.4 | 0.0 / 2.4 |
| 16 fresh real-repository scenarios: work-losing flagged / false alarms / "fails?" right | 3/3 · 1/13 · 16/17 | 3/3 · 1/13 · 16/17 |
| Never-trained worlds, answers the actions change — one prompt per question / read-once, % | 81.2 / 78.0 | 81.4 / 78.1 |
ftest_family, answers the actions change, % | 81.9 | 81.4 |
| The 240-question git comparison — seen / never-seen types, % | 95.8 / 85.8 | 95.8 / 85.8 |
| ECE on git (lower is better) | 0.024 | 0.026 |
| Same answer as bf16 — all / bf16-confident (≥ 0.9), % | — | 98.2 / 99.9 |
| One check, one GPU — latency / checks per second with 16 in parallel | 0.088 s / 23 | 0.074 s / 37 |
| Forecasts of 10–30 actions, one GPU — median seconds one at a time / per minute with 16 in parallel (one prompt per question) | 5.0 s / 14.9 | 3.3 s / 24.5 |
| Same, reading the state once per step | 4.1 s / 28.2 | 3.1 s / 41.2 |
It meets every criterion of the pre-registered quality gate.The gate was written before the first builds existed, for V42's (PREREG_quantized.md in the repository:
accuracy within 1 point of bf16; every work-losing real-repository scenario flagged, and flag and false-alarm rates
within 1 point; ECE within 0.01; at least 97% of answers unchanged, 99% where bf16 is confident), and is applied
unchanged to this release's builds, against this release's bf16. It informs; every build that works is released, so
each machine can run the one that fits.
| Build | Size | Runs on | Speed (one RTX PRO 6000) |
|---|---|---|---|
| Ekbasis-27B (bf16) | 55 GB | GPUs with 80 GB | the reference |
| Ekbasis-27B-FP8 | 30.4 GB | GPUs with 48 GB; fastest with FP8 kernels (Ada, Hopper, Blackwell) | 1.6× the throughput of bf16 |
| Ekbasis-27B-INT4 | 18.6 GB | GPUs with 32 GB; 24 GB with text only and a 4k context (command in its card) | about bf16's |
| Ekbasis-27B-MLX-4bit | 15 GB | Macs with Apple Silicon (32 GB or more recommended) | — |
GPU sizes were tested by limiting vLLM to that much memory on one RTX PRO 6000 and checking that the answers match the build at full memory; the MLX build was validated with MLX on a Linux GPU, not on a Mac.
Apache-2.0 (LICENSE). Built on Eikos-27B (MIT, LICENSE-Eikos) and Qwen3.8-27B (Apache-2.0, LICENSE-Qwen);
attributions in NOTICE.
See the Ekbasis-27B card: every entry checked against its source.
Questions, collaborations and commercial use: [email protected]. Found a way to make the guard approve a destructive action? Please email it before publishing (SECURITY.md in the repository).
Ekbasis was built by Caio Vicentino (OpenInterp) with the help of Claude Opus 5.5 (Anthropic). Claude worked on the project with him:
Papers: Look When Unsure, Check When Sure · DOI 10.5281/zenodo.23146970 · When Does a Consequence Model Make AI Agents Safer? · DOI 10.5281/zenodo.23197341 · Code: github.com/OpenInterpretability/ekbasis · Site: openinterp.org/ekbasis
FP8 build of Ekbasis-27B, an open world model for agents: given the current state and an action, it predicts what the action will do — will this lose work? will it fail? what will X be afterwards? — as calibrated, typed answers, in one forward pass. Not a model that thinks, nor one that judges: a model that foresees.
Weights in FP8 (per channel) with FP8 activations computed per token at run time, made with llm-compressor (FP8_DYNAMIC, compressed-tensors format), with no calibration data. The vision tower, MTP layers, embeddings and LM head stay in bf16. The prompt format, the letter readout and the calibration (calib.json, T = 1) are the same as the bf16
model. See the Ekbasis-27B card for what the model is, how it was
trained, the full evaluation, its limits, and how to let it look at the real state when unsure (predict, observe,
correct) over long sequences.
Context window: 262,144 tokens, as the bf16 model (the architecture's); the serving commands set 16,384, and the evaluation's prompts average 0.7–1.6K tokens per state.
Requires vLLM ≥ 0.30.0. One GPU; the API and the client are the same as for Ekbasis-27B.
hf download caiovicentino1/Ekbasis-27B-FP8 --local-dir Ekbasis-27B-FP8
bash Ekbasis-27B-FP8/serve_vllm.sh Ekbasis-27B-FP8 8001 # vLLM on 127.0.0.1:8001
python Ekbasis-27B-FP8/serve.py --model Ekbasis-27B-FP8 --vllm-url http://127.0.0.1:8001 --port 8000 # the API
pip install "git+https://github.com/OpenInterpretability/ekbasis" # the client
EKBASIS_URL=http://127.0.0.1:8000 ekbasis git-check -- "git reset --hard"
The same sets and scripts as the release evaluation, the build served the same way (vLLM 0.30, one RTX PRO 6000).
| Ekbasis-27B (bf16) | Ekbasis-27B-FP8 | |
|---|---|---|
| Size | 55 GB | 30.4 GB |
| Git, all 3,105 questions (seen + never-seen command types), % | 96.6 | 96.3 |
| "Will work be lost?" AUROC — seen / never-seen types | 0.998 / 0.989 | 0.997 / 0.987 |
| Work-losing flagged at P ≥ 0.2 — seen / never-seen, % | 98.0 / 95.2 | 98.0 / 95.2 |
| False alarms at P ≥ 0.2 — seen / never-seen, % | 0.0 / 2.4 | 0.0 / 2.4 |
| 16 fresh real-repository scenarios: work-losing flagged / false alarms / "fails?" right | 3/3 · 1/13 · 16/17 | 3/3 · 1/13 · 16/17 |
| Never-trained worlds, answers the actions change — one prompt per question / read-once, % | 81.2 / 78.0 | 81.4 / 78.1 |
ftest_family, answers the actions change, % | 81.9 | 81.4 |
| The 240-question git comparison — seen / never-seen types, % | 95.8 / 85.8 | 95.8 / 85.8 |
| ECE on git (lower is better) | 0.024 | 0.026 |
| Same answer as bf16 — all / bf16-confident (≥ 0.9), % | — | 98.2 / 99.9 |
| One check, one GPU — latency / checks per second with 16 in parallel | 0.088 s / 23 | 0.074 s / 37 |
| Forecasts of 10–30 actions, one GPU — median seconds one at a time / per minute with 16 in parallel (one prompt per question) | 5.0 s / 14.9 | 3.3 s / 24.5 |
| Same, reading the state once per step | 4.1 s / 28.2 | 3.1 s / 41.2 |
It meets every criterion of the pre-registered quality gate.The gate was written before the first builds existed, for V42's (PREREG_quantized.md in the repository:
accuracy within 1 point of bf16; every work-losing real-repository scenario flagged, and flag and false-alarm rates
within 1 point; ECE within 0.01; at least 97% of answers unchanged, 99% where bf16 is confident), and is applied
unchanged to this release's builds, against this release's bf16. It informs; every build that works is released, so
each machine can run the one that fits.
| Build | Size | Runs on | Speed (one RTX PRO 6000) |
|---|---|---|---|
| Ekbasis-27B (bf16) | 55 GB | GPUs with 80 GB | the reference |
| Ekbasis-27B-FP8 | 30.4 GB | GPUs with 48 GB; fastest with FP8 kernels (Ada, Hopper, Blackwell) | 1.6× the throughput of bf16 |
| Ekbasis-27B-INT4 | 18.6 GB | GPUs with 32 GB; 24 GB with text only and a 4k context (command in its card) | about bf16's |
| Ekbasis-27B-MLX-4bit | 15 GB | Macs with Apple Silicon (32 GB or more recommended) | — |
GPU sizes were tested by limiting vLLM to that much memory on one RTX PRO 6000 and checking that the answers match the build at full memory; the MLX build was validated with MLX on a Linux GPU, not on a Mac.
Apache-2.0 (LICENSE). Built on Eikos-27B (MIT, LICENSE-Eikos) and Qwen3.8-27B (Apache-2.0, LICENSE-Qwen);
attributions in NOTICE.
See the Ekbasis-27B card: every entry checked against its source.
Questions, collaborations and commercial use: [email protected]. Found a way to make the guard approve a destructive action? Please email it before publishing (SECURITY.md in the repository).
Ekbasis was built by Caio Vicentino (OpenInterp) with the help of Claude Opus 5.5 (Anthropic). Claude worked on the project with him: