−2 logit bias on Bonsai 2 27B (RTX 5060 Laptop)

Run by Joseph Murray Adams.

Date: 2026-10-02
Result: on a fixed 50-question MATH-500 sample, a −2 logit bias on “wait”, “maybe”, and “perhaps” changed accuracy from 44/50 to 43/50 and average length from 845.2 to 872.3 tokens. Tokens per second stayed 29.3. This does not reproduce the Qwen3.5-4B report of higher accuracy and fewer tokens.

Suggested post title: −2 logit bias on Bonsai 2 27B: 44/50 → 43/50 on MATH-500, +3% tokens

Reproduction record

Run A is the server above with no logit bias. Run B is the same server plus:

--logit-bias 11158-2 --logit-bias 3655-2 --logit-bias 13784-2 --logit-bias 35542-2 --logit-bias 6970-2 --logit-bias 20734-2 --logit-bias 63068-2 --logit-bias 8106-2 --logit-bias 30442-2

Each form was one token (/tokenize, add_special=false, with_pieces=true). The bias is −2, not a ban.

Prompt, both arms:

Solve this problem. Put the final answer in \boxed{}.

<problem>

Question order: .NET Random(42) shuffle of the 500-row test split, first 50, saved as questions-50.json and reused for run B. Runner: run_math500.ps1 in this folder. Set $root (line 10) to your own folder before running it.

Prism’s published thinking recipe is temperature 1.0, top-k 20, min-p 0.05. Their benchmark tables used min-p 0.0. This test holds temperature 0 so the bias is the only difference. These accuracy numbers are not a Prism leaderboard row.

Results

Accuracy −2 points. Length +3.2% (845.2 → 872.3). Speed unchanged.

Forty-seven outcomes matched. Three changed:

Misses that stayed misses: number_theory/598, algebra/2253, prealgebra/1742, intermediate_algebra/662, intermediate_algebra/1454.

Per question, completion tokens. A t marks a reply that hit the 3072 cap.

Files in this repository

What this is not

A LocalLLaMA report on Qwen3.5-4B saw higher MATH-500 accuracy and fewer tokens from a −2 bias on these words. The paper “Wait, We Don’t Need to ‘Wait’!” shortened traces and held accuracy, and it did not use a flat −2 bias. A −2 bias only makes the tokens less likely.

Simon Willison’s 20–44 tokens/s figures are Apple Silicon. This run is a Windows laptop at about 30 tokens/s while answering, and 31.69 tokens/s on llama-bench TG128.

Separate speed row

llama-bench on the same file, same GPU, same day, AC power, 85 W cap:

.\llama-bench.exe -m C:\Qu_models\bonsai\Ternary-Bonsai-2-27B-PTQ1_0.gguf -p 512 -n 128 -ngl 99 -fa 1

The bench line names the architecture inside the GGUF as qwen35 27B PTQ1_0. The file is Ternary-Bonsai-2-27B-PTQ1_0.gguf. PQ2_0 was not tested. This row is for Prism’s community-benchmarks page, not for the logit-bias thread. There is no RTX 5060 Laptop entry in that table. Template: https://github.com/PrismML-Eng/Bonsai-demo/blob/main/community-benchmarks/bonsai2/TEMPLATE-llama-cpp.md