GLM-5.3 REAP Fidelity Study

The evidence behind the GLM-5.3 REAP model series: how each pruned cut was measured, and why it prunes the way it does.

What we measured, and how

Every number here is KL divergence vs the full BF16 model — a token-by-token “how differently do these two models predict the next token” score over a sealed 25-prompt panel across the full 154,880-token vocabulary. 0 = identical, lower = closer. It is a far more sensitive test than benchmark accuracy: it catches quality loss long before a leaderboard would.

Finding 1 — which experts to keep matters more than anything

Pruning to the same size (keep 192 of 256 experts), only changing how experts are ranked:

Ranking experts by their largest share of any single domain’s work (so every domain keeps its specialists) beats the frequency-based choice almost 2×. That is the whole idea behind the series.

Finding 2 — the size ladder

Same criterion, different number of experts kept:

Finding 3 — the prune dominates the quant format

At the same expert count, EXL3 (3-bit) and W4A16 (INT4) land in the same place — 0.361 vs 0.357 at keep-192, 0.511 vs 0.506 at keep-168. The pruning drives the fidelity; the quant format barely moves it. So pick the format your hardware wants (EXL3 for Blackwell/consumer, W4A16 for Hopper) and choose the size by how much fidelity you can spend.

Contents

• kld/ — every KL run: the criterion sweep, the size ladder, EXL3 vs W4A16, NVFP4.
• plans/ — the exact kept-expert lists per layer.
• gpqa/ — GPQA-Diamond runs for spot-checking downstream accuracy.
• panel.json, frontier_table.txt, *_summary.json — the sealed eval panel and roll-ups.

Inputs come from glm-5.3-reap-observations-v1.

Credits

• REAP: Cerebras Research — arXiv:2510.13999.
• GLM-5.3: Z.AI / zai-org. EXL3: turboderp / exllamav3.
• Measured on 8× NVIDIA RTX PRO 6000 Blackwell.