Qwen3-Coder-REAP-25B-A3B AWQ 4-bit

AWQ 4-bit quantization of Cerebras Qwen3-Coder-REAP-25B-A3B — a REAP-pruned (arxiv:2510.13999) variant of Qwen3-Coder-30B-A3B-Instruct — calibrated with thinking + code data, optimized for AMD RDNA4 (gfx1201) inference with SGLang.

Model Details

Performance (2x AMD Radeon AI PRO R9700, TP=2, fp8 KV)

sglang.bench_serving, single user, FP8 KV cache, --disable-cuda-graph:

Flat ~22.5 tok/s decode across the full 131K range — A3B MoE stays bandwidth-bound, no attention scaling cliff.

Notes

This is REAP, not REAM. REAP prunes experts based on router-aware impact scores; REAM (Samsung SAIL) instead merges similar experts. Both shrink MoE models, but with different algorithms and tradeoffs — they're not interchangeable. The base Cerebras prune drops 32 of 128 experts (25% reduction).

The CT (compressed-tensors) format from llmcompressor was converted to native AWQ via convert_moe_ct_to_awq.py — on ROCm the AWQ Triton GEMM kernel is 6× faster than the compressed-tensors path on the same weights.

shared_expert.{gate,up,down}_proj and mlp.gate (router) are preserved in BF16 to avoid the always-on residual / routing path going through INT4. shared_expert_gate (output dim 1) auto-falls-back to BF16 in the converter since AWQ packing requires divisibility by 8.

Usage with SGLang

Tested on the RDNA4 inference stack (SGLang v0.5.10 + 16 RDNA4 patches):

git clone https://github.com/mattbucci/2x-R9700-RDNA4-GFX1201-sglang-inference
cd 2x-R9700-RDNA4-GFX1201-sglang-inference
./scripts/setup.sh
MODEL=mattbucci/Qwen3-Coder-REAP-25B-A3B-AWQ scripts/launch.sh coder-reap-25b

The coder-reap-25b preset auto-detects the AWQ format and uses --quantization moe_wna16 with FP8 KV cache for 131K context single-user.

For other inference engines, this is a standard AWQ 4-bit checkpoint (group_size=128, asymmetric, fused MoE) and should load via vllm / transformers + autoawq without modification.

Hardware

Calibrated and benchmarked on 2× AMD Radeon AI PRO R9700 (gfx1201, RDNA4, 64 GB total VRAM) with ROCm 7.2 and SGLang v0.5.10 + RDNA4 patches. Per-GPU: ~6 GB weights + ~4 GB FP8 KV at 131K + overhead.