Qwen3.5-4B — QUASAR NVFP4 W4A4 for Blackwell

Project page · Code

39% lower KL to BF16 than cosmicproc's public Qwen3.5-4B W4A4 build in our same-harness comparison (.046 vs .075), with +1.9 points top-1 agreement.

4.2 GB · all 200 decoder projections in NVFP4 · W4A4 activations · vision + MTP retained · native Blackwell FP4 path · serves directly with vLLM.

Also in this family: W4A16 / vLLM · W4A4 / Blackwell (this repo) · Q4_0 GGUF / llama.cpp · collection · paper

Same bit-exact QUASAR-trained weights as the W4A16 release, plus calibrated activation scales for native W4A4 execution on Blackwell. The W4A16 sibling contains the full reasoning benchmark head-to-head.

Run it

pip install "vllm>=0.26"

vllm serve QUASAR-QAT/Qwen3.5-4B-QUASAR-NVFP4-W4A4 --max-model-len 262144 --reasoning-parser qwen3

Target runtime is Blackwell (compute capability 10.0+), where vLLM uses the native NVFP4 W4A4 path. On Hopper, vLLM falls back to weight-only Marlin; we verified identical greedy outputs to the W4A16 release on H100.

Fidelity under FP4 activations

Per-token forward KL(BF16 ‖ quantized) and top-1 agreement on 734 held-out prompts of the BF16 model's own responses (934k tokens), with vLLM's NVFP4 activation quantization applied to the inputs of the same 200 projections; each W4A4 row uses its own calibrated scales. Methodology and audit trail: EVAL.md.

FP4 activations account for most of the remaining gap to BF16: the activation-only reference is .037 KL, while QUASAR W4A4 is .046. Under the same evaluation harness, cosmicproc is .075.

What is quantized

Weights: NVFP4 on all 200 major decoder projections; bit-exact to the QUASAR-trained W4A16 weights. Activations: NVFP4 group-16 with dynamic local scales and calibrated static per-tensor global scales. BF16: embeddings, norms, conv1d kernels, small delta-net gate vectors, vision tower and MTP drafter.

Related

• QUASAR-QAT/Qwen3.5-4B-QUASAR-Q4_0-GGUF — native Q4_0 QAT for llama.cpp / Ollama / LM Studio.
• QUASAR-QAT/Qwen3.5-4B-QUASAR-NVFP4 — the W4A16 release with the full benchmark head-to-head.
• QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 — the 27B W4A4 sibling.
• Qwen3.5-4B & Qwen3.8-27B — QUASAR 4-bit QAT — the collection · Paper: arXiv 2608.13966.

Citation

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}