Muse Glimmer 30B — native Q4_0 QAT for llama.cpp

Native Q4_0 quantization-aware training with QUASAR. On held-out response tokens, this checkpoint is closer to BF16 than Meta's official Q4_K_M GGUF: 0.0213 vs 0.0228 KL and 95.9% vs 95.6% top-1 agreement, while training all 416 decoder projections directly on the standard Q4_0 lattice.

NVFP4 / vLLM · W4A4 / Blackwell · GGUF / llama.cpp (this repo) · Muse Glimmer collection · QUASAR paper

Measured on 948 held-out agentic prompts and 1.19M response tokens with the same fidelity harness.

Run it

ollama run hf.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF

llama-server -hf QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF \
  -c 131072 -fa on --jinja

For LM Studio, search for QUASAR-QAT Muse-Glimmer.

Supports 131,072-token context with the embedded Muse chat template for reasoning and tool calls. Requires a llama.cpp build with Muse Glimmer support (9a4843c, September 2026, or newer). This GGUF is text-only.

Selected benchmarks

Both models were served through the same llama.cpp build with identical prompts and decoding. Full same-harness fidelity and downstream results are available in eval/.

Technical details

All 416 decoder projections were trained directly on the Q4_0 lattice with QUASAR: INT4 codes with one scale per 32-weight block. The GGUF packs the trained codes and scales directly rather than applying a separate post-training quantization pass.

The checkpoint is 19.6 GB, with Q4_0 decoder projections, F16 token embeddings and output head, F32 norms, and the Muse chat template embedded. Export fidelity is documented in native_q4_0_receipt.json.

QUASAR is loss-aware quantization-aware training: it improves low-bit reconstruction during training and exports to standard deployment formats with zero inference overhead.

Related

NVFP4 / vLLM · NVFP4 W4A4 / Blackwell · Muse Glimmer 30B — QUASAR 4-bit QAT · Base model · QUASAR paper

Running this checkpoint? Share benchmark results, deployment notes, or integrations in Discussions.

Citation

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}