Gemma 4 E4B — 2× lower KL than Google's official Q4_0 QAT

Native QUASAR Q4_0 for llama.cpp / Ollama / LM Studio: 0.044 KL to BF16 vs 0.088 for Google's official Q4_0 QAT, with standard Q4_0 runtime.

QUASAR Q4_0 also has lower KL than Unsloth's Q4_0 repack of Google's QAT (0.071; the UD-Q4_K_XL file is Q4_0 tensors despite its name) and LM Studio's Q4_K_M (0.114), a more expressive format. Same BF16 reference, prompts, llama.cpp build and backend for every row.

The W4A16 twin shows the same pattern under vLLM: KL 0.022 to BF16 vs 0.064 for Google's W4A16 QAT.

ollama run hf.co/QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF
llama-cli -hf QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF

Downstream benchmarks

Higher point estimates on all 8 downstream benchmarks than Google's QAT model (measured on the W4A16 twin, which carries the same trained codes).

Same evaluation harness; full configs, BF16 results and paired-bootstrap CIs are in EVAL.md.

Native QAT GGUF

Native QAT checkpoint: the GGUF stores QUASAR's trained INT4 codes and scales directly, with no post-training re-quantization. All 258 quantized tensors are verified bit-exact against the W4A16 twin; see native_q4_0_receipt.json and the included verifier (scripts/export_gemma4_gguf_q4_0.py --verify).

Standard Q4_0 runtime in llama.cpp, Ollama and LM Studio. Vision is included through the BF16 projector.

Run it

• Ollama: ollama run hf.co/QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF — the vision projector is picked up automatically.
• LM Studio: search QUASAR-QAT gemma-4-E4B in Discover.
• llama.cpp, vision: llama-mtmd-cli -hf QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF --image photo.jpg -p "Describe this image." --jinja
• Thinking: on by default in the embedded Gemma 4 chat template; toggle with --chat-template-kwargs '{"enable_thinking": false}' (llama-cli / llama-server).
• Context: 131,072 tokens.
• Files: gemma-4-E4B-it-QUASAR-Q4_0.gguf (5.23 GB — Q4_0 linears with trained group-64 scales, Q6_K token embeddings, chat template embedded) and mmproj-gemma-4-E4B-it-BF16.gguf (992 MB, the unmodified BF16 vision projector from google/gemma-4-E4B-it).

QUASAR

QUASAR is loss-aware quantization-aware training. It improves the low-bit reconstruction during training, then exports to standard deployment formats with zero inference overhead.

This checkpoint was healed against the BF16 teacher for 626M tokens: one epoch over ~398k prompts self-distilled from BF16 Gemma 4 E4B-it with thinking on, fp32 master weights, INT4 group-64.

Related

• W4A16 compressed-tensors (vLLM) twin of this checkpoint: QUASAR-QAT/gemma-4-E4B-it-QUASAR-W4A16-G64
• Gemma 4 12B, same recipe: GGUF · W4A16 · collection · Gemma 4 E4B & 12B — QUASAR 4-bit QAT
• Base model: google/gemma-4-E4B-it · Paper: arXiv:2608.13966 · Org: huggingface.co/QUASAR-QAT

License

Based on Gemma 4 by Google DeepMind. Upstream LICENSE/NOTICE apply.

Citation

arxiv.org/abs/2608.13966

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}