GLM-5.3-Flash on DGX Spark

Run GLM-5.3-Flash (320B params, 18B active) on a single NVIDIA DGX Spark with one Docker command. Every recipe includes measured quality numbers against the original BF16 model.

What's here

Two ways to run the model, both tested on real hardware:

Recipe A is smaller, faster to download, closer to the original model, and supports full context. Recipe B is kept for reproducibility.

Quick start

Prerequisites: one DGX Spark (128 GB), Docker, ~85 GB disk for weights.

# 1. Download the weights (one time, ~85 GB)
git clone https://github.com/0xSero/GLM-5.3-Flash-DGX-Spark.git
cd GLM-5.3-Flash-DGX-Spark/recipes/turboderp-2p05-sglang-mul1
./fetch-weights.sh ./model

# 2. Start the server
docker run --gpus all --ipc=host --shm-size 16g -p 8000:8000 \
  -v "$PWD/model:/model:ro" \
  ghcr.io/0xsero/glm53-flash-exl3-plain@sha256:85cb3fa86d31a781b94dcf10ee168adf096cfeaac14d2f1e6c560504e58e4eed

# 3. Use it
curl http://localhost:8000/v1/models
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"/model","messages":[{"role":"user","content":"Hello"}]}'

What's measured

Every claim has a receipt in recipes/turboderp-2p05-sglang-mul1/MEASURED.md:

• Quality: 78.9% top-1 agreement with the BF16 original across 65,504 test positions (KL divergence 0.384)
• Long context: correctly retrieves facts from 262,016-token prompts
• Structured output: generates valid JSON 1-300 at every context length up to 200k
• Pre-registered benchmark: 15/20 tasks pass (retrieval 5/5, structured output 5/5, reasoning 4/5)
• Vision: 6/6 synthetic image/video tests pass
• No looping: 5/5 repetition-safety checks pass

Not yet claimed: MTP/speculative decoding, speed benchmarks with CUDA graphs, full vision evaluation.

Why not prune a bigger model instead?

We tested this. Taking the 3bpw variant and removing the 25% least-important experts (ranked by routing mass) to fit the same ~90 GB budget produced a worse model: 66.8% top-1 / KL 0.905 vs the 2bpw unpruned at 78.9% / 0.384. You can't remove a quarter of the experts and compensate with more precision per survivor. At equal bytes, keep all experts and compress harder.

How Recipe A works

The stock kernels couldn't run this model. Three things were built and validated:

1. mul1 codebook + 2-bit support in sparkinfer (sparkinfer-patches/) — the trellis decoder only spoke "mcg" at 3-6 bits; now it handles the "mul1" codebook at 2-6 bits, verified bit-exact against exllamav3
2. exllamav3 ARM64 port (build/exllamav3-arm64/) — the reference runtime didn't compile on ARM; now it does (CPU expert offload and native-TP all-reduce are the only losses)
3. SGLang adapter (adapter/) — nine patches that teach SGLang's exl3 quantization method to load and serve the artifact, including fixes for two silent-corruption bugs (skipped KDA conv weights, unscaled MoE routing)

Each patch is in build/*.diff with a plain-language description in the recipe README.

Credits

• turboderp — the EXL3 quantizations (MIT)
• zai-org — GLM-5.3-Flash (MIT); cite arXiv:2602.15763
• malaiwah — the quality measurement standard
• Luke Alonso — sparkinfer (Apache-2.0)
• NVIDIA — DGX Spark hardware

MIT license. See LICENSE.