Qwen3.6-35B-A3B — ROCmFP4 STRIX_LEAN quant for AMD Strix Halo

Q4_0_ROCMFP4_STRIX_LEAN GGUF quantization of Qwen/Qwen3.6-35B-A3B (Qwen3.5-MoE architecture, ~3B active, multimodal), produced for AMD Strix Halo (Ryzen AI Max, gfx1151).

⚠️ These files use ROCmFP4 tensor types (ggml type 106) — stock llama.cpp will reject them (invalid ggml type). Use either:

• kyuz0/amd-strix-halo-toolboxes rocm-7.2.4-rocmfp4 image (public, maintained), or
• the Hal0ai/Hal0_ROCmFPX llama.cpp fork.

First published STRIX_LEAN of this model as far as I can tell (a ROCMFP4_FAST and an MTP-focused ROCmFP4 exist elsewhere — the STRIX_LEAN recipe trades a hair of file size for the Strix attn-K/V quality layout + Q5_K token embeddings).

Files

Measured performance

AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X), ROCm backend, kyuz0 rocm-7.2.4-rocmfp4 toolbox, llama-bench -fa 1 --mmap 0:

For reference, the same model as UD-Q4_K_XL runs 48 tok/s on identical hardware — the ROCmFP4 kernels are worth ~+31% decode on Strix Halo.

How it was made

BF16 GGUF source and imatrix from unsloth/Qwen3.6-35B-A3B-MTP-GGUF:

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Qwen3.6-35B-A3B-BF16-00001-of-00002.gguf \
  Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

(nextn/MTP metadata from the source survives conversion, though speculative decoding showed no speedup on Strix Halo in my testing — plain decode is recommended.)

Serving

llama-server -m Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-BF16.gguf -ngl 999 -fa on --jinja -c 65536

Companion repos

• gsrunion/Ornith-1.0-35B-ROCmFPX-GGUF
• gsrunion/Ornith-1.0-9B-ROCmFPX-GGUF

Credits

• Base model: Qwen — Qwen3.6-35B-A3B (Apache-2.0)
• BF16 GGUF conversion + imatrix: unsloth
• ROCmFP4 quant formats: Hal0ai/Hal0_ROCmFPX; maintained builds: kyuz0