🔧 Runtime: build the ROCmFPX fork below

Stock llama.cpp will not load this file. You need both the muse-glimmer architecture and the ROCmFP4 tensor types in one tree. Upstream charlie12345/ROCmFPX has the ROCmFP4 types but not muse-glimmer. Our fork has both:

kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

git clone https://github.com/kingjones30/ROCmFPX.git
cd ROCmFPX
cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server llama-quantize -j$(nproc)

Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a muse-glimmer ROCmFP4 GGUF from this family and generates coherent text.

Muse-Glimmer-30B — ROCmFP4 for AMD Strix Halo (gfx1151)

✅ the complete muse-glimmer port — text graph, vision projector and chat parser — ships as a single applyable patch in this repo, alongside 4 ROCmFP4 ftypes, the DFlash drafter and the vision projector

muse-glimmer is not an upstream llama.cpp architecture. Running it end-to-end takes three independent pieces of work; all three are in patches/muse-glimmer-complete.patch (20 files, 81,968 bytes, git apply --check clean). Variant count verified against Hugging Face repository metadata for the public ROCmFP4 builds of this base model — file facts only, no third-party build was benchmarked here.

Six quantisations of Muse-Glimmer-30B — four ROCmFP4 and two 8-bit ROCmFPX — built for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, bundled with the DFlash speculative drafter and vision projector. ROCmFP4 is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp; Muse Glimmer support exists only in current upstream — this build ports the model forward into the ROCmFPX base so the two can meet.

Why this build?

• 20.31 tok/s on the FAST variant vs 16.65 tok/s for Meta's fastest official GGUF (kquant-17gb) — same box, same flags, same drafter: 1.22×
• 13.80 GiB vs Meta's 15.61 GiB — 1.8 GiB smaller and faster
• Four ftypes published so you can pick the size/speed/verbosity point you want
• Text, vision and DFlash speculative decoding all work from one download
• Chat-format parser ported, so no to=self<|message|> control tokens leak into output
• Every published file validated before upload; nothing shipped unverified

Which file should I use?

Ryzen AI Max+ 395, ROCm 7.2.4, DFlash drafter at --spec-draft-n-max 15, -fa on, ctx 32768, batch 1, temperature 0. Warm medians of 9 generations; the first call after load is discarded.

Why two different speed figures? The comparison table above is a controlled A/B: every build ran the same fixed prompt set, so the numbers are directly comparable to each other and to Meta's reference (that is where the 1.22x comes from). Real-world decode on the FAST build varies with workload by about 2.6x - measured on the live deployment at ~45 tok/s on code-transform/edit work, ~17 tok/s on freeform prose, and ~30 tok/s in typical mixed use. Quote the range, not a single number.

Start with FAST. BASE is both the slowest and the largest — it is published for completeness, not because anyone should choose it.

⚠ The faster files write shorter answers

Some of the speed comes from terser output, not only from faster decode. Median words per answer on identical prompts:

The quality check is substring-based and cannot distinguish "more concise" from "less thorough." If answer depth matters more than throughput, prefer STRIX. This is a real trade, not a free win.

Across the whole family — all six model quants

⚠️ Decode on this model is workload-dominated, not variant-dominated. DFlash proposes long runs on repetitive and code-like text and very little on freeform prose, so a single tok/s figure is misleading — mean accepted length moves 2.5 → 7.1 on the same binary and the same weights. Quote a range for this model, not a point.

Measured on one Ryzen AI MAX+ 395, median of 3, with the DFlash head (--spec-type draft-dflash --model-draft dflash-ROCmFP4-STRIX.gguf --spec-draft-ngl 99):

Without a drafter the 8-bit builds measure 7.48 (111) and 7.22 (115) tok/s — medians of 3, raw runs 7.54 / 7.48 / 7.35 and 7.22 / 7.21 / 7.26.

⛔ Serve it with the draft head. Without --model-draft the 8-bit build drops 11.62 → 7.65 (−34%). The two 8-bit builds are within noise of each other: AGENT routing lifts draft acceptance on MTP models, and this model uses DFlash, so there is nothing for it to win here.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF --local-dir muse

llama-server \
  -m muse/muse-glimmer-30B-ROCmFP4-FAST.gguf \
  --mmproj muse/mmproj-kquant.gguf \
  --spec-type draft-dflash \
  --model-draft muse/dflash-kquant.gguf \
  --spec-draft-n-max 15 \
  --chat-template-kwargs '{"reasoning_strength":"medium"}' \
  -ngl 999 -fa off -dio --jinja \
  -c 32768 --host 127.0.0.1 --port 8080

Requires a llama.cpp built with ROCmFP4 support (ggml types 100–106). Stock llama.cpp rejects these tensor types.

Three flags that matter more than which file you pick

⛔ --reasoning-budget does not work on this model. Values 256 and -1 produced byte-identical runs at temperature 0 — the flag is not enforced on peg-native format. Use reasoning_strength instead.

Verified hardware

Vulkan is impossible, not merely slow. The backend rejects these files at parse time:

gguf_init_from_reader: tensor 'output.weight' has invalid ggml type 101. should be in [0, 43)

Vulkan's type table ends at 43; ROCmFP4 types are 100–106. No flag changes this. For reference, Meta's k-quant does run on Vulkan and measured 6.10 tok/s versus 16.65 on ROCm on this box, so Vulkan is not a useful path for this model in any case.

Speculative decoding (DFlash)

Muse Glimmer has no MTP tensors — zero in both the base checkpoint and the quants. It speculates using DFlash against a separate 5-layer drafter, which is what Meta's own recipe prescribes.

⚠️ Per-token acceptance is a misleading statistic for a block drafter. DFlash proposes 15 tokens in one forward pass; ~21% acceptance means ~4.13 tokens land per target pass, which is healthy. Judge block drafters on tokens-per-pass.

n-gram speculation is not a substitute here. Measured on the same build: ngram-map-k reached 42.9% acceptance — double DFlash's — yet ran 45% slower on code-transform work (15.20 vs 27.49 tok/s), because it proposes far fewer tokens per pass.

A ROCmFP4 drafter is included but is not the default. It measured 1.008× against Meta's k-quant drafter on a quiet box — inside noise, acceptance unchanged. The drafter is ~1.5 GiB of a ~17 GiB working set, so shrinking it 8.5% moves total memory traffic by well under 1%. Shipped because it is valid, not because it is faster.

Tool calling

7-case suite, run against this build and against Meta's k-quant on the upstream binary as a reference:

The parallel-call failure is the model's, not the quantisation's — Meta's own weights on upstream's own parser fail identically. Sequential agent loops are unaffected.

Raw example:

{"name": "book_flight",
 "arguments": {"passenger": {"name": "Alice Smith", "age": 34},
               "route": "LHR-JFK", "cabin": "business"}}

Agentic loop

A multi-step loop (list → move → observe → finish) over a directory of loose files, 3 runs at temperature 0.7: 3/3 completed the task correctly, 0 cases of claiming an action without emitting a tool call.

Vision

The vision path needs stage 2 of the port below — it registers PROJECTOR_TYPE_MUSE_GLIMMER in clip/mtmd. A build carrying only the 9-file text-graph patch reports unknown projector type: muse-glimmer when a projector is passed.

⚠️ Minimum useful image size is 28×28 px. The preprocessor snaps to patch 14 × merge 2, so anything smaller collapses to a single merge token and carries no spatial signal. Feed 256×256 or larger; llama-mtmd-cli behaves identically — this is preprocessing geometry, not the projector.

Works, and is verified for spatial correctness rather than plausible-sounding output: a four-quadrant colour image is scored on whether each colour lands in the right corner. A misapplied attention mask names colours confidently but places them wrongly, so this test distinguishes a working port from a fluent-but-broken one. 3/3.

Requires -fa off — ggml_flash_attn_ext aborts on Muse's per-layer sparse-window masks.

Quantization methodology

# 1. convert BF16 safetensors -> GGUF (upstream tree; only it has the muse-glimmer converter)
python convert_hf_to_gguf.py <MODEL_DIR> --outtype bf16 --outfile muse-glimmer-30B-BF16.gguf

# 2. quantize with the ROCmFPX build (only it has ggml types 100-106)
llama-quantize muse-glimmer-30B-BF16.gguf muse-glimmer-30B-ROCmFP4-FAST.gguf 103

Source: meta-models/Muse-Glimmer-30B BF16 safetensors, 1436 tensors → 55.7 GB BF16 GGUF (731 text tensors) → ROCmFP4.

The model was ported forward into the ROCmFPX base in three stages:

1. Text graph, arch registration and converter. Three API gaps bridged: is_swa_impl → swa_layers, n_layer() from method to field, and the NVFP4-only output-scale argument (null on the ROCmFP4 path).
2. Vision tower — required teaching the older base's build_vit to accept per-layer attention masks at all; it previously took no mask parameter. Added as an overload so the ~32 other vision models calling it are untouched.
3. Chat-format parser, so harmony-style channel output is parsed rather than leaking to=self<|message|> into content.

Files

Ten files, three distinct networks. llama.cpp loads them via --model, --model-draft and --mmproj — there is no merged single-file format.

Model weights — all six quants live in this repo

Drafter, projector and patches

Total 117.12 GiB. Download a single quant rather than the whole repo:

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF \
  --include "muse-glimmer-30B-ROCmFP4-FAST.gguf" --local-dir muse

⚠️ hf download silently ignores --include when given more than one pattern — issue one call per file.

Not yet measured

Listed explicitly so nobody mistakes absence for a pass. These are genuine gaps, not claims:

Measurement conditions: the tok/s figures were taken on a machine that also served other traffic during the run. The ordering across builds is wide enough to be reliable; the exact ratios are not trustworthy to three significant figures. A re-run on a quiesced box is planned.

Quality caveat: the 3/3 figure is a smoke check over factual recall, arithmetic and instruction-following, scored by substring match. It is a regression guard against a broken quantisation, not a benchmark suite, and it does not measure answer depth. No claim of "no quality loss" is made — that would require the perplexity and standardized evaluations listed above.

Independent results

None yet. If you run this build, please open a discussion with: hardware, GPU, OS, ROCm version, runtime commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent reproductions will be listed separately from author benchmarks and carry more weight.

Known issues

1. Vulkan/CUDA/CPU cannot load these files — ROCmFP4 is a ROCm-only tensor format.
2. Vision requires -fa off, costing ~21% on text at 32K context.
3. Parallel tool calls fail — model-level, reproduced identically on Meta's own weights.
4. Small max_tokens returns empty content — the budget goes to reasoning_content. Allow several hundred tokens.
5. --reasoning-budget is not enforced on this model; use reasoning_strength.

License and attribution

Base model, DFlash drafter and vision projector are Meta's, under the base model's licence. ROCmFP4 quantisation types are from the ROCmFPX fork of llama.cpp. This repository contains the quantised weights and the measurements above.

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Base model: meta-models/Muse-Glimmer-30B. Generated from Hub metadata; download counts move over time.

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

🩹 The muse-glimmer architecture port — complete, three stages

patches/muse-glimmer-complete.patch — 81,968 bytes, 20 files, git apply --check clean against charlie12345/ROCmFPX.

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git apply --check ../patches/muse-glimmer-complete.patch && \
git apply           ../patches/muse-glimmer-complete.patch
cmake -S . -B build-muse -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
      -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build-muse --target llama-server --target llama-quantize \
      --target llama-mtmd-cli -j 6

muse-glimmer is not an upstream architecture. Running it end-to-end takes three independent pieces of work, and all three are in this patch.

Stage 1 — text graph and conversion

⚠️ convert_hf_to_gguf.py does not know MuseGlimmer. The converter is conversion/muse_glimmer.py, driven through conversion.get_model_class.

Stage 2 — vision projector (clip / mtmd)

This stage is what makes --mmproj work. This repo ships mmproj-kquant.gguf (1.30 GiB); a BF16 projector built through this stage — mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors, clip.projector_type = muse-glimmer, merge 2, patch 14, image_size 896 — is published alongside the 8-bit builds in Muse-Glimmer-30B-ROCmFPX-Q8_0-GGUF.

Stage 3 — chat parser

common/chat.cpp — common_chat_params_init_muse_glimmer, PEG_NATIVE.

Without it the model's to=self<|message|> control sequence is emitted into content. With it, content is clean: "The capital of Japan is Tokyo."

Build note

cmake's source GLOB is configure-time. After the patch adds src/models/muse-glimmer.cpp you must re-run the cmake -S . -B build-muse configure step, not just --build. -DLLAMA_BUILD_WEBUI=OFF avoids a node/npm requirement.

Applying to a different base commit

The patch header names commit 3edc3d3, and it applies cleanly to later revisions (verified on b41ce12). On trees where cohere2moe and bailing_hybrid model sources are absent, their factory cases in llama-model.cpp reference symbols that do not exist in that tree — build those two out, or apply on 3edc3d3 where their .cpp files are present. The muse-glimmer factory case and graph are independent of both.