Method and full tables: docs/BENCHMARKS.md.

The recipe serves its own checkpoint, about 98 GB. A fresh download took about 3 h 15 min on my link (~8.6 MB/s). Download it on the Spark before the first boot, together with the two files of the base checkpoint that the recipe uses for PP (2.6 GiB):

hf download ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE --revision 03f4a0570496bbe741189f4a2a7d611679e85406
hf download local-inference-lab/Qwen3.8-Flash-Next-NVFP4 --revision 7c4f1bc1a2d6847e0cbc01ac6b823f00251de8dd \
  model-00035-of-00036.safetensors model.safetensors.index.json

sparkrun run returns before the engine is ready. Wait for health, then check that the reply has both reasoning and an answer:

H=http://<spark>:8000
until [ "$(curl -s -o /dev/null -w '%{http_code}' $H/health)" = 200 ]; do sleep 10; done
curl -s $H/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model":"qwen3.8-flash-next","max_tokens":1024,
  "messages":[{"role":"user","content":"Write a Python function that reverses a string."}]}' \
| python3 -c "import json,sys; m=json.load(sys.stdin)['choices'][0]['message']
print('reasoning:', len(m.get('reasoning') or m.get('reasoning_content') or ''), 'chars')
print('content  :', (m.get('content') or '')[:300])"

Empty content with long reasoning means max_tokens ran out inside thinking; garbled text means a checkpoint mismatch (see Troubleshooting under Details below). Optional long-context check (stdlib Python), expect exact 20 at both depths:

git clone https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark && cd qwen3.8-flash-next-1x-dgx-spark
python3 scripts/fidelity_probe.py --base $H --model qwen3.8-flash-next --depths 8000,32000

The server binds 0.0.0.0 with no API key: keep it on a trusted network or put a proxy with auth in front. Stop with sparkrun stop --all. To upgrade, run sparkrun registry update qwen38-flashnext-1x first: sparkrun caches registries and does not refresh them on run.

Moved here on 2026-10-08 from qwen3.8-flash-next-dgx-spark-tp-2, which keeps a copy of this recipe so existing sparkrun run setups keep working. If both registries are added, pick this one with sparkrun run @qwen38-flashnext-1x/qwen3.8-flash-next-1x-dgx-spark --hosts <spark> --solo.

<sub>Left: llama-benchy task mode (agent coding turn), ISL/OSL 2048/512, T=1.0, top-p 0.95, top-k 20, thinking on, prefix caching. Aggregate = all output tokens / wall time, PP included; per request = TG rate of one stream after its first token.
1x Spark: release v2.1.0, 2026-10-08, llama-benchy task mode, mean ± sd over runs, one boot
2x Spark TP=2: release v2.0.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271, mean ± sd of 3 runs, one boot
2x Spark DP=2: release v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271, mean ± sd of 3 runs, one boot
Versions are numbered per setup. Data: docs/data/capability.csv, with the source file of every point.

Right: c1, cold prefix (no cache hit). Each point: mean of 1 to 4 samples of one run (n per point in the CSV).
1x Spark: release v2.4.0, 2026-10-16, cold.py (k101)
2x Spark TP=2: release v2.0.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271; v1.4.0, 2026-10-07, fidelity_probe.py
1x Spark: not measured above 128K yet.
Data: docs/data/capability.csv, with the source file of every point.</sub>

<sub>Left: c1, cold prefix (no cache hit). Each point: mean of 1 to 4 samples of one run (n per point in the CSV).
1x Spark: release v2.4.0, 2026-10-16, cold.py (k101)
2x Spark TP=2: release v2.0.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271
Data: docs/data/capability.csv, with the source file of every point.

Right: 1x Spark: release v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271, mean ± sd of 2 Sparks x 3 runs, one boot each (older release; shipped v2.4.0 not measured on this test yet). Server default sampling. Each point: ISL/OSL 2048/512 on top of the cached context, same harness as the concurrency chart. Concurrency levels measured at one depth only are left out.
Data: docs/data/capability.csv, with the source file of every point.</sub>

_{1x Spark v2.1.0, 2x Spark TP=2 v2.0.0, 2x Spark DP=2 v2.1.0; llama-benchy task mode.}

Release v2.2.0. llama-benchy task mode, ISL/OSL 2048/512, T=1.0, thinking on; mean ± sd, where sd is between runs or between boots as the CSV stat column says for each run.

_{TG, coding c1 (median, n=36, T=0): 1x v2.1.0, 2026-10-10, coding_probe.py, up to 768 tokens out; TP=2 v2.0.0, 2026-10-10, coding_probe.py, up to 768 tokens out; not measured: 2x Spark DP=2.
TG c1, defaults: 1x v2.2.0, 2026-10-10, llama-benchy task mode; TP=2 v2.0.0, 2026-10-10, llama-benchy task mode; DP=2 v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271.
TG, aggregate c8, defaults: 1x v2.2.0, 2026-10-10, llama-benchy task mode; TP=2 v2.0.0, 2026-10-10, llama-benchy task mode; DP=2 v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271.
TTFT, ISL 16K, cold prefix: 1x v2.4.0, 2026-10-16, cold.py (k101); TP=2 v2.0.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271; not measured: 2x Spark DP=2.
KV cache: concurrent 262K-token requests: 1x v2.1.0, 2026-10-08, serve log; TP=2 v2.0.0, 2026-10-09, serve log; DP=2 v2.1.0, 2026-10-08, serve log.
TG c1/c8: llama-benchy task mode, ISL/OSL 2048/512, server defaults. KV row: vLLM's 'Maximum concurrency for 262,144 tokens per request' at boot (DP=2: two replicas, one pool each).
Data: docs/data/capability.csv, with the source file of every point.}

_{All sessions start at once; each turn resends the full conversation (prefix cache on), tools on, T=0.6, thinking off.
2x Spark TP=2: two-Spark v1.5.0 (old name b1.6), 2026-10-08, agent replay (drive.py), one boot
2x Spark DP=2: DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-08, agent replay (drive.py), one boot
Data: docs/data/capability.csv, with the source file of every point.}

_{TG: llama-benchy task mode, ISL/OSL 2048/512, T=1.0, thinking on, from each release's own first run (usually its promotion A/B; the runs line below names each), unpaired across releases, so day-to-day drift is included; paired A/B per release in VERSIONS.md.
TG runs: v1.0.0 2026-10-02, v1.1.0 2026-10-04, v1.2.0 2026-10-04, v1.3.0 2026-10-05, v2.0.0 2026-10-05 (llama-benchy task mode, mean ± sd over 3 runs, one boot); v2.1.0 2026-10-08 (llama-benchy task mode, mean ± sd over runs, one boot); v2.2.0 2026-10-10 (llama-benchy task mode, mean of 6 boots, sd between boots).
Hardmode: promotion gate of each release (b0: gate on the pinned checkpoint), v1.1.0 2026-10-04, v1.2.0 2026-10-04, v1.3.0 2026-10-05, v2.0.0 2026-10-05, v2.1.0 2026-10-07, v2.2.0 2026-10-09, v2.3.0 2026-10-13, v2.4.0 2026-10-16; v1.0.0: score not in the data.
Data: docs/data/capability.csv, with the source file of every point.}

Every number and where it comes from

The shipped releases were measured as a full grid in k76 on 2026-10-10 (tp-2 #128); where a cell has no k76 value, the table shows the newest release that has one. TG rows are aggregate unless marked "per request"; sampling is the server default (temperature 1.0, thinking on) unless the run says otherwise. The charts and tables are written by uv run scripts/make_charts.py from docs/data/capability.csv, where every point lists its raw file (1x: paths are in this repo, 2x: paths in the two-Spark repo). Full grids for every build: docs/BENCHMARKS.md.

Releases and runs behind the numbers:

• ^a one-Spark v2.2.0, 2026-10-10, k86-boot-noise-power, llama-benchy task mode (files)
• ^b two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, llama-benchy 0.4.1.dev4+g0d4de4271 (files)
• ^c DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llama-benchy 0.4.1.dev4+g0d4de4271 (files)
• ^d one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llama-benchy 0.4.1.dev4+g0d4de4271 (files)
• ^e one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, coding_probe.py, up to 768 tokens out (files)
• ^f two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, coding_probe.py, up to 768 tokens out (files)
• ^g one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, copy-heavy benchmark, 1,500 tokens out (files)
• ^h two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, copy-heavy benchmark, 1,500 tokens out (files)
• ⁱ one-Spark v2.4.0, 2026-10-16, promotion A/B, llama-benchy task mode (files)
• ^j one-Spark v2.4.0, 2026-10-16, promotion A/B, cold.py (k101) (files)
• ^k one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llm-inference-bench 0.7.6 (files)
• ^l two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, llm-inference-bench 0.7.6 (files)
• ^m DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llm-inference-bench 0.7.6 (files)
• ⁿ one-Spark v2.0.0 (old name v3d), 2026-10-05, depth and PP sweep, llm-inference-bench 0.7.6 (files)
• ^o two-Spark v1.4.0 (old name b1.4), 2026-10-05, depth and PP sweep, llm-inference-bench 0.7.6 (files)
• ^p one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, vLLM serve log (files)
• ^q two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, vLLM serve log (files)
• ^r DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, vLLM serve log (files)
• ^s one-Spark v2.1.0 (old name v3e), 2026-10-08, serve log (files)
• ^t two-Spark v2.0.0, 2026-10-09, serve log (files)
• ^u DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-08, serve log (files)
• ^v one-Spark v1.3.0 (old name v3c), 2026-10-05, kv-capacity page accounting, same 14 GiB pool in 1× v2.0.0 and v2.1.0 (files)
• ^w one-Spark v2.4.0, 2026-10-16, promotion gate (files)
• ^x two-Spark v2.0.0, 2026-10-09, promotion gate (files)
• ^y DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-08, quality gate through the router (files)

Checkpoint: ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE @ 03f4a057 (v2.2.0, drafter D3; v2.1.0: 16c9bd54, drafter D1). It is the NVFP4 checkpoint local-inference-lab/Qwen3.8-Flash-Next-NVFP4 @ 7c4f1bc1 with the GDN projections in weight-only NVFP4 and, since 16c9bd54, a retrained MTP drafter. Coming from 16c9bd54, only model-00034-of-00036.safetensors (4.5 GB) is new. Its model card covers what changed, how it was built and the license (Qwen Community License 1.0). The recipe also fetches shard 35 and the index of 7c4f1bc1 (2.6 GiB) for the MXFP8 PP copy. Image and video input are not tested.

• The 14 GiB KV pool holds ~7.5 requests at 128K, so 8 concurrent 128K requests can wait for KV space. 8 × 64K fits.
• On a first boot without a usable plan seed, it autotunes and compiles every kernel: ~30 min, with host MemAvailable down to 3.8 GiB for about a minute (earlyoom triggers at ~2.4 GiB). The images ship the plan seed and compile cache, so a normal first boot skips this. Close other memory-heavy work during the first boot.
• In steady state there is ~14 GiB MemAvailable (lowest 13.3 GiB in the benchmark run), most of it PLE page cache.
• The b12x plan seed and the compile cache are keyed by the model's snapshot path, so they match only the pinned checkpoint revision. Serving another revision or a local copy of the files autotunes and compiles on its first boot.
• The recipe and its rollback serve the same checkpoint revision but keep one vLLM compile cache root each (VLLM_CACHE_ROOT): their torch AOT keys are the same and the artifacts are not. The first boot of each copies the AOT seed from its own image; the b12x plan cache is shared.
• The image's Triton kernel cache was built on one Spark. On another host the first boot recompiles a few Triton kernels (~2 min once: 277 s instead of 147 s) and logs W-level TritonBundler tracebacks ("Cubin file saved by TritonBundler not found"); serving is not affected (#39).
• Hardmode still fails a few multi-step scenarios (e.g. TC-30, TC-68, TC-74, TC-88) on every release.
• Above 8 requests: a max_num_seqs 32 run (quality gate not run at that cap, v2.0.0) reached 189.3 tok/s aggregate TG at c32 (llama-benchy task mode, mean of 3 runs); a boot at max_num_seqs 16 was stopped by the 4 GiB memory guard (high concurrency).

Each recipe's header lists every change since the first release with its measured delta. Earlier recipes: archive/recipes/. Renames: recipes/RENAMES.md.

Releases use semantic versions per repo; VERSIONS.md lists every shipped release with its old build name, date, what changed, its measured delta and its manifest (image digest, vLLM and b12x commits, checkpoint, drafter, plan seed). Each release's own A/B against the one before it is in the release history and in docs/BENCHMARKS.md. Arms that were screened and not promoted: results/README.md.

• Candidate builds are screened with Thunderdome: one arm per Spark, control and candidate booted alternately, 2 passes, T=0 TG probes at c1/c4/c8 and at 16K context, plus llama-benchy at temperature 1.0. Noise band = the control's boot-to-boot spread, at least 1%.
• To be promoted, a build must pass the gate, be faster beyond noise in at least one coding or counting cell, and be slower beyond noise in no cell at 1–4 requests (scripts/arm_verdict.py).
• MTP acceptance per draft position is compared cell by cell as a numerics canary; logprob agreement against the previous build must sit within self-noise.
• Copy-heavy and counting workloads accept nearly every draft and show the TG ceiling of MTP; the coding grid (llama-benchy task mode, thinking on) is the rate an agent sees. Every table states its workload.

Index of every run and verdict: results/README.md. The issue history up to 2026-10-08 is in the tp-2 repo; open single-Spark issues are here.

• The 26.8 GiB PLE n-gram table is read through the page cache from the checkpoint files (VLLM_PLE_MMAP=1), with a WILLNEED pass before each TG gather and a 50 ms NVMe keepalive, so ~72 GiB of other weights fit on one GB10.
• One GDN PP staging buffer shared by all 36 GDN layers frees ~8.8 GiB, which goes to a 14 GiB KV pool, and compact MTP draft records cut each request's GDN state blocks from 185 to 37.
• The GDN projections run TG from weight-only NVFP4 and switch to an MXFP8 copy for calls of 41+ rows.
• b12x kernels cover NVFP4 MoE, MXFP8 linears, GDN (36 layers) and QSA sparse attention (12 layers), with an autotuned plan cache and the torch compile cache baked into each image (docker/b0-warm/).
• MTP ×4 uses probabilistic drafts over a 131k-id draft vocabulary; rejection sampling keeps the output distribution unchanged. The drafter was retrained on the served model's own outputs: D1 in v2.1.0, D3 (more data, a longer run) in v2.2.0 (tools/mtp_refit/).

Every flag and environment variable, with the reason for it, is in the recipe header. Engine-wide configuration, image provenance and rejected experiments shared with the two-Spark recipe: REFERENCE.md and ENGINEERING.md in the tp-2 repo.

The recipe sets the server default to reasoning_effort: medium. On my DevOps task set (two Sparks), b1.2 at the template default xhigh passed 42.5% of checks with 23/42 runaway-thinking runs and a 246 s median per task; b1.4 at medium passed 95.9% with 0/42 runaway and 33 s. A request can still ask for xhigh or low; "reasoning_effort": "none" turns thinking off. For hard reasoning problems xhigh per request helps: on the hotel-lights check, v3c (old name) went from 21/32 at medium to 31/31 completed runs; allow a long client timeout, since most of those answers took over 30 minutes at 8 concurrent requests. Override table: REFERENCE.md.