Method and full tables: docs/BENCHMARKS.md.

sparkrun run returns before the engine is ready; wait for health, then check that a reply has both reasoning and an answer:

H=http://<head>:8000
until [ "$(curl -s -o /dev/null -w '%{http_code}' $H/health)" = 200 ]; do sleep 10; done
curl -s $H/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model":"qwen3.8-flash-next","max_tokens":1024,
  "messages":[{"role":"user","content":"Write a Python function that reverses a string."}]}' \
| python3 -c "import json,sys; m=json.load(sys.stdin)['choices'][0]['message']
print('reasoning:', len(m.get('reasoning') or m.get('reasoning_content') or ''), 'chars')
print('content  :', (m.get('content') or '')[:300])"

Empty content with long reasoning means max_tokens ran out inside thinking; garbled text means a checkpoint mismatch (see Troubleshooting under Details below). Optional long-context check (stdlib Python), expect exact 20 at both depths:

git clone https://github.com/ursuciprian/qwen3.8-flash-next-dgx-spark-tp-2 && cd qwen3.8-flash-next-dgx-spark-tp-2
python3 scripts/fidelity_probe.py --base $H --model qwen3.8-flash-next --depths 8000,32000

The server binds 0.0.0.0 with no API key: keep it on a trusted network or put a proxy with auth in front. Stop with sparkrun stop --all. To upgrade, run sparkrun registry update qwen38-flashnext first: sparkrun caches registries and does not refresh them on run.

This repo still has copies of the single-Spark recipes qwen3.8-flash-next-1x-dgx-spark and -previous, so existing setups keep working; new one-Spark releases ship only in the one-Spark repo. With both registries added, use @qwen38-flashnext-1x/qwen3.8-flash-next-1x-dgx-spark.

<sub>Left: llama-benchy task mode (agent coding turn), ISL/OSL 2048/512, T=1.0, top-p 0.95, top-k 20, thinking on, prefix caching. Aggregate = all output tokens / wall time, PP included; per request = TG rate of one stream after its first token.
1x Spark: release v2.1.0, 2026-10-08, llama-benchy task mode, mean ± sd over runs, one boot
2x Spark TP=2: release v2.0.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271, mean ± sd of 3 runs, one boot
2x Spark DP=2: release v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271, mean ± sd of 3 runs, one boot
Versions are numbered per setup. Data: docs/data/capability.csv, with the source file of every point.

Right: c1, cold prefix (no cache hit). Each point: mean of 1 to 4 samples of one run (n per point in the CSV).
1x Spark: release v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271
2x Spark TP=2: release v2.2.0, 2026-10-15, cold.py (k101); v1.4.0, 2026-10-07, fidelity_probe.py
1x Spark: not measured above 128K yet.
Data: docs/data/capability.csv, with the source file of every point.</sub>

<sub>Left: c1, cold prefix (no cache hit). Each point: mean of 1 to 4 samples of one run (n per point in the CSV).
1x Spark: release v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271
2x Spark TP=2: release v2.2.0, 2026-10-15, cold.py (k101)
Data: docs/data/capability.csv, with the source file of every point.

Right: 2x Spark TP=2: release v2.0.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271, mean ± sd of 3 runs, one boot (older release; shipped v2.2.0 not measured on this test yet). Server default sampling. Each point: ISL/OSL 2048/512 on top of the cached context, same harness as the concurrency chart. Concurrency levels measured at one depth only are left out.
Data: docs/data/capability.csv, with the source file of every point.</sub>

_{1x Spark v2.1.0, 2x Spark TP=2 v2.0.0, 2x Spark DP=2 v2.1.0; llama-benchy task mode.}

Release v2.0.0. llama-benchy task mode, ISL/OSL 2048/512, T=1.0, thinking on; mean ± sd, where sd is between runs or between boots as the CSV stat column says for each run.

c1, cold prefix.

_{TG, coding c1 (median, n=36, T=0): 1x v2.1.0, 2026-10-10, coding_probe.py, up to 768 tokens out; TP=2 v2.0.0, 2026-10-10, coding_probe.py, up to 768 tokens out; not measured: 2x Spark DP=2.
TG c1, defaults: 1x v2.2.0, 2026-10-10, llama-benchy task mode; TP=2 v2.0.0, 2026-10-10, llama-benchy task mode; DP=2 v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271.
TG, aggregate c8, defaults: 1x v2.2.0, 2026-10-10, llama-benchy task mode; TP=2 v2.0.0, 2026-10-10, llama-benchy task mode; DP=2 v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271.
TTFT, ISL 16K, cold prefix: 1x v2.1.0, 2026-10-10, llama-benchy 0.4.1.dev4+g0d4de4271; TP=2 v2.2.0, 2026-10-15, cold.py (k101); not measured: 2x Spark DP=2.
KV cache: concurrent 262K-token requests: 1x v2.1.0, 2026-10-08, serve log; TP=2 v2.0.0, 2026-10-09, serve log; DP=2 v2.1.0, 2026-10-08, serve log.
TG c1/c8: llama-benchy task mode, ISL/OSL 2048/512, server defaults. KV row: vLLM's 'Maximum concurrency for 262,144 tokens per request' at boot (DP=2: two replicas, one pool each).
Data: docs/data/capability.csv, with the source file of every point.}

_{All sessions start at once; each turn resends the full conversation (prefix cache on), tools on, T=0.6, thinking off.
2x Spark TP=2: two-Spark v1.5.0 (old name b1.6), 2026-10-08, agent replay (drive.py), one boot
2x Spark DP=2: DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-08, agent replay (drive.py), one boot
Data: docs/data/capability.csv, with the source file of every point.}

_{TG: llama-benchy task mode, ISL/OSL 2048/512, T=1.0, thinking on, from each release's own first run (usually its promotion A/B; the runs line below names each), unpaired across releases, so day-to-day drift is included; paired A/B per release in VERSIONS.md.
Shaded (b0 to v1.4.0): chat template default reasoning_effort xhigh; later releases at the recipe default, medium. Different reasoning lengths, so not like-for-like across the shade.
TG runs: b0 2026-09-23 (llama-benchy task mode, mean ± sd over runs, one boot); v1.0.0 2026-09-25, v1.1.0 2026-09-26, v1.2.0 2026-09-27, v1.3.0 2026-09-29 (llama-benchy task mode, mean of 2 candidate boots x 3 runs; sd not recorded); v1.4.0 2026-10-01 (llama-benchy task mode, mean of 2 boots x 3 runs, sd between boots); v1.5.0 2026-10-08, v2.0.0 2026-10-09 (llama-benchy task mode, mean of 2 boots x 4 runs, sd between boots).
Hardmode: promotion gate of each release (b0: gate on the pinned checkpoint), b0 2026-09-23, v1.0.0 2026-09-25, v1.1.0 2026-09-26, v1.2.0 2026-09-27, v1.3.0 2026-09-29, v1.4.0 2026-10-01, v1.5.0 2026-10-08, v2.0.0 2026-10-09, v2.1.0 2026-10-12, v2.2.0 2026-10-16.
Data: docs/data/capability.csv, with the source file of every point.}

Every number and where it comes from

The shipped releases were measured as a full grid in k76 on 2026-10-10 (#128); where a cell has no k76 value, the table shows the newest release that has one. TG rows are aggregate unless marked "per request"; sampling is the server default (temperature 1.0, thinking on) unless the run says otherwise. The charts and tables are written by uv run scripts/make_charts.py from docs/data/capability.csv, where every point lists its raw file. Full grids for every build: docs/BENCHMARKS.md.

Releases and runs behind the numbers:

• ^a one-Spark v2.2.0, 2026-10-10, k86-boot-noise-power, llama-benchy task mode (files)
• ^b two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, llama-benchy 0.4.1.dev4+g0d4de4271 (files)
• ^c DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llama-benchy 0.4.1.dev4+g0d4de4271 (files)
• ^d one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llama-benchy 0.4.1.dev4+g0d4de4271 (files)
• ^e one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, coding_probe.py, up to 768 tokens out (files)
• ^f two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, coding_probe.py, up to 768 tokens out (files)
• ^g one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, copy-heavy benchmark, 1,500 tokens out (files)
• ^h two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, copy-heavy benchmark, 1,500 tokens out (files)
• ⁱ two-Spark v2.2.0, 2026-10-15, promotion A/B, llama-benchy task mode (files)
• ^j two-Spark v2.2.0, 2026-10-15, promotion A/B, cold.py (k101) (files)
• ^k one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llm-inference-bench 0.7.6 (files)
• ^l two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, llm-inference-bench 0.7.6 (files)
• ^m DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, llm-inference-bench 0.7.6 (files)
• ⁿ one-Spark v2.0.0 (old name v3d), 2026-10-05, depth and PP sweep, llm-inference-bench 0.7.6 (files)
• ^o two-Spark v1.4.0 (old name b1.4), 2026-10-05, depth and PP sweep, llm-inference-bench 0.7.6 (files)
• ^p one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, vLLM serve log (files)
• ^q two-Spark v2.0.0, 2026-10-10, k76-capability-matrix, vLLM serve log (files)
• ^r DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-10, k76-capability-matrix, vLLM serve log (files)
• ^s one-Spark v2.1.0 (old name v3e), 2026-10-08, serve log (files)
• ^t two-Spark v2.0.0, 2026-10-09, serve log (files)
• ^u DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-08, serve log (files)
• ^v one-Spark v1.3.0 (old name v3c), 2026-10-05, kv-capacity page accounting, same 14 GiB pool in 1× v2.0.0 and v2.1.0 (files)
• ^w one-Spark v2.3.0, 2026-10-13, promotion gate (1x repo)
• ^x two-Spark v2.2.0, 2026-10-16, promotion gate (files)
• ^y DP=2 on one-Spark v2.1.0 (old name v3e), 2026-10-08, quality gate through the router (files)

Checkpoint: ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE @ 16c9bd54 (v2.0.0): local-inference-lab/Qwen3.8-Flash-Next-NVFP4 @ 7c4f1bc1 with the GDN projections requantized to weight-only NVFP4 and the retrained drafter D1 (NVFP4 experts; MXFP8 dense and attention; 98 GiB). PP-sized GDN calls (41+ rows) use the MXFP8 GDN weights of 7c4f1bc1, read from its shard 35, which the recipe downloads on the first boot. Image and video input are not tested.

• By vLLM's count the KV pool fits about 13.5 requests at 262,144 tokens, under the 16-request cap. Four different ~256K contexts at once used 28% of the pool; more than four at once have not been run (long contexts at once).
• v2.0.0 reads the MXFP8 PP weights from shard 35 of the 7c4f1bc1 snapshot in the HF cache by its full path. Deleting the HF cache means downloading that shard (2.8 GB) again on the next boot, which the recipe does by itself.
• The v2.0.0 A/B measured c1, c4 and c8 against v1.5.0; c16 was not part of the A/B. k76 measured the shipped v2.0.0 at c16: 231.3 ± 4.9 tok/s aggregate (v1.4.0 A/B: 242.8). At c9-c16 the GDN layers run the same MXFP8 weights as v1.5.0.
• c1 TG varies between boots (earlier builds showed two levels, ~95 to 100 and ~85 to 88 tok/s on counting).
• Hardmode still fails a few multi-step scenarios (e.g. TC-30, TC-68, TC-74, TC-88) on every release.
• Above 16 requests: a max_num_seqs 32 run (quality gate not run at that cap) reached 285.5 tok/s aggregate TG at c32 (llama-benchy task mode, mean of 3 runs, v1.4.0; high concurrency).

Rolling back to -previous (v2.1.0) needs no download: it serves the same checkpoint (ursuciprian/Qwen3.8-Flash-Next-NVFP4-GDN-MSE @ 03f4a057 plus shard 35 of 7c4f1bc1) on the v2.1.0 image. v2.2.0 keeps its torch compile cache in a root of its own (VLLM_CACHE_ROOT), so switching does not recompile: in the v2.2.0 check the v2.2.0 boot loaded its cache from its own root, and the v2.1.0 boot after it left the shared root unchanged. The v2.0.0 recipe is archived in archive/recipes/. Renames: recipes/RENAMES.md.

Releases use semantic versions per repo; VERSIONS.md lists every shipped release with its old build name, date, what changed and its manifest (image digest, vLLM and b12x commits, checkpoint, drafter, plan seed). Each release's own A/B against the one before it is in the release history and in docs/BENCHMARKS.md. The single-Spark releases have their own list in the one-Spark repo.

• Candidate builds are screened with Thunderdome: control and candidate booted alternately (control, candidate, candidate, control), T=0 TG probes at c1/c4/c8 and at 16K context, plus llama-benchy at temperature 1.0. Noise band = the control's boot-to-boot spread, at least 1%.
• To be promoted, a build must pass the gate, be faster beyond noise in at least one cell, and be slower beyond noise in no cell at 1 to 4 requests; a loss at 5 to 16 requests is published as a caveat (scripts/arm_verdict.py).
• MTP acceptance per draft position is compared cell by cell as a numerics canary.
• Copy-heavy and counting workloads accept nearly every draft and show the TG ceiling of MTP; the coding grid (llama-benchy task mode, thinking on) is the rate an agent sees. Every table states its workload.

Index of every run and verdict: results/README.md.

• TP=2 over RoCE, one rank per GB10; all-reduces over the CX-7 link take ~4% of a c1 TG step.
• b12x kernels cover NVFP4 MoE, MXFP8 linears, GDN (36 layers) and QSA sparse attention (12 layers), with an autotuned plan cache baked into each image.
• MTP ×4 uses probabilistic drafts over a 131k-id draft vocabulary; rejection sampling keeps the output distribution unchanged.
• The 1-request step (~43 ms) splits into MoE 30%, dense MXFP8 31%, MTP draft + head 19%, idle 6%, all-reduce 4%, GDN 3%. MoE reads ~175 GB/s of the ~250 GB/s the GB10 reaches.

Flags, environment variables and the reason for each: docs/REFERENCE.md. Kernel and engine notes, rejected experiments: docs/ENGINEERING.md.

The recipes set the server default to reasoning_effort: medium. On my DevOps task set, two-Spark v1.2.0 (old name b1.2) at the template default xhigh passed 42.5% of checks with 23/42 runaway-thinking runs and a 246 s median per task; v1.4.0 (old name b1.4) at medium passed 95.9% with 0/42 runaway and 33 s. A request can still ask for xhigh or low; "reasoning_effort": "none" turns thinking off. For hard reasoning problems xhigh per request helps: on the hotel-lights check, the one-Spark v3c (old name) went from 21/32 at medium to 31/31 completed runs (#88); allow a long client timeout, since most of those answers took over 30 minutes at 8 concurrent requests. Override table: docs/REFERENCE.md.