This is local-inference-lab/Qwen3.8-Flash-Next-NVFP4
(revision 7c4f1bc1) with two changes:
16c9bd54 carries the first retrained drafter D1, and 244cb6fe keeps the
original drafter.Everything else is byte-identical to that checkpoint, including the routed and shared experts, attention, PLE tables, the MTP experts, vision encoder, tokenizer and chat template.
It is the served checkpoint of the single-DGX-Spark recipe qwen3.8-flash-next-1x-dgx-spark in
ursuciprian/qwen3.8-flash-next-1x-dgx-spark:
release v2.2.0 serves this revision (drafter D3), and the -previous recipe serves v2.1.0 (old name v3e) at
16c9bd54 (drafter D1).
| Tensors | Base checkpoint | This checkpoint |
|---|---|---|
linear_attn.in_proj_qkv, in_proj_z, out_proj in all 36 GDN layers (108 tensors) | MXFP8 (frozen during distillation) | NVFP4 W4A16: E2M1 values, E4M3 scale per 16, FP32 global scale |
24 dense BF16 tensors of the MTP drafter (mtp.*: fc, norms, hyper-connection mixers, attention, shared expert, router gate) | original | retrained (D3 since revision 3, D1 in revision 2) |
| Everything else | unchanged | unchanged |
How the GDN weights were built:
de4b8e4d).in_proj_qkv and in_proj_z of a layer share one global scale, because vLLM fuses them into one linear.Only model-00035-of-00036.safetensors, config.json, hf_quant_config.json and model.safetensors.index.json
differ from the base revision for the GDN change, and model-00034-of-00036.safetensors for the drafter.
During decode the GDN projections run at 1 to 40 rows per step. At those sizes a W4A16 NVFP4 GEMM reads about half the weight bytes of MXFP8, so each decode step is faster.
For prefill-sized calls the v3d recipe keeps an MXFP8 copy of the same weights. It loads that copy from the base
checkpoint, model-00035-of-00036.safetensors at revision 7c4f1bc1. Calls of 41 or more rows use the copy and
smaller calls use NVFP4 (VLLM_B12X_NVFP4_MXFP8_MIN_TOKENS=41), so prefill speed does not change. The copy uses
about 1.1 GiB more memory.
Paired screen, with control and candidate booted alternately on the same Spark, at temperature 0. Every cell not listed was within its noise band.
| Cell | Change | Noise band |
|---|---|---|
| decode, 8 fresh requests | +4.4% tok/s | 1.9% |
| decode at 16K context, 4 requests | +5.9% tok/s | 3.0% |
| counting task, 8 requests | +7.8% tok/s | 2.1% |
| tg512, 1 request (llama-benchy) | 50.0 -> 59.7 tok/s | 4.3% |
| pp2048, 1 request (llama-benchy) | -0.9% | 1.0% |
MTP acceptance per draft position moved by at most 0.005.
Quality gate on the v3d recipe:
D3 continues from D1 (below) with more data and a longer run. Only model-00034-of-00036.safetensors and this card
differ from revision 16c9bd54; inside that shard only the retrained mtp.* tensors changed.
Offline acceptance per draft position on the held-out set, temperature 0, D1 to D3: 0.905 to 0.911, 0.797 to 0.810, 0.699 to 0.718, 0.614 to 0.640, 0.542 to 0.574, 0.480 to 0.517 (+2.2 points per position on average). At temperature 1: 0.876 to 0.885, 0.713 to 0.733, 0.551 to 0.581, 0.409 to 0.446, 0.294 to 0.334, 0.206 to 0.245. Every prompt category improved.
Live screen on two DGX Sparks, D3 against D1 on the same image: one arm per Spark, control and candidate booted alternately, 6 boots per side per Spark over two runs (the second run repeated only the cells the first left in doubt), temperature 0 for the probes. Acceptance per draft position, D1 to D3:
| Spark | pos 1 | pos 2 | pos 3 | pos 4 |
|---|---|---|---|---|
| dgx-01 | 0.849 to 0.860 | 0.699 to 0.720 | 0.575 to 0.598 | 0.472 to 0.498 |
| dgx-02 | 0.852 to 0.857 | 0.703 to 0.717 | 0.578 to 0.596 | 0.474 to 0.493 |
Speed, D3 vs D1 (noise band in brackets). No cell was worse beyond its noise band on either Spark:
| Cell | dgx-01 | dgx-02 |
|---|---|---|
| decode, 1 fresh request | +1.5% (3.3) | +1.6% (8.3) |
| decode, 4 fresh requests | +3.4% (2.4) | +1.9% (4.8) |
| decode, 8 fresh requests | +3.3% (1.0) | +1.3% (2.8) |
| decode at 16K context, 4 requests | +4.6% (2.5) | +2.0% (3.9) |
| counting task, 8 requests | +0.7% (1.0) | +1.0% (1.9) |
| 16K context, 8 requests, wall time | +0.4% (1.0) | +0.4% (1.0) |
| pp2048, 1 request (llama-benchy) | +0.5% (2.9) | -0.7% (4.5) |
| tg512, 1 request (llama-benchy) | +0.7% (11.0) | +3.0% (10.2) |
| tg512, 8 requests (llama-benchy) | +1.2% (8.7) | +2.3% (8.8) |
Beyond noise on dgx-01: 4 and 8 fresh requests and 16K context with 4 requests. On dgx-02 every cell moved up but stayed inside its noise band.
Quality gate with D3: hard-mode suite 93/100, TC-45 tool calling 100/100, long-context retrieval 20/20 at 8K, 32K, 64K and 128K plus two extra 128K seeds at 20/20, straggler probe at 8, 12 and 16 requests with no preemptions.
The recipe drafts 4 tokens per step with the model's MTP layer. The original MTP layer was trained against the BF16 model, but it is served on a quantized target (NVFP4 with this GDN requant) and in quantized drafter formats, so its drafts are accepted less often. I retrained only the drafter's dense BF16 tensors; its routed experts and the draft head stay at the served NVFP4 values.
Offline acceptance per draft position on the held-out set, temperature 0, original to retrained: 0.878 to 0.905, 0.750 to 0.797, 0.635 to 0.699, 0.534 to 0.614, 0.449 to 0.542, 0.377 to 0.480. Every prompt category improved.
Live screen on two DGX Sparks (one arm per Spark, original and retrained drafter booted alternately, two passes, temperature 0). Acceptance per draft position, original to retrained:
| Spark | pos 1 | pos 2 | pos 3 | pos 4 |
|---|---|---|---|---|
| dgx-01 | 0.819 to 0.856 | 0.661 to 0.713 | 0.535 to 0.593 | 0.431 to 0.489 |
| dgx-02 | 0.818 to 0.854 | 0.666 to 0.708 | 0.539 to 0.589 | 0.438 to 0.488 |
Speed, retrained vs original (noise band in brackets). No cell was worse beyond its noise band:
| Cell | dgx-01 | dgx-02 |
|---|---|---|
| decode, 1 fresh request | +5.7% (4.7) | +2.8% (8.5) |
| decode, 4 fresh requests | +4.9% (3.1) | +7.9% (2.8) |
| decode, 8 fresh requests | +6.8% (3.6) | +8.1% (2.6) |
| decode at 16K context, 4 requests | +5.5% (3.7) | +5.7% (3.2) |
| counting task, 8 requests | +1.2% (1.0) | -0.1% (2.8) |
| 16K context, 8 requests, wall time | +2.9% (1.0) | +2.0% (1.0) |
| pp2048, 1 request (llama-benchy) | +0.0% (1.0) | -0.9% (2.0) |
| tg512, 1 request (llama-benchy) | +4.3% (7.6) | +1.6% (10.3) |
| tg512, 8 requests (llama-benchy) | +3.7% (4.9) | +1.2% (3.6) |
Quality gate with the retrained drafter, on both Sparks: hard-mode suite 91/100, TC-45 tool calling 100/100, long-context retrieval 20/20 at 8K, 32K, 64K and 128K (one of the three extra 128K seeds on dgx-01 scored 19/20), straggler probe at 8, 12 and 16 requests with no preemptions. The drafter only proposes tokens and the target model verifies them, so the output distribution does not depend on the drafter; the gate checks the serving stack.
The checkpoint needs a vLLM build with the b12x W4A16 NVFP4 linear backend and the Qwen3.8-Flash-Next model code. The recipe repository above pins both in its container image. With the recipe:
sparkrun registry add https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark
sparkrun run qwen3.8-flash-next-1x-dgx-spark --hosts <spark> --solo
Qwen Community License 1.0, the license of the base model; see LICENSE and NOTICE. The license conditions
apply to this derivative, including the separate-license requirement for Model-as-a-Service and AI Work Assistant
businesses and the model-name display requirement above the stated user and revenue thresholds.
This is local-inference-lab/Qwen3.8-Flash-Next-NVFP4
(revision 7c4f1bc1) with two changes:
16c9bd54 carries the first retrained drafter D1, and 244cb6fe keeps the
original drafter.Everything else is byte-identical to that checkpoint, including the routed and shared experts, attention, PLE tables, the MTP experts, vision encoder, tokenizer and chat template.
It is the served checkpoint of the single-DGX-Spark recipe qwen3.8-flash-next-1x-dgx-spark in
ursuciprian/qwen3.8-flash-next-1x-dgx-spark:
release v2.2.0 serves this revision (drafter D3), and the -previous recipe serves v2.1.0 (old name v3e) at
16c9bd54 (drafter D1).
| Tensors | Base checkpoint | This checkpoint |
|---|---|---|
linear_attn.in_proj_qkv, in_proj_z, out_proj in all 36 GDN layers (108 tensors) | MXFP8 (frozen during distillation) | NVFP4 W4A16: E2M1 values, E4M3 scale per 16, FP32 global scale |
24 dense BF16 tensors of the MTP drafter (mtp.*: fc, norms, hyper-connection mixers, attention, shared expert, router gate) | original | retrained (D3 since revision 3, D1 in revision 2) |
| Everything else | unchanged | unchanged |
How the GDN weights were built:
de4b8e4d).in_proj_qkv and in_proj_z of a layer share one global scale, because vLLM fuses them into one linear.Only model-00035-of-00036.safetensors, config.json, hf_quant_config.json and model.safetensors.index.json
differ from the base revision for the GDN change, and model-00034-of-00036.safetensors for the drafter.
During decode the GDN projections run at 1 to 40 rows per step. At those sizes a W4A16 NVFP4 GEMM reads about half the weight bytes of MXFP8, so each decode step is faster.
For prefill-sized calls the v3d recipe keeps an MXFP8 copy of the same weights. It loads that copy from the base
checkpoint, model-00035-of-00036.safetensors at revision 7c4f1bc1. Calls of 41 or more rows use the copy and
smaller calls use NVFP4 (VLLM_B12X_NVFP4_MXFP8_MIN_TOKENS=41), so prefill speed does not change. The copy uses
about 1.1 GiB more memory.
Paired screen, with control and candidate booted alternately on the same Spark, at temperature 0. Every cell not listed was within its noise band.
| Cell | Change | Noise band |
|---|---|---|
| decode, 8 fresh requests | +4.4% tok/s | 1.9% |
| decode at 16K context, 4 requests | +5.9% tok/s | 3.0% |
| counting task, 8 requests | +7.8% tok/s | 2.1% |
| tg512, 1 request (llama-benchy) | 50.0 -> 59.7 tok/s | 4.3% |
| pp2048, 1 request (llama-benchy) | -0.9% | 1.0% |
MTP acceptance per draft position moved by at most 0.005.
Quality gate on the v3d recipe:
D3 continues from D1 (below) with more data and a longer run. Only model-00034-of-00036.safetensors and this card
differ from revision 16c9bd54; inside that shard only the retrained mtp.* tensors changed.
Offline acceptance per draft position on the held-out set, temperature 0, D1 to D3: 0.905 to 0.911, 0.797 to 0.810, 0.699 to 0.718, 0.614 to 0.640, 0.542 to 0.574, 0.480 to 0.517 (+2.2 points per position on average). At temperature 1: 0.876 to 0.885, 0.713 to 0.733, 0.551 to 0.581, 0.409 to 0.446, 0.294 to 0.334, 0.206 to 0.245. Every prompt category improved.
Live screen on two DGX Sparks, D3 against D1 on the same image: one arm per Spark, control and candidate booted alternately, 6 boots per side per Spark over two runs (the second run repeated only the cells the first left in doubt), temperature 0 for the probes. Acceptance per draft position, D1 to D3:
| Spark | pos 1 | pos 2 | pos 3 | pos 4 |
|---|---|---|---|---|
| dgx-01 | 0.849 to 0.860 | 0.699 to 0.720 | 0.575 to 0.598 | 0.472 to 0.498 |
| dgx-02 | 0.852 to 0.857 | 0.703 to 0.717 | 0.578 to 0.596 | 0.474 to 0.493 |
Speed, D3 vs D1 (noise band in brackets). No cell was worse beyond its noise band on either Spark:
| Cell | dgx-01 | dgx-02 |
|---|---|---|
| decode, 1 fresh request | +1.5% (3.3) | +1.6% (8.3) |
| decode, 4 fresh requests | +3.4% (2.4) | +1.9% (4.8) |
| decode, 8 fresh requests | +3.3% (1.0) | +1.3% (2.8) |
| decode at 16K context, 4 requests | +4.6% (2.5) | +2.0% (3.9) |
| counting task, 8 requests | +0.7% (1.0) | +1.0% (1.9) |
| 16K context, 8 requests, wall time | +0.4% (1.0) | +0.4% (1.0) |
| pp2048, 1 request (llama-benchy) | +0.5% (2.9) | -0.7% (4.5) |
| tg512, 1 request (llama-benchy) | +0.7% (11.0) | +3.0% (10.2) |
| tg512, 8 requests (llama-benchy) | +1.2% (8.7) | +2.3% (8.8) |
Beyond noise on dgx-01: 4 and 8 fresh requests and 16K context with 4 requests. On dgx-02 every cell moved up but stayed inside its noise band.
Quality gate with D3: hard-mode suite 93/100, TC-45 tool calling 100/100, long-context retrieval 20/20 at 8K, 32K, 64K and 128K plus two extra 128K seeds at 20/20, straggler probe at 8, 12 and 16 requests with no preemptions.
The recipe drafts 4 tokens per step with the model's MTP layer. The original MTP layer was trained against the BF16 model, but it is served on a quantized target (NVFP4 with this GDN requant) and in quantized drafter formats, so its drafts are accepted less often. I retrained only the drafter's dense BF16 tensors; its routed experts and the draft head stay at the served NVFP4 values.
Offline acceptance per draft position on the held-out set, temperature 0, original to retrained: 0.878 to 0.905, 0.750 to 0.797, 0.635 to 0.699, 0.534 to 0.614, 0.449 to 0.542, 0.377 to 0.480. Every prompt category improved.
Live screen on two DGX Sparks (one arm per Spark, original and retrained drafter booted alternately, two passes, temperature 0). Acceptance per draft position, original to retrained:
| Spark | pos 1 | pos 2 | pos 3 | pos 4 |
|---|---|---|---|---|
| dgx-01 | 0.819 to 0.856 | 0.661 to 0.713 | 0.535 to 0.593 | 0.431 to 0.489 |
| dgx-02 | 0.818 to 0.854 | 0.666 to 0.708 | 0.539 to 0.589 | 0.438 to 0.488 |
Speed, retrained vs original (noise band in brackets). No cell was worse beyond its noise band:
| Cell | dgx-01 | dgx-02 |
|---|---|---|
| decode, 1 fresh request | +5.7% (4.7) | +2.8% (8.5) |
| decode, 4 fresh requests | +4.9% (3.1) | +7.9% (2.8) |
| decode, 8 fresh requests | +6.8% (3.6) | +8.1% (2.6) |
| decode at 16K context, 4 requests | +5.5% (3.7) | +5.7% (3.2) |
| counting task, 8 requests | +1.2% (1.0) | -0.1% (2.8) |
| 16K context, 8 requests, wall time | +2.9% (1.0) | +2.0% (1.0) |
| pp2048, 1 request (llama-benchy) | +0.0% (1.0) | -0.9% (2.0) |
| tg512, 1 request (llama-benchy) | +4.3% (7.6) | +1.6% (10.3) |
| tg512, 8 requests (llama-benchy) | +3.7% (4.9) | +1.2% (3.6) |
Quality gate with the retrained drafter, on both Sparks: hard-mode suite 91/100, TC-45 tool calling 100/100, long-context retrieval 20/20 at 8K, 32K, 64K and 128K (one of the three extra 128K seeds on dgx-01 scored 19/20), straggler probe at 8, 12 and 16 requests with no preemptions. The drafter only proposes tokens and the target model verifies them, so the output distribution does not depend on the drafter; the gate checks the serving stack.
The checkpoint needs a vLLM build with the b12x W4A16 NVFP4 linear backend and the Qwen3.8-Flash-Next model code. The recipe repository above pins both in its container image. With the recipe:
sparkrun registry add https://github.com/ursuciprian/qwen3.8-flash-next-1x-dgx-spark
sparkrun run qwen3.8-flash-next-1x-dgx-spark --hosts <spark> --solo
Qwen Community License 1.0, the license of the base model; see LICENSE and NOTICE. The license conditions
apply to this derivative, including the separate-license requirement for Model-as-a-Service and AI Work Assistant
businesses and the model-name display requirement above the stated user and revenue thresholds.