DeepSeek-V4-Flash DSpark drafter (Q2K-Q8 GGUF)

The DSpark block drafter for DeepSeek-V4-Flash, packaged for the ds4 C+CUDA inference engine — specifically the Entrpi/ds4 batched-serving fork (release v0.1.1+), where it drives lossless speculative decode at a 9-workload suite mean of 1.38x plain decode on a DGX Spark (GB10), peaking at 1.71x on structured content. Output is lossless by construction: the target model's verify forward is the sole token source.

What's in the file

The drafter's 3 full DeepSeek-V4 transformer layers (attention + 256-expert MoE), the target-fusing projection (main_proj/main_norm over target layers 40/41/42), the rank-256 Markov head, the confidence head, and the output head. embed_tokens/lm_head are shared from the target model at load time and are not duplicated here.

Quant policy: Q2_K routed experts (the ds4 v0.1.1 ship default — measured equal to Q4_K in acceptance and throughput, 4.2 GiB smaller), Q8_0 attention/projections/shared experts, F32 norms/gates, F16 Markov/confidence heads. ~6.5 GiB.

Extracted from the official checkpoint's drafter shards with gguf-tools/dspark_extract.py (FP8 e4m3/ue8m0 dequant identical to the engine's C implementation), routed experts requantized to Q2_K.

Files

The two MarkovQ8 files (added 2026-09-03) store the Markov head's markov_w2 table as Q8_0 instead of F16; every other tensor is byte-identical to the F16 file of the same checkpoint. The engine reads that table in full four times per draft block, and in Q8_0 it takes the existing Q8_0 mat-vec path: 0.18 ms per call instead of 0.63 on a GB10, about 1.8 ms per draft block, with the file 31 MB smaller. Acceptance is unchanged: identical per-record accept and the same mean commit (3.70) replaying a recorded trace through both forms, and within run-to-run noise on a live A/B. Decoding stays lossless in either form because the target verifies every draft.

A ds4 build from 2026-09-03 or later is needed to load the MarkovQ8 files (its loader accepts F16 or Q8_0 for that tensor); its launch defaults prefer the MarkovQ8 file beside the base GGUF and fall back to the F16 one. Older builds keep using the F16 files, which stay here unchanged. Generations never mix: the engine refuses a Vision-Exp drafter beside a 0731 base and the reverse.

Usage

One-command serving on a DGX Spark via ds4-on-spark:

curl -sSL https://raw.githubusercontent.com/entrpi/ds4-on-spark/main/install.sh | bash -s -- --start

Or by hand, alongside the base + MTP GGUFs from antirez/deepseek-v4-gguf:

env DS4_CONT_MTP_MODE=2 DS4_CONT_DSPARK=1 \
    DS4_DSPARK_MODEL=DSpark-drafter-Q2K-Q8-MarkovQ8-0731.gguf \
  ./ds4-server --cuda -m <base.gguf> --mtp <mtp.gguf> -c 32768 --port 8000

Speculation is content-dependent (52-91% draft acceptance across workloads). Since ds4 fork v0.1.1 that variance is bounded by default: a terminal yield-quench controller turns speculation off per request when acceptance can't pay the verify cost (worst case ~0.96x plain), and a kv-depth gate hands

64k-context requests to plain decode. Benchmarks, the two-corpus context frontier, and the break-even analysis: see the ds4-on-spark README.