DeepSeek V4 Flash 0731 GGUF for one GX10

This is a community GGUF quantization of deepseek-ai/DeepSeek-V4-Flash-0731, built to run fully resident on one 128 GB NVIDIA GB10 system such as an ASUS Ascent GX10 or DGX Spark.

It is not an official DeepSeek release.

Artifact

The GGUF contains the 45-shard base target model. It does not include shards 46 to 48 or enable the optional DSpark speculative module.

Quantization layout

The layout follows the proven one-GB10 V4 Flash recipe:

• routed gate and up experts: IQ2_XXS
• routed down experts: Q2_K
• attention projections: Q8_0
• shared experts: Q8_0
• output head: Q8_0
• token embedding: F16
• sensitive norms and control tensors: F16 or F32 according to the pinned template

The calibration imatrix is Antirez's 1.5M-token routed-MoE dataset artifact from antirez/deepseek-v4-gguf, revision a88c423b511666d7ff7a4dcaee651669312bea97.

Where the 80.76 GiB goes

A header-only GGUF audit reconciled every serialized tensor byte without reading tensor payloads:

The model routes 6 of 256 experts per layer. This does not imply that only 6/256 of stored expert bytes are transferred by the runtime, but it explains why selective low-bit expert storage matters more than uniformly reducing every tensor. See measurement/header-byte-accounting.json.

Important limitation: the imatrix was collected for the earlier V4 Flash checkpoint and reused for 0731. This release has not established broad quality equivalence to the official mixed-precision checkpoint.

Measured GX10 result

Measured on one NVIDIA GB10 with 128 GB unified memory using target-only decoding. Both checkpoints used the same DS4 binary, prompts, tokenizer/template path, context, sampler settings, seed, power setting, and GPU budget.

Protocol:

• 1,024-token context
• 256-token output cap
• greedy decoding, temperature 0
• seed 260729
• non-thinking mode
• one warm-up per arm and workload
• three measured runs per arm and workload
• balanced ABBAAB measured order

Across all 16 warm-up and measured runs:

• all runs completed
• zero safety breaches
• zero swap growth
• minimum available memory was approximately 14.0 GiB
• maximum sampled temperature was 67 C
• maximum sampled power was 41.69 W
• all three measured outputs were byte-identical within each checkpoint and workload

The output text differed between the two checkpoints, as expected after post-training. The benchmark prompts were capped at 256 generated tokens and therefore do not support a broad quality-retention claim. See benchmark/protocol.json and benchmark/summary.json.

The matched benchmark ran before a same-length provenance-only metadata sanitization. That edit replaced the local quantize.imatrix.file string without changing tensor payload bytes, tensor offsets, or file size. The uploaded artifact was rehashed, header-audited, and smoke-tested afterward.

Verified smoke test

The final GGUF loaded through the CUDA backend on an NVIDIA GB10 and generated:

This sanitized local model response was generated on an NVIDIA GX10.

For that post-sanitization one-shot smoke test:

• CUDA model preparation: 19.933 seconds
• resident model spans: 80.76 GiB
• planned model plus 1,024-token KV memory: 80.86 GiB
• prefill: 40.52 tok/s
• generation: 15.56 tok/s

The matched three-run benchmark above is the better throughput reference.

Download

hf download tekosML/DeepSeek-V4-Flash-0731-GGUF-GX10 \
  DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf \
  --local-dir .

Run with DS4

Build DS4 at the tested commit using its documented CUDA Spark target, then run:

./ds4 \
  -m DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf \
  --prompt-file prompt.txt \
  --ctx 1024 \
  -n 256 \
  --temp 0 \
  --seed 260729 \
  --nothink \
  --power 100 \
  --gpu-vram 105

Start conservatively. Larger context windows require more KV memory and were not validated for this release.

Reproducibility and lineage

See:

• provenance.json for revisions, sizes, and hashes
• READY.json for the exercised smoke-test result
• benchmark/protocol.json and benchmark/summary.json for the matched GX10 benchmark
• measurement/header-byte-accounting.json for exact GGUF payload and tensor-family accounting
• conversion/resume-notes.md for the boundary-validated write-resume provenance after the original converter stopped at tensor 133

The resume helper changed file continuation behavior only. It validated the complete GGUF header, every planned tensor's name, shape, type, size, and offset, and required the existing file to end at an exact tensor boundary before appending the remaining tensors. The final artifact was independently hashed after completion.

License

The source checkpoint is MIT licensed. The original license is included in this repository. Preserve the license and attribution when redistributing this derivative.