sixteen-miles-labs/sparklab

Run frontier open-weight models privately on NVIDIA DGX Spark.

stars

168

commits

Python

primary language

Sep 10, 2026

updated

cuda

dgx-spark

gb10

inference

local-ai

mixture-of-experts

nvidia

open-weight-models

sparklab

Browse cluster: GPU-Accelerated ML & CUDA Kernels →

README

SparkLab running frontier models on NVIDIA DGX Spark

Built for NVIDIA DGX Spark (GB10)

SparkLab

Run frontier open-weight models privately on NVIDIA DGX Spark.

SparkLab is developed by SixteenMiles Labs, a research lab under Oakmind AI.

SparkLab is a GB10-native inference product for local, single-system deployments. It combines immutable model recipes, unified-memory admission, resumable checkpoint acquisition, FTW preparation, NVMe-backed MoE execution, and OpenAI- and Anthropic-compatible APIs.

Supported production target

SparkLab deliberately supports one narrow hardware profile:

NVIDIA GB10 Grace Blackwell Superchip (SM121)
128 GB coherent unified memory
ARM64 Linux or DGX OS
Python 3.10 or newer
NVIDIA driver r580 or newer and CUDA 13 toolkit
Local NVMe storage for checkpoints, FTW artifacts, and disk-backed experts
One local DGX Spark; multi-node and high-concurrency serving are outside the Beta scope

Recipe launch fails closed on failed platform, memory, swap, dependency, and storage checks. Storage whose NVMe backing cannot be established produces a warning requiring review; sparklab doctor --strict also returns non-zero for warnings. Unsupported hardware may still work through native runtime fallbacks, but it is not a SparkLab support claim.

Why SparkLab

GB10 admission: sparklab doctor validates architecture, CUDA, unified memory, swap, dependencies, NVMe backing, and free capacity with human and JSON output.
Immutable recipes: acquisition pins model revisions, records manifests, validates prepared artifacts, and rejects mismatched provenance.
Unified-memory planning: runtime admission budgets weights, expert cache, KV and recurrent state, workspaces, and an operating-system reserve without counting swap as capacity.
Frontier models beyond memory: FTW expert banks and bounded NVMe-backed caching let selected MoE checkpoints exceed physical memory.
Stable local APIs: OpenAI Chat Completions, Responses, and Anthropic Messages share one loopback endpoint with streaming, reasoning, and tool-call support.
Evidence-bound claims: model status and performance point to versioned, complete-checkpoint GB10 records rather than inferred capability.

Model portfolio

Model	Parameters	Quantization	Status	tok/s	Warm TTFT (s)	Run
Fast — routine chat, editing, and short agent loops
Qwen3.6-35B-A3B	35B total / 3B active	NVFP4 · FTW + optional MTP2	Experimental	80.55	0.367	Instructions
Qwen3.8-27B	27B dense	NVFP4 · FTW + optional DFlash2-12	Experimental	45.88	0.152	Instructions
Frontier — quality-first coding, reasoning, and long agent work
Qwen3.8-Flash-Next	125B LM + 55B auxiliary / 6B active	NVFP4 · FTW + opt-in fast MTP3	Experimental	42.14	0.244	Instructions
DeepSeek V4 Flash	284B total / 13B active	DS-FP4 · FTW + optional DSpark5	Preview	14.02	0.515	Instructions
GLM-5.3 Flash	320B total / 18B active	NVFP4 + KDA FP8 · FTW + optional MTP3	Experimental	7.77	6.395	Instructions
Research — bounded execution outside the interactive envelope
GLM-5.3	753B total / 40B active	NVFP4 + resident FP8 · FTW	Experimental	0.81	2.530	Instructions
Kimi K3	2.8T total / 16 of 896 experts	ModelOpt NVFP4/FP8 · FTW	Experimental	0.16	395.405	Instructions

Status meanings:

Experimental: implementation or measured evidence exists, but required gates remain incomplete or failed.
Preview: a bounded supported path exists, but the full release promise is incomplete.
Certified: the exact recipe, revision, artifact, and release environment passed all required correctness, parser, agent, context, latency, memory, NVMe, and endurance gates.

Run sparklab models --json for exact recipe versions, checkpoint revisions, artifact fingerprints, implementation state, evidence IDs, and known constraints.

Documentation

Start with installation, then follow the quick start or a model's run instructions. The current Qwen3.8-Flash-Next recipe requires a source installation; the released 0.1.2 wheel does not include its required runtime support.

Contribution

SparkLab is stewarded in public by SixteenMiles Labs. Oakmind AI provides organizational backing, legal stewardship, and commercial support.

Credits and citation

SparkLab incorporates source and research contributions from FreeToken and builds on open inference projects including mini-sglang, SGLang, vLLM, FlashInfer, flash-linear-attention, LightLLM, and llama.cpp.

If you use SparkLab, cite the software:

@software{sixteenmileslabs2026sparklab,
  title={SparkLab: Frontier Open-Weight Model Inference on NVIDIA DGX Spark},
  author={{SixteenMiles Labs}},
  year={2026},
  url={https://github.com/sixteen-miles-labs/sparklab},
  license={Apache-2.0}
}

For work that builds on SparkLab's FreeToken-derived execution techniques, also cite the FreeToken paper:

@article{yang2026freetoken,
  title={FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution},
  author={Yang, Shuo and Fan, Xiaoze and Pan, Melissa and Xi, Haocheng and Wang, Zhe and Sun, Shanlin and Keutzer, Kurt and Han, Song and Zaharia, Matei and Xu, Chenfeng and Stoica, Ion},
  journal={arXiv preprint arXiv:2608.16157},
  year={2026}
}