Xolotl (formerly Strata-DSV4.1): DeepSeek-V4.1-Flash, all 552B, on one gaming PC (RTX 4090 + 90 GB RAM) at ~27 tok/s, plus GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B. A heavily extended fork of Strata: MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.
See the code
DeepSeek-V4.1-Flash - all 552 billion parameters - on one gaming PC, at 27 tok/s.
Plus GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B: one engine, one OpenAI / Anthropic-compatible server.
Built on Strata and heavily extended · Linux · NVIDIA · MIT
A Strata fork. Xolotl keeps Strata's engine, server and kernels at its core (merged at 0.1.44.1) and adds ~71,000 lines on top: the DeepSeek-V4.1 and V4-Flash engines, the GLM-5.3-Flash engine and the Qwen3.8-27B dense port, each with its own quantisation and expert-tiering work.
Starred this as Strata-DSV4.1? You are in the right place. Strata-DSV4.1 grew into Xolotl: the same DeepSeek-V4.1 engine (faster now), plus GLM-5.3-Flash and the Qwen3.8 models, in one tree. Old links and clones redirect here; nothing DeepSeek-V4.1 was removed.
One RTX 4090 (24 GB) · Ryzen 7 5700X · 90 GB DDR4 · one NVMe drive. Every number is measured on that machine, through the real server, while it solved real tasks (21 HumanEval problems, 10 tool-call tasks, 3 agentic bug-fix loops), greedy, thinking on with a 32K-token thinking budget.
| Model | Size | Prefill | Decode | HumanEval | Tool calls | Agentic | Context |
|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash | 552B MoE | 180-320 tok/s | 26.8 tok/s | 21/21 | 10/10 | 3/3 | 512K |
| GLM-5.3-Flash | 320B MoE | 290-375 tok/s | 25.3 tok/s | 19/21 | 10/10 | 3/3 | 512K |
| Qwen3.8-Flash-Next | 125B MoE | 3,000-3,430 tok/s | 127.8 tok/s | 21/21 | 10/10 | 3/3 | 200K |
| Qwen3.8-27B (dense) | 27B | 1,240-1,390 tok/s | 98.5 tok/s | 21/21 | 10/10 | 3/3 | 200K |
Decode = generation speed during those tasks. Prefill = a fresh 6K-17K-token prompt through the server
(tools/throughput.py; Qwen3.8-Flash-Next: 3,430 / 3,005 tok/s at 6.5K / 16.7K, Qwen3.8-27B: 1,385 / 1,244 tok/s,
DeepSeek-V4.1: 178 / 268 tok/s at 6.1K / 16.5K - its prompt batch is sized from free VRAM, ~6-9K tokens per sweep
on a 24 GB card with the speculative-decode draft loaded).
DeepSeek-V4.1 with the speculative-decode draft off the card during long prompts (the default since 2026-10-17):
187 / 317 tok/s at 6.4K / 16.7K. GLM-5.3-Flash: soft prune + --route-bias 0.25
(tools/glm/serve/strata-glm-maya-abl-512k-bias.json), prefill 301 / 374 tok/s at 6.1K / 15.6K and 368-376 on fresh
30K / 44K prompts. How close the DeepSeek quant gets to DeepSeek's own API on the same tasks:
docs/xolotl/BENCHMARKS.md.
What that means in practice:
Linux, an NVIDIA GPU with 24 GB, CUDA 12.x/13.x, and RAM for the arena (90 GB was used above).
git clone https://github.com/Indras-Mirror/Xolotl.git xolotl && cd xolotl
bash tools/xolotl/fetch-llama.sh # llama.cpp at the pinned commit + tools/xolotl/ggml/xolotl-ggml.patch -> third_party/llama.cpp
cmake -S . -B build-xolotl -G Ninja -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_GGML_CUDA=ON \
-DSTRATA_MMQ_KQUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=89 -DSTRATA_GGML_DIR=$PWD/third_party/llama.cpp -DSTRATA_DSV41_FQ_OP=ON \
-DSTRATA_DSV41_HC_MIX=ON
ninja -C build-xolotl # strata, strata-dense, glm_generate, ds41_generate, ds4_generate
Get a model: DeepSeek-V4.1-Flash is IndrasMirror/DeepSeek-V4.1-Flash-GSQ-Uncensored (152 GiB, our GSQ mixed ternary / 2-bit / 3-bit quant - the card has the download and Engram steps). Then serve it (edit the model paths in its config first):
python3 tools/v41/serve/serve_ds41.py --engine strata --config tools/v41/serve/strata-ds41.json --port 8150 # DeepSeek-V4.1-Flash
python3 tools/glm/serve/serve_glm.py --engine strata --config tools/glm/serve/strata-glm-maya-abl-512k.json --port 8144 # GLM-5.3-Flash
python3 -m serve.server --engine strata --config <your-qwen-flash.json> --port 8130 # Qwen3.8-Flash-Next
python3 -m serve.server --engine strata --config <your-27b.json> --port 8090 # Qwen3.8-27B (exe: build-xolotl/strata-dense)
Then point any OpenAI- or Anthropic-compatible client at http://127.0.0.1:<port>/v1.
A mixture-of-experts model only uses a few experts per token. Xolotl keeps every expert somewhere - the hottest in VRAM, tens of GiB in a pinned RAM arena the GPU reads by DMA, the rest on NVMe read at the drive's ceiling - and moves the ones each token's router asks for, a layer ahead. Each model adds its own tricks: DeepSeek-V4.1 gets a calibrated mixed-precision quantisation, speculative decoding with exact sampling, a bit-exact 4-bit KV cache for 512K context and an arena that adapts to the conversation; GLM-5.3-Flash gets its abliteration LoRA on every path and soft pruning; the Qwen models get MTP speculative decoding. Read the full write-up.
| How Xolotl works | the three memory tiers, per-model engines, the server, the agent-loop work |
| Benchmarks | every measured number, exactness checks, how to measure your own PC |
| DeepSeek-V4.1-Flash engine | GSQ quantisation, DSpark, the fp4 KV cache, Engram |
| GLM-5.3-Flash engine | KDA + MLA attention, the LoRA on routed experts, soft pruning |
| Qwen3.8 / upstream Strata | the Strata engine this grew from |
| Brand kit | logo, banner, palette |
Built on Strata by Niko1221 (its engine, server and kernels, merged at 0.1.44.1) and ggml / llama.cpp; the Qwen3.8-27B dense port is perronemirko's. The DeepSeek-V4.1 and GLM-5.3-Flash engines, the GSQ quant and the unification are Indras Mirror's. Model weights: DeepSeek, Zhipu AI, Qwen, with quants and fine-tunes by peasantsmith, ukisai, d0xin and gcsa-abliterix.
Xolotl is Quetzalcoatl's dark twin, who carries the dead up from the underworld - as the engine carries experts up from RAM and NVMe. Research code from one developer, measured on one PC; reports from other hardware welcome. MIT.
Xolotl (formerly Strata-DSV4.1): DeepSeek-V4.1-Flash, all 552B, on one gaming PC (RTX 4090 + 90 GB RAM) at ~27 tok/s, plus GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B. A heavily extended fork of Strata: MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.
See the code
DeepSeek-V4.1-Flash - all 552 billion parameters - on one gaming PC, at 27 tok/s.
Plus GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B: one engine, one OpenAI / Anthropic-compatible server.
Built on Strata and heavily extended · Linux · NVIDIA · MIT
A Strata fork. Xolotl keeps Strata's engine, server and kernels at its core (merged at 0.1.44.1) and adds ~71,000 lines on top: the DeepSeek-V4.1 and V4-Flash engines, the GLM-5.3-Flash engine and the Qwen3.8-27B dense port, each with its own quantisation and expert-tiering work.
Starred this as Strata-DSV4.1? You are in the right place. Strata-DSV4.1 grew into Xolotl: the same DeepSeek-V4.1 engine (faster now), plus GLM-5.3-Flash and the Qwen3.8 models, in one tree. Old links and clones redirect here; nothing DeepSeek-V4.1 was removed.
One RTX 4090 (24 GB) · Ryzen 7 5700X · 90 GB DDR4 · one NVMe drive. Every number is measured on that machine, through the real server, while it solved real tasks (21 HumanEval problems, 10 tool-call tasks, 3 agentic bug-fix loops), greedy, thinking on with a 32K-token thinking budget.
| Model | Size | Prefill | Decode | HumanEval | Tool calls | Agentic | Context |
|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash | 552B MoE | 180-320 tok/s | 26.8 tok/s | 21/21 | 10/10 | 3/3 | 512K |
| GLM-5.3-Flash | 320B MoE | 290-375 tok/s | 25.3 tok/s | 19/21 | 10/10 | 3/3 | 512K |
| Qwen3.8-Flash-Next | 125B MoE | 3,000-3,430 tok/s | 127.8 tok/s | 21/21 | 10/10 | 3/3 | 200K |
| Qwen3.8-27B (dense) | 27B | 1,240-1,390 tok/s | 98.5 tok/s | 21/21 | 10/10 | 3/3 | 200K |
Decode = generation speed during those tasks. Prefill = a fresh 6K-17K-token prompt through the server
(tools/throughput.py; Qwen3.8-Flash-Next: 3,430 / 3,005 tok/s at 6.5K / 16.7K, Qwen3.8-27B: 1,385 / 1,244 tok/s,
DeepSeek-V4.1: 178 / 268 tok/s at 6.1K / 16.5K - its prompt batch is sized from free VRAM, ~6-9K tokens per sweep
on a 24 GB card with the speculative-decode draft loaded).
DeepSeek-V4.1 with the speculative-decode draft off the card during long prompts (the default since 2026-10-17):
187 / 317 tok/s at 6.4K / 16.7K. GLM-5.3-Flash: soft prune + --route-bias 0.25
(tools/glm/serve/strata-glm-maya-abl-512k-bias.json), prefill 301 / 374 tok/s at 6.1K / 15.6K and 368-376 on fresh
30K / 44K prompts. How close the DeepSeek quant gets to DeepSeek's own API on the same tasks:
docs/xolotl/BENCHMARKS.md.
What that means in practice:
Linux, an NVIDIA GPU with 24 GB, CUDA 12.x/13.x, and RAM for the arena (90 GB was used above).
git clone https://github.com/Indras-Mirror/Xolotl.git xolotl && cd xolotl
bash tools/xolotl/fetch-llama.sh # llama.cpp at the pinned commit + tools/xolotl/ggml/xolotl-ggml.patch -> third_party/llama.cpp
cmake -S . -B build-xolotl -G Ninja -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_GGML_CUDA=ON \
-DSTRATA_MMQ_KQUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=89 -DSTRATA_GGML_DIR=$PWD/third_party/llama.cpp -DSTRATA_DSV41_FQ_OP=ON \
-DSTRATA_DSV41_HC_MIX=ON
ninja -C build-xolotl # strata, strata-dense, glm_generate, ds41_generate, ds4_generate
Get a model: DeepSeek-V4.1-Flash is IndrasMirror/DeepSeek-V4.1-Flash-GSQ-Uncensored (152 GiB, our GSQ mixed ternary / 2-bit / 3-bit quant - the card has the download and Engram steps). Then serve it (edit the model paths in its config first):
python3 tools/v41/serve/serve_ds41.py --engine strata --config tools/v41/serve/strata-ds41.json --port 8150 # DeepSeek-V4.1-Flash
python3 tools/glm/serve/serve_glm.py --engine strata --config tools/glm/serve/strata-glm-maya-abl-512k.json --port 8144 # GLM-5.3-Flash
python3 -m serve.server --engine strata --config <your-qwen-flash.json> --port 8130 # Qwen3.8-Flash-Next
python3 -m serve.server --engine strata --config <your-27b.json> --port 8090 # Qwen3.8-27B (exe: build-xolotl/strata-dense)
Then point any OpenAI- or Anthropic-compatible client at http://127.0.0.1:<port>/v1.
A mixture-of-experts model only uses a few experts per token. Xolotl keeps every expert somewhere - the hottest in VRAM, tens of GiB in a pinned RAM arena the GPU reads by DMA, the rest on NVMe read at the drive's ceiling - and moves the ones each token's router asks for, a layer ahead. Each model adds its own tricks: DeepSeek-V4.1 gets a calibrated mixed-precision quantisation, speculative decoding with exact sampling, a bit-exact 4-bit KV cache for 512K context and an arena that adapts to the conversation; GLM-5.3-Flash gets its abliteration LoRA on every path and soft pruning; the Qwen models get MTP speculative decoding. Read the full write-up.
| How Xolotl works | the three memory tiers, per-model engines, the server, the agent-loop work |
| Benchmarks | every measured number, exactness checks, how to measure your own PC |
| DeepSeek-V4.1-Flash engine | GSQ quantisation, DSpark, the fp4 KV cache, Engram |
| GLM-5.3-Flash engine | KDA + MLA attention, the LoRA on routed experts, soft pruning |
| Qwen3.8 / upstream Strata | the Strata engine this grew from |
| Brand kit | logo, banner, palette |
Built on Strata by Niko1221 (its engine, server and kernels, merged at 0.1.44.1) and ggml / llama.cpp; the Qwen3.8-27B dense port is perronemirko's. The DeepSeek-V4.1 and GLM-5.3-Flash engines, the GSQ quant and the unification are Indras Mirror's. Model weights: DeepSeek, Zhipu AI, Qwen, with quants and fine-tunes by peasantsmith, ukisai, d0xin and gcsa-abliterix.
Xolotl is Quetzalcoatl's dark twin, who carries the dead up from the underworld - as the engine carries experts up from RAM and NVMe. Research code from one developer, measured on one PC; reports from other hardware welcome. MIT.