README
Qwen3.8-27B-Splash-HQ (Native 8-bit)
[!IMPORTANT]
⚠️ Runtime Package Compatibility Notice
This model is packed in the
splash-packed-q8(schema_version 5) format developed for high-precision Apple Silicon evaluation.
- Official upstream Splash 1.0 (
incoai/splash) only recognizes 4-bit formats (splash-packed-q4, schema 3/4) and will reject Q8 packages withrepository is not a supported Splash runtime package.- To run this model, use the Q8-enabled Metal runtime fork available at: 👉 https://github.com/npanj/splash/tree/q8
All 81 model files (target layers, embedding, head, draft model, vision model, tokenizer, and manifest) are 100% complete and verified.
Overview
This repository contains the uncompressed native 8-bit weights of Qwen3.8-27B with built-in Multi-Token Prediction (MTP) heads, optimized for high-throughput speculative decoding on Apple Silicon.
All 64 layers are packed with true uncompressed 8-bit tiled weights (MDFL0008), preserving exact associative reasoning while running speculative verification via Splash's compiled Metal kernel pipeline.
Performance Benchmark vs Other Engines (Apple M-Series, 64 GiB)
| Model / Engine | Format | Speculative Scheme | Decode Speed (tok/s) | Speedup vs Stock AR |
|---|---|---|---|---|
| Qwen3.8-27B-Splash-HQ | Native 8-bit | Splash Metal MTP | 36.9 t/s (peak 54.8) | 3.73x |
| Splash-Q8 | Transcoded 8-bit | Splash Metal MTP | 36.5 t/s (peak 52.7) | 3.69x |
| Splash-Q4 | 4-bit | Splash Metal MTP | 60.7 t/s (peak 83.3) | 6.13x |
| MTPLX-Optimized-Quality | 8-bit | MTPLX DraftCore D3 | 26.5 t/s (peak 28.8) | 2.68x |
| MLX-Community 8bit | 8-bit | None (Autoregressive) | 9.9 t/s | 1.00x |
| llama.cpp Q8_0 GGUF | Q8_0 | None (Autoregressive) | 9.9 t/s | 1.00x |
Step-by-Step Setup Recipe
1. Clone the Q8 Splash Engine Fork
git clone https://github.com/npanj/splash.git -b q8
cd splash
2. Build the Metal Kernels
make -j4
3. Serve the Model
./splash serve --model nitinpanj/Qwen3.8-27B-Splash-HQ --port 8000
(On first launch, this automatically verifies and downloads the model into install/models/nitinpanj/Qwen3.8-27B-Splash-HQ and starts the OpenAI-compatible API on http://127.0.0.1:8000).
4. Connect Coding Agents
# Connect with Oh My Pi (OMP)
omp --model splash/nitinpanj/Qwen3.8-27B-Splash-HQ
# Or connect with any OpenAI-compatible client at http://127.0.0.1:8000/v1
For the complete benchmark report, evaluation scripts, and raw JSON logs: 👉 https://github.com/npanj/splash-plus
Similar repos
README
Qwen3.8-27B-Splash-HQ (Native 8-bit)
[!IMPORTANT]
⚠️ Runtime Package Compatibility Notice
This model is packed in the
splash-packed-q8(schema_version 5) format developed for high-precision Apple Silicon evaluation.
- Official upstream Splash 1.0 (
incoai/splash) only recognizes 4-bit formats (splash-packed-q4, schema 3/4) and will reject Q8 packages withrepository is not a supported Splash runtime package.- To run this model, use the Q8-enabled Metal runtime fork available at: 👉 https://github.com/npanj/splash/tree/q8
All 81 model files (target layers, embedding, head, draft model, vision model, tokenizer, and manifest) are 100% complete and verified.
Overview
This repository contains the uncompressed native 8-bit weights of Qwen3.8-27B with built-in Multi-Token Prediction (MTP) heads, optimized for high-throughput speculative decoding on Apple Silicon.
All 64 layers are packed with true uncompressed 8-bit tiled weights (MDFL0008), preserving exact associative reasoning while running speculative verification via Splash's compiled Metal kernel pipeline.
Performance Benchmark vs Other Engines (Apple M-Series, 64 GiB)
| Model / Engine | Format | Speculative Scheme | Decode Speed (tok/s) | Speedup vs Stock AR |
|---|---|---|---|---|
| Qwen3.8-27B-Splash-HQ | Native 8-bit | Splash Metal MTP | 36.9 t/s (peak 54.8) | 3.73x |
| Splash-Q8 | Transcoded 8-bit | Splash Metal MTP | 36.5 t/s (peak 52.7) | 3.69x |
| Splash-Q4 | 4-bit | Splash Metal MTP | 60.7 t/s (peak 83.3) | 6.13x |
| MTPLX-Optimized-Quality | 8-bit | MTPLX DraftCore D3 | 26.5 t/s (peak 28.8) | 2.68x |
| MLX-Community 8bit | 8-bit | None (Autoregressive) | 9.9 t/s | 1.00x |
| llama.cpp Q8_0 GGUF | Q8_0 | None (Autoregressive) | 9.9 t/s | 1.00x |
Step-by-Step Setup Recipe
1. Clone the Q8 Splash Engine Fork
git clone https://github.com/npanj/splash.git -b q8
cd splash
2. Build the Metal Kernels
make -j4
3. Serve the Model
./splash serve --model nitinpanj/Qwen3.8-27B-Splash-HQ --port 8000
(On first launch, this automatically verifies and downloads the model into install/models/nitinpanj/Qwen3.8-27B-Splash-HQ and starts the OpenAI-compatible API on http://127.0.0.1:8000).
4. Connect Coding Agents
# Connect with Oh My Pi (OMP)
omp --model splash/nitinpanj/Qwen3.8-27B-Splash-HQ
# Or connect with any OpenAI-compatible client at http://127.0.0.1:8000/v1
For the complete benchmark report, evaluation scripts, and raw JSON logs: 👉 https://github.com/npanj/splash-plus