Hy3 295B-A21B — NVFP4 for veloGB10 (NVIDIA DGX Spark / GB10)
0
2 commits
1 linked in READMEs
updated Jul 22, 2026
README
Hy3 295B-A21B — NVFP4 for veloGB10 (NVIDIA DGX Spark / GB10)
Original model: tencent/Hy3 by Tencent's Hunyuan team, Apache-2.0. This artifact contains modified weights (NVFP4 quantization); the original model is © its authors.
NVFP4-quantized variant of Hy3 (295B total / ~21B activated per token, MoE). This quantized
artifact exists first and foremost to run with the veloGB10
inference engine (gb10_inference) on NVIDIA DGX Spark / GB10 systems — it is produced,
validated, and tuned for veloGB10's kernels. The weights use the standard HuggingFace
compressed-tensors layout (nvfp4-pack-quantized), so this artifact can be used for any
purpose, with any framework that reads the format. It is in particular fit and proven to work
with veloGB10 — two-node TP=2 serving, native NextN speculative decoding, and the engine's
greedy-lossless correctness gates all pass on this artifact.
Specifications
| Base model | tencent/Hy3 |
| Architecture | MoE transformer, 80 layers, hidden 4096 |
| Attention | GQA — 64 Q / 8 KV heads, head_dim 128, per-head qk-norm |
| MoE | 192 routed experts, top-8 + 1 shared (expert inter 1536); sigmoid router with aux-loss-free bias, ×2.826 scaling |
| Dense layers | first 1 layer dense MLP (inter 13312) |
| MTP head | native NextN prediction layer (used for speculative decoding) |
| Context | 262,144 tokens |
| Vocabulary | 120,832 |
| Quantization | all,-router,-embed,-lmhead — all GEMMs NVFP4 except router, embeddings, and LM head |
| Format | nvfp4-pack-quantized (compressed-tensors compatible) |
| On-disk size | ~166 GB — requires 2× GB10 (TP=2) (~83 GB per node); does not fit one 128 GB box |
Running with veloGB10
Full instructions: veloGB10 README (build or download a
release binary, then). The binary loads its PTX kernel artifacts from the current directory — run
it from wherever the binary + *.ptx live. TP=2 is required for this model; the node needs no
model copy and no configuration — the head ships weights, settings, and calibration at sync.
On the second GB10:
gb10_inference --node --port 29500
On the head node:
gb10_inference --server --model-dir=/path/to/hy3-nvfp4 --tp \
--nodes <peer-ip>:29500 --port=9000 --max-seq-len=32768 --prefix-cache=on
Throughput and tool-eval numbers for this artifact are tracked in the veloGB10 README benchmarks section.
Notes
- Tokenizer, chat template, and generation config are included as shipped by the quantization
pipeline (chat format: Hunyuan
hy_*tags with<think>blocks). - Quantization can degrade output quality relative to the bf16 original; validate on your own workload. Report issues via the veloGB10 issue tracker.
- Intended for research and local/agentic serving use, under the base model's Apache-2.0 terms.
Similar repos
Linked in READMEs
Hy3 295B-A21B — NVFP4 for veloGB10 (NVIDIA DGX Spark / GB10)
0
2 commits
1 linked in READMEs
updated Jul 22, 2026
README
Hy3 295B-A21B — NVFP4 for veloGB10 (NVIDIA DGX Spark / GB10)
Original model: tencent/Hy3 by Tencent's Hunyuan team, Apache-2.0. This artifact contains modified weights (NVFP4 quantization); the original model is © its authors.
NVFP4-quantized variant of Hy3 (295B total / ~21B activated per token, MoE). This quantized
artifact exists first and foremost to run with the veloGB10
inference engine (gb10_inference) on NVIDIA DGX Spark / GB10 systems — it is produced,
validated, and tuned for veloGB10's kernels. The weights use the standard HuggingFace
compressed-tensors layout (nvfp4-pack-quantized), so this artifact can be used for any
purpose, with any framework that reads the format. It is in particular fit and proven to work
with veloGB10 — two-node TP=2 serving, native NextN speculative decoding, and the engine's
greedy-lossless correctness gates all pass on this artifact.
Specifications
| Base model | tencent/Hy3 |
| Architecture | MoE transformer, 80 layers, hidden 4096 |
| Attention | GQA — 64 Q / 8 KV heads, head_dim 128, per-head qk-norm |
| MoE | 192 routed experts, top-8 + 1 shared (expert inter 1536); sigmoid router with aux-loss-free bias, ×2.826 scaling |
| Dense layers | first 1 layer dense MLP (inter 13312) |
| MTP head | native NextN prediction layer (used for speculative decoding) |
| Context | 262,144 tokens |
| Vocabulary | 120,832 |
| Quantization | all,-router,-embed,-lmhead — all GEMMs NVFP4 except router, embeddings, and LM head |
| Format | nvfp4-pack-quantized (compressed-tensors compatible) |
| On-disk size | ~166 GB — requires 2× GB10 (TP=2) (~83 GB per node); does not fit one 128 GB box |
Running with veloGB10
Full instructions: veloGB10 README (build or download a
release binary, then). The binary loads its PTX kernel artifacts from the current directory — run
it from wherever the binary + *.ptx live. TP=2 is required for this model; the node needs no
model copy and no configuration — the head ships weights, settings, and calibration at sync.
On the second GB10:
gb10_inference --node --port 29500
On the head node:
gb10_inference --server --model-dir=/path/to/hy3-nvfp4 --tp \
--nodes <peer-ip>:29500 --port=9000 --max-seq-len=32768 --prefix-cache=on
Throughput and tool-eval numbers for this artifact are tracked in the veloGB10 README benchmarks section.
Notes
- Tokenizer, chat template, and generation config are included as shipped by the quantization
pipeline (chat format: Hunyuan
hy_*tags with<think>blocks). - Quantization can degrade output quality relative to the bf16 original; validate on your own workload. Report issues via the veloGB10 issue tracker.
- Intended for research and local/agentic serving use, under the base model's Apache-2.0 terms.