README
MoonViT K2.6
This repository packages the Kimi K2.6 MoonViT Tower and Projector as a self-contained Transformers model. It does not download the full 64-shard checkpoint or fetch architecture code from another repository at runtime.
Contents
| File | Tensors | What it holds |
|---|---|---|
model.safetensors | 329 | MoonViT Tower |
projector.safetensors | 6 | Kimi K2.6 Projector |
projector_config.json, projector.py | Projector shapes and loader | |
config.json, configuration_moonvit.py, modeling_moonvit.py | Standalone Transformers model | |
preprocessor_config.json, kimi_k25_vision_processing.py, media_utils.py | Image preprocessing |
Architecture
| Component | Details |
|---|---|
| Tower | 27 layers, 1152 hidden, 16 heads, 4304 intermediate, patch size 14 |
| Token compression | 2x2 spatial regrouping plus temporal pooling, no learned parameters |
| Projector | LayerNorm(1152), flatten four patches, Linear(4608, 4608), GELU, Linear(4608, 7168) |
The Tower loads through AutoModel.from_pretrained(..., trust_remote_code=True). The
processor accepts the standard images=... interface, and projector.py maps the merged
tokens to the Kimi K2.6 language width.
Usage
See examples/inference.py for image-to-projected-features
inference.
Validation
The release tests compare all 335 packaged tensors with shards 63 and 64 of the pinned
Moonshot checkpoint using torch.equal. Fixed-image merged and projected outputs also
match the parent implementation bit-for-bit on CPU and in BF16 on an NVIDIA A100.
Reproduction
The export script
splits the BF16 weights published by
exolabs/Kimi-K2.6-vision into separate
Tower and Projector files. It derives a vision-only loader and processor from the pinned
Moonshot implementation.
Credits
Moonshot AI released Kimi K2.6 and its modeling code. Exolabs published the combined vision-only weight file used by the exporter.
License
Kimi K2.6 License, the same license as the source model. The upstream third-party notices are included.
Contributors
2 commits
Similar repos
Linked in READMEs
README
MoonViT K2.6
This repository packages the Kimi K2.6 MoonViT Tower and Projector as a self-contained Transformers model. It does not download the full 64-shard checkpoint or fetch architecture code from another repository at runtime.
Contents
| File | Tensors | What it holds |
|---|---|---|
model.safetensors | 329 | MoonViT Tower |
projector.safetensors | 6 | Kimi K2.6 Projector |
projector_config.json, projector.py | Projector shapes and loader | |
config.json, configuration_moonvit.py, modeling_moonvit.py | Standalone Transformers model | |
preprocessor_config.json, kimi_k25_vision_processing.py, media_utils.py | Image preprocessing |
Architecture
| Component | Details |
|---|---|
| Tower | 27 layers, 1152 hidden, 16 heads, 4304 intermediate, patch size 14 |
| Token compression | 2x2 spatial regrouping plus temporal pooling, no learned parameters |
| Projector | LayerNorm(1152), flatten four patches, Linear(4608, 4608), GELU, Linear(4608, 7168) |
The Tower loads through AutoModel.from_pretrained(..., trust_remote_code=True). The
processor accepts the standard images=... interface, and projector.py maps the merged
tokens to the Kimi K2.6 language width.
Usage
See examples/inference.py for image-to-projected-features
inference.
Validation
The release tests compare all 335 packaged tensors with shards 63 and 64 of the pinned
Moonshot checkpoint using torch.equal. Fixed-image merged and projected outputs also
match the parent implementation bit-for-bit on CPU and in BF16 on an NVIDIA A100.
Reproduction
The export script
splits the BF16 weights published by
exolabs/Kimi-K2.6-vision into separate
Tower and Projector files. It derives a vision-only loader and processor from the pinned
Moonshot implementation.
Credits
Moonshot AI released Kimi K2.6 and its modeling code. Exolabs published the combined vision-only weight file used by the exporter.
License
Kimi K2.6 License, the same license as the source model. The upstream third-party notices are included.
Similar repos
Linked in READMEs
Contributors
2 commits