README
Qwen3.8-27B Vision
This repository packages the Tower and learned merger from Qwen3.8-27B.
Contents
| File | Tensors | What it holds |
|---|---|---|
model.safetensors | 333 | Tower and learned merger, extracted from source shard 1 |
config.json | Vision-only Qwen3_5VisionConfig | |
preprocessor_config.json | Qwen image preprocessing configuration |
Qwen implements the learned merger inside Qwen3_5VisionModel, so this repository does
not need a separate Projector file.
Architecture
| Component | Details |
|---|---|
| Tower | 27 layers, 1152 hidden, 16 heads, 4304 intermediate, patch size 16 |
| Token compression | 2x2 spatial grouping |
| Learned merger | LayerNorm(1152), Linear(4608, 4608), GELU, Linear(4608, 5120) |
last_hidden_state contains the raw Tower tokens. pooler_output contains the merged
features at the language-model width.
Usage
See examples/inference.py for image feature extraction.
Validation
The release tests compare all 333 tensors with the pinned parent checkpoint using
torch.equal. Fixed-image Tower and merged outputs also match the parent implementation
bit-for-bit on CPU and in BF16 on an NVIDIA A100.
Reproduction
The export script
reads model.visual.* from shard 1 of Qwen/Qwen3.8-27B, removes the prefix, and writes
the original BF16 tensors. The script pins the parent revision.
Credits
Qwen released the Qwen3.8-27B weights and the native Transformers implementation.
License
Apache License 2.0, the same license as the source model.
Contributors
3 commits
Similar repos
Linked in READMEs
README
Qwen3.8-27B Vision
This repository packages the Tower and learned merger from Qwen3.8-27B.
Contents
| File | Tensors | What it holds |
|---|---|---|
model.safetensors | 333 | Tower and learned merger, extracted from source shard 1 |
config.json | Vision-only Qwen3_5VisionConfig | |
preprocessor_config.json | Qwen image preprocessing configuration |
Qwen implements the learned merger inside Qwen3_5VisionModel, so this repository does
not need a separate Projector file.
Architecture
| Component | Details |
|---|---|
| Tower | 27 layers, 1152 hidden, 16 heads, 4304 intermediate, patch size 16 |
| Token compression | 2x2 spatial grouping |
| Learned merger | LayerNorm(1152), Linear(4608, 4608), GELU, Linear(4608, 5120) |
last_hidden_state contains the raw Tower tokens. pooler_output contains the merged
features at the language-model width.
Usage
See examples/inference.py for image feature extraction.
Validation
The release tests compare all 333 tensors with the pinned parent checkpoint using
torch.equal. Fixed-image Tower and merged outputs also match the parent implementation
bit-for-bit on CPU and in BF16 on an NVIDIA A100.
Reproduction
The export script
reads model.visual.* from shard 1 of Qwen/Qwen3.8-27B, removes the prefix, and writes
the original BF16 tensors. The script pins the parent revision.
Credits
Qwen released the Qwen3.8-27B weights and the native Transformers implementation.
License
Apache License 2.0, the same license as the source model.
Similar repos
Linked in READMEs
Contributors
3 commits