README
UniTok: A Unified Tokenizer for Visual Generation and Understanding
This repository contains UniTok, a unified visual tokenizer for both image generation and understanding tasks, as presented in UniTok: A Unified Tokenizer for Visual Generation and Understanding.
Project Page: https://foundationvision.github.io/UniTok/
Code: https://github.com/FoundationVision/UniTok
UniTok encodes fine-grained details for generation and captures high-level semantics for understanding. It's compatible with autoregressive generative models (e.g., LlamaGen), multimodal understanding models (e.g., LLaVA), and unified MLLMs (e.g., Chameleon and Liquid).
Built upon UniTok, we construct an MLLM capable of both multimodal generation and understanding, which sets a new state-of-the-art among unified autoregressive MLLMs. The weights of our MLLM will be released soon.
Performance
| Method | #Tokens | rFID ↓ | Accuracy |
|---|---|---|---|
| VQVAE Model | |||
| VQ-GAN | 256 | 4.98 | -- |
| RQ-VAE | 256 | 1.30 | -- |
| VAR | 680 | 0.90 | -- |
| CLIP Model | |||
| CLIP | 256 | -- | 76.2 |
| SigLIP | 256 | -- | 80.5 |
| ViTamin | 256 | -- | 81.2 |
| Unified Model | |||
| TokenFlow † | 680 | 1.37 | -- |
| VILA-U † | 256 | 1.80 | 73.3 |
| UniTok | 256 | 0.39 | 70.5 |
| UniTok † | 256 | 0.38 | 78.6 |
This repo is used for hosting UniTok's checkpoints.
For more details or tutorials see https://github.com/FoundationVision/UniTok.
Citation
@article{unitok,
title={UniTok: A Unified Tokenizer for Visual Generation and Understanding},
author={Ma, Chuofan and Jiang, Yi and Wu, Junfeng and Yang, Jihan and Yu, Xin and Yuan, Zehuan and Peng, Bingyue and Qi, Xiaojuan},
journal={arXiv preprint arXiv:2502.20321},
year={2025}
}
Similar repos
README
UniTok: A Unified Tokenizer for Visual Generation and Understanding
This repository contains UniTok, a unified visual tokenizer for both image generation and understanding tasks, as presented in UniTok: A Unified Tokenizer for Visual Generation and Understanding.
Project Page: https://foundationvision.github.io/UniTok/
Code: https://github.com/FoundationVision/UniTok
UniTok encodes fine-grained details for generation and captures high-level semantics for understanding. It's compatible with autoregressive generative models (e.g., LlamaGen), multimodal understanding models (e.g., LLaVA), and unified MLLMs (e.g., Chameleon and Liquid).
Built upon UniTok, we construct an MLLM capable of both multimodal generation and understanding, which sets a new state-of-the-art among unified autoregressive MLLMs. The weights of our MLLM will be released soon.
Performance
| Method | #Tokens | rFID ↓ | Accuracy |
|---|---|---|---|
| VQVAE Model | |||
| VQ-GAN | 256 | 4.98 | -- |
| RQ-VAE | 256 | 1.30 | -- |
| VAR | 680 | 0.90 | -- |
| CLIP Model | |||
| CLIP | 256 | -- | 76.2 |
| SigLIP | 256 | -- | 80.5 |
| ViTamin | 256 | -- | 81.2 |
| Unified Model | |||
| TokenFlow † | 680 | 1.37 | -- |
| VILA-U † | 256 | 1.80 | 73.3 |
| UniTok | 256 | 0.39 | 70.5 |
| UniTok † | 256 | 0.38 | 78.6 |
This repo is used for hosting UniTok's checkpoints.
For more details or tutorials see https://github.com/FoundationVision/UniTok.
Citation
@article{unitok,
title={UniTok: A Unified Tokenizer for Visual Generation and Understanding},
author={Ma, Chuofan and Jiang, Yi and Wu, Junfeng and Yang, Jihan and Yu, Xin and Yuan, Zehuan and Peng, Bingyue and Qi, Xiaojuan},
journal={arXiv preprint arXiv:2502.20321},
year={2025}
}