MIV-XJTU/FLAME

[CVPR 2025] PyTorch implementation of paper "FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training"

stars

commits

Python

primary language

Jul 8, 2025

updated

README

CVPR 2025 | FLAME

FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training
Anjia Cao, Xing Wei, Zhiheng Ma

📰 News

[2025/03/09] Release training codes and scripts.
[2025/03/08] Recaptioned CC3M and Recaptioned YFCC15M on Hugging Face.
[2025/02/27] Accepted by CVPR 2025.
[2024/11/28] Model on Hugging Face.
[2024/11/28] Release evaluation codes and scripts.
[2024/11/18] Paper on arXiv.

💡 Highlights

🔥 Leveraging frozen LLMs to naturally process long text inputs.
🔥 Generalizing from monolingual training to multilingual evaluation.
🔥 Strong improvement on long/short-context image-text retrieval, image classification, and multilingual scenarios.

📅 TODO Roadmap

Release training code and data.
Release evaluation code.
Release pre-trained checkpoints.

🛠️ Get Started

Setup

git clone https://github.com/MIV-XJTU/FLAME.git
cd FLAME
conda create -n flame python=3.10 -y
conda activate flame
make install
make install-training
make install-test

Training

See Training.md.

Evaluation

See Evaluation.md.

📁 Datasets

Dataset	Link
CC3M-ReCap	Hugging Face
YFCC15M-ReCap	Hugging Face

🔐 Pre-trained Checkpoints

Dataset	Model	Link
CC3M	Mistral-Nemo-ViT-B/16	Hugging Face

🛂 License

The project is under a standard Creative Common CC-BY-4.0 License.

📖 Citation

If you find our work helpful for your research, please consider giving a star and citation.

@inproceedings{cao2025flame,
  title={FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training},
  author={Cao, Anjia and Wei, Xing and Ma, Zhiheng},
  booktitle={CVPR},
  year={2025}
}

🫡 Acknowledgements

This project is based on open_clip, and thanks for the nice work! We also thank CLIP_benchmark, DreamLIP, Long-CLIP, PromptEOL, and MiniCPM-V for their codes.

Contributors

CAOANJIA

24 commits

Similar repos

caj/FLAME-ReCap-YFCC15M-MiniCPM-Llama3-V-2_5

Dataset

caj/FLAME-Mistral-Nemo-ViT-B-16-CC3M

Model

caj/FLAME-ReCap-CC3M-MiniCPM-Llama3-V-2_5

Dataset

mlfoundations/open_flamingo

An open-source framework for training large multimodal models.

4,114

wusize/F-LMM

[CVPR2025] Code Release of F-LMM: Grounding Frozen Large Multimodal Models

115

geyaokai/F-LMM

chs20/fuselip

FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens

TRI-ML/prismatic-vlms

A flexible and efficient codebase for training visually-conditioned language models (VLMs)

1,016

Languages

Python

99.9%

MIV-XJTU/FLAME

[CVPR 2025] PyTorch implementation of paper "FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training"

stars

commits

Python

primary language

Jul 8, 2025

updated

README

CVPR 2025 | FLAME

FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training
Anjia Cao, Xing Wei, Zhiheng Ma

📰 News

[2025/03/09] Release training codes and scripts.
[2025/03/08] Recaptioned CC3M and Recaptioned YFCC15M on Hugging Face.
[2025/02/27] Accepted by CVPR 2025.
[2024/11/28] Model on Hugging Face.
[2024/11/28] Release evaluation codes and scripts.
[2024/11/18] Paper on arXiv.

💡 Highlights

🔥 Leveraging frozen LLMs to naturally process long text inputs.
🔥 Generalizing from monolingual training to multilingual evaluation.
🔥 Strong improvement on long/short-context image-text retrieval, image classification, and multilingual scenarios.

📅 TODO Roadmap

Release training code and data.
Release evaluation code.
Release pre-trained checkpoints.

🛠️ Get Started

Setup

git clone https://github.com/MIV-XJTU/FLAME.git
cd FLAME
conda create -n flame python=3.10 -y
conda activate flame
make install
make install-training
make install-test

Training

See Training.md.

Evaluation

See Evaluation.md.

📁 Datasets

Dataset	Link
CC3M-ReCap	Hugging Face
YFCC15M-ReCap	Hugging Face

🔐 Pre-trained Checkpoints

Dataset	Model	Link
CC3M	Mistral-Nemo-ViT-B/16	Hugging Face

🛂 License

The project is under a standard Creative Common CC-BY-4.0 License.

📖 Citation

If you find our work helpful for your research, please consider giving a star and citation.

@inproceedings{cao2025flame,
  title={FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training},
  author={Cao, Anjia and Wei, Xing and Ma, Zhiheng},
  booktitle={CVPR},
  year={2025}
}

🫡 Acknowledgements

This project is based on open_clip, and thanks for the nice work! We also thank CLIP_benchmark, DreamLIP, Long-CLIP, PromptEOL, and MiniCPM-V for their codes.