Awesome-Continual-Learning-in-Generative-Models

✨ Motivation

The remarkable progress of generative models has equipped AI systems with human-level capabilities in content generation. Yet, their practical deployment is hindered by catastrophic forgetting—a fundamental issue where learning new tasks erases previously acquired knowledge. Despite growing interest, no comprehensive survey exists to systematically categorize and analyze continual learning methods for mainstream generative models (e.g., Large Language Models, Multimodal Large Language Models, Vision-Language Action Models and Diffusion Models). This work fills the gap by:

• Classifying​​ solutions into architecture-based, regularization-based, and replay-based paradigms, aligning with human-like memory mechanisms.
• Analyzing​​ task adaptations, benchmarks, and model backbones to reveal key insights.
• Prospecting​​ future directions for continual learning in generative models, paving the way for scalable and adaptable intelligence.

📰 News

• 2026.06: 🔥🔥🔥 Community Highlight: Check out MCITlib, an open-source framework for Multimodal Continual Instruction Tuning. It provides out-of-the-box training and evaluation pipelines for 10+ methods across both image and video modalities, fully compatible with 4 diverse base models.
• 2026.01: We have updated the repository to include relevant papers accepted to ICLR 2026. If you notice any omissions or have any questions, please feel free to open an issue!
• 2025.12: We have released MCITlib, the first complete open-source codebase providing benchmarks and methods for Multimodal Continual Instruction Tuning. The code is open sourced here.
• 2025.07: Check out our new work: "Federated Continual Instruction Tuning" (ICCV 2025). The code is open sourced here.
• 2025.07: We have updated recent public work on continual learning in generative models. If you notice any omissions, please feel free to contact us!
• 2025.06: We released our survey paper "A Comprehensive Survey on Continual Learning in Generative Models". Feel free to cite or open pull requests!
• 2025.06: We released a repository on continual learning in generative models, and a corresponding survey will be available soon.

📖 Framework

• Continual Learning in Large Language Model
• Continual Learning in Multimodal Large Language Model
• Continual Learning in Vision-Language Action Model
• Continual Learning in Diffusion Model

⚖️ Benchmarks for Continual Learning in Generative Models

Large Language Model

• SuperNI Benchmark [Paper]
• Long Sequence Benchmark [Paper]
• Standard CL Benchmark [Paper]

Multimodal Large Language Model

• MCITlib Benchmark [Paper]
• MLLM-CL Benchmark [Paper]
• UCIT Benchmark [Paper]
• CoIN Benchmark [Paper]
• UCo-VQA Benchmark [Paper] [Code]
• AndroidControl-CL / Android-CL Benchmark [Paper]

Vision-Language Action Model

• LIBERO [Paper]

Diffusion Model

• T2I-ConBench [Paper]

🔖 Continual Learning in Large Language Model

Architecture-based Approaches

Regularization-based Approaches

Replay-based Approaches

RL / RFT-based Approaches

👓 Continual Learning in Multimodal Large Language Model

Architecture-based Approaches

Regularization-based Approaches

Replay-based Approaches

Preference-Optimization-based Approaches

RL / RFT-based Approaches

Evaluation / Benchmark

Knowledge / Safety Retention

Video-Language Continual Learning

🤖 Continual Learning in Vision-Language Action Model

Architecture-based Approaches

Regularization-based Approaches

Replay-based Approaches

Lifecycle / System Framework

🖌️ Continual Learning in Diffusion Model

Architecture-based Approaches

Regularization-based Approaches

Replay-based Approaches

🌞 Citation

@article{guo2025comprehensive,
  title={A Comprehensive Survey on Continual Learning in Generative Models},
  author={Guo, Haiyang and Zeng, Fanhu and Zhu, Fei and Wang, Jiayi and Wang, Xukai and Zhou, Jingang and Zhao, Hongbo and Liu, Wenzhuo and Ma, Shijie and Zhang, Xu-Yao and others},
  journal={arXiv preprint arXiv:2506.13045},
  year={2025}
}