CSU-JPG/Awesome-VLM-Reasoning

24 commits

updated May 19, 2025

README

Awesome VLM Reasoning

A carefully curated collection of papers and resources exploring methods to advance reasoning in Vision-Language Models (VLMs).

🗂️ Table of Contents

Multi-modality COT Reasoning
Benchmarks
Reinforce Learning
Datasets
Other Resources
Survey

Multi-modality COT Reasoning

Title	Venue	Date	Code	Star
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning	arXiv	2024-03-16	GitHub
Multimodal Chain-of-Thought Reasoning in Language Models	arXiv	2024-05-20	GitHub
Visual SKETCHPAD: Sketching as a Visual Chain of Thought for Multimodal Language Models	arXiv	2024-06-09	GitHub
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization	arXiv	2024-11-15	GitHub
Imagine while Reasoning in Space: Multimodal Visualization-of-Thought	arXiv	2025-01-07	-	-
Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search	arXiv	2025-02-02	GitHub
AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO	arXiv	2025-02-14	-	-
VOILA: EVALUATION OF MLLMS FOR PERCEPTUAL UNDERSTANDING AND ANALOGICAL REASONING	arXiv	2025-03-04	GitHub
GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation	arXiv	2025-04-10	GitHub
Kimi-VL Technical Report	arXiv	2025-04-15	GitHub
Accelerating Large Language Model Reasoning via Speculative Search	arXiv	2025-05-03	-	-

↑ Back to Top ↑

Benchmarks

Title	Venue	Date	Code	Star
MindGYM: Enhancing Vision-Language Models via Synthetic Self-Challenging Questions	arXiv	2025-03-12	GitHub
MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models	arXiv	2025-02-02	GitHub
VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity	arXiv	2025-03-14	ProjectPage	-
Graph-to-Vision: Multi-graph Understanding and Reasoning using Vision-Language Models	arXiv	2025-03-27	-	-
PointArena: Probing Multimodal Grounding Through Language-Guided Pointing	arXiv	2025-05-15	GitHub

↑ Back to Top ↑

Reinforce Learn

🔤Based On DPO

Title	Venue	Date	Code	Star
A General Theoretical Paradigm to Understand Learning from Human Preferences	arXiv	2023-11-22	-	-
UNA: UNIFYING ALIGNMENTS OF RLHF/PPO, DPOAND KTO BY A GENERALIZED IMPLICIT REWARDFUNCTION	arXiv	2024-09-30	-	-
Accelerated Preference Optimization for Large Language Model Alignment	arXiv	2024-10-08	-	-
AS SIMPLE AS FINE-TUNING: LLM ALIGNMENT VIABIDIRECTIONAL NEGATIVE FEEDBACK LOSS	arXiv	2024-10-25	GitHub
KTO: Model Alignment as Prospect Theoretic Optimization	arXiv	2024-11-19	GitHub
MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples	arXiv	2024-12-13	-	-
Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization	arXiv	2024-12-24	-	-
Gradient Imbalance in Direct Preference Optimization	arXiv	2025-02-28	-	-

🧠Based On PPO

Title	Venue	Date	Code	Star
Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward	arXiv	2024-11-22	-	-
Can We Generate Images with CoT? Let’s Verify and Reinforce Image Generation Step by Step	arXiv	2025-01-23	GitHub
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization	arXiv	2025-02-11	-	-
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown	arXiv	2025-02-12	-	-
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance	arXiv	2025-02-24	-	-
Q♯: Provably Optimal Distributional RL for LLM Post-Training	arXiv	2025-02-27	GitHub

🤏Based On RLHF

Title	Venue	Date	Code	Star
Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback	arXiv	2024-12-06	-	-
RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation	arXiv	2025-02-10	-	-

👀Based On SFT

Title	Venue	Date	Code	Star
Continual SFT Matches Multimodal RLHF with Negative Supervision	arXiv	2024-11-22	-	-
ORPO: Monolithic Preference Optimization without Reference Model	arXiv	2024-03-14	-	-

⚙️Based On GRPO

Title	Venue	Date	Code
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning	arXiv	2025-04-13	GitHub
TimeZero: Temporal Video Grounding with Reasoning-Guided LVLM	arXiv	2025-03-17	GitHub
Understanding R1-Zero-Like Training: A Critical Perspective	arXiv	2025-03-26	GitHub
DRA-GRPO: Exploring Diversity-Aware Reward Adjustment for R1-Zero-Like Training of Large Language Models	arXiv	2025-05-14	GitHub

🏆About Reward

Title	Venue	Date	Code	Star
R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning	arXiv	2025-05-05	GitHub

🔗Others-RL

Title	Venue	Date	Code	Star
Learning Future Representation with Synthetic Observations for Sample-efficient Reinforcement Learning	arXiv	2024-05-20	-	-
The Perfect Blend: Redefining RLHF with Mixture of Judges	arXiv	2024-09-30	-	-
Fast Best-of-N Decoding via Speculative Rejection	arXiv	2024-10-31	GitHub
Distributionally Robust Optimization	arXiv	2024-11-04	-	-
MM-EUREKA: EXPLORING VISUAL AHA MOMENT WITH RULE-BASED LARGE-SCALE REINFORCEMENTLEARNING	arXiv	2025-03-10	GitHub

↑ Back to Top ↑

Datasets

Name	Paper	Date	Download	Star
MMPR	Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization	2024-11-15	HuggingFace
V1: Toward Multimodal Reasoning by Designing Auxiliary Tasks	-	2025-03-17	HuggingFace

↑ Back to Top ↑

Other Resources

Title	Date	Code	Star
R1-V: Reinforcing Super Generalization Ability in Vision Language Models with Less Than $3	2025-02-03	GitHub

↑ Back to Top ↑

Survey

Title	Venue	Date	Code	Star
A COMPREHENSIVE SURVEY OF LLM ALIGNMENT TECHNIQUES:RLHF, RLAIF, PPO, DPO AND MORE	arXiv	2024-07-23	-	-
Personalized Generation In Large Model Era: A Survey	arXiv	2025-03-04	-	-
A Survey on Post-training of Large Language Models	arXiv	2025-03-08	-	-
Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models	arXiv	2025-03-13	GitHub
Multimodal Chain-of-Thought Reasoning:A Comprehensive Survey	arXiv	2025-03-16	GitHub
Awesome-Large-Multimodal-Reasoning-Models	arXiv	2025-05-08	GitHub

↑ Back to Top ↑

Contributors

Carrot0729

22 commits

dongxingMao

2 commits

Similar repos

NOVAglow646/LLM-MLLM-paper-list

关于LLM和Multimodal LLM的paper list

The-Martyr/Awesome-Multimodal-Reasoning

Latest Advances on (RL based) Multimodal Reasoning and Generation in Multimodal LLMs

103

lwpyh/Awesome-MLLM-Reasoning-Collection

A collection of multimodal reasoning papers, codes, datasets, benchmarks and resources.

opendilab/awesome-multi-modal-reinforcement-learning

A curated list of Multi-Modal Reinforcement Learning resources (continually updated)

620

yunlong10/Awesome-Video-LMM-Post-Training

🔥🔥🔥 Latest Papers, Codes and Datasets on Video-LMM Post-Training

305

HJYao00/Awesome-Agentic-MLLMs

Agentic MLLMs

215

SuperBruceJia/Awesome-Large-Vision-Language-Model

Awesome Large Vision-Language Model: A Curated List of Large Vision-Language Model

yaotingwangofficial/Awesome-MCoT

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

1,026

CSU-JPG/Awesome-VLM-Reasoning

24 commits

updated May 19, 2025

See the code

README

Awesome VLM Reasoning

A carefully curated collection of papers and resources exploring methods to advance reasoning in Vision-Language Models (VLMs).

🗂️ Table of Contents

Multi-modality COT Reasoning
Benchmarks
Reinforce Learning
Datasets
Other Resources
Survey

Multi-modality COT Reasoning

Title	Venue	Date	Code	Star
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning	arXiv	2024-03-16	GitHub
Multimodal Chain-of-Thought Reasoning in Language Models	arXiv	2024-05-20	GitHub
Visual SKETCHPAD: Sketching as a Visual Chain of Thought for Multimodal Language Models	arXiv	2024-06-09	GitHub
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization	arXiv	2024-11-15	GitHub
Imagine while Reasoning in Space: Multimodal Visualization-of-Thought	arXiv	2025-01-07	-	-
Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search	arXiv	2025-02-02	GitHub
AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO	arXiv	2025-02-14	-	-
VOILA: EVALUATION OF MLLMS FOR PERCEPTUAL UNDERSTANDING AND ANALOGICAL REASONING	arXiv	2025-03-04	GitHub
GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation	arXiv	2025-04-10	GitHub
Kimi-VL Technical Report	arXiv	2025-04-15	GitHub
Accelerating Large Language Model Reasoning via Speculative Search	arXiv	2025-05-03	-	-

↑ Back to Top ↑

Benchmarks

Title	Venue	Date	Code	Star
MindGYM: Enhancing Vision-Language Models via Synthetic Self-Challenging Questions	arXiv	2025-03-12	GitHub
MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models	arXiv	2025-02-02	GitHub
VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity	arXiv	2025-03-14	ProjectPage	-
Graph-to-Vision: Multi-graph Understanding and Reasoning using Vision-Language Models	arXiv	2025-03-27	-	-
PointArena: Probing Multimodal Grounding Through Language-Guided Pointing	arXiv	2025-05-15	GitHub

↑ Back to Top ↑

Reinforce Learn

🔤Based On DPO

Title	Venue	Date	Code	Star
A General Theoretical Paradigm to Understand Learning from Human Preferences	arXiv	2023-11-22	-	-
UNA: UNIFYING ALIGNMENTS OF RLHF/PPO, DPOAND KTO BY A GENERALIZED IMPLICIT REWARDFUNCTION	arXiv	2024-09-30	-	-
Accelerated Preference Optimization for Large Language Model Alignment	arXiv	2024-10-08	-	-
AS SIMPLE AS FINE-TUNING: LLM ALIGNMENT VIABIDIRECTIONAL NEGATIVE FEEDBACK LOSS	arXiv	2024-10-25	GitHub
KTO: Model Alignment as Prospect Theoretic Optimization	arXiv	2024-11-19	GitHub
MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples	arXiv	2024-12-13	-	-
Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization	arXiv	2024-12-24	-	-
Gradient Imbalance in Direct Preference Optimization	arXiv	2025-02-28	-	-

🧠Based On PPO

Title	Venue	Date	Code	Star
Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward	arXiv	2024-11-22	-	-
Can We Generate Images with CoT? Let’s Verify and Reinforce Image Generation Step by Step	arXiv	2025-01-23	GitHub
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization	arXiv	2025-02-11	-	-
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown	arXiv	2025-02-12	-	-
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance	arXiv	2025-02-24	-	-
Q♯: Provably Optimal Distributional RL for LLM Post-Training	arXiv	2025-02-27	GitHub

🤏Based On RLHF

Title	Venue	Date	Code	Star
Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback	arXiv	2024-12-06	-	-
RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation	arXiv	2025-02-10	-	-

👀Based On SFT

Title	Venue	Date	Code	Star
Continual SFT Matches Multimodal RLHF with Negative Supervision	arXiv	2024-11-22	-	-
ORPO: Monolithic Preference Optimization without Reference Model	arXiv	2024-03-14	-	-

⚙️Based On GRPO

Title	Venue	Date	Code
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning	arXiv	2025-04-13	GitHub
TimeZero: Temporal Video Grounding with Reasoning-Guided LVLM	arXiv	2025-03-17	GitHub
Understanding R1-Zero-Like Training: A Critical Perspective	arXiv	2025-03-26	GitHub
DRA-GRPO: Exploring Diversity-Aware Reward Adjustment for R1-Zero-Like Training of Large Language Models	arXiv	2025-05-14	GitHub

🏆About Reward

Title	Venue	Date	Code	Star
R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning	arXiv	2025-05-05	GitHub

🔗Others-RL

Title	Venue	Date	Code	Star
Learning Future Representation with Synthetic Observations for Sample-efficient Reinforcement Learning	arXiv	2024-05-20	-	-
The Perfect Blend: Redefining RLHF with Mixture of Judges	arXiv	2024-09-30	-	-
Fast Best-of-N Decoding via Speculative Rejection	arXiv	2024-10-31	GitHub
Distributionally Robust Optimization	arXiv	2024-11-04	-	-
MM-EUREKA: EXPLORING VISUAL AHA MOMENT WITH RULE-BASED LARGE-SCALE REINFORCEMENTLEARNING	arXiv	2025-03-10	GitHub

↑ Back to Top ↑

Datasets

Name	Paper	Date	Download	Star
MMPR	Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization	2024-11-15	HuggingFace
V1: Toward Multimodal Reasoning by Designing Auxiliary Tasks	-	2025-03-17	HuggingFace

↑ Back to Top ↑

Other Resources

Title	Date	Code	Star
R1-V: Reinforcing Super Generalization Ability in Vision Language Models with Less Than $3	2025-02-03	GitHub

↑ Back to Top ↑

Survey

Title	Venue	Date	Code	Star
A COMPREHENSIVE SURVEY OF LLM ALIGNMENT TECHNIQUES:RLHF, RLAIF, PPO, DPO AND MORE	arXiv	2024-07-23	-	-
Personalized Generation In Large Model Era: A Survey	arXiv	2025-03-04	-	-
A Survey on Post-training of Large Language Models	arXiv	2025-03-08	-	-
Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models	arXiv	2025-03-13	GitHub
Multimodal Chain-of-Thought Reasoning:A Comprehensive Survey	arXiv	2025-03-16	GitHub
Awesome-Large-Multimodal-Reasoning-Models	arXiv	2025-05-08	GitHub

↑ Back to Top ↑

Similar repos

NOVAglow646/LLM-MLLM-paper-list

关于LLM和Multimodal LLM的paper list

The-Martyr/Awesome-Multimodal-Reasoning

Latest Advances on (RL based) Multimodal Reasoning and Generation in Multimodal LLMs

103

lwpyh/Awesome-MLLM-Reasoning-Collection

A collection of multimodal reasoning papers, codes, datasets, benchmarks and resources.

opendilab/awesome-multi-modal-reinforcement-learning

A curated list of Multi-Modal Reinforcement Learning resources (continually updated)

620

yunlong10/Awesome-Video-LMM-Post-Training

🔥🔥🔥 Latest Papers, Codes and Datasets on Video-LMM Post-Training

305

HJYao00/Awesome-Agentic-MLLMs

Agentic MLLMs

215

SuperBruceJia/Awesome-Large-Vision-Language-Model

Awesome Large Vision-Language Model: A Curated List of Large Vision-Language Model

yaotingwangofficial/Awesome-MCoT

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

1,026

Contributors

Carrot0729

22 commits

dongxingMao

2 commits