falonss703/Awesome-Uncertainty-based-Reinforcement-Learning

🔥🔥🔥Latest Papers, Codes on Uncertainty-based RL

25 commits

updated Aug 24, 2025

README

Awesome Uncertainty-based RL Papers

This repository targets researchers exploring recent advances in reinforcement learning that utilize uncertainty to improve the reasoning capabilities of large language models.

📋 Table of Contents

Awesome Uncertainty-based RL Papers

📜 History

Recent Updates

2025-08-24: Added "Agentic Reinforced Policy Optimization"
2025-08-19: Added "IBRO, GTPO, GRPO-S"
2025-08-12: Added "EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity"
2025-07-23: Added "Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR"
2025-07-01: Added "SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning"
2025-06-20: Added “Reasoning with Exploration: An Entropy Perspective”
2025-06-20: Added “Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning”
2025-06-18: Added “Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models”
2025-06-14: Repository made public with comprehensive collection of uncertainty-based RL papers

How is Uncertainty used in RL?

🎯 Uncertainty as an Optimization Objective

Title	Method	Date	Metric	Task Domain	Code	Venue
Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization	EMPO	04/2025	Semantic Entropy	Math & General	code	arXiv
SLOT: Sample-specific Language Model Optimization at Test-time	SLOT	05/2025	Token Entropy	Math	code	arXiv
The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning	EM-FT, EM-INF	05/2025	Trajectory/Token Entropy	Math & Code	code	arXiv
One-shot Entropy Minimization	EM	05/2025	Token Entropy	Math, Code, Logic	code	arXiv
Maximizing Confidence Alone Improves Reasoning	RENT	05/2025	Token Entropy	Math	project page	arXiv
Verbalized Confidence Triggers Self-Verification: Emergent Behavior Without Explicit Reasoning Supervision	CSFT	06/2025	Self-Confidence	Math	-	arXiv

💰 Uncertainty as a Reward Signal

Title	Method	Date	Metric	Task Domain	Code	Venue
The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning	EM-RL	05/2025	Trajectory/Token Entropy	Math & Code	code	arXiv
Learning to Reason without External Rewards	INTUITOR	05/2025	Self-Certainty	Math & Code	code	arXiv
Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models	RLSC	06/2025	Self-Confidence	Math	-	arXiv
Consistent Paths Lead to Truth Self-Rewarding Reinforcement Learning for LLM Reasoning	CoVo	06/2025	Trajectory Features	Math	code	arXiv

🧭 Uncertainty as a Optimization Guide

Title	Method	Date	Metric	Task Domain	Code	Venue
SEED-GRPO: Semantic entropy enhanced GRPO for uncertainty-aware policy optimization	SEED-GRPO	05/2025	Semantic Entropy	Math	-	arXiv
The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models	Clip-Cov, KL-Cov	05/2025	Policy Entropy	Math	code	arXiv
Reinforcing Video Reasoning with Focused Thinking	TW-GRPO	05/2025	Intra-Group Information Entropy	Video	code	arXiv
Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning	Forking Tokens	06/2025	Token Entropy	Math	project page	arXiv
Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning	MI peaks	06/2025	Mutual Information	Math	code	arXiv
Reasoning with Exploration: An Entropy Perspective	Entropy-Based Advantage Shaping	06/2025	Token Entropy	Math	-	arXiv
SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning	SRFT	06/2025	Token Entropy	Math	poject page	arXiv
Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR	Archer	07/2025	Token Entropy	Math & Code	code	arXiv
Revisiting LLM Reasoning via Information Bottleneck	IBRO	07/2025	Information Bottleneck	Math	-	arXiv
Agentic Reinforced Policy Optimization	ARPO	07/2025	Token Entropy	Math & General & Deep Search	code	arXiv
EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity	EDGE-GRPO	07/2025	Token Entropy	Math	code	arXiv
GTPO: Trajectory-Based Policy Optimization in Large Language Models	GTPO	08/2025	Token Entropy	Math	-	arXiv
GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy	GTPO, GRPO-S	08/2025	Token Entropy	Math	-	arXiv

Contributing

We welcome everyone to contribute to this repository and help improve it. You can submit pull requests to add new papers, projects, and helpful materials, or to correct any errors that you may find. Please make sure that your pull requests follow the format in the tables above. Thank you for your valuable contributions!

🌟 Star this repo if you find it helpful! 🌟

🌟 Star History

llm-reasoning

mllm-reasoning

rainforcement-learning

uncertainty-analysis

unsupervised-learning

Contributors

falonss703

24 commits

zitian-gao

1 commits

Similar repos

TsinghuaC3I/Awesome-RL-for-LRMs

A Survey of Reinforcement Learning for Large Reasoning Models

2,489

mbzuai-oryx/Awesome-LLM-Post-training

Awesome Reasoning LLM Tutorial/Survey/Guide

2,561

YsTvT/Awesome-Agentic-RL-Papers

107

QingyangZhang/Label-Free-RLVR

312

benjaminzwhite/reasoning-models

Experiments with reasoning models, training techniques, papers

blacksnail789521/Agentic-RL-Training-Recipes

Training Recipes for Agentic Reinforcement Learning in LLMs: A Survey

ventr1c/Awesome-RL-based-Agentic-Search-Papers

The official repository of "A Comprehensive Survey on Reinforcement Learning-based Agentic Search:…

292

HHHHHejia/Awesome-AgenticLLM-RL-Papers

1,899