Awesome Search RL

:star::star::star: If you find this repo useful, please star it!

🔥 for papers with >50 citations or repositories with >200 stars.
📖 for papers accepted by reputed conferences/journals.

✨ Search-RL Related

1. 🔥 Search-R1 Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning [Code] (12 Mar 2025)

   Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, Jiawei Han

2. 🔥 DeepResearcher DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments [Code] (4 Apr 2025)

   Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, Pengfei Liu

3. 🔥 R1-Searcher R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning [Code] (7 Mar 2025)

   Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, Ji-Rong Wen

4. Multimodal-Search-R1 Multimodal-Search-R1: Incentivizing LMMs to Search [Code]

   Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Zejun Ma

5. 🔥 ZeroSearch ZeroSearch: Incentivize the Search Capability of LLMs without Searching [Code] (7 May 2025)

   Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, Jingren Zhou

6. IKEA Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent [Code] (12 May 2025)

   Ziyang Huang, Xiaowei Yuan, Yiming Ju, Jun Zhao, Kang Liu

7. AutoRefine Search and Refine During Think: Autonomous Retrieval-Augmented Reasoning of LLMs [Code] (16 May 2025)

   Yaorui Shi, Sihang Li, Chang Wu, Zhiyuan Liu, Junfeng Fang, Hengxing Cai, An Zhang, Xiang Wang

8. O2-Searcher O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answering [Code] (22 May 2025)

   Jianbiao Mei, Tao Hu, Daocheng Fu, Licheng Wen, Xuemeng Yang, Rong Wu, Pinlong Cai, Xinyu Cai, Xing Gao, Yu Yang, Chengjun Xie, Botian Shi, Yong Liu, Yu Qiao

9. MaskSearch MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability [Code] (26 May 2025)

   Weiqi Wu, Xin Guan, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, Jiuxin Cao, Hai Zhao, Jingren Zhou

10. 🔥 WebDancer WebDancer: Towards Autonomous Information Seeking Agency [Code] (28 May 2025)

    Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou

⚒️ RL Related

1. 🔥 GRPO DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models [Code] (5 Feb 2024)

   Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y.K. Li, Y. Wu, Daya Guo

2. 🔥 DAPO DAPO: An Open-Source LLM Reinforcement Learning System at Scale [Code] (18 Mar 2025)

   Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, Mingxuan Wang

3. 🔥 DR.GRPO Understanding R1-Zero-Like Training: A Critical Perspective [Code] (26 Mar 2025)

   Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, Min Lin

4. GPG GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning [Code] (3 Apr 2025)

   Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, Yong Wang

5. EMPO EMPO: A Clustering-Based On-Policy Algorithm for Offline Reinforcement Learning

   Jongeui Park, Myungsik Cho, Youngchul Sung

6. 🔥 TTRL TTRL: Test-Time Reinforcement Learning [Code] (22 Apr 2025)

   Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biqing Qi, Youbang Sun, Zhiyuan Ma, Lifan Yuan, Ning Ding, Bowen Zhou

7. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model (18 Apr 2025)

   Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, Gao Huang

8. 🔥 One-shot-RL Reinforcement Learning for Reasoning in Large Language Models with One Training Example [Code] (29 Apr 2025)

   Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, Yelong Shen

9. ScPO Self-Consistency Preference Optimization (29 Apr 2025)

   Archiki Prasad, Weizhe Yuan, Richard Yuanzhe Pang, Jing Xu, Maryam Fazel-Zarandi, Mohit Bansal, Sainbayar Sukhbaatar, Jason Weston, Jane Yu

10. The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning (21 May 2025)

    Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng

↑ Back to Top ↑

🍺 Contributing

All issues and pull requests are warmly welcomed to contribute related papers to this curated list! Feel free to submit any relevant additions to help expand and enhance this collection.