🚀 Introduction

• This repository provides a curated collection of papers and resources on the intersection of Large Language Models (LLMs) and Multi-Armed Bandits (MABs). As modern LLMs grow more capable, bandit algorithms offer efficient tools for improving prompt design, training, inference optimization, and personalization. Conversely, LLMs enable more expressive and context-aware bandit decision-making.

• Our goal is to give researchers a clear entry point into this emerging area, covering both LLM-enhanced bandits and bandit-enhanced LLMs, along with key insights, methods, and open challenges.

• We welcome any contributions and suggestions to our repository or the addition of your own work. Feel free to make a pull request or leave your comments!!

📋 Contents

• 🚀 Introduction

• 📋 Contents

• 💘 Tips

• 🔍 Search Strategy

• 📍 Bandit-enhancements for LLMs

  • Pre-training
  • Fine-tuning
  • Alignment
  • Prompt Design and Selection
  • Tool and Function Calling
  • Contextual Understanding
  • Retrieval-Augmented Generation
  • Inference Optimization
  • Decoding Strategies
  • Adaptation and Personalizing

• 📍 LLM-enhancements for Multi-armed Bandits

  • Regret Minimization Objective
  • Arm Definition
  • Environment
  • Reward Formulation
  • Sampling Strategy
  • Action Decision

• 👨‍💻 Team

• 😉 Citation

💘 Tips

• ✅ Paper searching via catatogue: directly clicking the content of the catatogue to select the area of your research and browse related papers.
• ✅ Paper searching via author name: Free feel to search papers of a specific author via ctrl + F and then type the author name. The dropdown list of authors will automatically expand when searching.
• ✅ Paper searching via tag: You can also search the related papers via the following tags: customization, iteractive, human motion generation tokenizer. (More tags are ongoing)

🔍 Search Strategy

This repository is curated following a systematic methodology to ensure comprehensive and reproducible coverage of research at the intersection of Large Language Models (LLMs) and Bandit Algorithms.

1. Search Framework

We adopted the PCC (Population, Concept, Context) framework as recommended in 'The Systematic Review: An Overview' by Aromataris & Pearson (2014) to structure our search strings.

2. Search Queries

Our search strategy is organized into four hierarchical layers to balance Sensitivity (finding all relevant papers) and Specificity (filtering out noise).

A. Core Intersection (High-level)

Broad terms used to identify foundational literature at the cross-section.

1. "Large Language Model" AND "Bandit"
2. "LLM" AND "Multi-armed Bandit"
3. "Generative AI" AND "Contextual Bandit"
4. "Foundation Model" AND "Sequential Decision Making"
5. "Autoregressive Model" AND "Online Learning"

B. LLM Component-Based Search

Focusing on where Bandit algorithms enhance specific LLM stages.

6. Pre-training: ("Pre-training" AND ("Bandit" OR "Exploration"))

7. Fine-tuning: ("Fine-tuning" AND ("Bandit" OR "Online optimization"))

8. Alignment: (("Alignment" OR "RLHF" OR "Preference learning") AND "Bandit")

9. Prompt Design and Selection: (("Prompt selection" OR "Prompt optimization" OR "In-context learning") AND "Bandit")

10. Tool and Function Calling: (("Tool calling" OR "Function calling" OR "Action invocation") AND "Bandit")

11. Context Understanding: (("Context management" OR "Long context" OR "Context window") AND "Bandit")

12. Retrieval-Augmented Generation (RAG): (("Retrieval-augmented generation" OR "RAG") AND "Bandit")

13. Inference Optimization: (("Inference optimization" OR "Resource allocation") AND "Bandit")

14. Decoding Strategies: (("Decoding strategy" OR "Sampling strategy" OR "Generation control") AND "Bandit")

15. Adaptation and Personalization: (("Personalization" OR "Model adaptation" OR "User preference modeling") AND "Bandit")

C. Bandit Component-Based Search

Focusing on how LLMs are integrated into Bandit framework elements.

16. Regret Minimization Objective: (("Regret minimization" OR "Reward maximization") AND "LLM")

17. Arm Definition: (("Action space" OR "Arm representation") AND "LLM")

18. Environment Modeling: (("Environment modeling" OR "Dynamic environment" OR "Simulator") AND "LLM")

19. Reward Formulation: (("Reward modeling" OR "Reward shaping") AND "LLM")

20. Sampling Strategy: (("Thompson Sampling" OR "UCB" OR "Exploration strategy") AND "LLM")

21. Action Decision: (("Action selection" OR "Decision making") AND "LLM")

D. Specialized & Emerging Interaction Terms

Capturing specific techniques and interdisciplinary applications.

22. "RLHF" AND "Bandit"
23. "Combinatorial Bandit" AND "Text Generation"
24. "Neural Bandit" AND "Transformer"
25. "Bayesian Optimization" AND "LLM"
26. "Active Learning" AND "LLM" AND "Bandit"
27. "Interactive NLP" AND "Bandit"
28. "Resource allocation" AND "LLM inference" AND "Bandit"
29. "Query selection" AND "RAG" AND "Bandit"
30. "Automated prompt engineering" AND "Multi-armed Bandit"
31. "Speculative decoding" AND "Multi-armed Bandit"
32. "User preference modeling" AND "LLM" AND "Bandit"

📍 Bandit-enhancements for LLMs

Pre-training

• Multi-armed bandits for resource efficient, online optimization of language model pre-training: the use case of dynamic masking (2022)

  Inigo Urteaga, Moulay Zaidane Draidia, Tomer Lancewicki, et al.

  Inigo Urteaga, Moulay Zaidane Draidia, Tomer Lancewicki, Shahram Khadivi
  

• Efficient online data mixing for language model pre-training (2023)

  Alon Albalak, Liangming Pan, Colin Raffel, et al.

  Alon Albalak, Liangming Pan, Colin Raffel, William Yang Wang
  

• Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning (2024)

  Subhojyoti Mukherjee, Josiah P Hanna, Qiaomin Xie, et al.

  Subhojyoti Mukherjee, Josiah P Hanna, Qiaomin Xie, Robert Nowak
  

• Harnessing Diversity for Important Data Selection in Pretraining Large Language Models (2024)

  Chi Zhang, Huaping Zhong, Kuan Zhang, et al.

  Chi Zhang, Huaping Zhong, Kuan Zhang, Chengliang Chai, Rui Wang, Xinlin Zhuang, Tianyi Bai, Jiantao Qiu, Lei Cao, Ye Yuan, others
  

• Actor-Critic based Online Data Mixing For Language Model Pre-Training (2025)

  Jing Ma, Chenhao Dang, Mingjie Liao

  Jing Ma, Chenhao Dang, Mingjie Liao
  

• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (2025)

  Allen Nie, Yi Su, Bo Chang, et al.

  Allen Nie, Yi Su, Bo Chang, Jonathan Lee, Ed H. Chi, Quoc V Le, Minmin Chen
  

Fine-tuning

• RL-NMT: Reinforcement Learning Fine-tuning for Improved Neural Machine Translation of Burmese Dialects (2023)

  Ye Kyaw Thu, Thazin Myint Oo, Thepchai Supnithi

  Ye Kyaw Thu, Thazin Myint Oo, Thepchai Supnithi
  

• Reflect-RL: Two-Player Online RL Fine-Tuning for LMs (2024)

  Runlong Zhou, Simon S Du, Beibin Li

  Runlong Zhou, Simon S Du, Beibin Li
  

• Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts (2024)

  Tong Zhu, Daize Dong, Xiaoye Qu, et al.

  Tong Zhu, Daize Dong, Xiaoye Qu, Jiacheng Ruan, Wenliang Chen, Yu Cheng
  

• Iterative data smoothing: Mitigating reward overfitting and overoptimization in rlhf (2024)

  Banghua Zhu, Michael I Jordan, Jiantao Jiao

  Banghua Zhu, Michael I Jordan, Jiantao Jiao
  

• Sharp Analysis for KL-Regularized Contextual Bandits and RLHF (Unknown Year)

  Heyang Zhao, Chenlu Ye, Quanquan Gu, et al.

  Heyang Zhao, Chenlu Ye, Quanquan Gu, Tong Zhang
  

• Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Bandits (2024)

  Yu Xia, Fang Kong, Tong Yu, et al.

  Yu Xia, Fang Kong, Tong Yu, Liya Guo, Ryan A Rossi, Sungchul Kim, Shuai Li
  

• Convergence-aware online model selection with time-increasing bandits (2024)

  Yu Xia, Fang Kong, Tong Yu, et al.

  Yu Xia, Fang Kong, Tong Yu, Liya Guo, Ryan A Rossi, Sungchul Kim, Shuai Li
  

• Preference fine-tuning of LLMs should leverage suboptimal, on-policy data (2024)

  Fahim Tajwar, Anikait Singh, Archit Sharma, et al.

  Fahim Tajwar, Anikait Singh, Archit Sharma, Rafael Rafailov, Jeff Schneider, Tengyang Xie, Stefano Ermon, Chelsea Finn, Aviral Kumar
  

• Sample-efficient alignment for llms (2024)

  Zichen Liu, Changyu Chen, Chao Du, et al.

  Zichen Liu, Changyu Chen, Chao Du, Wee Sun Lee, Min Lin
  

• Chunks as arms: Multi-armed bandit-guided sampling for long-context llm preference optimization (2025)

  Shaohua Duan, Xinze Li, Zhenghao Liu, et al.

  Shaohua Duan, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
  

• DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections (2025)

  Haebin Shin, Lei Ji, Xiao Liu, et al.

  Haebin Shin, Lei Ji, Xiao Liu, Zhiwei Yu, Qi Chen, Yeyun Gong
  

Alignment

• Nash Learning from Human Feedback (2024)

  Remi Munos, Michal Valko, Daniele Calandriello, et al.

  Remi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Côme Fiegel, Andrea Michi, Marco Selvi, Sertan Girgin, Nikola Momchev, Olivier Bachem, Daniel J Mankowitz, Doina Precup, Bilal Piot
  

• Training a helpful and harmless assistant with reinforcement learning from human feedback (2022)

  Yuntao Bai, Andy Jones, Kamal Ndousse, et al.

  Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al.
  

• Aligning large language models with human: A survey (2023)

  Yufei Wang, Wanjun Zhong, Liangyou Li, et al.

  Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, Qun Liu
  

• Training language models to follow instructions with human feedback (2022)

  Long Ouyang, Jeffrey Wu, Xu Jiang, et al.

  Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, Ryan Lowe
  

• Tokenized Bandit for LLM Decoding and Alignment (2025)

  Suho Shin, Chenghao Yang, Haifeng Xu, et al.

  Suho Shin, Chenghao Yang, Haifeng Xu, Mohammad T Hajiaghayi
  

• Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023)

  Rafael Rafailov, Archit Sharma, Eric Mitchell, et al.

  Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, Chelsea Finn
  

Prompt Design and Selection

• Prompt Optimization with Human Feedback (2024)

  Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, et al.

  Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low
  

• Best arm identification for prompt learning under a limited budget (2024)

  Chengshuai Shi, Kun Yang, Jing Yang, et al.

  Chengshuai Shi, Kun Yang, Jing Yang, Cong Shen
  

• Prompt optimization with EASE? efficient ordering-aware automated selection of exemplars (2024)

  Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, et al.

  Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low
  

• Prompt Optimization with Logged Bandit Data (2024)

  Haruka Kiyohara, Yuta Saito, Daniel Yiming Cao, et al.

  Haruka Kiyohara, Yuta Saito, Daniel Yiming Cao, Thorsten Joachims
  

• Prompt-based Code Completion via Multi-Retrieval Augmented Generation (2024)

  Hanzhuo Tan, Qi Luo, Ling Jiang, et al.

  Hanzhuo Tan, Qi Luo, Ling Jiang, Zizheng Zhan, Jing Li, Haotian Zhang, Yuqun Zhang
  

• Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers (2024)

  Xiaoqiang Lin, Zhaoxuan Wu, Zhongxiang Dai, et al.

  Xiaoqiang Lin, Zhaoxuan Wu, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low
  

• Efficient prompt optimization through the lens of best arm identification (2024)

  Chengshuai Shi, Kun Yang, Zihan Chen, et al.

  Chengshuai Shi, Kun Yang, Zihan Chen, Jundong Li, Jing Yang, Cong Shen
  

• FedPOB: Sample-Efficient Federated Prompt Optimization via Bandits (2025)

  Pingchen Lu, Zhi Hong, Zhiwei Shang, et al.

  Pingchen Lu, Zhi Hong, Zhiwei Shang, Zhiyong Wang, Yikun Ban, Yao Shu, Min Zhang, Shuang Qiu, Zhongxiang Dai
  

• Prompt Tuning Decision Transformers with Structured and Scalable Bandits (2025)

  Finn Rietz, Oleg Smirnov, Sara Karimi, et al.

  Finn Rietz, Oleg Smirnov, Sara Karimi, Lele Cao
  

• Meta-prompt optimization for llm-based sequential decision making (2025)

  Mingze Kong, Zhiyong Wang, Yao Shu, et al.

  Mingze Kong, Zhiyong Wang, Yao Shu, Zhongxiang Dai
  

• Bandit-Based Prompt Design Strategy Selection Improves Prompt Optimizers (2025)

  Rin Ashizawa, Yoichi Hirose, Nozomu Yoshinari, et al.

  Rin Ashizawa, Yoichi Hirose, Nozomu Yoshinari, Kento Uchida, Shinichi Shirakawa
  

• Prompt Optimization with Logged Bandit Data (2025)

  Haruka Kiyohara, Daniel Yiming Cao, Yuta Saito, et al.

  Haruka Kiyohara, Daniel Yiming Cao, Yuta Saito, Thorsten Joachims
  

• LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization (2025)

  Yuanchen Wu, Saurabh Verma, Justin Lee, et al.

  Yuanchen Wu, Saurabh Verma, Justin Lee, Fangzhou Xiong, Poppy Zhang, Amel Awadelkarim, Xu Chen, Yubai Yuan, Shawndra Hill
  

Tool and Function Calling

• Reinforcing multi-turn reasoning in llm agents via turn-level credit assignment (2025)

  Siliang Zeng, Quan Wei, William Brown, et al.

  Siliang Zeng, Quan Wei, William Brown, Oana Frunza, Yuriy Nevmyvaka, Yang Katie Zhao, Mingyi Hong
  

• Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees (2024)

  Sijia Chen, Yibo Wang, Yi-Feng Wu, et al.

  Sijia Chen, Yibo Wang, Yi-Feng Wu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang
  

• Semantic Context for Tool Orchestration (2025)

  Robert Müller

  Robert Müller
  

• Learning Evolving Tools for Large Language Models (2025)

  Guoxin Chen, Zhong Zhang, Xin Cong, et al.

  Guoxin Chen, Zhong Zhang, Xin Cong, Fangda Guo, Yesai Wu, Yankai Lin, Wenzheng Feng, Yasheng Wang
  

• Verltool: Towards holistic agentic reinforcement learning with tool use (2025)

  Dongfu Jiang, Yi Lu, Zhuofeng Li, et al.

  Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, Ping Nie, Haozhe Wang, Alex Su, Hui Chen, Kai Zou, Chao Du, et al.
  

• AgentEvolver: Towards Efficient Self-Evolving Agent System (2025)

  Yunpeng Zhai, Shuchang Tao, Cheng Chen, et al.

  Yunpeng Zhai, Shuchang Tao, Cheng Chen, Anni Zou, Ziqian Chen, Qingxu Fu, Shinji Mai, Li Yu, Jiaji Deng, Zouying Cao, et al.
  

Contextual Understanding

• Can large language models explore in-context? (2024)

  Akshay Krishnamurthy, Keegan Harris, Dylan J Foster, et al.

  Akshay Krishnamurthy, Keegan Harris, Dylan J Foster, Cyril Zhang, Aleksandrs Slivkins
  

• Code Repair with LLMs gives an Exploration-Exploitation Tradeoff (2024)

  Hao Tang, Keya Hu, Jin Peng Zhou, et al.

  Hao Tang, Keya Hu, Jin Peng Zhou, Sicheng Zhong, Wei-Long Zheng, Xujie Si, Kevin Ellis
  

• Efficient exploration for llms (2024)

  Vikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao, et al.

  Vikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao, Benjamin Van Roy
  

• Enhancing Sequential Recommendations through Multi-Perspective Reflections and Iteration (2024)

  Weicong Qin, Yi Xu, Weijie Yu, et al.

  Weicong Qin, Yi Xu, Weijie Yu, Chenglei Shen, Xiao Zhang, Ming He, Jianping Fan, Jun Xu
  

• Controlling Large Language Model Agents with Entropic Activation Steering (2024)

  Nate Rahn, Pierluca D'Oro, Marc G Bellemare

  Nate Rahn, Pierluca D'Oro, Marc G Bellemare
  

• Chunks as arms: Multi-armed bandit-guided sampling for long-context llm preference optimization (2025)

  Shaohua Duan, Xinze Li, Zhenghao Liu, et al.

  Shaohua Duan, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
  

• Efficient jailbreak attack sequences on large language models via multi-armed bandit-based context switching (2025)

  Aditya Ramesh, Shivam Bhardwaj, Aditya Saibewar, et al.

  Aditya Ramesh, Shivam Bhardwaj, Aditya Saibewar, Manohar Kaul
  

• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (2025)

  Allen Nie, Yi Su, Bo Chang, et al.

  Allen Nie, Yi Su, Bo Chang, Jonathan Lee, Ed H. Chi, Quoc V Le, Minmin Chen
  

• Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolution (2025)

  Manhin Poon, XiangXiang Dai, Xutong Liu, et al.

  Manhin Poon, XiangXiang Dai, Xutong Liu, Fang Kong, John Lui, Jinhang Zuo
  

• Context Attribution with Multi-Armed Bandit Optimization (2025)

  Deng Pan, Keerthiram Murugesan, Nuno Moniz, et al.

  Deng Pan, Keerthiram Murugesan, Nuno Moniz, Nitesh Chawla
  

Retrieval-Augmented Generation

• M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions (2024)

  Zheng Wang, Shu Xian Teo, Jieer Ouyang, et al.

  Zheng Wang, Shu Xian Teo, Jieer Ouyang, Yongjun Xu, Wei Shi
  

• AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generation (2024)

  Jia Fu, Xiaoting Qin, Fangkai Yang, et al.

  Jia Fu, Xiaoting Qin, Fangkai Yang, Lu Wang, Jue Zhang, Qingwei Lin, Yubo Chen, Dongmei Zhang, Saravan Rajmohan, Qi Zhang
  

• Advances in Neural Information Processing Systems (2024)

  Qinggang Zhang, Junnan Dong, Hao Chen, et al.

  Qinggang Zhang, Junnan Dong, Hao Chen, Daochen Zha, Zailiang Yu, Xiao Huang
  

• MBA-RAG: a Bandit Approach for Adaptive Retrieval-Augmented Generation through Question Complexity (2025)

  Xiaqiang Tang, Qiang Gao, Jian Li, et al.

  Xiaqiang Tang, Qiang Gao, Jian Li, Nan Du, Qi Li, Sihong Xie
  

• IEEE INFOCOM 2025 - IEEE Conference on Computer Communications (2025)

  Tao Ouyang, Guihang Hong, Kongyange Zhao, et al.

  Tao Ouyang, Guihang Hong, Kongyange Zhao, Zhi Zhou, Weigang Wu, Zhaobiao Lv, Xu Chen
  

• Adapting to non-stationary environments: Multi-armed bandit enhanced retrieval-augmented generation on knowledge graphs (2025)

  Xiaqiang Tang, Jian Li, Nan Du, et al.

  Xiaqiang Tang, Jian Li, Nan Du, Sihong Xie
  

• Context Attribution with Multi-Armed Bandit Optimization (2025)

  Deng Pan, Keerthiram Murugesan, Nuno Moniz, et al.

  Deng Pan, Keerthiram Murugesan, Nuno Moniz, Nitesh Chawla
  

Inference Optimization

• LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing (2025)

  Yang Li

  Yang Li
  

• Adaptive llm routing under budget constraints (2025)

  Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu, et al.

  Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu, Sho Takemori, Vishal Sharma
  

• Dynamic and Cost-Efficient Deployment of Large Language Models Using Uplift Modeling and Multi Armed Bandits (2025)

  Ninad Tongay

  Ninad Tongay
  

• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs (2025)

  Wang Wei, Tiankai Yang, Hongjie Chen, et al.

  Wang Wei, Tiankai Yang, Hongjie Chen, Yue Zhao, Franck Dernoncourt, Ryan A. Rossi, Hoda Eldardiry
  

• LLM Cache Bandit Revisited: Addressing Query Heterogeneity for Cost-Effective LLM Inference (2025)

  Hantao Yang, Hong Xie, Defu Lian, et al.

  Hantao Yang, Hong Xie, Defu Lian, Enhong Chen
  

Decoding Strategies

• BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms (2025)

  Yunlong Hou, Fengzhuo Zhang, Cunxiao Du, et al.

  Yunlong Hou, Fengzhuo Zhang, Cunxiao Du, Xuan Zhang, Jiachun Pan, Tianyu Pang, Chao Du, Vincent YF Tan, Zhuoran Yang
  

• Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism (2024)

  Jiahao Liu, Qifan Wang, Jingang Wang, et al.

  Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai
  

• Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs (2025)

  Hongyi Liu, Jiaji Huang, Zhen Jia, et al.

  Hongyi Liu, Jiaji Huang, Zhen Jia, Youngsuk Park, Yu-Xiang Wang
  

• Tokenized Bandit for LLM Decoding and Alignment (2025)

  Suho Shin, Chenghao Yang, Haifeng Xu, et al.

  Suho Shin, Chenghao Yang, Haifeng Xu, Mohammad T Hajiaghayi
  

Adaptation and Personalization

• Personalizing natural language understanding using multi-armed bandits and implicit feedback (2020)

  Fabian Moerchen, Patrick Ernst, Giovanni Zappella

  Fabian Moerchen, Patrick Ernst, Giovanni Zappella
  

• User Feedback-based Online Learning for Intent Classification (2023)

  Kaan G"on\cc, Baturay Sa\uglam, Onat Dalmaz, et al.

  Kaan G"on\cc, Baturay Sa\uglam, Onat Dalmaz, Tolga \cCukur, Serdar Kozat, Hamdi Dibeklioglu
  

• Online personalizing white-box llms generation with neural bandits (2024)

  Zekai Chen, Po-Yu Chen, Francois Buet-Golfouse

  Zekai Chen, Po-Yu Chen, Francois Buet-Golfouse
  

• LLMs Are In-Context Bandit Reinforcement Learners (2024)

  Giovanni Monea, Antoine Bosselut, Kiant'e Brantley, et al.

  Giovanni Monea, Antoine Bosselut, Kiant'e Brantley, Yoav Artzi
  

• Cost-Effective Online Multi-LLM Selection with Versatile Reward Models (2024)

  Xiangxiang Dai, Jin Li, Xutong Liu, et al.

  Xiangxiang Dai, Jin Li, Xutong Liu, Anqi Yu, John Lui
  

• Tokenized Reinforcement Learning for LLM Generation: A Survey (2025)

  Minseok Shin, Anji Gao, Xiang Li, et al.

  Minseok Shin, Anji Gao, Xiang Li, Tianyi Xie, Wei Wang
  

• LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing (2025)

  Yang Li

  Yang Li
  

• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs (2025)

  Wang Wei, Tiankai Yang, Hongjie Chen, et al.

  Wang Wei, Tiankai Yang, Hongjie Chen, Yue Zhao, Franck Dernoncourt, Ryan A. Rossi, Hoda Eldardiry
  

• Comparing Exploration-Exploitation Strategies of LLMs and Humans: Insights from Standard Multi-armed Bandit Tasks (2025)

  Ziyuan Zhang, Darcy Wang, Ningyuan Chen, et al.

  Ziyuan Zhang, Darcy Wang, Ningyuan Chen, Rodrigo Mansur, Vahid Sarhangian
  

📍 LLM-enhancements for Multi-armed Bandits

Regret Minimization Objective

• On Bits and Bandits: Quantifying the Regret-Information Trade-off (2024)

  Itai Shufaro, Nadav Merlis, Nir Weinberger, et al.

  Itai Shufaro, Nadav Merlis, Nir Weinberger, Shie Mannor
  

• Beyond Numeric Awards: In-Context Dueling Bandits with LLM Agents (2024)

  Fanzeng Xia, Hao Liu, Yisong Yue, et al.

  Fanzeng Xia, Hao Liu, Yisong Yue, Tongxin Li
  

• Do llm agents have regret? a case study in online learning and games (2024)

  Chanwoo Park, Xiangyu Liu, Asuman Ozdaglar, et al.

  Chanwoo Park, Xiangyu Liu, Asuman Ozdaglar, Kaiqing Zhang
  

• LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations (2023)

  Ali Baheri, Aman Hosseini, Mehdi Jiang, et al.

  Ali Baheri, Aman Hosseini, Mehdi Jiang, Peter Chin
  

• Neural Dueling Bandits (2024)

  Arun Verma, Zhongxiang Dai, Xiaoqiang Lin, et al.

  Arun Verma, Zhongxiang Dai, Xiaoqiang Lin, Patrick Jaillet, Bryan Kian Hsiang Low
  

• LLM-informed multi-armed bandit strategies for non-stationary environments (2023)

  J De Curt`o, Irene de Zarz`a, Gemma Roig, et al.

  J De Curt`o, Irene de Zarz`a, Gemma Roig, Juan Carlos Cano, Pietro Manzoni, Carlos T Calafate
  

Arm Definition

• Neural Dueling Bandits (2024)

  Arun Verma, Zhongxiang Dai, Xiaoqiang Lin, et al.

  Arun Verma, Zhongxiang Dai, Xiaoqiang Lin, Patrick Jaillet, Bryan Kian Hsiang Low
  

• Beyond Numeric Awards: In-Context Dueling Bandits with LLM Agents (2024)

  Fanzeng Xia, Hao Liu, Yisong Yue, et al.

  Fanzeng Xia, Hao Liu, Yisong Yue, Tongxin Li
  

• LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations (2023)

  Ali Baheri, Aman Hosseini, Mehdi Jiang, et al.

  Ali Baheri, Aman Hosseini, Mehdi Jiang, Peter Chin
  

• Guiding pretraining in reinforcement learning with large language models (2023)

  Yuqing Du, Olivia Watkins, Zihan Wang, et al.

  Yuqing Du, Olivia Watkins, Zihan Wang, C'edric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, Jacob Andreas
  

• Multi-Armed Bandit Approach for Optimizing Training on Synthetic Data (2024)

  Abdulrahman Kerim, Leandro Soriano Marcolino, Erickson R. Nascimento, et al.

  Abdulrahman Kerim, Leandro Soriano Marcolino, Erickson R. Nascimento, Richard Jiang
  

• Investigating the Relationship Between Physical Activity and Tailored Behavior Change Messaging: Connecting Contextual Bandit with Large Language Models (2025)

  Haochen Song, Dominik Hofer, Rania Islambouli, et al.

  Haochen Song, Dominik Hofer, Rania Islambouli, Laura Hawkins, Ananya Bhattacharjee, Meredith Franklin, Joseph Jay Williams
  

Environment

• LLM-informed multi-armed bandit strategies for non-stationary environments (2023)

  J De Curt`o, Irene de Zarz`a, Gemma Roig, et al.

  J De Curt`o, Irene de Zarz`a, Gemma Roig, Juan Carlos Cano, Pietro Manzoni, Carlos T Calafate
  

• Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization (2024)

  Yunfan Zhao, Nikhil Behari, Edward Hughes, et al.

  Yunfan Zhao, Nikhil Behari, Edward Hughes, Edwin Zhang, Dheeraj Nagaraj, Karl Tuyls, Aparna Taneja, Milind Tambe
  

• Jump Starting Bandits with LLM-Generated Prior Knowledge (2024)

  Parand Alamdari, Yanshuai Cao, Kevin Wilson

  Parand Alamdari, Yanshuai Cao, Kevin Wilson
  

• In-context impersonation reveals Large Language Models' strengths and biases (2024)

  Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, et al.

  Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, Zeynep Akata
  

• Multi-Armed Bandit Approach for Optimizing Training on Synthetic Data (2024)

  Abdulrahman Kerim, Leandro Soriano Marcolino, Erickson R. Nascimento, et al.

  Abdulrahman Kerim, Leandro Soriano Marcolino, Erickson R. Nascimento, Richard Jiang
  

• EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents (2024)

  Abhay Zala, Jaemin Cho, Han Lin, et al.

  Abhay Zala, Jaemin Cho, Han Lin, Jaehong Yoon, Mohit Bansal
  

• Balancing act: prioritization strategies for llm-designed restless bandit rewards (2025)

  Shresth Verma, Niclas Boehmer, Lingkai Kong, et al.

  Shresth Verma, Niclas Boehmer, Lingkai Kong, Milind Tambe
  

• Investigating the Relationship Between Physical Activity and Tailored Behavior Change Messaging: Connecting Contextual Bandit with Large Language Models (2025)

  Haochen Song, Dominik Hofer, Rania Islambouli, et al.

  Haochen Song, Dominik Hofer, Rania Islambouli, Laura Hawkins, Ananya Bhattacharjee, Meredith Franklin, Joseph Jay Williams
  

• Toward efficient exploration by large language model agents (2025)

  Dilip Arumugam, Thomas L Griffiths

  Dilip Arumugam, Thomas L Griffiths
  

• Prior-informed optimization of treatment recommendation via bandit algorithms trained on large language model-processed historical records (2025)

  Saman Nessari, Ali Bozorgi-Amiri

  Saman Nessari, Ali Bozorgi-Amiri
  

Reward Formulation

• Statistical and computational trade-off in multi-agent multi-armed bandits (2024)

  Filippo Vannella, Alexandre Proutiere, Jaeseong Jeong

  Filippo Vannella, Alexandre Proutiere, Jaeseong Jeong
  

• On the Importance of Uncertainty in Decision-Making with Large Language Models (2024)

  Nicol`o Felicioni, Lucas Maystre, Sina Ghiassian, et al.

  Nicol`o Felicioni, Lucas Maystre, Sina Ghiassian, Kamil Ciosek
  

• A decision-language model (DLM) for dynamic restless multi-armed bandit tasks in public health (2024)

  Nikhil Behari, Edwin Zhang, Yunfan Zhao, et al.

  Nikhil Behari, Edwin Zhang, Yunfan Zhao, Aparna Taneja, Dheeraj Nagaraj, Milind Tambe
  

• Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization (2024)

  Yunfan Zhao, Nikhil Behari, Edward Hughes, et al.

  Yunfan Zhao, Nikhil Behari, Edward Hughes, Edwin Zhang, Dheeraj Nagaraj, Karl Tuyls, Aparna Taneja, Milind Tambe
  

• Do llm agents have regret? a case study in online learning and games (2024)

  Chanwoo Park, Xiangyu Liu, Asuman Ozdaglar, et al.

  Chanwoo Park, Xiangyu Liu, Asuman Ozdaglar, Kaiqing Zhang
  

• Balancing act: prioritization strategies for llm-designed restless bandit rewards (2025)

  Shresth Verma, Niclas Boehmer, Lingkai Kong, et al.

  Shresth Verma, Niclas Boehmer, Lingkai Kong, Milind Tambe
  

• Large Language Model-Enhanced Multi-Armed Bandits (2025)

  Jiahang Sun, Zhiyong Wang, Runhan Yang, et al.

  Jiahang Sun, Zhiyong Wang, Runhan Yang, Chenjun Xiao, John C. S. Lui, Zhongxiang Dai
  

• Prior-informed optimization of treatment recommendation via bandit algorithms trained on large language model-processed historical records (2025)

  Saman Nessari, Ali Bozorgi-Amiri

  Saman Nessari, Ali Bozorgi-Amiri
  

• Prioritization Strategies for LLM-Designed Restless Bandit Rewards in Public Health (2024)

  Shresth Verma, Niclas Boehmer, Lingkai Kong, et al.

  Shresth Verma, Niclas Boehmer, Lingkai Kong, Milind Tambe
  

• Beyond Numeric Rewards: In-Context Dueling Bandits with LLM Agents (2025)

  Fanzeng Xia, Hao Liu, Yisong Yue, et al.

  Fanzeng Xia, Hao Liu, Yisong Yue, Tongxin Li
  

• When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training (2025)

  Sanxing Chen, Xiaoyin Chen, Yukun Huang, et al.

  Sanxing Chen, Xiaoyin Chen, Yukun Huang, Roy Xie, Bhuwan Dhingra
  

Sampling Strategy

• Towards optimizing with large language models (2023)

  Pei-Fu Guo, Ying-Hsuan Chen, Yun-Da Tsai, et al.

  Pei-Fu Guo, Ying-Hsuan Chen, Yun-Da Tsai, Shou-De Lin
  

• On Bits and Bandits: Quantifying the Regret-Information Trade-off (2024)

  Itai Shufaro, Nadav Merlis, Nir Weinberger, et al.

  Itai Shufaro, Nadav Merlis, Nir Weinberger, Shie Mannor
  

• Efficient Sequential Decision Making with Large Language Models (2024)

  Dingyang Chen, Qi Zhang, Yinglun Zhu

  Dingyang Chen, Qi Zhang, Yinglun Zhu
  

• Jump Starting Bandits with LLM-Generated Prior Knowledge (2024)

  Parand Alamdari, Yanshuai Cao, Kevin Wilson

  Parand Alamdari, Yanshuai Cao, Kevin Wilson
  

• Cost-Effective Online Multi-LLM Selection with Versatile Reward Models (2024)

  Xiangxiang Dai, Jin Li, Xutong Liu, et al.

  Xiangxiang Dai, Jin Li, Xutong Liu, Anqi Yu, John Lui
  

• Large Language Model-Enhanced Multi-Armed Bandits (2025)

  Jiahang Sun, Zhiyong Wang, Runhan Yang, et al.

  Jiahang Sun, Zhiyong Wang, Runhan Yang, Chenjun Xiao, John C. S. Lui, Zhongxiang Dai
  

• When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training (2025)

  Sanxing Chen, Xiaoyin Chen, Yukun Huang, et al.

  Sanxing Chen, Xiaoyin Chen, Yukun Huang, Roy Xie, Bhuwan Dhingra
  

• Llms are greedy agents: Effects of rl fine-tuning on decision-making abilities (2025)

  Thomas Schmied, J"org Bornschein, Jordi Grau-Moya, et al.

  Thomas Schmied, J"org Bornschein, Jordi Grau-Moya, Markus Wulfmeier, Razvan Pascanu
  

• Evaluation of LLM Powered Agentic AI for Solving Multi-Arm Bandit Problems (2025)

  Jawad Hazime, Junaid Farooq

  Jawad Hazime, Junaid Farooq
  

• TextBandit: Evaluating Probabilistic Reasoning in LLMs Through Language-Only Decision Tasks (2025)

  Jimin Lim, Arjun Damerla, Arthur Jiang, et al.

  Jimin Lim, Arjun Damerla, Arthur Jiang, Nam Le
  

• Toward efficient exploration by large language model agents (2025)

  Dilip Arumugam, Thomas L Griffiths

  Dilip Arumugam, Thomas L Griffiths
  

Action Decision

• Using large language models for hyperparameter optimization (2023)

  Michael R Zhang, Nishkrit Desai, Juhan Bae, et al.

  Michael R Zhang, Nishkrit Desai, Juhan Bae, Jonathan Lorraine, Jimmy Ba
  

• Towards optimizing with large language models (2023)

  Pei-Fu Guo, Ying-Hsuan Chen, Yun-Da Tsai, et al.

  Pei-Fu Guo, Ying-Hsuan Chen, Yun-Da Tsai, Shou-De Lin
  

• LLMs for User Interest Exploration: A Hybrid Approach (2024)

  Jianling Wang, Haokai Lu, Yifan Liu, et al.

  Jianling Wang, Haokai Lu, Yifan Liu, He Ma, Yueqi Wang, Yang Gu, Shuzhou Zhang, Shuchao Bi, Lexi Baugher, Ed Chi, others
  

• Large language models as evolutionary optimizers (2024)

  Shengcai Liu, Caishun Chen, Xinghua Qu, et al.

  Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, Yew-Soon Ong
  

• Large Language Model-Enhanced Multi-Armed Bandits (2025)

  Jiahang Sun, Zhiyong Wang, Runhan Yang, et al.

  Jiahang Sun, Zhiyong Wang, Runhan Yang, Chenjun Xiao, John C. S. Lui, Zhongxiang Dai
  

• When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training (2025)

  Sanxing Chen, Xiaoyin Chen, Yukun Huang, et al.

  Sanxing Chen, Xiaoyin Chen, Yukun Huang, Roy Xie, Bhuwan Dhingra
  

• Evaluation of LLM Powered Agentic AI for Solving Multi-Arm Bandit Problems (2025)

  Jawad Hazime, Junaid Farooq

  Jawad Hazime, Junaid Farooq
  

• Llms are greedy agents: Effects of rl fine-tuning on decision-making abilities (2025)

  Thomas Schmied, J"org Bornschein, Jordi Grau-Moya, et al.

  Thomas Schmied, J"org Bornschein, Jordi Grau-Moya, Markus Wulfmeier, Razvan Pascanu
  

• TextBandit: Evaluating Probabilistic Reasoning in LLMs Through Language-Only Decision Tasks (2025)

  Jimin Lim, Arjun Damerla, Arthur Jiang, et al.

  Jimin Lim, Arjun Damerla, Arthur Jiang, Nam Le
  

• Toward efficient exploration by large language model agents (2025)

  Dilip Arumugam, Thomas L Griffiths

  Dilip Arumugam, Thomas L Griffiths
  

👨‍💻 Team

Here is the list of our contributors in each modality of this repository.

😉 Citation

If you find this work useful in your research, Please cite the paper as below:

@article{xxx,
    
}