Multi-Agent Reinforcement Learning papers

This is a collection of Multi-Agent Reinforcement Learning (MARL) papers. Each category is a potential start point for you to start your research. Some papers are listed more than once because they belong to multiple categories.

For MARL papers with code and MARL resources, please refer to MARL Papers with Code and MARL Resources Collection.

I will continually update this repository and I welcome suggestions. (missing important papers, missing categories, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.

Update (2026): A large batch of recent papers (2022–2026) has been added across most categories, plus three new sections — LLM-based Multi-Agent, Zero-Shot Coordination and Generalization — and a Recent Surveys (2022–2026) list.

This repository is not for commercial purposes.

My email: chenhao915@mails.ucas.ac.cn

Overview

• Reviews
• Environments
• Dealing With Credit Assignment Issue
• Policy Gradient
• Communication
• Emergent
• Opponent Modeling
• Game Theoretic
• Hierarchical
• Ad Hoc Teamwork
• League Training
• Curriculum Learning
• Mean Field
• Transfer Learning
• Meta Learning
• Fairness
• Exploration
• Graph Neural Network
• Model-based
• NAS
• Safe Multi-Agent Reinforcement Learning
• From Single-Agent to Multi-Agent
• Discrete-Continuous Hybrid Action Spaces / Parameterized Action Space
• Role
• Diversity
• Sparse Reward
• Large Scale
• DTDE
• Decision Transformer
• Offline MARL
• Generalization
• Adversarial
• LLM-based Multi-Agent
• Zero-Shot Coordination
• Generalization
• Multi-Agent Path Finding
• To be Categorized
• TODO

Reviews

Recent Reviews (Since 2019)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
• A Review of Cooperative Multi-Agent Deep Reinforcement Learning
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Recent Surveys (2022–2026)

• Multi-agent Reinforcement Learning: A Comprehensive Survey
• A Survey of Progress on Cooperative Multi-Agent Reinforcement Learning in Open Environments
• A Survey of Multi-Agent Deep Reinforcement Learning with Communication (AAMAS 2024)
• A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives
• A Survey on Large Language Model based Multi-Agent Systems: Recent Advances and New Frontiers in Application
• Multi-Agent Collaboration Mechanisms: A Survey of LLMs
• A Survey of Safe Reinforcement Learning and Constrained MDPs: Single-Agent and Multi-Agent Safety
• Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

Other Reviews (Before 2019)

• If multi-agent learning is the answer, what is the question?
• Multiagent learning is not the answer. It is the question
• Is multiagent deep reinforcement learning the answer or the question? A brief survey Note that A Survey and Critique of Multiagent Deep Reinforcement Learning is an updated version of this paper with the same authors.
• Evolutionary Dynamics of Multi-Agent Learning: A Survey
• (Worth reading although they're not recent reviews.)

Environments

Dealing With Credit Assignment Issue

Value Decomposition

Other Methods

Policy Gradient

Communication

Communication Without Bandwidth Constraint

Communication Under Limited Bandwidth

Emergent

Opponent Modeling

Game Theoretic

Hierarchical

Ad Hoc Teamwork

League Training

Curriculum Learning

Mean Field

Transfer Learning

Meta Learning

Fairness

Exploration

Dense Reward Exploration

Sparse Reward Exploration

Uncategorized

Graph Neural Network

Model-based

NAS

Safe Multi-Agent Reinforcement Learning

From Single-Agent to Multi-Agent

Discrete-Continuous Hybrid Action Space / Parameterized Action Space

Role

Diversity

Sparse Reward

Large Scale

DTDE

Decision Transformer

Offline MARL

Adversarial

For more adversarial / robust RL papers (single-agent and multi-agent), see Adversarial Reinforcement Learning Papers.

Multi-Agent Path Finding

• TODO

LLM-based Multi-Agent

Large Language Model (LLM) based multi-agent systems and the intersection of LLMs with (multi-agent) reinforcement learning. This is an emerging and fast-moving area.

Zero-Shot Coordination

Generalization

To be Categorized

TODO

• Multi-Agent Path Finding
• Generalization in MARL

Citation

If you find this repository useful, please cite our repo:

@misc{chen2021multi,
  author={Chen, Hao},
  title={Multi-Agent Reinforcement Learning Papers},
  year={2021}
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers}}
}