MARL Resources Collection

This is a collection of Multi-Agent Reinforcement Learning (MARL) Resources. The purpose of this repository is to give beginners a better understanding of MARL and accelerate the learning process. Note that some of the resources are written in Chinese and only important papers that have a lot of citations were listed.

I will continually update this repository and I welcome suggestions. (missing important papers, missing important resources, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.

Update (2026): Added recent open-source environments/benchmarks (JaxMARL, SMACv2, Melting Pot 2.0, Gigastep, POGEMA, ...), a new MARL Libraries / Frameworks section, a new LLM-based Multi-Agent Systems section, and a Recent Surveys (2022–2026) list.

This repository is not for commercial purposes.

My email: chenhao915@mails.ucas.ac.cn

Overview

• Courses
• Important Conferences
• Reviews
• Books
• Open Source Environments
• MARL Libraries / Frameworks
• LLM-based Multi-Agent Systems
• Research Groups
• Companies
• Paper List
• Talks
• Useful Resources
• TODO

Courses

• RLChina
• UCL Multi-agent AI
• SJTU Multi-Agent Reinforcement Learning Tutorial
• SJTU Reinforcement Learning

Important Conferences

• AAMAS, AAAI, IJCAI, ICLR, ICML, NIPS
• Sorted by difficulty (roughly)

Reviews

Recent Reviews (Since 2019)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
• A Review of Cooperative Multi-Agent Deep Reinforcement Learning
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Recent Surveys (2022–2026)

• Multi-agent Reinforcement Learning: A Comprehensive Survey
• A Survey of Progress on Cooperative Multi-Agent Reinforcement Learning in Open Environments
• A Survey of Multi-Agent Deep Reinforcement Learning with Communication (AAMAS 2024)
• A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives
• A Survey of Safe Reinforcement Learning and Constrained MDPs: Single-Agent and Multi-Agent Safety
• Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
• Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey
• A Survey on Large Language Model based Multi-Agent Systems: Recent Advances and New Frontiers in Application
• Multi-Agent Collaboration Mechanisms: A Survey of LLMs

Other Reviews (Before 2019)

• If multi-agent learning is the answer, what is the question?
• Multiagent learning is not the answer. It is the question
• Is multiagent deep reinforcement learning the answer or the question? A brief survey Note that A Survey and Critique of Multiagent Deep Reinforcement Learning is an updated version of this paper with the same authors.
• Evolutionary Dynamics of Multi-Agent Learning: A Survey
• (Worth reading although they're not recent reviews.)

Books

• Multiagent systems: Algorithmic, game-theoretic, and logical foundations
• Multi‐Agent Machine Learning A Reinforcement Approach

Open Source Environments

• StarCraft Micromanagement Environment
  • pymarl is the original environment mentioned in the paper The StarCraft Multi-Agent Challenge. Note that pymarl is based on SMAC.
  • MARL-Algorithms is a simplified implementation of pymarl
  • EPyMARL is a extended python MARL framework with more environments (Level Based Foraging, Multi-Robot Warehouse, Multi-Agent Particle Environment) and more algorithms. Paper
  • pymarl2 added code-level tricks to the original pymarl. Paper
• Multi-Agent Particle Environment PyTorch Implementation
• Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
• OpenSpiel: A Framework for Reinforcement Learning in Games
• Hanabi-learning-environment
• RoboCup 2D Half Field Offense
• Pommerman
• Multi-agent-emergence-environments
• Google Research Football
• MAgent Note that the original project is no longer maintained.
• DI-engine
• MARLlib is a MARL Extension for RLlib
• Multiagent Mujoco
• PettingZoo website
• Safe Policy Optimization (SafePO)
• (I personally recommend the first two environments for beginners, especially EPyMARL.)

Newer Environments / Benchmarks (2022–2026)

• SMACv2 — an improved, procedurally-generated version of the StarCraft Multi-Agent Challenge. Paper
• JaxMARL — a large collection of GPU-accelerated MARL environments (including SMAX, an SMAC re-implementation) and baseline algorithms in JAX; extremely fast. Paper
• Melting Pot 2.0 — DeepMind's suite for evaluating generalization to novel social situations (cooperation, competition, deception). Paper
• Overcooked-AI — a benchmark for human-AI coordination; the de-facto standard for zero-shot coordination research. Paper
• Gigastep — one billion steps per second multi-agent RL; large-scale aerial/ground combat scenarios. Paper
• POGEMA — a fast benchmark platform for cooperative multi-agent pathfinding (MAPF). Paper
• MOMAland — Farama's benchmarks for multi-objective multi-agent RL. Paper
• MARBLER — standardized evaluation of multi-robot RL algorithms on the Robotarium. Paper
• MABIM / ReplenishmentEnv — a versatile MARL benchmark for inventory management. Paper

MARL Libraries / Frameworks

Training libraries and algorithm frameworks (as opposed to task environments).

• MARLlib — a Ray/RLlib-based library unifying many MARL algorithms across a wide range of environments through a standardized interface. Paper
• BenchMARL — a TorchRL-backed benchmarking library for reproducible, standardized comparison of MARL algorithms, tasks and models. Paper
• EPyMARL — Extended PyMARL adding more algorithms (IPPO, MAA2C, etc.) and environments.
• pymarlzooplus — a further-extended PyMARL/EPyMARL benchmarking suite for complex fully-cooperative tasks. Paper
• Mava — InstaDeep's research-friendly JAX framework for scalable MARL.
• JaxMARL — bundles both JAX environments and baseline MARL algorithms.
• TorchRL — PyTorch-native RL library with first-class multi-agent support (MAPPO, IPPO, QMIX, MADDPG); the backend of BenchMARL.

LLM-based Multi-Agent Systems

The intersection of Large Language Models and multi-agent systems has become a very active area. A few widely-used open-source frameworks:

• CAMEL — communicative agents / role-playing for LLM societies. Paper
• AutoGen — Microsoft's framework for building multi-agent LLM applications via conversation. Paper
• MetaGPT — a multi-agent framework encoding SOPs for collaborative software teams. Paper
• ChatDev — communicative agents that collaborate to develop software. Paper
• AgentVerse — a framework for multi-agent collaboration and emergent behavior. Paper
• Generative Agents — interactive simulacra of human behavior. Paper
• See also the LLM-based Multi-Agent section in Multi-Agent-Reinforcement-Learning-papers.

Research Groups

Companies

• DeepMind
• OpenAI
• Facebook
• Tencent
• NetEase
• Huawei
• Parametrix.ai
• Inspir.ai

Paper Lists

• https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers
• https://github.com/TimeBreaker/MARL-papers-with-code
• https://github.com/TimeBreaker/Adversarial-Reinforcement-Learning-Papers
• https://github.com/LantaoYu/MARL-Papers

Talks

In English

• https://www.youtube.com/watch?v=W_9kcQmaWjo
• https://www.youtube.com/watch?v=TMTT2z8lifA
• https://www.youtube.com/watch?v=Yd6HNZnqjis
• https://www.youtube.com/watch?v=ufFue5_gR4c

In Chinese

• https://www.techbeat.net/talk-info?id=501
• https://www.bilibili.com/video/av457780236/
• https://space.bilibili.com/551888585/channel/detail?cid=167587
• https://www.bilibili.com/video/BV1ig4y1v7xU
• https://www.bilibili.com/video/BV18z411q7Kc
• https://www.bilibili.com/video/BV1k5411V7ue

Useful Resources

In English

• https://dblp.uni-trier.de/
• https://paperswithcode.com/
• https://www.connectedpapers.com
• https://deeplearn.org
• https://spinningup.openai.com/
• https://github.com/openai/spinningup
• https://github.com/Jinjiarui/hrl-papers

In Chinese

• http://www.neurondance.com/
• https://www.zhihu.com/question/376068768
• https://www.zhihu.com/question/323584412
• https://zhuanlan.zhihu.com/p/372558232
• https://space.bilibili.com/4801051?spm_id_from=333.788.b_765f7570696e666f.2
• https://www.zhihu.com/people/tian-yuan-dong
• https://www.zhihu.com/people/eyounx
• https://www.zhihu.com/people/wan-shang-zhu-ce-de
• Wechat public account: AIORHHC; RLCN
• https://www.bilibili.com/video/av925922430/
• https://www.bilibili.com/video/av626777400/
• https://github.com/NeuronDance/DeepRL

TODO

• The Research Groups part needs to be completed
• The Companies part needs to be completed
• The Useful Resources part needs to be perfected

Citation

If you find this repository useful, please cite our repo:

@misc{chen2021collection,
  author={Chen, Hao},
  title={A Collection of Multi-Agent Reinforcement Learning Resources},
  year={2021}
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/TimeBreaker/MARL-resources-collection}}
}