[NeurIPS 2023 Spotlight] LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios (awesome MCTS)
See the code
Updated on 2026.03.11 LightZero-v0.2.0
LightZero is a lightweight, efficient, and easy-to-understand open-source algorithm toolkit that combines Monte Carlo Tree Search (MCTS) and Deep Reinforcement Learning (RL).
English | 简体中文(Simplified Chinese) | Documentation | LightZero Paper | UniZero Paper | ReZero Paper | 🔥ScaleZero Paper
The integration of Monte Carlo Tree Search and Deep Reinforcement Learning,
exemplified by AlphaZero and MuZero,
has achieved unprecedented performance levels in various games, including Go and Atari.
This advanced methodology has also made significant strides in scientific domains like protein structure prediction and the search for matrix multiplication algorithms.
The following is an overview of the historical evolution of the Monte Carlo Tree Search algorithm series:

LightZero is an open-source algorithm toolkit that combines Monte Carlo Tree Search (MCTS) and Reinforcement Learning (RL) for PyTorch. It supports a range of MCTS-based RL algorithms and applications, offering several key advantages:
For further details, please refer to Features, Framework Structure and Integrated Algorithms.
LightZero aims to promote the standardization of the MCTS+RL algorithm family to accelerate related research and applications. A performance comparison of all implemented algorithms under a unified framework is presented in the Benchmark.
Lightweight: LightZero integrates multiple MCTS algorithm families and can solve decision-making problems with various attributes in a lightweight framework. The algorithms and environments LightZero implemented can be found here.
Efficient: LightZero uses mixed heterogeneous computing programming to improve computational efficiency for the most time-consuming part of MCTS algorithms.
Easy-to-understand: LightZero provides detailed documentation and algorithm framework diagrams for all integrated algorithms to help users understand the algorithm's core and compare the differences and similarities between algorithms under the same paradigm. LightZero also provides function call graphs and network structure diagrams for algorithm code implementation, making it easier for users to locate critical code. All the documentation can be found here.
The above picture is the framework pipeline of LightZero. We briefly introduce the three core modules below:
Model:
Model is used to define the network structure, including the __init__ function for initializing the network structure and the forward function for computing the network's forward propagation.
Policy:
Policy defines the way the network is updated and interacts with the environment, including three processes: the learning process, the collecting process, and the evaluation process.
MCTS:
MCTS defines the structure of the Monte Carlo search tree and the way it interacts with the Policy. The implementation of MCTS includes two languages: Python and C++, implemented in ptree and ctree, respectively.
For the file structure of LightZero, please refer to lightzero_file_structure.
LightZero is a library with a PyTorch implementation of MCTS algorithms (sometimes combined with cython and cpp), including:
The environments and algorithms currently supported by LightZero are shown in the table below:
| Env./Algo. | AlphaZero | MuZero | Sampled MuZero | EfficientZero | Sampled EfficientZero | Gumbel MuZero | Stochastic MuZero | UniZero | Sampled UniZero | ReZero |
|---|---|---|---|---|---|---|---|---|---|---|
| TicTacToe | ✔ | ✔ | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | ✔ | 🔒 | 🔒 |
| Gomoku | ✔ | ✔ | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | ✔ | 🔒 | ✔ |
| Connect4 | ✔ | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | ✔ |
| 2048 | --- | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | ✔ | ✔ | 🔒 | 🔒 |
| Chess | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| Go | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| CartPole | --- | ✔ | 🔒 | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | ✔ |
| Pendulum | --- | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | ✔ | 🔒 |
| LunarLander | --- | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 |
| BipedalWalker | --- | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 |
| Atari | --- | ✔ | 🔒 | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | ✔ |
| DeepMind Control | --- | --- | ✔ | --- | ✔ | 🔒 | 🔒 | 🔒 | ✔ | 🔒 |
| MuJoCo | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| MiniGrid | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 | 🔒 |
| Bsuite | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 | 🔒 |
| Memory | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 | 🔒 |
| SumToThree (billiards) | --- | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| MetaDrive | --- | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
(1): "✔" means that the corresponding item is finished and well-tested.
(2): "🔒" means that the corresponding item is in the waiting-list (Work In Progress).
(3): "---" means that this algorithm doesn't support this environment.
You can install the latest LightZero in development from the GitHub source codes with the following command:
git clone https://github.com/opendilab/LightZero.git
cd LightZero
pip3 install -e .
Kindly note that LightZero currently supports compilation only on Linux and macOS platforms.
We are actively working towards extending this support to the Windows platform.
Your patience during this transition is greatly appreciated.
We also provide a Dockerfile that sets up an environment with all dependencies needed to run the LightZero library. This Docker image is based on Ubuntu 20.04 and installs Python 3.8, along with other necessary tools and libraries. Here's how to use our Dockerfile to build a Docker image, run a container from this image, and execute LightZero code inside the container.
mkdir lightzero-docker
mv Dockerfile lightzero-docker/
cd lightzero-docker/
docker build -t ubuntu-py38-lz:latest -f ./Dockerfile .
docker run -dit --rm ubuntu-py38-lz:latest /bin/bash
python ./LightZero/zoo/classic_control/cartpole/config/cartpole_muzero_config.py
Train a MuZero agent to play CartPole:
cd LightZero
python3 -u zoo/classic_control/cartpole/config/cartpole_muzero_config.py
Train a MuZero agent to play Pong:
cd LightZero
python3 -u zoo/atari/config/atari_muzero_segment_config.py
Train an experimental Ray async MuZero agent to play Pong:
cd LightZero
python3 -u zoo/atari/config/atari_muzero_segment_async_config.py
Train a MuZero agent to play TicTacToe:
cd LightZero
python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
Train a UniZero agent to play Pong:
cd LightZero
python3 -u zoo/atari/config/atari_unizero_segment_config.py
The Atari segment async pipeline uses Ray collector/evaluator actors while keeping the learner and replay buffer in the driver process. It supports MuZero-family segment policies and UniZero segment policies through the explicit --async-pipeline config flag. In a same-window Pong MuZero rjob comparison (2026-07-02, 2 collector actors, ~6.2 h wall-clock), async collected 809.5k env steps (≈36.3 env steps/s) versus 499.8k env steps (≈22.4 env steps/s) for the synchronous baseline — about 1.6× end-to-end throughput, with eval rewards statistically comparable at this early training stage. For design details and UniZero differences, see Async Pipeline and 中文说明.
The LightZero documentation can be found here. It contains tutorials and the API reference.
For those interested in customizing environments and algorithms, we provide relevant guides:
Should you have any questions, feel free to contact us for support.
Factored/Gaussian policy representation on three classic continuous action space games: Pendulum-v1, LunarLanderContinuous-v2, BipedalWalker-v3
and two MuJoCo continuous action space games: Hopper-v3, Walker2d-v3."Factored Policy" indicates that the agent learns a policy network that outputs a categorical distribution. After manual discretization, the dimensions of the action space for the five environments are 11, 49 (7^2), 256 (4^4), 64 (4^3), and 4096 (4^6), respectively. On the other hand, "Gaussian Policy" refers to the agent learning a policy network that directly outputs parameters (mu and sigma) for a Gaussian distribution.
The following are the detailed paper notes (in Chinese) of the above algorithms:
You can also refer to the relevant Zhihu column (in Chinese): In-depth Analysis of MCTS+RL Frontier Theories and Applications.
The following are the overview MCTS principle diagrams of the above algorithms:
Here is a collection of research papers about Monte Carlo Tree Search. This Section will be continuously updated to track the frontier of MCTS.
File an issue on Github
Open or participate in our discussion forum
Discuss on LightZero discord server
Contact our email (opendilab@pjlab.org.cn)
We appreciate all the feedback and contributions to improve LightZero, both algorithms and system designs.
@article{niu2024lightzero,
title={LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios},
author={Niu, Yazhe and Pu, Yuan and Yang, Zhenjie and Li, Xueyan and Zhou, Tong and Ren, Jiyuan and Hu, Shuai and Li, Hongsheng and Liu, Yu},
journal={Advances in Neural Information Processing Systems},
volume={36},
year={2024}
}
@article{puunizero,
title={UniZero: Generalized and Efficient Planning with Scalable Latent World Models},
author={Pu, Yuan and Niu, Yazhe and Yang, Zhenjie and Ren, Jiyuan and Li, Hongsheng and Liu, Yu},
journal={Transactions on Machine Learning Research}
}
@article{xuan2024rezero,
title={ReZero: Boosting MCTS-based Algorithms by Backward-view and Entire-buffer Reanalyze},
author={Xuan, Chunyu and Niu, Yazhe and Pu, Yuan and Hu, Shuai and Liu, Yu and Yang, Jing},
journal={arXiv preprint arXiv:2404.16364},
year={2024}
}
@article{pu2025one,
title={One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning},
author={Pu, Yuan and Niu, Yazhe and Tang, Jia and Xiong, Junyu and Hu, Shuai and Li, Hongsheng},
journal={arXiv preprint arXiv:2509.07945},
year={2025}
}
This project has been developed partially based on the following pioneering works on GitHub repositories. We express our profound gratitude for these foundational resources:
We would like to extend our special thanks to the following contributors @PaParaZz1, @karroyan, @nighood, @jayyoung0802, @timothijoe, @TuTuHuss, @HarryXuancy, @puyuan1996, @HansBug for their valuable contributions and support to this algorithm library.
Thanks to all who contributed to this project:
All code within this repository is under Apache License 2.0.
Python
93.3%
C++
4.9%
[NeurIPS 2023 Spotlight] LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios (awesome MCTS)
See the code
Updated on 2026.03.11 LightZero-v0.2.0
LightZero is a lightweight, efficient, and easy-to-understand open-source algorithm toolkit that combines Monte Carlo Tree Search (MCTS) and Deep Reinforcement Learning (RL).
English | 简体中文(Simplified Chinese) | Documentation | LightZero Paper | UniZero Paper | ReZero Paper | 🔥ScaleZero Paper
The integration of Monte Carlo Tree Search and Deep Reinforcement Learning,
exemplified by AlphaZero and MuZero,
has achieved unprecedented performance levels in various games, including Go and Atari.
This advanced methodology has also made significant strides in scientific domains like protein structure prediction and the search for matrix multiplication algorithms.
The following is an overview of the historical evolution of the Monte Carlo Tree Search algorithm series:

LightZero is an open-source algorithm toolkit that combines Monte Carlo Tree Search (MCTS) and Reinforcement Learning (RL) for PyTorch. It supports a range of MCTS-based RL algorithms and applications, offering several key advantages:
For further details, please refer to Features, Framework Structure and Integrated Algorithms.
LightZero aims to promote the standardization of the MCTS+RL algorithm family to accelerate related research and applications. A performance comparison of all implemented algorithms under a unified framework is presented in the Benchmark.
Lightweight: LightZero integrates multiple MCTS algorithm families and can solve decision-making problems with various attributes in a lightweight framework. The algorithms and environments LightZero implemented can be found here.
Efficient: LightZero uses mixed heterogeneous computing programming to improve computational efficiency for the most time-consuming part of MCTS algorithms.
Easy-to-understand: LightZero provides detailed documentation and algorithm framework diagrams for all integrated algorithms to help users understand the algorithm's core and compare the differences and similarities between algorithms under the same paradigm. LightZero also provides function call graphs and network structure diagrams for algorithm code implementation, making it easier for users to locate critical code. All the documentation can be found here.
The above picture is the framework pipeline of LightZero. We briefly introduce the three core modules below:
Model:
Model is used to define the network structure, including the __init__ function for initializing the network structure and the forward function for computing the network's forward propagation.
Policy:
Policy defines the way the network is updated and interacts with the environment, including three processes: the learning process, the collecting process, and the evaluation process.
MCTS:
MCTS defines the structure of the Monte Carlo search tree and the way it interacts with the Policy. The implementation of MCTS includes two languages: Python and C++, implemented in ptree and ctree, respectively.
For the file structure of LightZero, please refer to lightzero_file_structure.
LightZero is a library with a PyTorch implementation of MCTS algorithms (sometimes combined with cython and cpp), including:
The environments and algorithms currently supported by LightZero are shown in the table below:
| Env./Algo. | AlphaZero | MuZero | Sampled MuZero | EfficientZero | Sampled EfficientZero | Gumbel MuZero | Stochastic MuZero | UniZero | Sampled UniZero | ReZero |
|---|---|---|---|---|---|---|---|---|---|---|
| TicTacToe | ✔ | ✔ | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | ✔ | 🔒 | 🔒 |
| Gomoku | ✔ | ✔ | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | ✔ | 🔒 | ✔ |
| Connect4 | ✔ | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | ✔ |
| 2048 | --- | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | ✔ | ✔ | 🔒 | 🔒 |
| Chess | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| Go | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| CartPole | --- | ✔ | 🔒 | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | ✔ |
| Pendulum | --- | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | ✔ | 🔒 |
| LunarLander | --- | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 |
| BipedalWalker | --- | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 |
| Atari | --- | ✔ | 🔒 | ✔ | ✔ | ✔ | ✔ | ✔ | 🔒 | ✔ |
| DeepMind Control | --- | --- | ✔ | --- | ✔ | 🔒 | 🔒 | 🔒 | ✔ | 🔒 |
| MuJoCo | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| MiniGrid | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 | 🔒 |
| Bsuite | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 | 🔒 |
| Memory | --- | ✔ | 🔒 | ✔ | ✔ | 🔒 | 🔒 | ✔ | 🔒 | 🔒 |
| SumToThree (billiards) | --- | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
| MetaDrive | --- | 🔒 | 🔒 | 🔒 | ✔ | 🔒 | 🔒 | 🔒 | 🔒 | 🔒 |
(1): "✔" means that the corresponding item is finished and well-tested.
(2): "🔒" means that the corresponding item is in the waiting-list (Work In Progress).
(3): "---" means that this algorithm doesn't support this environment.
You can install the latest LightZero in development from the GitHub source codes with the following command:
git clone https://github.com/opendilab/LightZero.git
cd LightZero
pip3 install -e .
Kindly note that LightZero currently supports compilation only on Linux and macOS platforms.
We are actively working towards extending this support to the Windows platform.
Your patience during this transition is greatly appreciated.
We also provide a Dockerfile that sets up an environment with all dependencies needed to run the LightZero library. This Docker image is based on Ubuntu 20.04 and installs Python 3.8, along with other necessary tools and libraries. Here's how to use our Dockerfile to build a Docker image, run a container from this image, and execute LightZero code inside the container.
mkdir lightzero-docker
mv Dockerfile lightzero-docker/
cd lightzero-docker/
docker build -t ubuntu-py38-lz:latest -f ./Dockerfile .
docker run -dit --rm ubuntu-py38-lz:latest /bin/bash
python ./LightZero/zoo/classic_control/cartpole/config/cartpole_muzero_config.py
Train a MuZero agent to play CartPole:
cd LightZero
python3 -u zoo/classic_control/cartpole/config/cartpole_muzero_config.py
Train a MuZero agent to play Pong:
cd LightZero
python3 -u zoo/atari/config/atari_muzero_segment_config.py
Train an experimental Ray async MuZero agent to play Pong:
cd LightZero
python3 -u zoo/atari/config/atari_muzero_segment_async_config.py
Train a MuZero agent to play TicTacToe:
cd LightZero
python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
Train a UniZero agent to play Pong:
cd LightZero
python3 -u zoo/atari/config/atari_unizero_segment_config.py
The Atari segment async pipeline uses Ray collector/evaluator actors while keeping the learner and replay buffer in the driver process. It supports MuZero-family segment policies and UniZero segment policies through the explicit --async-pipeline config flag. In a same-window Pong MuZero rjob comparison (2026-07-02, 2 collector actors, ~6.2 h wall-clock), async collected 809.5k env steps (≈36.3 env steps/s) versus 499.8k env steps (≈22.4 env steps/s) for the synchronous baseline — about 1.6× end-to-end throughput, with eval rewards statistically comparable at this early training stage. For design details and UniZero differences, see Async Pipeline and 中文说明.
The LightZero documentation can be found here. It contains tutorials and the API reference.
For those interested in customizing environments and algorithms, we provide relevant guides:
Should you have any questions, feel free to contact us for support.
Factored/Gaussian policy representation on three classic continuous action space games: Pendulum-v1, LunarLanderContinuous-v2, BipedalWalker-v3
and two MuJoCo continuous action space games: Hopper-v3, Walker2d-v3."Factored Policy" indicates that the agent learns a policy network that outputs a categorical distribution. After manual discretization, the dimensions of the action space for the five environments are 11, 49 (7^2), 256 (4^4), 64 (4^3), and 4096 (4^6), respectively. On the other hand, "Gaussian Policy" refers to the agent learning a policy network that directly outputs parameters (mu and sigma) for a Gaussian distribution.
The following are the detailed paper notes (in Chinese) of the above algorithms:
You can also refer to the relevant Zhihu column (in Chinese): In-depth Analysis of MCTS+RL Frontier Theories and Applications.
The following are the overview MCTS principle diagrams of the above algorithms:
Here is a collection of research papers about Monte Carlo Tree Search. This Section will be continuously updated to track the frontier of MCTS.
File an issue on Github
Open or participate in our discussion forum
Discuss on LightZero discord server
Contact our email (opendilab@pjlab.org.cn)
We appreciate all the feedback and contributions to improve LightZero, both algorithms and system designs.
@article{niu2024lightzero,
title={LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios},
author={Niu, Yazhe and Pu, Yuan and Yang, Zhenjie and Li, Xueyan and Zhou, Tong and Ren, Jiyuan and Hu, Shuai and Li, Hongsheng and Liu, Yu},
journal={Advances in Neural Information Processing Systems},
volume={36},
year={2024}
}
@article{puunizero,
title={UniZero: Generalized and Efficient Planning with Scalable Latent World Models},
author={Pu, Yuan and Niu, Yazhe and Yang, Zhenjie and Ren, Jiyuan and Li, Hongsheng and Liu, Yu},
journal={Transactions on Machine Learning Research}
}
@article{xuan2024rezero,
title={ReZero: Boosting MCTS-based Algorithms by Backward-view and Entire-buffer Reanalyze},
author={Xuan, Chunyu and Niu, Yazhe and Pu, Yuan and Hu, Shuai and Liu, Yu and Yang, Jing},
journal={arXiv preprint arXiv:2404.16364},
year={2024}
}
@article{pu2025one,
title={One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning},
author={Pu, Yuan and Niu, Yazhe and Tang, Jia and Xiong, Junyu and Hu, Shuai and Li, Hongsheng},
journal={arXiv preprint arXiv:2509.07945},
year={2025}
}
This project has been developed partially based on the following pioneering works on GitHub repositories. We express our profound gratitude for these foundational resources:
We would like to extend our special thanks to the following contributors @PaParaZz1, @karroyan, @nighood, @jayyoung0802, @timothijoe, @TuTuHuss, @HarryXuancy, @puyuan1996, @HansBug for their valuable contributions and support to this algorithm library.
Thanks to all who contributed to this project:
All code within this repository is under Apache License 2.0.
Python
93.3%
C++
4.9%