RolandGao/gobench

GoBench: Evaluating LLMs on the Game of Go

Python

1

12 commits

updated Sep 17, 2026

See the code

See what people are saying (1)

SourceMessageScoreDate

GoBench: Evaluating LLMs on the game of Go [R] (r/MachineLearning)

GoBench evaluates LLMs on 9x9 Go games against a ladder of KataGo opponents, from random to superhuman. It measures general reasoning ability, strongly correlates with ARC-AGI 2 (r=0.83 correlation), and remains highly unsaturated. GPT-6 Astra max achieves 2500 Elo, much lower than the best KataGo,…

14

Sep 16, 2026

README

GoBench

paper: https://rolandgao.com/gobench.pdf

Active leaderboard: https://rolandgao.com/blog/gobench/ Screenshot 2026-09-15 at 8 09 28 PM

Setup

Use Python 3.12 or newer. From the repository root:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .

Run

Edit the CONFIG = ArenaConfig(...) block near the top of arena.py, then run:

python arena.py

Summary

Generate a summary of the configured historical runs:

python arena.py --summary

Citation

@misc{gao2026gobench,
  title = {{GoBench}: Evaluating {LLMs} on the Game of {Go}},
  author = {Gao, Roland},
  year = {2026},
  url = {https://rolandgao.com/gobench.pdf}
}

License

MIT License

Contributors

RolandGao

12 commits

RolandGao/gobench

GoBench: Evaluating LLMs on the Game of Go

Python

1

12 commits

updated Sep 17, 2026

See the code

See what people are saying (1)

SourceMessageScoreDate

GoBench: Evaluating LLMs on the game of Go [R] (r/MachineLearning)

GoBench evaluates LLMs on 9x9 Go games against a ladder of KataGo opponents, from random to superhuman. It measures general reasoning ability, strongly correlates with ARC-AGI 2 (r=0.83 correlation), and remains highly unsaturated. GPT-6 Astra max achieves 2500 Elo, much lower than the best KataGo,…

14

Sep 16, 2026

README

GoBench

paper: https://rolandgao.com/gobench.pdf

Active leaderboard: https://rolandgao.com/blog/gobench/ Screenshot 2026-09-15 at 8 09 28 PM

Setup

Use Python 3.12 or newer. From the repository root:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .

Run

Edit the CONFIG = ArenaConfig(...) block near the top of arena.py, then run:

python arena.py

Summary

Generate a summary of the configured historical runs:

python arena.py --summary

Citation

@misc{gao2026gobench,
  title = {{GoBench}: Evaluating {LLMs} on the Game of {Go}},
  author = {Gao, Roland},
  year = {2026},
  url = {https://rolandgao.com/gobench.pdf}
}

License

MIT License

Contributors

RolandGao

12 commits

Languages

Python

95.6%

TeX

4.4%