Why boonta

boonta is built to scale reinforcement learning up, in compute and in models. Training plays thousands of environments at once, keeps every device busy and grows from one GPU to a cluster without touching the algorithm. Models scale the same way, from a small network learning CartPole from scratch to LLMs and VLAs fine-tuned as agents, all trained by the same algorithms. And because the algorithm, the environment and the training loop stay out of each other's way, offline data, multi-agent self-play and curricula slot in without rewriting anything.

Podracers

🏎️ Anakin is for JAX environments. Acting, stepping and learning compile into one program that runs on every device.

🔧 Sebulba is for slow or non-JAX environments. Actors collect rollouts on some devices while learners train on the rest.

📼 Quadinaros is for offline RL and BC. It trains on a fixed dataset and only steps the environment to evaluate.

In the hangar

Algorithms

• On-policy: PPO · REPPO · PQN · GRPO
• Off-policy: DQN · SAC
• Offline: BC · IQL
• Recurrent: PPO · PuPO · PQN · GRPO · DQN · SAC · BC
• Multi-agent: IPPO · MAPPO · MMD · PSRO

Environments

• Control: Gymnax · Brax · MuJoCo Playground · Isaac Lab
• Grid worlds and open-ended: Jumanji · XLand-MiniGrid · Craftax
• Levels: Kinetix · JaxUED
• Multi-agent: JaxMARL · Mapox · BluffJAX · ConnectX
• Games: Atari · Pokémon Red on Peanut-GB · Super Mario Bros. · Wordle
• Anything else: Gymnasium vector environments

Curricula

• PLR replays the levels the agent learns most from
• League trains against a pool of earlier policies, picked by PSRO meta-solvers

Install

boonta is not on PyPI. Install it from a checkout with uv:

git clone https://github.com/noahfarr/boonta && cd boonta
uv sync                  # the library
uv sync --extra hangar   # the library, plus Hydra to run the configs in hangar

The library never imports hangar, Hydra or OmegaConf. Environment suites that need their own packages come as extras too, such as craftax. uv sync installs exactly the extras it is given, so pass every one you use in one command, for example uv sync --extra hangar --extra craftax.

Guides

• hangar: run training from the command line, config groups, several machines, sweeps
• Podracers: the training loops, and which one to pick
• Algorithms: the algorithm interface, and how to add an algorithm
• Networks: heads, recurrent torsos and stacks, mixed precision
• Environments: the environment interface, wrappers, and how to add an environment
• Datasets: offline data for quadinaros, and how to add a dataset
• Recipes: how a run is put together from its parts