Overview

We’re entering the era of experience, where large language models (LLMs) learn not just from static datasets, but from interactive experience gathered in complex, expressive environments.

As a step toward this, we introduce GEM — a General Experience Maker for LLMs — an open-source environment suite designed for training agentic LLMs via online reinforcement learning.

Like OpenAI Gym for traditional RL, GEM provides a standardized API and a growing collection of diverse environments. It is training framework-agnostic and supports seamless integration with six popular RL training frameworks including Oat and Tinker, offering:

• 🧩 Clean, composable environment APIs
• ⚙️ Async vectorized execution for high-throughput simulation
• 🔧 Tool integration & custom wrappers
• 🧠 Multi-environment training
• 🎈 Ready-to-use benchmark environments and algorithms

Links

• 📜 Initial Blog
• 🚀 Blog release tweet
• 📄 Paper
• 📘 Documentation

Installation

pip install -U gem-llm

Or install from source for the latest version:

git clone https://github.com/axon-rl/gem.git
cd gem
pip install -e .

Please check Getting Started for more setup details.

🔥 You can jump into examples to quickly start your agentic RL training with GEM & your favorite training framework.

Interface

GEM's interface closely follows OpenAI-Gym's API. Here's an example using the game:GuessTheNumber-v0 environment:

import gem

# List all supported environments
gem.print_envs()

# Initialize the environment
env = gem.make("game:GuessTheNumber-v0")

# Reset the environment to generate the first observation
observation, info = env.reset()

# Start the agent-environment loop
while True:
    action = env.sample_random_action() # insert policy here, e.g.,
    # (pseudocode) action = llm.generate(observation)

    # apply action and receive next observation, reward
    # and whether the episode has ended
    next_observation, reward, terminated, truncated, info = env.step(action)
    print("OBS", observation)
    print("ACT", action)

    # update the policy (online) here
    # e.g., policy = learn(policy, observation, action, reward, info)

    observation = next_observation
    # Exit when the episode terminates
    if terminated or truncated:
        break

Features

1. Environments consist of tasks and (optional) tools. Tool-calling is achieved via an environment wrapper, as demonstrated here.
2. GEM is training framework-agnostic, and we demonstrate its integration with six popular RL training frameworks.
3. We provide implementations and benchmarking results for different algorithms across a diverse set of environments.

Supported Tasks

Supported Tools

Supported Frameworks

Examples of training agents on GEM environments with all above frameworks can be found in here!

Supported Algorithms

Please check out our paper for a more detailed description for each algorithm and empirical results showing their tradeoffs.

Contributing

We welcome all forms of contribution — from adding new environments to integrating additional training frameworks. We're planning to write a community-driven technical report, and major contributors will be recognized with authorship. Join discord to discuss more!

Acknowledgement

• This work is supported by Sea AI Lab for computing resources.
• Our code learns from and builds on several awesome projects such as gym, rllm, TextArena, Search-R1, ReasoningGym.
• The training example code is built on Oat, Tinker, Verl, RL2, ROLL, OpenRLHF.

Citation

If you find our works useful for your research, please consider citing:

• GEM paper (please prioritize citing the paper unless you believe the blog is a better fit):

  @article{liu2025gem,
    title={GEM: A Gym for Agentic LLMs},
    author={Liu, Zichen and Sims, Anya and Duan, Keyu and Chen, Changyu and Yu, Simon and Zhou, Xiangxin and Xu, Haotian and Xiong, Shaopan and Liu, Bo and Tan, Chenmien and others},
    journal={arXiv preprint arXiv:2510.01051},
    year={2025}
  }
  

• GEM blog:

  @misc{liu2025gemblog,
    title={GEM: A Gym for Generalist LLMs},
    author={Liu, Zichen and Sims, Anya and Duan, Keyu and Chen, Changyu and Yang, Diyi and Lee, Wee Sun and Lin, Min},
    year={2025},
    howpublished={\url{https://axon-rl.notion.site/gem}},
    note={Notion Blog},
  }