NeMo Gym

Requirements • Quick Start • Environment Tutorials • Available Environments • Documentation & Resources • Community & Support • Citations

NeMo Gym is a library for evaluating and improving models and agents using environments. NeMo Gym provides infrastructure to develop environments, scalably run evaluation and training, and a collection of popular benchmarks and training environments.

An environment is the complete system an agent interacts with to complete a task. It consists of a dataset (tasks to solve), an agent harness (how the model interacts with the world), a verifier (task completion scoring), and state (per-task execution context).

🎯 When to Use NeMo Gym

• You need to evaluate models or agents in stateful environments (e.g. code execution, tool calling, sandboxes)
• You want reproducible evaluation across teams using shared environments and verifiers
• You need to use environments at scale — multiple repeats per task, or thousands of concurrent requests for training
• You want to seamlessly transition between evaluation, agent optimization, and training

If you're scoring model outputs with a stateless check and don't need scale or training, a script is probably sufficient.

🏆 What NeMo Gym Provides

• Modular, extensible interfaces for agents, environments, tasks, and verifiers
• Environment hub of popular benchmarks and training environments
• Use your own agents or choose from built-in harnesses
• Scale to thousands of concurrent environments
• Train with the RL framework of your choice
• Optional OpenTelemetry tracing across the agent, model, and resources servers
• Battle-tested in production Nemotron training

🌎 Ecosystem

NeMo Gym is a component of NVIDIA NeMo, a GPU-accelerated platform for training generative AI models and optimizing AI agents. NeMo Gym is integrated with the broader agentic ecosystem - see the Ecosystem page for more details.

Environment Libraries: Seamlessly combine environments and benchmarks from other libraries alongside NeMo Gym environments. Examples: Aviary • Harbor • OpenEnv • Reasoning Gym • Verifiers

Training Framework Libraries: Use environments for SFT and RL training. NeMo RL • Unsloth • VeRL

Agent Harnesses: Agent harnesses for evaluation and training available out of the box. Examples: OpenHands • Mini SWE Agent • LangGraph

[!IMPORTANT] NeMo Gym is currently in early development. You should expect evolving APIs, incomplete documentation, and occasional bugs. We welcome contributions and feedback - for any changes, please open an issue first to kick off discussion!

📣 News

• [09/03/2026] Release v0.6.0: Highlights:
  • Use supported external agent harnesses during RL training while preserving exact token IDs across multi-step runs
  • Compare fixed and routed model strategies on the same benchmark with Switchyard
  • Validate and debug rollouts with automatic health checks, traces, and token, tool-call, turn, and latency diagnostics
  • Evaluate multiple agents and datasets in one run with task-level harness routing
  • Scale vLLM evaluation jobs across GPUs or Slurm nodes for higher rollout concurrency

Previous News

• [08/06/2026] Release v0.5.0: Highlights:

  • Seven sandbox providers: Docker, Daytona, ECS Fargate, Enroot, and OpenShell join OpenSandbox and Apptainer; large-scale OpenSandbox reliability significantly improved
  • Four new agent harnesses: Codex CLI, KiloCode, RemoteAgent, and anyswe_agent
  • Recompute rewards from stored rollouts without re-running inference with gym eval reverify
  • Rollout observability joined end-to-end: model-call capture, agent observations, and a standardized ng_trajectory schema
  • 21 new environments across six domains: Agentic, Knowledge and instruction following, Long context, Science and coding, Translation and multilingual, and Reasoning

• [07/01/2026] Release v0.4.0: Unified gym CLI, BLADE diagnostics, agent skill evaluation, pluggable sandboxes, more agent harnesses (OpenCode, OpenClaw, Pi), hosted inference providers, and new benchmarks.

• [06/04/2026] Release v0.3.0: 70+ new environments, Nemotron 3 Ultra training datasets, VeRL integration, and out-of-the-box harnesses including Claude Code and Hermes.

📋 Requirements

NeMo Gym is designed to run on standard development machines:

Additional Requirements

• API Keys: OpenAI API key with available credits (for the quickstart examples)
  • Other model providers supported (Azure OpenAI, self-hosted models via vLLM)
• Ray: Automatically installed as a dependency (no separate setup required)

🚀 Quick Start

Requires Python 3.13.14+ on x86_64 or ARM64 (Linux, macOS, Windows via WSL2). No GPU required. See the Getting Started docs for a more comprehensive walkthrough.

Install NeMo Gym:

Requires uv and Python 3.13.14+.

git clone git@github.com:NVIDIA-NeMo/Gym.git
cd Gym
uv venv --python 3.13.14 && source .venv/bin/activate
uv sync

Configure your model:

This quickstart uses OpenAI. NeMo Gym supports local and hosted inference — see Configure Model for vLLM, Fireworks, OpenRouter, and others.

Create env.yaml in the project root:

policy_base_url: https://api.openai.com/v1
policy_api_key: <your-openai-api-key>
policy_model_name: gpt-4.1-2025-04-14

Run Evaluation

Run your agent on a set of tasks and score the results. This example uses a simple tool calling agent simple_agent with the mcqa (multiple-choice Q&A) environment and its included example data.

1. Start servers

NeMo Gym uses local servers to coordinate your model, agent, and task verification. Start them first:

gym env start \
    --resources-server mcqa \
    --model-type openai_model

You should see three server instances starting:

[1] mcqa (resources_servers/mcqa)
[2] mcqa_simple_agent (responses_api_agents/simple_agent)
[3] policy_model (responses_api_models/openai_model)

2. Evaluate your agent

In a new terminal, run your agent on a single task to verify everything works:

source .venv/bin/activate

gym eval run --no-serve \
    --agent mcqa_simple_agent \
    --input resources_servers/mcqa/data/example.jsonl \
    --output results/mcqa_rollouts.jsonl \
    --limit 5 \
    --num-repeats 1

You should see a progress bar followed by aggregate metrics:

Collecting rollouts: 100%|██████| 5/5 [01:22<00:00, 16.44s/it]

Key metrics for mcqa_simple_agent:
{
    "mean/reward": 0.8,
    "pass@1[avg-of-1]/accuracy": 80.0,
    "pass@1/accuracy": 80.0
}
Finished rollout collection! View results at:
Fully materialized inputs: results/mcqa_rollouts_materialized_inputs.jsonl
Rollouts: results/mcqa_rollouts.jsonl
Aggregate metrics: results/mcqa_rollouts_aggregate_metrics.json

For per-task pass rates, see the gym eval profile command.

Using the NeMo-Gym Container with VLM or Audio/Video Benchmarks

The NeMo-Gym container omits packages with bundled codec libraries (opencv-python-headless, torchvision, torchaudio) to avoid shipping royalty-bearing binaries. If you are running VLM or audio/video benchmarks inside the container, restore them first:

bash docker/install_codec_deps.sh

This installs the packages at the same versions used during the container build. It is safe to run multiple times.

Next Steps

• Browse Environments — Browse available environments for evaluation and training.
• Agents — Explore available agent harnesses and learn how to integrate your own.
• Training — Improve your agent or model with RL or fine-tuning.
• Build Custom Environments — Create your own evaluation or training environments.

🧭 Environment Tutorials

Learn how to build custom environments through hands-on tutorials. Here are popular starting points:

See all environment tutorials for additional patterns and advanced topics.

📦 Available Environments

Environments for training and evaluation.

Each resources server includes example data, configuration files, and tests. See each server's README for details.

The Dataset column links to publicly available datasets (e.g., on HuggingFace). A - means the train/validation data has not been publicly released yet, or that it is procedurally generated using a provided script. If no data is released yet, new data can be generated, or the environment can be used as a reference. Each server includes 5 example tasks in data/example.jsonl.