⚠️ Development paused: Development in Forge has paused. LLM training at PyTorch is being consolidated in torchtitan.

torchforge

A PyTorch-native agentic RL library that lets you focus on algorithms—not infra.

Overview

The primary purpose of the torchforge ecosystem is to separate infra concerns from model concerns thereby making RL experimentation easier. torchforge delivers this by providing clear RL abstractions and one scalable implementation of these abstractions. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm.

Key features:

• Usability for rapid research (isolating the RL loop from infrastructure)
• Hackability for power users (all parts of the RL loop can be easily modified without interacting with infrastructure)
• Scalability (ability to shift between async and synchronous training and across thousands of GPUs)

📖 Documentation

View torchforge's hosted documentation: https://meta-pytorch.org/torchforge.

Tutorials

You can also find our notebook tutorials (coming soon)

Installation

torchforge requires PyTorch 2.9.0 with Monarch, vLLM, and torchtitan.

Conda

Install torchforge with:

conda create -n forge python=3.12
conda activate forge
./scripts/install.sh

ROCm Installation

ROCm users can install with the dedicated script:

conda create -n forge python=3.12
conda activate forge
./scripts/install_rocm.sh

Notes:

• PYTORCH_ROCM_ARCH is auto-detected when possible; set it manually if detection fails (example: PYTORCH_ROCM_ARCH=gfx942).
• ROCM_VERSION is auto-detected when possible; set it manually if detection fails (example: ROCM_VERSION=6.4).
• For ROCm 7.x, PyTorch stable 2.9.0 is not available. The script defaults to nightly wheels. You can override with PYTORCH_CHANNEL=stable|nightly.
• ROCm builds install Monarch with USE_TENSOR_ENGINE=0, so RDMA and distributed tensor features are disabled for now.
• Optional flags: --use-sudo (system packages) and --skip-amdsmi (skip amdsmi install).

The install script installs system dependencies along with torchforge. Note that this install script uses DNF, but could be easily extended to other Linux OS.

Optional: By default, the packages installation uses conda. If you want to install system packages on the target machine instead of conda, you can pass the --use-sudo flag to the installation script: ./scripts/install.sh --use-sudo.

Pixi

Pixi combines benefits of uv with access to conda forge for system dependencies. pixi.toml provides a manifest with build tasks with install as a the combined install all task.

Install pixi:

curl -fsSL https://pixi.sh/install.sh | bash

Install torchforge with pixi:

pixi run install

Note: We are actively working on enabling pure uv installation. Currently, Conda is the recommended approach. uv support is not fully working at the moment but is being tracked in issue #494.

After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 2 GPU devices):

python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml

Quick Start

To run SFT on a Llama3 8B model, run

python -m apps.sft.main --config apps/sft/llama3_8b.yaml

Citation

License

Source code is made available under a BSD 3 license, however you may have other legal obligations that govern your use of other content linked in this repository, such as the license or terms of service for third-party data and models.