Vime

中文版 · Repository

Vime is an LLM post-training framework for RL scaling, built on slime. It keeps slime's training stack and data-generation design while using vLLM (with vllm-router) as the default rollout backend. Vime provides two core capabilities:

1. High-performance training: Efficient training in various modes by connecting Megatron with vLLM;
2. Flexible data generation: Arbitrary training data generation workflows through custom data generation interfaces and server-based engines.

Vime inherits broad model support from slime, including:

• Qwen series (Qwen3.6, Qwen3.5, Qwen3Next, Qwen3MoE, Qwen3, Qwen2.5);
• DeepSeek V3 series (DeepSeek V3, V3.1, DeepSeek R1);
• Llama 3.

Discussion channels:

• slack
• wechat group

Positioning

The vLLM community horizontally supports many LLM post-training frameworks, including (in alphabetical order) NeMo RL, OpenRLHF, prime-rl, SkyRL, verl, and so on. We built the Vime project to seamlessly bring slime's proven training paradigm into the vLLM ecosystem, offering a production-ready bridge that aligns both projects' rapid release cycles. We hope that users with different needs can find the right vLLM-ecosystem choice for their workflows. The vLLM community will continue to support the vLLM integration in these post-training frameworks.

Table of Contents

• Vime
  • Positioning
  • Table of Contents
  • Architecture Overview
  • Quick Start
    • Agentic RL examples
  • Arguments Walkthrough
  • Engine Deployment
  • Correctness, Stability, and CI
  • Code Reading Path
  • Developer Guide
  • slime doc
  • FAQ
  • Acknowledgements
  • Citation

Architecture Overview

Module Descriptions:

• training (Megatron): Responsible for the main training process, reads data from the Data Buffer, and synchronizes parameters to the rollout module after training.
• rollout (vLLM + router): Launches vLLM inference engines and routes generation requests; custom generate functions can wrap generation with multi-turn loops, tool calls, environment/sandbox interaction, and verifier-based rewards.
• data buffer: A bridge module that manages prompt initialization, custom data, and rollout generation methods, including agentic workflows that produce samples through the same interface.

Quick Start

For a comprehensive quick start guide covering environment setup, data preparation, training startup, and key code analysis, please refer to:

• Quick Start Guide

We also provide examples for some use cases not covered in the quick start guide; please check examples.

Agentic RL examples

Agentic workloads use the standard rollout / Data Buffer loop through Vime's customization interfaces; they are not a separate framework:

• examples/multi_agent: Multi-agent generation through --custom-generate-function-path.
• examples/fully_async: Fully asynchronous rollout for long-tail agent generation.
• examples/coding_agent_rl: End-to-end coding-agent RL with Claude Code or Codex, sandboxed tool use, test-based rewards, and token-correct trajectory segments.

See the Agentic RL Training Roadmap and Customization Guide. The coding-agent example ships an E2B-compatible backend, while the shared vime.agent.sandbox.Sandbox contract can be implemented for Docker, Modal, or local VMs.

Arguments Walkthrough

Arguments in Vime are divided into three categories:

1. Megatron arguments: Vime reads all arguments in Megatron. You can configure Megatron by passing arguments like --tensor-model-parallel-size 2.
2. vLLM arguments: vLLM server and engine options are exposed with a --vllm- prefix (for example, --vllm-gpu-memory-utilization). Router options live under two prefixes: vllm-router's native options are passed with --router- (for example, --router-policy round_robin, --router-request-timeout-secs), while Vime-side orchestration knobs that tell Vime where the router lives use --vllm-router- (--vllm-router-ip, --vllm-router-port). See vime/backends/vllm_utils/arguments.py for the full surface.
3. Framework-specific arguments: Shared Vime orchestration flags (rollout GPUs, data paths, RL algorithms, etc.). Please refer to vime/utils/arguments.py.

--rollout-num-gpus-per-engine sets the tensor parallel size of each vLLM engine. The default rollout entry is vime.rollout.vllm_rollout.generate_rollout.

For complete usage instructions, please refer to the Usage Documentation.

Engine Deployment

Vime keeps the Megatron and vLLM control surfaces close to the upstream engines while adding the RL dataflow around them. Beyond the argument pass-through described above, see:

• vLLM Config for optional YAML topology configuration, heterogeneous server groups, multi-model serving, and per-group overrides;
• PD Disaggregation for multi-turn and agentic workloads with different prefill/decode resource needs;
• router policies such as session affinity for multi-turn agents (see vLLM Config);
• Delta Weight Sync for disk-based updates of disaggregated rollout engines;
• External Rollout Engines for serving managed outside the training job. Serving can use an independent environment; disk transport avoids an NCCL group between training and serving. Different GPU models or vendors still require compatible model formats, precision, and vLLM hardware support.

Correctness, Stability, and CI

RL bugs can be silent. Vime keeps the dataflow explicit and supports separate rollout-only and train-only debugging paths. CPU unit tests, customization-hook contract tests, and GPU end-to-end suites protect different parts of this workflow. Buildkite runs always-on CPU checks; GPU suites require the manual gate, so a green CPU build is not GPU validation.

Useful engineering docs:

• CI
• Debugging
• Reproducibility
• Fault Tolerance
• Trace Viewer
• Profiling

Code Reading Path

Start from the training loop and follow the calls only as deep as needed:

train.py: train
├─ vime/ray/placement_group.py       Ray resource and worker initialization
├─ vime/ray/rollout.py              RolloutManager.generate: rollout orchestration
│  └─ vime/rollout/vllm_rollout.py  Sample generation and reward computation
└─ vime/ray/actor_group.py          RayTrainGroup.async_train: training dispatch
   └─ vime/backends/megatron_utils/actor.py
      ├─ model.py                    Megatron model execution
      └─ loss.py                     RL losses and advantages

On a first pass, treat vime/utils/arguments.py as the configuration entry point. The deployment details in vime/backends/vllm_utils/ and the weight-sync implementations under vime/backends/megatron_utils/update_weight/ can wait until you need to change those areas.

Developer Guide

• Contributions are welcome! If you have suggestions for new features, performance tuning, or feedback on user experience, feel free to submit an Issue or PR.

• Use pre-commit to ensure code style consistency for your commits:

apt install pre-commit -y
pre-commit install

# run pre-commit to ensure code style consistency
pre-commit run --all-files --show-diff-on-failure --color=always

• For debugging tips, please refer to the Debugging Guide

slime doc

Vime is derived from slime. The following upstream resources and in-repo guides still use the slime naming and remain the reference for shared concepts (Megatron integration, customization, advanced topics):

• Upstream repository: THUDM/slime
• English docs in this repo: docs/en/
• Chinese docs in this repo: docs/zh/

FAQ

For frequently asked questions, please see the Q&A

Acknowledgements

Vime builds on ideas and infrastructure from the open-source RL ecosystem. We especially thank the slime community, whose great work Vime is directly built on. We also thank SkyRL and verl, whose excellent work we referenced. Vime is maintained by the vLLM community.

Citation

@misc{vime,
  author       = {Vime Contributors},
  title        = {Vime: An LLM post-training framework with vLLM for RL Scaling},
  year         = {2026},
  howpublished = {\url{https://github.com/vllm-project/vime}},
  urldate      = {2026-06}
}