Verl-Tool

VerlTool: A unified and easy-to-extend tool-agent training framework based on verl.

| Paper | Quick Start | Training Recipes | DeepWiki | WeChat Group | Discord |

---

News

• [2026/06/01] 🏆 Our paper has been accepted by TMLR 2026!
• [2026/05/01] 🏆 Our paper received the Best Paper Award at ICLR 2026 SPOT!
• [2025/11/10] VerlTool has re-organized its codebase to improve modularity and maintainability, supporting to the latest verl (0.6.0) and vllm (0.11.0) versions. Please refer to the verl-tool v0.6.0.dev Upgrade Notes for more details.
• [2025/09/02] VerlTool's tech report is out! See on Hugging Face Daily Paper!
• [2025/06/30] We reproduce Search-R1 with even higher performance on the same benchmarks! See PR and training README for more details.
• [2025/06/28] We support NL2SQL tool RL training. See NL2SQL README for more details.
• [2025/06/26] We support DAPO recipe training. See DAPO.md for more details.
• [2025/06/18] VerlTool now officially supports Trajectory-Level asynchronous, speeding up the rollout generation with tool calling by at least 2x! see asyncRL.md for more details.
• [2025/06/16] We have updated the verl submodule to the latest version (06/16) and modified some code to adapt to the new version.
• [2025/06/13] We integrated DeepWiki for Verl-Tool. Feel free to browse the AI-generated docs and chat with Verl-tool codes.
• [2025/06/06] We have updated a detailed design overview in the README, including how to add new tools, how to use the tool server, and how to train your own models with verl-tool.
• [2025/05/31] We released the Verl-tool training/evaluation code with ToRL training as an initial example (see X post). We are working on the paper and will release it very soon.

Features

• 🔧 Complete decoupling of actor rollout and environment interaction - We use verl as a submodule to benefit from ongoing verl repository updates. All tool calling is integrated via a unified API, allowing you to easily add new tools by simply adding a Python file and testing independently.
• 🌍 Tool-as-environment paradigm - Each tool interaction can modify the environment state. We store and reload environment states for each trajectory.
• ⚡ Native RL framework for tool-calling agents - verl-tool natively supports multi-turn interactive loops between agents and their tool environments.
• 📊 User-friendly evaluation suite - Launch your trained model with OpenAI API alongside the tool server. Simply send questions and get final outputs with all interactions handled internally. See benchmarks.

📚 Contents Link

• 📖 Installation Guide
• ⚡ Synchronous Rollout Design
• 🔄 Asynchronous Rollout Design
• 🛠️ Tool Server Design
• 🎯 Training Guide
• 📊 Evaluation Guide
• 🔧 Update Verl Submodule Version
• 📈 Existing Training Results
• 🤝 Contributing Guide

Core Contributors

Advisors

Acknowledgements

We thank the following open-source projects for making verl-tool possible:

• VLLM and SGLang for their fast LLM inference support!
• verl for the excellent RL framework design.
• SearchR1, RAGEN, and ToRL for their early-stage exploration of tool-agent RL training.

We thank Netmind.AI, SeaAI Lab, and Map for GPU support!

Community Projects Inspired by Verl-Tool

• AgentFlow: In-the-Flow Agentic System Optimization

Star History

Badge

Citation

@article{jiang2025verltool,
  title={VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use},
  author={Jiang, Dongfu and Lu, Yi and Li, Zhuofeng and Lyu, Zhiheng and Nie, Ping and Wang, Haozhe and Su, Alex and Chen, Hui and Zou, Kai and Du, Chao and others},
  journal={arXiv preprint arXiv:2509.01055},
  year={2025}
}