EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis

If you like our project, please give us a star ⭐ on GitHub. We greatly appreciate your support.

📣 Latest News

• [Oct 12, 2026]: 🚀 We released Agent-Reflex, teaching language agents to act reflectively under noisy environments. See the GitHub.
• [Aug 10, 2026]: 🚀 We released Agent-World, evolving general agent intelligence by scaling real-world environments. See the arXiv and GitHub.
• [July 02, 2026]: 🎉 Our paper EnvScaler has been accepted at ACL 2026 Findings!
• [Jan 12, 2026]: 📄 Our EnvScaler paper is now available on arXiv and Hugging Face Paper.
• [Jan 09, 2026]: 🚀🚀🚀 Full codebase of EnvScaler released, together with the 🤗 HF models (Qwen3-1.7B, 4B, and 8B) and datasets.

🎬 Demo

Env-Agent-User Interaction

Env-Agent Interaction

Building Environment From Scratch

To locally run the demo that interacting with Envs:

cd interact_with_env
python app.py

To locally run the demo that builing Envs from scratch:

cd skel_builder
python env_build_demo.py

📦 Dataset & Models

We provide EnvScaler’s data and models (after SFT+RL) as follows:

📑 Contents

• 🎬 Demo
• 📦 Dataset & Models
• 👀 Overview
• 📊 Results
• 📁 Project Structure
  • skel_builder/ - Environment Skeleton Construction
  • scen_generator/ - Scenario Generation
  • interact_with_env/ - Agent-Environment Interaction
  • sft/ - Supervised Fine-Tuning
  • rl/ - Reinforcement Learning
  • evaluation/ - Evaluation Guide
• 🚀 Quick Start
• 📚 Citation
• 📞 Contact

👀 Overview

EnvScaler is an automated, scalable framework that realizes executable, stateful, tool-interactive environments via programmatic synthesis, for training LLM agents.

Overview of EnvScaler.

SkelBuilder is the first stage of EnvScaler. It (1) mines potential Env descriptions from existing open-source textual tasks; (2) plans the corresponding state schema and business rules, and generates a fully-functional Python class whose methods expose tool interfaces; (3) performs a dual-agent loop for Env quality inspection (one agent invokes tools, the other checks code, return values, and state changes), guaranteeing quality and consistency.

Framework of SkelBuilder.

ScenGenerator is the second stage for synthesizing multiple Env scenarios. Given an Env skeleton, it first prompts LLMs to generate an initial state/database, then creates a challenging task that can be solved from that state. Finally, it decomposes the task into checklists, and converts each checkpoint into a Python Boolean function over the final state of the Env, providing rule-based, verifiable reward signals.

Framework of ScenGenerator.

📊 Results

With EnvScaler, we synthesized 191 environments and about 7K scenarios, and applied them to Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) for Qwen3 series models. Results on three benchmarks show that EnvScaler significantly improves LLMs' ability to solve tasks in complex environments involving multiturn, multi-tool interactions.

Statistics of 191 synthesized environments.

Performance comparison.

📁 Project Structure

EnvScaler/
├── skel_builder/              # Stage 1: Env Skeleton Construction
├── scen_generator/            # Stage 2: Scenario Generation
├── interact_with_env/         # Agent-Env Interaction
├── sft/                       # Supervised Fine-Tuning (SFT)
├── rl/                        # Reinforcement Learning (RL)
└── evaluation/                # Evaluation Guide

Module Description

💡 Tip: We provide detailed documentation under each module.

1. skel_builder/ – Env skeleton construction framework that automatically generates executable environment classes from existing tasks.
2. scen_generator/ – Scenario generation framework that produces state data, task scenarios, and checkpoint functions for an Env skeleton.
3. interact_with_env/ – Agent-Env interaction module supporting (1) collecting training data by interacting with synthesized Envs and (2) benchmark evaluation.
4. sft/ – Supervised fine-tuning implementation based on LlamaFactory.
5. rl/ – Reinforcement learning implementation based on the ROLL framework.
6. evaluation/ – Evaluation guide including BFCL, TauBench, and ACEBench.

🚀 Quick Start

1. Clone the repository

git clone https://github.com/RUC-NLPIR/EnvScaler 
cd EnvScaler

2. Install dependencies

pip install -r requirements.txt

💡 Note: Basic dependencies are included in requirements.txt. If you need SFT or RL training, please install extra dependencies following the corresponding sub-project documentation:

• SFT training: refer to sft/README.md to install LlamaFactory
• RL training: refer to rl/README.md to install the ROLL framework

3. Configure LLM service

Option 1: Use OpenAI API

Create a .env file in the project root and configure your OpenAI API key:

# .env
OPENAI_API_KEY=your-openai-api-key-here
OPENAI_BASE_URL=https://api.openai.com/v1 

Option 2: Use self-hosted model

You can deploy a local model with an OpenAI-compatible inference framework such as vLLM.

Deploy a model with vLLM:

vllm serve your-model-path \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code

⚠️ Important: Ensure the deployed model service supports Function Calling (FC) interface, see vLLM OpenAI-Compatible Server docs for details.

4. Verify configuration

Run the demo to verify your setup:

# Environment interaction demo
cd interact_with_env
python app.py

# Environment interaction Debug
cd interact_with_env
python run_main_debug.py

# Environment building demo
cd skel_builder
python env_build_demo.py

5. Start using

Now you can use each module of EnvScaler independently:

• Build environments: refer to skel_builder/README.md
• Generate scenarios: refer to scen_generator/README.md
• Collect training data: refer to interact_with_env/README.md
• Model training: refer to sft/README.md and rl/README.md
• Evaluation: refer to evaluation/README.md

📚 Citation

If you find our work helpful, please consider citing it. We greatly appreciate your support.

@article{song2026envscaler,
  title={EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis},
  author={Song, Xiaoshuai and Chang, Haofei and Dong, Guanting and Zhu, Yutao and Dou, Zhicheng and Wen, Ji-Rong},
  journal={arXiv preprint arXiv:2601.05808},
  year={2026}
}

@article{dong2026agent,
  title   = {Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence},
  author  = {Dong, Guanting and Lu, Junting and Huang, Junjie and Zhong, Wanjun and
             Liu, Longxiang and Huang, Shijue and Li, Zhenyu and Zhao, Yang and
             Song, Xiaoshuai and Li, Xiaoxi and Jin, Jiajie and Zhu, Yutao and
             Wang, Hanbin and Lei, Fangyu and Luo, Qinyu and Chen, Mingyang and
             Chen, Zehui and Feng, Jiazhan and Wen, Ji-Rong and Dou, Zhicheng},
  journal = {CoRR},
  volume  = {abs/2604.18292},
  year    = {2026},
  url     = {https://arxiv.org/abs/2604.18292}
}

📞 Contact

For any questions or feedback, please reach out to us at [email protected].