RecreationWorld

Scalable and Verifiable Environments for Hybrid Computer-Use Agents

Website · Report · Hugging Face · ModelScope

Results · Quickstart · Citation

Overview

RecreationWorld is a five-platform framework for studying and improving hybrid computer-use agents that autonomously interleave GUI exploration, implementation with coding tools, and visual verification of their own running artifacts. By framing recreation around a running reference as an executable oracle, it turns open-source applications into scalable, verifiable training experience that transfers beyond recreation, while RecreationBench provides 250 held-out tasks with reference-grounded programmatic and visual evaluation.

Recreation workflow

Each task gives the agent a high-level request, interactive access to a running reference, and both GUI-control and software-development tools. The agent decides when to explore the reference, implement source code, build and launch its candidate, inspect the result, and revise it—forming a recurring explore–implement–verify loop rather than a fixed sequence of stages.

The final candidate is evaluated by a frozen suite of reference-validated programmatic and visual assertions. Scoring depends on observable behavior rather than source-level similarity, so implementations remain free to use different languages, frameworks, and architectures.

RecreationBench Results

Scores are macro-averaged within each platform and then equally weighted across platforms. Average is the unweighted mean of Prog and VLM; estimated costs assume 90% cache reads.

Quickstart

Install uv, then run from the repository root:

uv sync
uv run rb run --help

A scored run also needs a matching frozen task bundle from Hugging Face or ModelScope, a prepared execution environment, and model and judge endpoints. Choose a platform for setup and batch runs:

The canonical task index is in tasks/.

To verify the checkout, run these offline checks; they do not require benchmark data or credentials:

uv run python scripts/release/smoke_providers.py
uv run python scripts/release/smoke_runtime.py

If you have any questions, please contact xiezhihui.xzh@alibaba-inc.com or gaochang.gao@alibaba-inc.com.

Citation

If you find this environment useful, please consider citing:

@misc{qwen2026recreationworld,
      title={RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents},
      author={Shuai Bai and Jiayong Deng and Sicheng Fan and Yikun Fu and Chang Gao and Xuhao Hu and Mianqiu Huang and Yizhen Jiang and Yuheng Jing and Dehui Kong and Keliang Li and Ning Li and Wanli Li and Dayiheng Liu and Dunjie Lu and Changwei Luo and Que Shen and Zheyuan Wang and Zijian Wang and Jie Wu and Gao Wu and Zhihui Xie and Rui Xie and Haiyang Xu and An Yang and Jiakang Yuan and Yanming Zhang and Jiajun Zhang and Xi Zhang and Zhenru Zhang and Zhuo Zhen and Mingkang Zhu and Bowen Zhou},
      year={2026},
      eprint={2609.22000},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2609.22000},
}

Released under the MIT License. Third-party components retain their upstream licenses; see THIRD_PARTY_NOTICES.md.