🌍 UI-Simulator: LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training

🔗 Paper link: https://arxiv.org/abs/2510.14969

🌐 Website: https://ui-simulator.notion.site/llms-as-scalable-digital-world-simulator

🤗 Model Weights & Datasets: https://huggingface.co/UI-Simulator

📧 Contact: da.yin9712@gmail.com, w10y20ming@gmail.com

---

🚀 Overview

🌍 UI-Simulator is a scalable, general-purpose simulator designed for digital agent training. It provides a robust and extensible simulation framework to enable efficient, data-rich interaction modeling and venues for extending to broader domains.

---

✨ Key Features

• 🔧 Scalable & General-Purpose: Generalized design adaptable to multiple digital environments, or even beyond.
• ⚙️ Targeted Scaling Strategy: Enables more rapid and data-efficient agent training.
• 🧩 Comprehensive Evaluation: Demonstrated strong results on WebArena and AndroidWorld benchmarks.

---

🧩 Installation

git clone https://github.com/WadeYin9712/UI-Simulator.git
cd UI-Simulator

conda create --name ui_simulator python=3.11
conda activate ui_simulator
pip install -r requirements.txt

If you wanna run evaluation on WebArena or AndroidWorld, please configure the venv according to their official installation guide. We refer the readers to the original repos.

---

🧪 Running

📊 Data Collection

We provides the shells for running the data collection here

export OPENAI_API_KEY=<YOUR_KEY>
export OPENAI_ORG_ID=<YOUR_ORG_ID>

# e.g. running to collect trajectories with retrieval-augmented simulation on web env, in gitlab domain
bash shells/web_collector/rag/gitlab.sh

🏃🏻‍♀️ Training

We used alignment-handbook as the codebase for our model training.

---

📚 Citation

If you find this work useful, please consider citing: [bibtex]

🧭 Contributors