ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
28
17 commits
updated Oct 14, 2026
📄 Paper
ExeWorldBench is a benchmark for evaluating world understanding and executable realization through code-based 3D world construction. Rather than merely reproducing what a world looks like, ExeWorldBench evaluates whether models can construct an executable world that works.
For harness-based evaluation, the t1/, t2/, and t3/ directories contain the task files and required assets.
Each task is organized in its own directory and can be executed directly using the corresponding agent harness.
Before running a task, replace the dataset path /xxx/ExeWorldBench in the task instructions with your own path to the dataset.
For direct API-based evaluation, use the prompts provided in the api_prompt/ directory.
All T1 instances share the same prompt templates, located in api_prompt/t1/:
prompt.txt: Replace <image> with the corresponding reference image.prompt_image.txt: Replace <image> and <frames> with the corresponding reference image and reference frames, respectively.The required images and frames can be found in the corresponding task directories under t1/.
All T2 instances share the same prompt templates, located in api_prompt/t2/:
prompt.txt: Replace <image> with the corresponding reference image.prompt_image.txt: Replace <image> and <frames> with the corresponding reference image and motion reference frames, respectively.The required images and frames can be found in the corresponding task directories under t2/.
Unlike T1 and T2, each T3 instance has its own task-specific prompt.
All individual prompts are provided in api_prompt/t3/, organized by task. Use the corresponding prompt for each instance directly, rather than applying a shared prompt template.
The prompts for the judge model are provided in the eval/ directory.
For questions, please contact:
If you find ExeWorldBench useful in your research, please consider citing our paper:
ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
28
17 commits
updated Oct 14, 2026
📄 Paper
ExeWorldBench is a benchmark for evaluating world understanding and executable realization through code-based 3D world construction. Rather than merely reproducing what a world looks like, ExeWorldBench evaluates whether models can construct an executable world that works.
For harness-based evaluation, the t1/, t2/, and t3/ directories contain the task files and required assets.
Each task is organized in its own directory and can be executed directly using the corresponding agent harness.
Before running a task, replace the dataset path /xxx/ExeWorldBench in the task instructions with your own path to the dataset.
For direct API-based evaluation, use the prompts provided in the api_prompt/ directory.
All T1 instances share the same prompt templates, located in api_prompt/t1/:
prompt.txt: Replace <image> with the corresponding reference image.prompt_image.txt: Replace <image> and <frames> with the corresponding reference image and reference frames, respectively.The required images and frames can be found in the corresponding task directories under t1/.
All T2 instances share the same prompt templates, located in api_prompt/t2/:
prompt.txt: Replace <image> with the corresponding reference image.prompt_image.txt: Replace <image> and <frames> with the corresponding reference image and motion reference frames, respectively.The required images and frames can be found in the corresponding task directories under t2/.
Unlike T1 and T2, each T3 instance has its own task-specific prompt.
All individual prompts are provided in api_prompt/t3/, organized by task. Use the corresponding prompt for each instance directly, rather than applying a shared prompt template.
The prompts for the judge model are provided in the eval/ directory.
For questions, please contact:
If you find ExeWorldBench useful in your research, please consider citing our paper: