Awesome 3D Generation for Embodied AI

A curated list of papers, datasets, benchmarks, simulators, and evaluation resources for 3D generation in embodied AI and robotic simulation.

This repository accompanies our survey on 3D generation for embodied AI and robotic simulation. It organizes the literature around how generative 3D methods support embodied systems, from simulation-ready asset creation, to interactive scene generation, to sim-to-real transfer. In addition to research papers, we collect resources that are useful for building and evaluating embodied 3D generation systems, including datasets, benchmarks, simulator platforms, and asset formats.

Contributions are welcome. Please open an issue or submit a pull request if you would like to add a paper, dataset, benchmark, simulator, or evaluation resource.

Highlights

• Survey-driven taxonomy for 3D generation in embodied AI
• Curated coverage of papers, datasets, benchmarks, simulators, and evaluation
• Emphasis on simulation-ready assets and environments rather than visual realism alone

Table of Contents

• Scope and Taxonomy
• Awesome List
  • Data Generator
  • Simulation Environments
  • Sim2Real Bridge
• Datasets and Benchmarks
• Simulators and Formats
• Evaluation
• Contributing
• Citation

Scope and Taxonomy

We focus on 3D generation methods that are useful for embodied AI, robot learning, and robotic simulation. Compared with general-purpose 3D generation, embodied settings require more than visual realism: generated assets and environments should be executable, physically meaningful, interactive, and compatible with simulation platforms.

We organize the field into three major roles:

1. Data Generator: generating simulation-ready objects and assets, including articulated, physically grounded, and deformable content.
2. Simulation Environments: generating interactive, controllable, and task-oriented 3D scenes for embodied training and evaluation.
3. Sim2Real Bridge: using 3D generation for digital twins, data augmentation, and synthetic demonstrations to support transfer to real robots.

Awesome List

In chronological order, from the earliest to the latest.

Data Generator

Articulated Object Generation

Physically-Grounded Object Generation

Deformable Object Generation

End-to-End Sim-Ready Asset Pipelines

Simulation Environments

Structure-Driven Scene Generation

Controllable Scene Generation

Agentic and Task-Oriented Scene Generation

Sim2Real Bridge

Digital Twin Construction

Data Augmentation

Synthetic Demonstration and Task Generation

Datasets and Benchmarks

Object-Level Datasets

Scene-Level Datasets

Robot Demonstration Datasets

Embodied Benchmarks

Simulators and Formats

Simulation Platforms

Asset and Scene Formats

Evaluation

We recommend evaluating embodied 3D generation at three levels:

• Geometric Fidelity: shape and appearance quality
• Physical Plausibility and Simulation Compatibility: stability, articulation correctness, collision validity, material realism, and simulator readiness
• Downstream Embodied Task Performance: grasp success, articulated manipulation success, navigation success, task completion, and sim-to-real transfer performance

Citation

If you find this repository useful, please consider citing our survey.

@article{to_be_updated,
  title   = {3D Generation for Embodied AI and Robotic Simulation: A Survey},
  author  = {To be updated},
  journal = {To be updated},
  year    = {2025}
}