Awesome VLA Benchmarks

If you want to add or update an entry, start with CONTRIBUTING.md. If this list is useful, consider giving the repository a star ⭐.

A curated list of benchmarks for evaluating Vision-Language-Action (VLA) models and closely related embodied agents across robotics, autonomous driving, GUI agents, interactive environments, and multimodal perception. This repository is benchmark-first and includes datasets, simulation platforms, and evaluation resources only when they directly support benchmark usage, comparison, or reproducibility.

---

Contents

• Awesome VLA Benchmarks
  • Contents
  • Scope
  • Embodied Robotics
    • Robot Manipulation
    • Dexterous Manipulation & Locomotion
    • Embodied Navigation
    • Embodied Multi-Task & Household
    • Social & Human-Robot Interaction
  • Autonomous Driving
  • GUI & Computer-Use Agents
  • Game & Interactive Environments
  • Multimodal Perception & Understanding
    • General Multimodal
    • Video & Temporal Understanding
    • Egocentric & Activity
    • 3D Scene Understanding
    • Language-Grounded Perception
  • Infrastructure & Resources
    • Simulation Environments & Platforms
    • Large-Scale Datasets & Model Hubs
    • Leaderboards & Evaluation Platforms
  • Contributing
  • Maintenance
  • License

---

Scope

• Benchmarks whose primary artifact is an evaluation suite, challenge, or task collection for VLA or embodied agents
• Task environments, benchmark datasets, and evaluation platforms that are directly used to measure agent performance
• Supporting resources such as simulators, model hubs, or leaderboards when they materially help readers run, compare, or reproduce benchmark results
• Benchmarks spanning robot manipulation, navigation, household interaction, autonomous driving, GUI agents, games, egocentric video, and multimodal perception

This list does not aim to track model-only papers, generic multimodal benchmarks with no action or embodied relevance, or infrastructure projects that do not directly support benchmark use.

Embodied Robotics

Benchmarks for embodied agents operating in physical or simulated robotic environments.

Robot Manipulation

Benchmarks for language-conditioned manipulation with robot arms, mobile manipulators, or bimanual systems.

Dexterous Manipulation & Locomotion

Benchmarks for dexterous hands, whole-body control, and locomotion-oriented embodied control.

Embodied Navigation

Benchmarks for navigation in 3D environments from language, visual, or object-goal instructions.

Embodied Multi-Task & Household

Benchmarks for household and open-world agents that combine navigation, manipulation, and task reasoning.

Social & Human-Robot Interaction

Benchmarks for socially aware, collaborative, or communicative robot behavior.

---

Autonomous Driving

Benchmarks for driving agents that ground perception, language, and decision-making in traffic scenes.

---

GUI & Computer-Use Agents

Benchmarks for agents that perceive graphical interfaces and act through keyboard and mouse inputs.

---

Game & Interactive Environments

Benchmarks for language-guided sequential decision-making in games and interactive simulated worlds.

---

Multimodal Perception & Understanding

Benchmarks for multimodal perception, temporal understanding, and grounding capabilities that support VLA systems.

General Multimodal

Benchmarks for general multimodal reasoning, instruction following, and cross-modal grounding.

Video & Temporal Understanding

Benchmarks for temporal video understanding, action reasoning, and long-context event interpretation.

Egocentric & Activity

Benchmarks for egocentric perception and activity understanding from first-person video.

3D Scene Understanding

Benchmarks for grounding language and reasoning in 3D scenes for embodied perception.

Language-Grounded Perception

Benchmarks for visual grounding, referring expressions, and spatial language understanding.

---

Infrastructure & Resources

Supporting resources used to run, compare, and reproduce VLA benchmark results.

Simulation Environments & Platforms

Simulation platforms and task environments used to build or execute VLA benchmarks.

Large-Scale Datasets & Model Hubs

Datasets and model hubs that provide training assets, benchmark data, or evaluation-facing artifacts for VLA work.

Leaderboards & Evaluation Platforms

Leaderboards and evaluation services that host benchmark results, competitions, or public comparisons.

---

Contributing

See CONTRIBUTING.md for repository scope, placement rules, row format, sorting expectations, and the pull request checklist.

---

Maintenance

• Use the issue templates for broken links, metadata corrections, benchmark suggestions, and reclassification requests.
• Open an issue before a pull request when changing top-level taxonomy, naming conventions, or repository-wide formatting rules.
• Keep entries concrete and linkable so the list stays useful as a reference rather than a broad survey of adjacent work.

---

License

This list is released under the CC0 1.0 Universal public domain dedication. You are free to copy, modify, and distribute this work, even for commercial purposes, without asking permission.