:sunglasses: Awesome VLA for Autonomous Driving

Autonomous driving has long relied on modular "Perception-Decision-Action" pipelines, whose hand-crafted interfaces and rule-based components often struggle in complex, dynamic, or long-tailed scenarios. Their cascaded structure also amplifies upstream perception errors, undermining downstream planning and control.

This survey reviews vision-action (VA) models and vision-language-action (VLA) models for autonomous driving. We trace the evolution from early VA approaches to modern VLA frameworks, and organize existing methods into two principal paradigms:

• End-to-End VLA, which integrates perception, reasoning, and planning within a single model.
• Dual-System VLA, which separates slow deliberation (via VLMs) from fast, safety-critical execution (via planners).

For more details, kindly refer to our :books: Paper, :globe_with_meridians: Project Page, and :hugs: HuggingFace Leaderboard.

:books: Citation

If you find this work helpful for your research, please kindly consider citing our paper:

@article{survey_vla4ad,
    title   = {Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future},
    author  = {Tianshuai Hu and Xiaolu Liu and Song Wang and Yiyao Zhu and Ao Liang and Lingdong Kong and Guoyang Zhao and Zeying Gong and Jun Cen and Zhiyu Huang and Xiaoshuai Hao and Linfeng Li and Hang Song and Xiangtai Li and Jun Ma and Shaojie Shen and Jianke Zhu and Dacheng Tao and Ziwei Liu and Junwei Liang},
    journal = {arXiv preprint arXiv:2512.16760},
    year    = {2025},
}

@article{survey_3d_4d_world_models,
    title   = {{3D} and {4D} World Modeling: A Survey},
    author  = {Lingdong Kong and Wesley Yang and Jianbiao Mei and Youquan Liu and Ao Liang and Dekai Zhu and Dongyue Lu and Wei Yin and Xiaotao Hu and Mingkai Jia and Junyuan Deng and Kaiwen Zhang and Yang Wu and Tianyi Yan and Shenyuan Gao and Song Wang and Linfeng Li and Liang Pan and Yong Liu and Jianke Zhu and Wei Tsang Ooi and Steven C. H. Hoi and Ziwei Liu},
    journal = {arXiv preprint arXiv:2509.07996},
    year    = {2025}
}

Table of Contents

• 1. Vision-Action Models
  • Action-Only Models
  • Perception-Action Models
  • Image-Based World Models
  • Occupancy-Based World Models
  • Latent-Based World Models
• 2. Vision-Language-Action Models
  • Textual Action Generator
  • Numerical Action Generator
  • Explicit Action Guidance
  • Implicit Representations Transfer
• 3. Datasets & Benchmarks
  • Vision-Action Datasets
  • Vision-Language-Action Datasets
• 4. Applications
• 5. Other Resources

1. Vision-Action Models

:one: Action-Only Models

:timer_clock: In chronological order, from the earliest to the latest.

:two: Perception-Action Models

:timer_clock: In chronological order, from the earliest to the latest.

:three: Image-Based World Models

:timer_clock: In chronological order, from the earliest to the latest.

:four: Occupancy-Based World Models

:timer_clock: In chronological order, from the earliest to the latest.

:five: Latent-Based World Models

:timer_clock: In chronological order, from the earliest to the latest.

2. Vision-Language-Action Models

:one: Textual Action Generator

:timer_clock: In chronological order, from the earliest to the latest.

:two: Numerical Action Generator

:timer_clock: In chronological order, from the earliest to the latest.

:three: Explicit Action Guidance

:timer_clock: In chronological order, from the earliest to the latest.

:four: Implicit Representations Transfer

:timer_clock: In chronological order, from the earliest to the latest.

3. Datasets & Benchmarks

:timer_clock: In chronological order, from the earliest to the latest.

:one: Vision-Action Datasets

:two: Vision-Language-Action Datasets

4. Applications

5. Other Resources