Welcome to Awesome VLA4AD—a curated, continuously updated collection of research papers and resources on Vision–Language–Action models for Autonomous Driving (VLA4AD). This repository tracks the latest advances in VLA4AD, from explanatory perception modules to end-to-end reasoning and control architectures.

Our latest survey is here. We invite your feedback and discussion!

⭐️ Follow & Star to stay up to date!
🤝 Contributions welcome—if you know of new papers, datasets, or tools, please open an issue or submit a PR.
📬 Questions or suggestions? Reach us at sicong.jiang@mail.mcgill.ca or qka23@mails.tsinghua.edu.cn.

---

📜 Citation

If this project is useful in your work, we'd appreciate a star 🌟 and a citation of our survey.

@article{jiang2025survey,
  title={A Survey on Vision-Language-Action Models for Autonomous Driving},
  author={Jiang, Sicong and Huang, Zilin and Qian, Kangan and Luo, Ziang and Zhu, Tianze and Zhong, Yang and Tang, Yihong and Kong, Menglin and Wang, Yunlong and Jiao, Siwen and others},
  journal={arXiv preprint arXiv:2506.24044},
  year={2025}
}

---

📚 Table of Contents

• 🔥 Motivation & Paradigm Shift
• 🚀 Overview of VLA4AD
• 🏆 Awesome VLA4AD Papers
  • 1️⃣ Pre-VLA: VLM as Explainers
  • 2️⃣ Modular VLA4AD
  • 3️⃣ End-to-End VLA4AD
  • 4️⃣ Reasoning-Augmented VLA4AD
• 📊 Datasets & Benchmarks
• ⚙️ Installation & Usage

---

🔥 Motivation & Paradigm Shift

The development of autonomous driving has progressed from modular pipelines to fully integrated systems. This survey summarizes the latest advances into three core paradigms:

• End-to-End AD: Direct sensor-to-control mapping—efficient but opaque and weak on rare scenarios.

  • Flow: Sensors → Network → Actions

• VLMs for AD: Adds language reasoning—boosts explainability but doesn’t drive the vehicle.

  • Flow: Sensors → VLM → Answers

• VLA for AD: Unifies vision, language, and control in one policy—understands instructions, reasons, acts, and explains.

  • Flow: Sensors → Multimodal Encoder → LLM/VLM → Decoder → Actions

Figure 1. (a) conventional end-to-end AD, (b) vision-language models as explainers, (c) full Vision–Language–Action systems.

---

🚀 Overview of VLA4AD

A typical VLA4AD model follows an “Input–Process–Output” flow, unifying environment perception, instruction understanding, and vehicle control.

Figure 2. Overview of VLA4AD, integrating vision, language, and action modules.

A snapshot of the field’s evolution through four successive stages—from VLM-as-explainer to augmented, reasoning-centric agents:

Figure 3. Progression of VLA4AD models: (1) VLMs as passive explainers; (2) Modular VLA with intermediate representations; (3) End-to-end VLA mapping sensors directly to actions; (4) Augmented VLA with long-horizon reasoning and tool use.

The following table shows the representative models of VLA4AD and their inside modules:

Table 1. Representative VLA4AD Models (2023–2025). Sensor Inputs: Single = single forward-facing camera input; Multi = multi-view camera input; State = vehicle state information & other sensor input. Outputs: LLC= low-level control, Traj.= future trajectory, Multi.= multiple tasks such as perception, prediction or planning.

---

🏆 Awesome VLA4AD Papers

1️⃣ Pre-VLA: VLM as Explainers

2️⃣ Modular VLA4AD

3️⃣ End-to-End VLA4AD

4️⃣ Reasoning-Augmented VLA4AD

---

📊 Datasets & Benchmarks

---

⚙️ Installation & Usage

git clone https://github.com/JohnsonJiang1996/Awesome-VLA4AD.git
cd Awesome-VLA4AD
# Browse papers, datasets & code samples in each folder

---