DelinQu/awesome-vision-language-action-model

Latest Advances on Vison-Language-Action Models.

145

5 commits

updated Mar 4, 2025

See the code

README

Awesome-vision-language-action-models Awesome

🔥 Latest Advances on Vison-Language-Action Models. (UNDERCONSTRUCTION 🔥)

Embodied intelligence is one of the most critical carrier for general artificial intelligence to reach the physical world, with significant implications for our daily lives. Recent years have seen tremendous advancements in robotics technology, and the AI community is increasingly turning its attention to robots. We're excitedly anticipating the emergence of a GPT-like breakthrough in robotics! 🤗

  • 🤗LeRoBot - Making AI for Robotics more accessible with end-to-end learning.

Table of Content

Milestone Papers

DatekeywordsInstitutePaperCode
2022-12TransformerGoogleRT-1: Robotics Transformer for real-world control at scalegoogle-research/robotics_transformer
2023-03General MultimodalGooglePaLM-E: An Embodied Multimodal Language Model-
2024-03Vector QuantizationNew York UniversityBehavior Generation with Latent ActionsjayLEE0301/vq_bet_official
2023-04DiffusionColumbia UniversityDiffusion Policy: Visuomotor Policy Learning via Action Diffusionreal-stanford/diffusion_policy
2023-04Action ChunkingStanfordLearning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (ACTtonyzhaozh/act
2023-07-GoogleRT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control-
2023-10Action-in-Video-OutUC BerkeleyUniSim: Learning Interactive Real-World Simulators-
2024-05Low-Cost/DesignGoogleALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationtonyzhaozh/aloha
2024-05DiffusionUC BerkeleyOcto: An Open-Source Generalist Robot Policyocto-models/octo
2024-06Open-SourceGoogleOpenVLA: An Open-Source Vision-Language-Action Modelopenvla/openvla
2024-09HeterogeneousMITHeterogenous Pre-trained Transformersliruiw/HPT
2024-10Bimanual DiffusionTHURDT-1B: a Diffusion Foundation Model for Bimanual Manipulationthu-ml/RoboticsDiffusionTransformer
2024-10Video-Language-ActionByteDanceGR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation-
2024-10Flow MatchingPhysical Intelligenceπ0: A Vision-Language-Action Flow Model for General Robot Control-
2025-01SpatialVLAShanghai AI LaboratorySpatialVLA: Exploring Spatial Representations for Visual-Language-Action Models[SpatialVLA](https://github.com/thu-ml/RoboticsDiffusionTransformer

Datasets & Benchmark

Tutorials and Courses

Contributing

This is an active repository and your contributions are always welcome!

The repository is underconstruction and I will keep some pull requests open if I'm not sure if they are awesome for VLA Model, you could vote for them by adding 👍 to them.


If you have any question about this opinionated list, do not hesitate to contact me 📮 dlqu22@m.fudan.edu.cn.

DelinQu/awesome-vision-language-action-model

Latest Advances on Vison-Language-Action Models.

145

5 commits

updated Mar 4, 2025

See the code

README

Awesome-vision-language-action-models Awesome

🔥 Latest Advances on Vison-Language-Action Models. (UNDERCONSTRUCTION 🔥)

Embodied intelligence is one of the most critical carrier for general artificial intelligence to reach the physical world, with significant implications for our daily lives. Recent years have seen tremendous advancements in robotics technology, and the AI community is increasingly turning its attention to robots. We're excitedly anticipating the emergence of a GPT-like breakthrough in robotics! 🤗

  • 🤗LeRoBot - Making AI for Robotics more accessible with end-to-end learning.

Table of Content

Milestone Papers

DatekeywordsInstitutePaperCode
2022-12TransformerGoogleRT-1: Robotics Transformer for real-world control at scalegoogle-research/robotics_transformer
2023-03General MultimodalGooglePaLM-E: An Embodied Multimodal Language Model-
2024-03Vector QuantizationNew York UniversityBehavior Generation with Latent ActionsjayLEE0301/vq_bet_official
2023-04DiffusionColumbia UniversityDiffusion Policy: Visuomotor Policy Learning via Action Diffusionreal-stanford/diffusion_policy
2023-04Action ChunkingStanfordLearning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (ACTtonyzhaozh/act
2023-07-GoogleRT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control-
2023-10Action-in-Video-OutUC BerkeleyUniSim: Learning Interactive Real-World Simulators-
2024-05Low-Cost/DesignGoogleALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationtonyzhaozh/aloha
2024-05DiffusionUC BerkeleyOcto: An Open-Source Generalist Robot Policyocto-models/octo
2024-06Open-SourceGoogleOpenVLA: An Open-Source Vision-Language-Action Modelopenvla/openvla
2024-09HeterogeneousMITHeterogenous Pre-trained Transformersliruiw/HPT
2024-10Bimanual DiffusionTHURDT-1B: a Diffusion Foundation Model for Bimanual Manipulationthu-ml/RoboticsDiffusionTransformer
2024-10Video-Language-ActionByteDanceGR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation-
2024-10Flow MatchingPhysical Intelligenceπ0: A Vision-Language-Action Flow Model for General Robot Control-
2025-01SpatialVLAShanghai AI LaboratorySpatialVLA: Exploring Spatial Representations for Visual-Language-Action Models[SpatialVLA](https://github.com/thu-ml/RoboticsDiffusionTransformer

Datasets & Benchmark

Tutorials and Courses

Contributing

This is an active repository and your contributions are always welcome!

The repository is underconstruction and I will keep some pull requests open if I'm not sure if they are awesome for VLA Model, you could vote for them by adding 👍 to them.


If you have any question about this opinionated list, do not hesitate to contact me 📮 dlqu22@m.fudan.edu.cn.