awesome-embodied-ai-ros

A curated list of repositories and papers on Embodied AI in ROS ecosystem.

Repositories

Frameworks

• rai - a generic AI Agent framework for ROS 2.
• ROSA - to inspect, diagnose, understand, and operate robots.
• ros4hri - umbrella for all the ROS packages, conventions and tools that help developing interactive robots with ROS.
• stretch_ai - designed to help researchers and developers build intelligent behaviors for the Stretch 3 mobile manipulator.

Tools

• ros_to_markdown - convert ROS systems to Markdown for LLM processing.
• git2md - tool for converting Git repository contents into Markdown format.

User tools

• ros2ai - command line interface extension, especially for those learning ROS.

Model integrations

• llama_ros - integrated llama.cpp
• whisper_ros - listen to speech and generate responses.
• yolo_ros - ROS 2 wrap for YOLO models
• piper_ros - set of ROS 2 packages to integrate piper TTS (Text-to-Speech)

Inference & Training

• mlc-llm - Universal LLM Deployment Engine with ML Compilation
• llama.cpp - LLM inference in C/C++

Projects

• rai-agriculture-demo + rai docs link - in a beautiful scene of a virtual orchard, RAI goes beyond obstacle detection to analyze best course of action for a given unexpected situation.
• rai-manipulation-demo + rai docs link - complete flexible manipulation tasks thanks to RAI and Grounded SAM 2
• rai-rosbot-xl-demo + rai docs link - demonstrate RAI's interaction with an autonomous mobile robot platform for navigation and control
• tabletop-handybot - AI-powered robot arm assistant.
• chatbot_ros - combines whisper_ros,llama_ros, and piper_rosfor conversational AI

Stale repositories

• turtlebot3_drlnav - A ROS 2 framework for DRL autonomous navigation on mobile robots with LiDAR.

Related awesome-* lists

• awesome-foundation-model-ros - a collection of ROS projects and resources utilizing or simplifying the application of foundation models in robotics.

Other EmbodiedAI Repositories

Interensting projects without ROS integration yet

Vision-Language-Action Models & Projects

• openpi - Physical Intelligence robot packages and models, including $\pi$ 0, a flow-based diffusion vision-language-action model.
• open-pi-zero - this repo implements the pi0 model from Physical Intelligence (Pi)

World Foundation Models

• Cosmos - developer-first world foundation model platform designed to help developers build their Physical AI systems

Papers

• RAI: Flexible Agent Framework for Embodied AI - presents RAI, a modular framework for building and evaluating embodied multi-agent systems integrated with robots, simulations, and language models.