动手做具身智能：从算法到实机部署的“通关”实践指南

作者：Brain实验室

一、机器人基础操作与硬件集成

• 具体教程：
  • 1.1 机器人的基本操作, link
  • 1.2 末端执行器安装, link
  • 1.3 感知传感器安装, link
  • 1.4 内部计算单元配置, link
• 硬件设备：
  • Unitree G1, is a humanoid robot, website / specs / development guide
  • BrainCo Revo2, is a dexterous hand, website / specs
  • HBVCAM-4M2214HD-2 V11, is a USB stereo vision camera module, specs
  • HBVCAM-F2209HD V11, is a USB camera module, specs

二、机器人全身运动控制

• 具体教程：
  • 2.1 基于Isaac Gym的基础步态训练, link
  • 2.2 运动数据采集（一）：基于人类演示视频的GVHMR, link
  • 2.3 运动数据采集（二）：基于动作捕捉的GMR, link
  • 2.4 运动策略生成（一）：基于ASAP & PBHC, link
  • 2.5 运动策略生成（二）：基于Beyond MIMIC, link
  • 2.6 运动策略板载部署：基于RoboJuDo, link
• 参考文献：
  • arXiv 2025.10, Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking, code
  • arXiv 2025.09, RSL-RL: A Learning Library for Robotics Research, website / code
  • arXiv 2025.09, KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Control, website / code
  • RSS 2025, ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills, website / code
  • SIGGRAPH 2024, World-Grounded Human Motion Recovery via Gravity-View Coordinates, website / code
  • CVPR 2019, Expressive Body Capture: 3D Hands, Face, and Body From a Single Image, website / code
• 软件工具：
  • unitree_rl_gym, is a repository for reinforcement learning implementation based on Unitree robots, supporting Unitree Go2, H1, H1_2, and G1, code
  • HumanoidVerse, is a multi-simulator framework for humanoid robot sim-to-real learning, code
• 硬件设备：
  • Unitree G1, is a humanoid robot, website / specs / development guide

三、机器人操作数据采集

• 具体教程：
  • 3.1 机器人半身遥操作：基于XR Teleoperate框架, link
  • 3.2 机器人全身遥操作：基于TWIST框架, link
• 参考文献：
  • arXiv 2024.07, Open-TeleVision: Teleoperation with Immersive Active Visual Feedback, website / code
• 软件工具：
  • xr_teleoperate, aims to implement teleoperation control of a Unitree humanoid robot using XR devices (such as Apple Vision Pro, PICO 4 Ultra Enterprise, or Meta Quest 3), code
• 硬件设备：
  • INSPIRE-ROBOTS RH56E2, is a dexterous hand integrating 17 tactile sensor in a single hand, which setup allows real-time acquisition of tactile information from various parts, website / specs
  • Orbbec Gemini 335, is a depth + RGB camera, website / specs
  • Apple Vision Pro, is a XR device for teleoperation control, website
  • PICO 4 Ultra Enterprise, is a XR device for teleoperation control, website

四、机器人环境感知与建模

• 具体教程：
  • 4.1 基于激光 SLAM 的环境建图, link
  • 4.2 场景自动化生成与增强, link
  • 4.3 室外环境下的机器人导航, link
• 参考文献：
  • arXiv 2025.05, Procedural Generation of Articulated Simulation-Ready Assets, website / code
  • arXiv 2024.07, GRUtopia: Dream General Robots in a City at Scale, website / code
  • TRO 2022, FAST-LIO2: Fast Direct LiDAR-inertial Odometry, code
• 软件工具：
  • ROS2, Robotic Operating System2, code
  • Navigation2, is a open-source navigation framework and system for ROS2, website / code
  • Autonomous Exploration Development Environment, is meant for leveraging system development and robot deployment for ground-based autonomous navigation and exploration, website / code
  • Isaac Sim, is a reference application built on NVIDIA Omniverse that enables developers to develop, simulate, and test AI-driven robots in physically-based virtual environments, website
• 硬件设备：
  • AgileX Robotics SCOUT 2.0, is a an unmanned ground vehicle (UGV) designed for robotics research and development, website / specs
  • LimX Dynamics TRON1, is a multi-modal biped robot, website / specs
  • UBTECH Walker Tienkung, is a full-size humanoid robot, website / specs
  • NVIDIA Jetson Orin NX, is a embedded computing platform for small, low-power autonomous machines, website
  • Livox AVIA, is a LiDAR which combines compact and lightweight design, featuring an FOV greater than 70°, website / specs
  • Unicore UM982, is a positioning and heading module designed to perform on-chip RTK positioning and dual-antenna heading, website / specs

五、机器人视觉语言导航

• 具体教程：
  • 5.1 基于OpenNav的视觉语言导航, link
  • 5.2 基于NaVILA的视觉语言导航, link
  • 5.3 基于InternNav的视觉语言导航, link
• 参考文献：
  • ICRA 2025, SpatialBot: Precise Spatial Understanding with Vision Language Models, code
  • RSS 2025, NaVILA: Legged Robot Vision-Language-Action Model for Navigation, website / code / model
  • 2025.09, InternVLA-N1: An Open Dual-System Vision-Language Navigation Foundation Model with Learned Latent Plans, website / code
  • arXiv 2025.05, NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance, website
  • arXiv 2024.09, Open-nav: Exploring zero-shot vision-and-language navigation in continuous environment with open-source llms, website / code
  • arXiv 2024.06, Depth Anything V2, website / code
  • CVPRW 2024, Recognize Anything: A Strong Image Tagging Model, website / code
  • CVPR 2022, Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation, code
  • ECCV 2020, Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments, website / code
• 软件工具：
  • Habitat-Sim, a flexible, high-performance 3D simulator for embodied AI research, website / code
  • Habitat-Lab, a modular high-level library to train embodied AI agents across a variety of tasks and environments, website / code

六、机器人视觉语言操作

• 具体教程：
  • 6.1 Unitree LeRobot框架介绍及配置, link
  • 6.2 Unitree LeRobot数据格式及转换, link
  • 6.3 Unitree LeRobot典型模型库介绍, link
  • 6.4 Unitree LeRobot训练与工程测试, link
  • 6.5 NVIDIA GR00T模型微调, link
  • 6.6 NVIDIA GR00T工程实践, link
• 参考文献：
  • arXiv 2025.03, : GR00T N1: An Open Foundation Model for Generalist Humanoid Robots, website / code
  • arXiv 2024.10, : $π_0$, A Vision-Language-Action Flow Model for General Robot Control, website / code
  • arXiv 2023.04, Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
• 软件工具：
  • LeRobot, aims to provide models, datasets, and tools for real-world robotics in PyTorch, website / code
  • unitree_IL_lerobot, is a modification of the LeRobot open-source training framework, enabling the training and testing of data collected using the dual-arm dexterous hands of Unitree's G1 robot, code
  • xr_teleoperate, aims to implement teleoperation control of a Unitree humanoid robot using XR devices (such as Apple Vision Pro, PICO 4 Ultra Enterprise, or Meta Quest 3), code

七、机器人人机交互系统

• 具体教程：
  • 7.1 宇树原生交互系统介绍, link
  • 7.2 外接大模型的集成方法, link
  • 7.3 本地知识库构建与接入方案, link
  • 7.4 机器人导游与导览应用概述, link
• 软件工具：
  • LangChain, is a framework for building agents and LLM-powered applications, website / code
• 硬件设备：
  • Livox Mid 360, is a LiDAR for low speed robotics, website / specs

联系方式