[!NOTE] 当前 GitHub 派生仓库为 JustinKe02/so101_lerobot， 仓库所有者为 JustinKe02。项目基于 Hugging Face LeRobot 开发，上游版权、许可证和引用信息保持不变。

LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry so that everyone can contribute to and benefit from shared datasets and pretrained models.

🤗 A hardware-agnostic, Python-native interface that standardizes control across diverse platforms, from low-cost arms (SO-100) to humanoids.

🤗 A standardized, scalable LeRobotDataset format (Parquet + MP4 or images) hosted on the Hugging Face Hub, enabling efficient storage, streaming and visualization of massive robotic datasets.

🤗 State-of-the-art policies that have been shown to transfer to the real-world ready for training and deployment.

🤗 Comprehensive support for the open-source ecosystem to democratize physical AI.

Quick Start

LeRobot can be installed directly from PyPI.

pip install lerobot
lerobot-info

[!IMPORTANT] For detailed installation guide, please see the Installation Documentation.

本仓库的 SO-101 模型工作

main 分支作为稳定入口，保留 SO-101 训练、标定和基础部署能力，主要包括：

• SO-101 数据集训练策略、checkpoint 评估和多随机种子动作质量检查。
• 同步推理与 RTC（Real-Time Chunking）真机 rollout，包括 actual-consumed 时序补偿。
• 动作滤波、机器人侧相对目标限幅、stall guard 和推理失败后的资源清理。
• PI0.5 TensorRT prefix 加速、PyTorch/TensorRT 一致性验证和性能分析。
• LeRobotDataset 非流式批量视频编码支持连续多个 batch，并在 finalize 时完整回填视频元数据。
• SO-101 标定、相机、训练和上机过程的中文实验记录。

较新的模型和推理框架在独立分支开发，避免实验代码直接影响 main 的稳定使用：

ACT 20k 训练与真机推理基线

本仓库保留了一套已经完成训练和两轮真机推理验证的 ACT 基线：

• 数据集：admin123/so101_test_data，40 个 episode，共 17,960 帧，30 FPS。
• 任务：Put the block in the bin，使用 top 和 wrist 两路 640 x 480 图像。
• 模型：ACT，chunk_size=100、n_action_steps=100、条件 VAE，约 5160 万参数。
• 初始化：没有加载预训练 ACT 检查点（pretrained_path=null），仅使用 ImageNet 预训练 ResNet18 初始化视觉骨干。
• 训练范围：全部 51,597,190 个参数均参与训练，其中 ResNet18 骨干的 11,166,912 个参数 没有冻结；未使用 LoRA、Adapter 或其他 PEFT 方法。
• 优化配置：AdamW，主体和视觉骨干学习率均为 1e-5，weight decay 为 1e-4， batch size 为 8。
• 训练时长：20,000 step，约 8.9 epoch，每 5,000 step 保存一次检查点。
• 最终指标：loss=0.119、l1_loss=0.105、kld_loss=0.001。
• 最终模型：outputs/train/act_so101_test_data_20k/checkpoints/020000/pretrained_model， model.safetensors 约 207 MB，继续保存在本机，不提交到 Git。

真机推理采用 lerobot-rollout 同步后端。第一轮每 10 步强制重规划，产生了明显的周期性卡顿； 第二轮改为 policy.n_action_steps=50 并移除强制重规划，最终 horizon_replans=0、clamp_replans=0。完整时序、延迟和动作块分析见 ACT 两轮真机推理日志报告。

Realtime-VLA V2 最新结果

realtime-vla-v2 分支已经完成基于 40 条回合数据的 PI0.5 RTC6 全量解冻训练：

• 训练 5613 步，共 10 轮，最终记录损失为 0.009。
• freeze_vision_encoder=false、train_expert_only=false，视觉编码器和动作专家均参与训练。
• 权重支持 0-6 步训练前缀，PyTorch/Triton 固定噪声一致性验证全部通过。
• Triton 完整模型平均推理延迟约 45-49 ms，PyTorch 约 136-138 ms，加速接近 3 倍。
• 两种后端的实际上机效果接近，说明 Triton 主要降低计算开销，没有明显改变策略行为。
• 当前推荐 Triton 直出并关闭未标定的 Realtime Executor；时间轴规划器开关和固定/滚动前缀 在本轮上机中的任务效果接近。

realtime-vla-v2 包含完整运行代码、推荐配置、中文报告和可审计机器日志。模型权重和原始轨迹 体积较大，继续保存在本机 outputs/ 或 model_assets/，不直接提交到 Git。

分支使用说明

• main 不包含 --inference.type=vlash，使用 VLASH 前请切换到 pi05-vlash。
• Realtime-VLA V2 的完整 Triton 后端和 RTC6 训练前缀能力位于 realtime-vla-v2。
• GR00T N1.7 的 checkpoint 兼容与 rollout 接入位于 groot-n1.7-inference。
• 不建议在不同实验分支之间直接混用配置或导出权重；运行前应核对分支、checkpoint 和相机顺序。

相关文档：

• SO-101 PI0.5 训练策略
• RTC 与 TensorRT 兼容计划
• PI0.5 TensorRT 故障排查
• TensorRT 视觉加速分析
• SO-101 PI0.5 阶段总结
• ACT 两轮真机推理日志报告
• VLASH 分支 README
• Realtime-VLA V2 中文训练与上机报告
• Realtime-VLA V2 推荐配置

Robots & Control

LeRobot provides a unified Robot class interface that decouples control logic from hardware specifics. It supports a wide range of robots and teleoperation devices.

from lerobot.robots.myrobot import MyRobot

# Connect to a robot
robot = MyRobot(config=...)
robot.connect()

# Read observation and send action
obs = robot.get_observation()
action = model.select_action(obs)
robot.send_action(action)

Supported Hardware: SO100, LeKiwi, Koch, HopeJR, OMX, EarthRover, Reachy2, Gamepads, Keyboards, Phones, OpenARM, Unitree G1, reBot B601.

While these devices are natively integrated into the LeRobot codebase, the library is designed to be extensible. You can easily implement the Robot interface to utilize LeRobot's data collection, training, and visualization tools for your own custom robot.

For detailed hardware setup guides, see the Hardware Documentation.

LeRobot Dataset

To solve the data fragmentation problem in robotics, we utilize the LeRobotDataset format.

• Structure: Synchronized MP4 videos (or images) for vision and Parquet files for state/action data.
• HF Hub Integration: Explore thousands of robotics datasets on the Hugging Face Hub.
• Tools: Seamlessly delete episodes, split by indices/fractions, add/remove features, and merge multiple datasets.

from lerobot.datasets.lerobot_dataset import LeRobotDataset

# Load a dataset from the Hub
dataset = LeRobotDataset("lerobot/aloha_mobile_cabinet")

# Access data (automatically handles video decoding)
episode_index=0
print(f"{dataset[episode_index]['action'].shape=}\n")

Learn more about it in the LeRobotDataset Documentation

SoTA Models

LeRobot implements state-of-the-art policies in pure PyTorch, covering Imitation Learning, Reinforcement Learning, Vision-Language-Action (VLA) models, World Models, and Reward Models, with more coming soon. It also provides you with the tools to instrument and inspect your training process.

Training a policy is as simple as running a script configuration:

lerobot-train \
  --policy.type=act \
  --dataset.repo_id=lerobot/aloha_mobile_cabinet

Similarly to the hardware, you can easily implement your own policy & leverage LeRobot's data collection, training, and visualization tools, and share your model to the HF Hub

For detailed policy setup guides, see the Policy Documentation. For GPU/RAM requirements and expected training time per policy, see the Compute Hardware Guide.

Inference & Evaluation

Evaluate your policies in simulation or on real hardware using the unified evaluation script. LeRobot supports standard benchmarks like LIBERO, MetaWorld and more to come.

# Evaluate a policy on the LIBERO benchmark
lerobot-eval \
  --policy.path=lerobot/pi0_libero_finetuned \
  --env.type=libero \
  --env.task=libero_object \
  --eval.n_episodes=10

Learn how to implement your own simulation environment or benchmark and distribute it from the HF Hub by following the EnvHub Documentation

Resources

• Documentation: The complete guide to tutorials & API.
• Chinese Tutorials: LeRobot+SO-ARM101中文教程-同济子豪兄 Detailed doc for assembling, teleoperate, dataset, train, deploy. Verified by Seed Studio and 5 global hackathon players.
• Discord: Join the LeRobot server to discuss with the community.
• X: Follow us on X to stay up-to-date with the latest developments.
• Robot Learning Tutorial: A free, hands-on course to learn robot learning using LeRobot.
• T-Shirt Folding Experiment: An end-to-end demonstration of folding t-shirts with LeRobot.
• LeLab: A web interface for LeRobot — teleoperate, calibrate, record datasets, replay, and train your SO arm from the browser, no CLI required.

Citation

If you use LeRobot in your project, please cite the GitHub repository to acknowledge the ongoing development and contributors:

@misc{cadene2024lerobot,
    author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Meftah, Khalil and Ellerbach, Maxime and Moss, Jess and Wolf, Thomas},
    title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
    howpublished = "\url{https://github.com/huggingface/lerobot}",
    year = {2024}
}

If you are referencing our research or the academic paper, please also cite our ICLR publication:

@inproceedings{cadenelerobot,
  title={LeRobot: An Open-Source Library for End-to-End Robot Learning},
  author={Cadene, Remi and Alibert, Simon and Capuano, Francesco and Aractingi, Michel and Zouitine, Adil and Kooijmans, Pepijn and Choghari, Jade and Russi, Martino and Pascal, Caroline and Palma, Steven and Shukor, Mustafa and Moss, Jess and Soare, Alexander and Aubakirova, Dana and Lhoest, Quentin and Gallou\'edec, Quentin and Wolf, Thomas},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://arxiv.org/abs/2602.22818}
}

Contribute

We welcome contributions from everyone in the community! To get started, please read our CONTRIBUTING.md guide. Whether you're adding a new feature, improving documentation, or fixing a bug, your help and feedback are invaluable. We're incredibly excited about the future of open-source robotics and can't wait to work with you on what's next—thank you for your support!