H2R-1M: Human-to-Robot Egocentric Video Dataset

H2R-1M is a family of large-scale robot-centric datasets generated via H2R visual data augmentation pipeline. It transforms human egocentric manipulation videos into robot-augmented versions by replacing human hands with simulated robot arms using accurate pose estimation and physically aligned rendering. The resulting data is suitable for robot pre-training from videos.

Dataset Structure

We release the following datasets constructed by combining different robot embodiments and egocentric video sources:

• SSv2-1M: Extracted images of the original Something-Something V2

• H2R-UR5-SSv2-1M-Gripper: UR5 robot with 2-finger gripper, augmented from Something-Something V2

• H2R-UR5-SSv2-1M-Leaphand: UR5 robot with 5-finger Leaphand

• H2R-Franka-SSv2-1M: Franka Emika Panda with 2-finger gripper

• H2R-SSv2-1M-without-Retarget: Human demonstration data from SSv2 that has not been retargeted to any robot.

• Ego4D-1M: Extracted images of the original Ego4D

• H2R-UR5-Ego4D-1M-Gripper: UR5 with gripper, from Ego4D video clips

• H2R-UR5-Ego4D-1M-Leaphand: UR5 with Leaphand

• H2R-Franka-Ego4D-1M: Franka gripper version

• H2R-Ego4D-1M-without-Retarget: Human demonstration data from Ego4D that has not been retargeted to any robot.