This dataset was created using LeRobot.

Dataset Description

• Homepage: unitreerobotics

• License: apache-2.0

• Task Objective: Organize and tidy the items on the table.

• Operation Duration: Each operation takes approximately 20 to 40 seconds.

• Recording Frequency: 30 Hz.

• Robot Type: 7-DOF dual-arm G1 robot.

• End Effector: Gripper.

• Dual-Arm Operation: Yes.

• Image Resolution: 128x128.

• Camera Positions: head-mounted (binocular cameras).

• Data Content:
  • Robot's current state.
  • Robot's next action.
  • Current camera view images.

• Robot Initial Posture: The first robot state in each dataset entry.

• Object Placement: Randomly placed within the robot arm's motion range and the field of view of the robot's head-mounted camera.

• Camera View: Follow the guidelines in Part 5 of AVP Teleoperation Documentation.

• Important Notes:

1. This is a G1 diversity dataset that can be used for video generation models, world models, and other applications [Lee et al., 2018].
2. If you want to use the lerobotv2.1 format, refer to this file for conversion: convert_v3_to_v2.py
3. Due to the inability to precisely describe spatial positions, adjust the scene to closely match the first frame of the dataset after installing the hardware as specified in Part 5 of AVP Teleoperation Documentation.
4. Data collection is not completed in a single session, and variations between data entries exist. Ensure these variations are accounted for during model training.

Dataset Structure

meta/info.json:

{
    "codebase_version": "v3.0",
    "robot_type": "Unitree_G1_Dex1",
    "total_episodes": 468,
    "total_frames": 331555,
    "total_tasks": 1,
    "chunks_size": 1000,
    "data_files_size_in_mb": 100,
    "video_files_size_in_mb": 500,
    "fps": 30,
    "splits": {
        "train": "0:468"
    },
    "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
    "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
    "features": {
        "observation.state": {
            "dtype": "float32",
            "shape": [
                16
            ],
            "names": [
                [
                    "kLeftShoulderPitch",
                    "kLeftShoulderRoll",
                    "kLeftShoulderYaw",
                    "kLeftElbow",
                    "kLeftWristRoll",
                    "kLeftWristPitch",
                    "kLeftWristYaw",
                    "kRightShoulderPitch",
                    "kRightShoulderRoll",
                    "kRightShoulderYaw",
                    "kRightElbow",
                    "kRightWristRoll",
                    "kRightWristPitch",
                    "kRightWristYaw",
                    "kLeftGripper",
                    "kRightGripper"
                ]
            ]
        },
        "action": {
            "dtype": "float32",
            "shape": [
                16
            ],
            "names": [
                [
                    "kLeftShoulderPitch",
                    "kLeftShoulderRoll",
                    "kLeftShoulderYaw",
                    "kLeftElbow",
                    "kLeftWristRoll",
                    "kLeftWristPitch",
                    "kLeftWristYaw",
                    "kRightShoulderPitch",
                    "kRightShoulderRoll",
                    "kRightShoulderYaw",
                    "kRightElbow",
                    "kRightWristRoll",
                    "kRightWristPitch",
                    "kRightWristYaw",
                    "kLeftGripper",
                    "kRightGripper"
                ]
            ]
        },
        "observation.images.cam_left_high": {
            "dtype": "video",
            "shape": [
                128,
                128,
                3
            ],
            "names": [
                "height",
                "width",
                "channel"
            ],
            "info": {
                "video.height": 128,
                "video.width": 128,
                "video.codec": "h264",
                "video.pix_fmt": "yuv420p",
                "video.is_depth_map": false,
                "video.fps": 30,
                "video.channels": 3,
                "has_audio": false
            }
        },
        "observation.images.cam_right_high": {
            "dtype": "video",
            "shape": [
                128,
                128,
                3
            ],
            "names": [
                "height",
                "width",
                "channel"
            ],
            "info": {
                "video.height": 128,
                "video.width": 128,
                "video.codec": "h264",
                "video.pix_fmt": "yuv420p",
                "video.is_depth_map": false,
                "video.fps": 30,
                "video.channels": 3,
                "has_audio": false
            }
        },
        "timestamp": {
            "dtype": "float32",
            "shape": [
                1
            ],
            "names": null
        },
        "frame_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        },
        "episode_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        },
        "index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        },
        "task_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        }
    }
}

Citation

BibTeX:

@article{lee2018stochastic,
  title={Stochastic Adversarial Video Prediction},
  author={Lee, Alex X. and Zhang, Richard and Ebert, Frederik and Abbeel, Pieter and Finn, Chelsea and Levine, Sergey},
  journal={arXiv preprint arXiv:1804.01523},
  year={2018},
  url={https://arxiv.org/abs/1804.01523}
}