Dataset Card for MimicGen Datasets

Dataset Summary

This repository contains the official release of datasets for the CoRL 2023 paper "MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations".

The datasets contain over 48,000 task demonstrations across 12 tasks, grouped into the following categories:

• source: 120 human demonstrations across 12 tasks used to automatically generate the other datasets
• core: 26,000 task demonstrations across 12 tasks (26 task variants)
• object: 2000 task demonstrations on the Mug Cleanup task with different mugs
• robot: 16,000 task demonstrations across 4 different robot arms on 2 tasks (4 task variants)
• large_interpolation: 6000 task demonstrations across 6 tasks that pose significant challenges for modern imitation learning methods

For more information please see the website, the paper, and the code.

Dataset Structure

Each dataset is an hdf5 file that is readily compatible with robomimic --- the structure is explained here.

As described in the paper, each task has a default reset distribution (D_0). Source human demonstrations (usually 10 demos) were collected on this distribution and MimicGen was subsequently used to generate large datasets (usually 1000 demos) across different task reset distributions (e.g. D_0, D_1, D_2), objects, and robots.

The datasets are split into different types:

• source: source human datasets used to generate all data -- this generally consists of 10 human demonstrations collected on the D_0 variant for each task.
• core: datasets generated with MimicGen for different task reset distributions. These correspond to the core set of results in Figure 4 of the paper.
• object: datasets generated with MimicGen for different objects. These correspond to the results in Appendix G of the paper.
• robot: datasets generated with MimicGen for different robots. These correspond to the results in Appendix F of the paper.
• large_interpolation: datasets generated with MimicGen using much larger interpolation segments. These correspond to the results in Appendix H in the paper.

Note: We found that the large_interpolation datasets pose a significant challenge for imitation learning, and have substantial room for improvement.

Citation

Please cite the MimicGen paper if you use these datasets in your work:

@inproceedings{mandlekar2023mimicgen,
    title={MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations},
    author={Mandlekar, Ajay and Nasiriany, Soroush and Wen, Bowen and Akinola, Iretiayo and Narang, Yashraj and Fan, Linxi and Zhu, Yuke and Fox, Dieter},
    booktitle={7th Annual Conference on Robot Learning},
    year={2023}
}