VT-MUSE Real-World Visuotactile Manipulation Dataset

This repository hosts the real-world tactile manipulation data contributed with VT-MUSE to the Open-X-Tactile (OXT) dataset.

The release follows the FTP-1-compatible Zarr organization required by OXT. Each task is distributed as an independently downloadable ZIP archive containing one standardized Zarr dataset.

Dataset contents

The standardized release contains 192,225 temporally aligned frames. It includes the 200 trajectories used for the final VT-MUSE real-world experiments (50 per task) and 451 additional valid real-world trajectories. The authoritative episode-level split is recorded in the OXT submission manifest.

Repository layout

insert_tube/
  insert_tube.zarr.zip
press_toaster/
  press_toaster.zarr.zip
pull_out_drawer/
  pull_out_drawer.zarr.zip
wipe_board/
  wipe_board.zarr.zip
README.md
supplementary_tactile_note.json

Sensors and observations

• Fixed third-person RGB camera, exported as camera_main_rgb.
• Robot state/action: seven arm joints and one gripper degree of freedom.
• Two GelSight Mini tactile pads, with aligned tactile RGB observations.
• Supplemental per-pad tactile depth maps are documented in supplementary_tactile_note.json.

Format and quality control

The archives use the OXT/FTP-1-compatible Zarr schema. Metadata and structure validation was run with the official OXT-QC toolkit. The four task-level Zarr datasets passed the metadata precheck with no reported errors or warnings.

For the data-format specification and the broader collection, visit the Open-X-Tactile website.

Institution and contact

Shanghai Jiao Tong University
Contact: acondaway@sjtu.edu.cn