SutureBot: A Precision Framework and Benchmark for Autonomous End-to-End Suturing

This dataset is part of the SutureBot project, a benchmark for developing and evaluating autonomous surgical robotic policies for end-to-end suturing. It contains high-resolution, multi-camera video and robot kinematics from real robot demonstrations on tissue phantoms, supporting research in imitation learning, vision-language-action modeling, and surgical autonomy.

project website | code repository.

Dataset Summary

The dataset includes:

• Stereo endoscope images (left_img_dir, right_img_dir)
• Wrist camera views (endo_psm1, endo_psm2)
• Low-level kinematics (end-effector pose, joint angles, jaw state, etc.) in ee_csv.csv
• Multiple tissues and multiple task types (e.g., needle pick-up, suturing)
• All data is organized by: tissue_[id]/[task_name]/[episode_timestamp]/

The following is a visualization of the dataset folder structure:

$PATH_TO_DATASET
├── [DATASET_NAME]       # the dataset base dir
|   └── tissue_1                      # data subset
|   |   ├── 1_[task_name]             # task name
|   |   |   ├── [episode]             # should be timestamp when the data was recorded
|   |   |   |      ├── left_img_dir   # left endoscope cam images (frame000000_left.jpg)
|   |   |   |      ├── right_img_dir  # right endoscope cam images (frame000000_right.jpg)
|   |   |   |      ├── endo_psm1      # right wrist cam images (frame000000_psm1.jpg)
|   |   |   |      ├── endo_psm2      # left wrist cam images (frame000000_psm2.jpg)
|   |   |   |      └── ee_csv.csv     # kinematics
|   └── tissue_2                      # data subset
|   |   ├── 1_[task_name]             # task name
|   |   |   ├── [episode]             # should be timestamp when the data was recorded
|   |   |   |      ├── left_img_dir   # left endoscope cam images (frame000000_left.jpg)
|   |   |   |      ├── right_img_dir  # right endoscope cam images (frame000000_right.jpg)
|   |   |   |      ├── endo_psm1      # right wrist cam images (frame000000_psm1.jpg)
|   |   |   |      ├── endo_psm2      # left wrist cam images (frame000000_psm2.jpg)
|   |   |   |      └── ee_csv.csv     # kinematics
...

Each episode is a complete trajectory from a real robotic execution using a da Vinci Research Kit (dVRK) platform.

---

Usage

This dataset is designed for training and evaluating autonomous policies that take as input multi-view RGB images (optionally language-conditioned), and output continuous control actions.

It supports a wide range of applications including:

• Behavioral cloning and offline RL
• Multi-modal policy learning (images + kinematics + language)
• Visual servoing and tool tracking
• Surgical robot benchmarking

---

File Structure

Each episode contains:

• left_img_dir/: Left endoscope camera images
• right_img_dir/: Right endoscope camera images
• endo_psm1/: Right wrist (PSM1) camera images
• endo_psm2/: Left wrist (PSM2) camera images
• ee_csv.csv: Kinematic data including joint states, poses, and jaw angles

---

Format

• Image format: .jpg, RGB
• Kinematics: CSV with columns such as:
• timestamp
• psm1_pose.position.{x,y,z}
• psm1_pose.orientation.{x,y,z,w}
• psm1_js[0-5], psm2_js[0-5], etc.
• ecm_pose.*, suj_pose.*, etc.

A complete list of columns is available in the dataset.croissant.json schema.

---

Citation

If you use this dataset, please cite:

@misc{suturebot2025,
title       = {SutureBot: A Precision Framework and Benchmark for Autonomous End-to-End Suturing},
author      = {Jesse Haworth, Juo-Tung Chen, Nigel Nelson, Ji Woong Kim, Masoud Moghani, Chelsea Finn, Axel Krieger},
year        = {2025},
note        = {Under review at NeurIPS 2025 Datasets and Benchmarks Track},
howpublished = {\url{https://huggingface.co/datasets/jchen396/suturebot}}
}

License

This dataset is licensed under CC-BY-4.0. Please credit the authors when using this dataset.

Contact

For questions or collaboration inquiries, please contact:

Juo-Tung Chen Johns Hopkins University 📧 jchen396@jh.edu