README
Awesome-Surgical-Video-Analysis
Keep updating. Welcome to contribute.
Contents
Surgical Instrument Recognition / Localization
Surgical Scene / Anatomy Recognition
Human / Activity in Operating Rooms
The Big Picture
| Title | Venue | Links |
|---|---|---|
| Surgomics: personalized prediction of morbidity, mortality and long‐term outcome in surgery using machine learning on multimodal data | Surgical Endoscopy 2020 | Paper |
| Surgical Data Science: from Concepts to Clinical Translation | Arxiv 2020 | Paper |
| Surgical Data Science: A Consensus Perspective | Arxiv 2018 | Paper |
| Surgical data science for next-generation interventions | NBE 2017 | Paper |
| CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions | IEEE 2019 | Paper |
| OR Black Box and Surgical Control Tower: Recording and Streaming data and Analytics to Improve Surgical Care | JVS 2021 | Paper |
| General-purpose foundation models for increased autonomy in robot-assisted surgery | arxiv 2024 | Paper |
Survey
| Title | Venue | Links |
|---|---|---|
| Machine learning for technical skill assessment in surgery: a systematic review | NPJ 2022 | Paper ｜ |
| A Review on Deep Learning in Minimally Invasive Surgery | IEEE Access 2021 | Paper |
| Deep neural networks for the assessment of surgical skills: A systematic review | IEEE Access 2021 | Paper |
| Current methods for assessing technical skill in cataract surgery | JCRS 2021 | Paper |
| Gesture Recognition in Robotic Surgery: a Review | TBE 2021 | Paper |
| Objective assessment of surgical technical skill and competency in the operating room | ARBE 2017 | Paper |
| Vision-based and marker-less surgical tool detection and tracking: a review of the literature | MIA 2017 | Paper |
| Surgical process modelling: a review | IJCARS 2013 | Paper |
Surgical Skill Assessment
| Title | Venue | Links |
|---|---|---|
| Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark | Arxiv 2021 | Paper |
| Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis | JAMA 2021 | Paper |
| Towards Unified Surgical Skill Assessment | CVPR 2021 | Paper Code |
| Clearness of Operating Field: A Surrogate for Surgical Skills on In-Vivo Clinical Data | IJCARS 2020 | Paper |
| Predicting the quality of surgical exposure using spatial and procedural features from laparoscopic videos | IJCARS 2020 | Paper |
| The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos | CVPR 2019 | Paper Code |
| Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field | MICCAI 2019 | Paper |
| Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating | AnnS 2019 | Paper |
| Towards Optimizing Convolutional Neural Networks for Robotic Surgery Skill Evaluation | IJCNN 2019 | Paper |
| Video-based surgical skill assessment using 3D convolutional neural networks | IJCARS 2019 | Paper Code |
| Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks | IJCARS 2019 | Paper Code |
| Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions | IJCARS 2019 | Paper |
| Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery | IJCARS 2019 | Paper |
| Objective classification of psychomotor laparoscopic skills of surgeons based on three different approaches | IJCARS 2019 | |
| Who’s Better? Who’s Best? Pairwise Deep Ranking for Skill Determination | CVPR 2018 | |
| Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks | WACV 2018 | |
| Evaluating surgical skills from kinematic data using convolutional neural networks | MICCAI 2018 | Paper Code |
| Automated surgical skill assessment in RMIS training | IJCARS 2018 | |
| Video and accelerometer-based motion analysis for automated surgical skills assessment | IJCARS 2018 | |
| Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery | IJCARS 2018 | |
| Surgical skills: Can learning curves be computed from recordings of surgical activities? | IJCARS 2018 | Paper |
| Automated robot‐assisted surgical skill evaluation: Predictive analytics approach | IJMRCAR 2018 | |
| Meaningful Assessment of Surgical Expertise: Semantic Labeling with Data and Crowds | MICCAI 2016 | |
| Automated video-based assessment of surgical skills for training and evaluation in medical schools | IJCARS 2016 | |
| Task-Level vs. Segment-Level Quantitative Metrics for Surgical Skill Assessment | Surg Educ 2016 | |
| Relative Hidden Markov Models for Video-Based Evaluation of Motion Skills in Surgical Training | TPAMI 2015 | |
| Automated Assessment of Surgical Skills Using Frequency Analysis | MICCAI 2015 | |
| Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty | IJCARS 2015 | |
| Automated Surgical OSATS Prediction From Videos | ISBI 2014 | |
| Pairwise Comparison-Based Objective Score for Automated Skill Assessment of Segments in a Surgical Task | IPCAI 2014 | |
| Video Based Assessment of OSATS Using Sequential Motion Textures | MICCAIW 2014 | |
| Augmenting Bag-of-Words: Data-Driven Discovery of Temporal and Structural Information for Activity Recognition | CVPR 2013 | |
| String Motif-Based Description of Tool Motion for Detecting Skill and Gestures in Robotic Surgery | MICCAI 2013 | |
| Robotic Path Planning for Surgeon Skill Evaluation in Minimally-Invasive Sinus Surgery | MICCAI 2012 | |
| An objective and automated method for assessing surgical skill in endoscopic sinus surgery using eye-tracking and tool-motion data | IFAR 2012 | |
| Sparse Hidden Markov Models for Surgical Gesture Classification and Skill Evaluation | IPCAI 2012 | |
| Towards integrating task information in skills assessment for dexterous tasks in surgery and simulation | ICRA 2011 | |
| Video-based Motion Expertise Analysis in Simulation-based Surgical Training Using Hierarchical Dirichlet Process Hidden Markov Model | MMAR 2011 | |
| Eye Metrics as an Objective Assessment of Surgical Skill | AnnS 2010 | |
| Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery | MICCAI 2010 | |
| Task versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery | MICCAI 2009 | |
| Data-derived models for segmentation with application to surgical assessment and training | MICCAI 2009 |
Surgical Phase Recognition
| Title | Venue | Links |
|---|---|---|
| Self-supervised representation learning for surgical activity recognition | IJCARS 2021 | Paper |
| Surgical Workflow Anticipation Using Instrument Interaction | MICCAI 2021 | Paper |
| Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark | Arxiv 2021 | Paper |
| OperA: Attention-Regularized Transformers for Surgical Phase Recognition | MICCAI 2021 | Paper |
| Temporal Memory Relation Network for Workflow Recognition from Surgical Video | TMI 2021 | Paper Code |
| Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass | IJCARS 2021 | Paper |
| Train one, Classify one, Teach one"–Cross-surgery transfer learning for surgical step recognition | MIDL 2021 | Paper |
| TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks | MICCAI 2020 | |
| Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: Experimental research | IJS 2020 | Paper |
| Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis | MIA 2020 | Paper Code |
| LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition | IJCARS 2020 | Paper Code |
| Assisted phase and step annotation for surgical videos | IJCARS 2020 | |
| Accurate Detection of Out of Body Segments in Surgical Video using Semi-Supervised Learning | MIDL 2020 | Paper |
| Impact of data on generalization of AI for surgical intelligence applications | SciReports 2020 | Paper |
| Hard Frame Detection and Online Mapping for Surgical Phase Recognition | MICCAI 2019 | |
| MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation | CVPR 2019 | |
| Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition | IPCAI 2019 | Paper |
| Machine and deep learning for workflow recognition during surgery | MITAT 2019 | |
| Assessment of automated identification of phases in videos of cataract surgery using machine learning and deep learning techniques | JAMA 2019 | Paper |
| Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos | Arxiv 2019 | Paper |
| SV-RCNet: Workflow Recognition from Surgical Videos using Recurrent Convolutional Network | TMI 2018 | Paper Code |
| DeepPhase: Surgical Phase Recognition in CATARACTS Videos | MICCAI 2018 | |
| Surgical activity recognition in robot-assisted radical prostatectomy using deep learning | MICCAI 2018 | Paper |
| Knowledge transfer for surgical activity prediction | IJCARS 2018 | Paper |
| Temporal coherence-based self-supervised learning for laparoscopic workflow analysis | MICCAIW 2018 | Paper |
| “Deep-Onto” network for surgical workflow and context recognition | IJCARS 2018 | Paper |
| Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks | Arxiv 2018 | |
| EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos | TMI 2017 | |
| Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis | Arxiv 2017 | Paper |
| System events: readily accessible features for surgical phase detection | IJCARS 2016 | Paper |
| Automatic data-driven real-time segmentation and recognition of surgical workflow | IJCARS 2016 | Paper |
| MICCAI Workflow Challenge: Convolutional neural networks with time smoothing and Hidden Markov Model for video frames classification | Arxiv 2016 | |
| Automatic phase prediction from low-level surgical activities | IJCARS 2015 | Paper |
| Lapontospm: an ontology for laparoscopic surgeries and its application to surgical phase recognition | IJCARS 2015 | Paper |
| Real-time segmentation and recognition of surgical tasks in cataract surgery videos | TMI 2014 | Paper |
| Statistical modeling and recognition of surgical workflow | MIA 2012 | |
| A framework for the recognition of high-level surgical tasks from video images for cataract surgeries | TBE 2012 | Paper |
| An application-dependent framework for the recognition of high-level surgical tasks in the OR | TBE 2011 | |
| Modeling and Segmentation of Surgical Workflow from Laparoscopic Video | MICCAI 2010 |
Surgical Gesture Recognition
| Title | Venue | Links |
|---|---|---|
| Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery | ICRA 2021 | Paper |
| Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search | ICRA 2020 | Paper |
| Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and Progress Prediction | ICRA 2020 | Paper |
| Automated surgical activity recognition with one labeled sequence | MICCAI 2019 | Paper |
| Using 3d convolutional neural networks to learn spatiotemporal features for automatic surgical gesture recognition in video | MICCAI 2019 | Paper |
| Segmenting and classifying activities in robot-assisted surgery with recurrent neural networks | IJCARS 2019 | Paper |
| Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification | MICCAI 2018 | Paper Code |
| Unsupervised learning for surgical motion by learning to predict the future | MICCAI 2018 | |
| Surgical motion analysis using discriminative interpretable patterns | AIM 2018 | Paper |
| A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery | TBE 2017 | |
| Temporal Convolutional Networks for Action Segmentation and Detection | CVPR 2017 | |
| Temporal convolutional networks: A unified approach to action segmentation | ECCVW 2016 | |
| Recognizing surgical activities with recurrent neural networks | MICCAI 2016 | |
| Segmental spatiotemporal cnns for fine-grained action segmentation | ECCV 2016 | |
| Learning convolutional action primitives for fine-grained action recognition | ICRA 2016 | |
| Unsupervised surgical data alignment with application to automatic activity annotation | ICRA 2016 | Paper |
| An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks | WACV 2015 | |
| Learning shared, discriminative dictionaries for surgical gesture segmentation and classification | MICCAIW 2015 | |
| JHU-ISI gesture and skill assessment working set (JIGSAWS): A surgical activity dataset for human motion modeling | MICCAIW 2014 | |
| Surgical gesture segmentation and recognition | MICCAI 2013 | |
| Surgical gesture classification from video and kinematic data | MIA 2013 | Paper |
| Sparse hidden markov models for surgical gesture classification and skill evaluation | IPCAI 2011 | |
| Data-derived models for segmentation with application to surgical assessment and training | MICCAI 2009 | |
| Automatic detection and segmentation of robot-assisted surgical motions | MICCAI 2005 |
Surgical Instrument Recognition / Localization
| Title | Venue | Links |
|---|---|---|
| hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries | MICCAI 2021 | Paper |
| Prototypical Interaction Graph for Unsupervised Domain Adaptation in Surgical Instrument Segmentation | MICCAI 2021 | Paper |
| Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data | MICCAI 2021 | Paper |
| One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video | ICRA 2021 | Paper |
| A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images | ICRA 2021 | Paper |
| Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation | IJCARS 2021 | Paper |
| Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery | MICCAI 2020 | Paper |
| Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion | MICCAI 2020 | Paper Code |
| Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video | MICCAI 2020 | Paper Code |
| Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery | MICCAI 2020 | Paper |
| BARNet: Bilinear Attention Network with Adaptive Receptive Field for Surgical Instrument Segmentation | Arxiv 2020 | |
| Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis | MIA 2020 | Paper Code |
| Real-time surgical needle detection using region-based convolutional neural networks | IJCARS 2020 | Paper |
| Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision | ML-CDS 2020 | Paper Code |
| Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video | MICCAI 2019 | Paper Code |
| Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos | IJCARS 2019 | |
| Self-supervised surgical tool segmentation using kinematic information | ICRA 2019 | |
| Learning Where to Look While Tracking Instruments in Robot-assisted Surgery | MICCAI 2019 | |
| Patch-based adaptive weighting with segmentation and scale (PAWSS) for visual tracking in surgical video | MIA 2019 | Paper |
| CATARACTS: Challenge on automatic tool annotation for cataRACT surgery | MIA 2019 | |
| 2017 robotic instrument segmentation challenge | Arxiv 2019 | |
| Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos | Arxiv 2019 | Paper |
| U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instruments from Laparoscopic Images | EMBC 2019 | |
| RASNet: Segmentation for tracking surgical instruments in surgical videos using refined attention segmentation network | EMBC 2019 | |
| CFCM: Segmentation via coarse to fine context memory | MICCAI 2018 | |
| Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks | MIA 2018 | Paper |
| 3D Pose Estimation of Articulated Instruments in Robotic Minimally Invasive Surgery | TMI 2018 | Paper |
| Exploiting the potential of unlabeled endoscopic video data with self-supervised learning | IJCARS 2018 | Paper |
| Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery | Arxiv 2018 | Paper |
| Weakly-supervised learning for tool localization in laparoscopic videos | MICCAIW 2018 | |
| Automatic instrument segmentation in robot-assisted surgery using deep learning | ICMLA 2018 | |
| Concurrent segmentation and localization for tracking of surgical instruments | MICCAI 2017 | |
| Simultaneous recognition and pose estimation of instruments in minimally invasive surgery | MICCAI 2017 | Paper |
| Toolnet: holistically-nested real-time segmentation of robotic surgical tools | IROS 2017 | |
| Real-time localization of articulated surgical instruments in retinal microsurgery | MIA 2016 | |
| Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery | IJCARS 2016 | Paper |
| Detecting surgical tools by modelling local appearance and global shape | TMI 2015 | |
| Surgical Tool Tracking and Pose Estimation in Retinal Microsurgery | MICCAI 2015 | Paper |
| Toward Detection and Localization of Instruments in Minimally Invasive Surgery | TBE 2013 | Paper |
| Unified detection and tracking of instruments during retinal microsurgery | TPAMI 2013 | Paper |
| Data-driven visual tracking in retinal microsurgery | MICCAI 2012 | Paper |
Surgical Scene / Anatomy Recognition
| Title | Venue | Links |
|---|---|---|
| Artificial Intelligence for Surgical Safety: Automatic Assessment of the Critical View of Safety in Laparoscopic Cholecystectomy using Deep Learning | AnnS 2020 | Paper |
| EasyLabels: weak labels for scene segmentation in laparoscopic videos | IJCARS 2019 | Paper |
| Kidney edge detection in laparoscopic image data for computer-assisted surgery | IJCARS 2019 | Paper |
| Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy | TBE 2018 | Paper |
| Long Term Safety Area Tracking (LT-SAT) with online failure detection and recovery for robotic minimally invasive surgery | MIA 2018 | Paper |
Human / Activity in Operating Rooms
| Title | Venue | Links |
|---|---|---|
| Self-supervision on Unlabelled OR Data for Multi-person 2D/3D Human Pose Estimation | MICCAI 2020 | Paper |
| Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery | MICCAI 2020 | Paper |
| Privacy-Preserving Human Pose Estimation on Low-Resolution Depth Images | MICCAI 2019 | Paper |
| Face Detection in the Operating Room: Comparison of State-of-the-art Methods and a Self-supervised Approach | IPCAI 2019 | Paper |
| A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms | WACV 2017 | Paper |
| Pictorial Structures on RGB-D Images for Human Pose Estimation in the Operating Room | MICCAI 2015 | Paper |
| Data-driven Spatio-temporal RGBD Feature Encoding for Action Recognition in Operating Rooms | IJCARS 2015 | Paper |
| Video based activity recognition in trauma resuscitation | FG 2013 | Paper |
Others
| Title | Venue | Links |
|---|---|---|
| Surgical Instruction Generation with Transformers | MICCAI 2021 | Paper |
| SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance | MICCAI 2021 | Paper |
| E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception | MICCAI 2021 | Paper |
| Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery | ICRA 2021 | |
| A Computer Vision Platform to Automatically Locate Critical Events in Surgical Videos | AnnS 2021 | Paper |
| Ethical implications of AI in robotic surgical training: A Delphi consensus statement | EUF 2021 | Paper |
| Future Frame Prediction for Robot-assisted Surgery | IPMI 2021 | Paper |
| Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge | Arxiv 2021 | Paper |
| Offline identification of surgical deviations in laparoscopic rectopexy | AIM 2020 | Paper |
| Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets | MICCAI 2020 | Paper |
| Orientation Matters: 6-DoF Autonomous Camera Movement for Minimally Invasive Surgery | Arxiv 2020 | Paper |
| Future-State Predicting LSTM for Early Surgery Type Recognition | TMI 2019 | Paper |
| Dense depth estimation in monocular endoscopy with self-supervised learning methods | TMI 2019 | |
| Real-time identification of blood regions for hemostasis support in laparoscopic surgery | SIVP 2019 | Paper |
| RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations | TMI 2018 | Paper |
| Learning to see forces: Surgical force prediction with rgb-point cloud temporal convolutional networks | MICCAIW 2018 | Paper |
| Deep Neural Networks Predict Remaining Surgery Duration from Cholecystectomy Videos | MICCAI 2017 | Paper |
| Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery | IJCARS 2017 | Paper |
| Surgical Soundtracks: Towards Automatic Musical Augmentation of Surgical Procedures | MICCAI 2017 | Paper |
| Distinguishing surgical behavior by sequential pattern discovery | JBI 2017 | Paper |
| Finding discriminative and interpretable patterns in sequences of surgical activities | AIM 2017 | Paper |
| Automatic matching of surgeries to predict surgeons’ next actions | AIM 2017 | Paper |
| Query-by-example surgical activity detection | IJCARS 2016 | Paper |
| Classification Approach for Automatic Laparoscopic Video Database Organization | IJCARS 2015 | Paper |
| Work domain constraints for modelling surgical performance | IJCARS 2015 | Paper |
| Smoke detection in endoscopic surgery videos: a first step towards retrieval of semantic events | IJMRCAS 2015 | Paper |
| Fisher Kernel Based Task Boundary Retrieval in Laparoscopic Database with Single Video Query | MICCAI 2014 | Paper |
| Non-linear temporal scaling of surgical processes | AIM 2014 | Paper |
| Surgical process modelling: a review | IJCARS 2014 | Paper |
| Real-time dense stereo reconstruction using convex optimisation with a cost-volume for image-guided robotic surgery | MICCAI 2013 | Paper |
| Vision-based proximity detection in retinal surgery | TBE 2012 | Paper |
| Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures | IJCARS 2012 | Paper |
| Similarity metrics for surgical process models | AIM 2012 | Paper |
Contributors
Similar repos
README
Awesome-Surgical-Video-Analysis
Keep updating. Welcome to contribute.
Contents
Surgical Instrument Recognition / Localization
Surgical Scene / Anatomy Recognition
Human / Activity in Operating Rooms
The Big Picture
| Title | Venue | Links |
|---|---|---|
| Surgomics: personalized prediction of morbidity, mortality and long‐term outcome in surgery using machine learning on multimodal data | Surgical Endoscopy 2020 | Paper |
| Surgical Data Science: from Concepts to Clinical Translation | Arxiv 2020 | Paper |
| Surgical Data Science: A Consensus Perspective | Arxiv 2018 | Paper |
| Surgical data science for next-generation interventions | NBE 2017 | Paper |
| CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions | IEEE 2019 | Paper |
| OR Black Box and Surgical Control Tower: Recording and Streaming data and Analytics to Improve Surgical Care | JVS 2021 | Paper |
| General-purpose foundation models for increased autonomy in robot-assisted surgery | arxiv 2024 | Paper |
Survey
| Title | Venue | Links |
|---|---|---|
| Machine learning for technical skill assessment in surgery: a systematic review | NPJ 2022 | Paper ｜ |
| A Review on Deep Learning in Minimally Invasive Surgery | IEEE Access 2021 | Paper |
| Deep neural networks for the assessment of surgical skills: A systematic review | IEEE Access 2021 | Paper |
| Current methods for assessing technical skill in cataract surgery | JCRS 2021 | Paper |
| Gesture Recognition in Robotic Surgery: a Review | TBE 2021 | Paper |
| Objective assessment of surgical technical skill and competency in the operating room | ARBE 2017 | Paper |
| Vision-based and marker-less surgical tool detection and tracking: a review of the literature | MIA 2017 | Paper |
| Surgical process modelling: a review | IJCARS 2013 | Paper |
Surgical Skill Assessment
| Title | Venue | Links |
|---|---|---|
| Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark | Arxiv 2021 | Paper |
| Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis | JAMA 2021 | Paper |
| Towards Unified Surgical Skill Assessment | CVPR 2021 | Paper Code |
| Clearness of Operating Field: A Surrogate for Surgical Skills on In-Vivo Clinical Data | IJCARS 2020 | Paper |
| Predicting the quality of surgical exposure using spatial and procedural features from laparoscopic videos | IJCARS 2020 | Paper |
| The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos | CVPR 2019 | Paper Code |
| Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field | MICCAI 2019 | Paper |
| Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating | AnnS 2019 | Paper |
| Towards Optimizing Convolutional Neural Networks for Robotic Surgery Skill Evaluation | IJCNN 2019 | Paper |
| Video-based surgical skill assessment using 3D convolutional neural networks | IJCARS 2019 | Paper Code |
| Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks | IJCARS 2019 | Paper Code |
| Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions | IJCARS 2019 | Paper |
| Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery | IJCARS 2019 | Paper |
| Objective classification of psychomotor laparoscopic skills of surgeons based on three different approaches | IJCARS 2019 | |
| Who’s Better? Who’s Best? Pairwise Deep Ranking for Skill Determination | CVPR 2018 | |
| Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks | WACV 2018 | |
| Evaluating surgical skills from kinematic data using convolutional neural networks | MICCAI 2018 | Paper Code |
| Automated surgical skill assessment in RMIS training | IJCARS 2018 | |
| Video and accelerometer-based motion analysis for automated surgical skills assessment | IJCARS 2018 | |
| Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery | IJCARS 2018 | |
| Surgical skills: Can learning curves be computed from recordings of surgical activities? | IJCARS 2018 | Paper |
| Automated robot‐assisted surgical skill evaluation: Predictive analytics approach | IJMRCAR 2018 | |
| Meaningful Assessment of Surgical Expertise: Semantic Labeling with Data and Crowds | MICCAI 2016 | |
| Automated video-based assessment of surgical skills for training and evaluation in medical schools | IJCARS 2016 | |
| Task-Level vs. Segment-Level Quantitative Metrics for Surgical Skill Assessment | Surg Educ 2016 | |
| Relative Hidden Markov Models for Video-Based Evaluation of Motion Skills in Surgical Training | TPAMI 2015 | |
| Automated Assessment of Surgical Skills Using Frequency Analysis | MICCAI 2015 | |
| Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty | IJCARS 2015 | |
| Automated Surgical OSATS Prediction From Videos | ISBI 2014 | |
| Pairwise Comparison-Based Objective Score for Automated Skill Assessment of Segments in a Surgical Task | IPCAI 2014 | |
| Video Based Assessment of OSATS Using Sequential Motion Textures | MICCAIW 2014 | |
| Augmenting Bag-of-Words: Data-Driven Discovery of Temporal and Structural Information for Activity Recognition | CVPR 2013 | |
| String Motif-Based Description of Tool Motion for Detecting Skill and Gestures in Robotic Surgery | MICCAI 2013 | |
| Robotic Path Planning for Surgeon Skill Evaluation in Minimally-Invasive Sinus Surgery | MICCAI 2012 | |
| An objective and automated method for assessing surgical skill in endoscopic sinus surgery using eye-tracking and tool-motion data | IFAR 2012 | |
| Sparse Hidden Markov Models for Surgical Gesture Classification and Skill Evaluation | IPCAI 2012 | |
| Towards integrating task information in skills assessment for dexterous tasks in surgery and simulation | ICRA 2011 | |
| Video-based Motion Expertise Analysis in Simulation-based Surgical Training Using Hierarchical Dirichlet Process Hidden Markov Model | MMAR 2011 | |
| Eye Metrics as an Objective Assessment of Surgical Skill | AnnS 2010 | |
| Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery | MICCAI 2010 | |
| Task versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery | MICCAI 2009 | |
| Data-derived models for segmentation with application to surgical assessment and training | MICCAI 2009 |
Surgical Phase Recognition
| Title | Venue | Links |
|---|---|---|
| Self-supervised representation learning for surgical activity recognition | IJCARS 2021 | Paper |
| Surgical Workflow Anticipation Using Instrument Interaction | MICCAI 2021 | Paper |
| Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark | Arxiv 2021 | Paper |
| OperA: Attention-Regularized Transformers for Surgical Phase Recognition | MICCAI 2021 | Paper |
| Temporal Memory Relation Network for Workflow Recognition from Surgical Video | TMI 2021 | Paper Code |
| Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass | IJCARS 2021 | Paper |
| Train one, Classify one, Teach one"–Cross-surgery transfer learning for surgical step recognition | MIDL 2021 | Paper |
| TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks | MICCAI 2020 | |
| Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: Experimental research | IJS 2020 | Paper |
| Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis | MIA 2020 | Paper Code |
| LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition | IJCARS 2020 | Paper Code |
| Assisted phase and step annotation for surgical videos | IJCARS 2020 | |
| Accurate Detection of Out of Body Segments in Surgical Video using Semi-Supervised Learning | MIDL 2020 | Paper |
| Impact of data on generalization of AI for surgical intelligence applications | SciReports 2020 | Paper |
| Hard Frame Detection and Online Mapping for Surgical Phase Recognition | MICCAI 2019 | |
| MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation | CVPR 2019 | |
| Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition | IPCAI 2019 | Paper |
| Machine and deep learning for workflow recognition during surgery | MITAT 2019 | |
| Assessment of automated identification of phases in videos of cataract surgery using machine learning and deep learning techniques | JAMA 2019 | Paper |
| Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos | Arxiv 2019 | Paper |
| SV-RCNet: Workflow Recognition from Surgical Videos using Recurrent Convolutional Network | TMI 2018 | Paper Code |
| DeepPhase: Surgical Phase Recognition in CATARACTS Videos | MICCAI 2018 | |
| Surgical activity recognition in robot-assisted radical prostatectomy using deep learning | MICCAI 2018 | Paper |
| Knowledge transfer for surgical activity prediction | IJCARS 2018 | Paper |
| Temporal coherence-based self-supervised learning for laparoscopic workflow analysis | MICCAIW 2018 | Paper |
| “Deep-Onto” network for surgical workflow and context recognition | IJCARS 2018 | Paper |
| Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks | Arxiv 2018 | |
| EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos | TMI 2017 | |
| Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis | Arxiv 2017 | Paper |
| System events: readily accessible features for surgical phase detection | IJCARS 2016 | Paper |
| Automatic data-driven real-time segmentation and recognition of surgical workflow | IJCARS 2016 | Paper |
| MICCAI Workflow Challenge: Convolutional neural networks with time smoothing and Hidden Markov Model for video frames classification | Arxiv 2016 | |
| Automatic phase prediction from low-level surgical activities | IJCARS 2015 | Paper |
| Lapontospm: an ontology for laparoscopic surgeries and its application to surgical phase recognition | IJCARS 2015 | Paper |
| Real-time segmentation and recognition of surgical tasks in cataract surgery videos | TMI 2014 | Paper |
| Statistical modeling and recognition of surgical workflow | MIA 2012 | |
| A framework for the recognition of high-level surgical tasks from video images for cataract surgeries | TBE 2012 | Paper |
| An application-dependent framework for the recognition of high-level surgical tasks in the OR | TBE 2011 | |
| Modeling and Segmentation of Surgical Workflow from Laparoscopic Video | MICCAI 2010 |
Surgical Gesture Recognition
| Title | Venue | Links |
|---|---|---|
| Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery | ICRA 2021 | Paper |
| Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search | ICRA 2020 | Paper |
| Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and Progress Prediction | ICRA 2020 | Paper |
| Automated surgical activity recognition with one labeled sequence | MICCAI 2019 | Paper |
| Using 3d convolutional neural networks to learn spatiotemporal features for automatic surgical gesture recognition in video | MICCAI 2019 | Paper |
| Segmenting and classifying activities in robot-assisted surgery with recurrent neural networks | IJCARS 2019 | Paper |
| Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification | MICCAI 2018 | Paper Code |
| Unsupervised learning for surgical motion by learning to predict the future | MICCAI 2018 | |
| Surgical motion analysis using discriminative interpretable patterns | AIM 2018 | Paper |
| A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery | TBE 2017 | |
| Temporal Convolutional Networks for Action Segmentation and Detection | CVPR 2017 | |
| Temporal convolutional networks: A unified approach to action segmentation | ECCVW 2016 | |
| Recognizing surgical activities with recurrent neural networks | MICCAI 2016 | |
| Segmental spatiotemporal cnns for fine-grained action segmentation | ECCV 2016 | |
| Learning convolutional action primitives for fine-grained action recognition | ICRA 2016 | |
| Unsupervised surgical data alignment with application to automatic activity annotation | ICRA 2016 | Paper |
| An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks | WACV 2015 | |
| Learning shared, discriminative dictionaries for surgical gesture segmentation and classification | MICCAIW 2015 | |
| JHU-ISI gesture and skill assessment working set (JIGSAWS): A surgical activity dataset for human motion modeling | MICCAIW 2014 | |
| Surgical gesture segmentation and recognition | MICCAI 2013 | |
| Surgical gesture classification from video and kinematic data | MIA 2013 | Paper |
| Sparse hidden markov models for surgical gesture classification and skill evaluation | IPCAI 2011 | |
| Data-derived models for segmentation with application to surgical assessment and training | MICCAI 2009 | |
| Automatic detection and segmentation of robot-assisted surgical motions | MICCAI 2005 |
Surgical Instrument Recognition / Localization
| Title | Venue | Links |
|---|---|---|
| hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries | MICCAI 2021 | Paper |
| Prototypical Interaction Graph for Unsupervised Domain Adaptation in Surgical Instrument Segmentation | MICCAI 2021 | Paper |
| Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data | MICCAI 2021 | Paper |
| One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video | ICRA 2021 | Paper |
| A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images | ICRA 2021 | Paper |
| Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation | IJCARS 2021 | Paper |
| Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery | MICCAI 2020 | Paper |
| Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion | MICCAI 2020 | Paper Code |
| Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video | MICCAI 2020 | Paper Code |
| Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery | MICCAI 2020 | Paper |
| BARNet: Bilinear Attention Network with Adaptive Receptive Field for Surgical Instrument Segmentation | Arxiv 2020 | |
| Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis | MIA 2020 | Paper Code |
| Real-time surgical needle detection using region-based convolutional neural networks | IJCARS 2020 | Paper |
| Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision | ML-CDS 2020 | Paper Code |
| Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video | MICCAI 2019 | Paper Code |
| Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos | IJCARS 2019 | |
| Self-supervised surgical tool segmentation using kinematic information | ICRA 2019 | |
| Learning Where to Look While Tracking Instruments in Robot-assisted Surgery | MICCAI 2019 | |
| Patch-based adaptive weighting with segmentation and scale (PAWSS) for visual tracking in surgical video | MIA 2019 | Paper |
| CATARACTS: Challenge on automatic tool annotation for cataRACT surgery | MIA 2019 | |
| 2017 robotic instrument segmentation challenge | Arxiv 2019 | |
| Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos | Arxiv 2019 | Paper |
| U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instruments from Laparoscopic Images | EMBC 2019 | |
| RASNet: Segmentation for tracking surgical instruments in surgical videos using refined attention segmentation network | EMBC 2019 | |
| CFCM: Segmentation via coarse to fine context memory | MICCAI 2018 | |
| Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks | MIA 2018 | Paper |
| 3D Pose Estimation of Articulated Instruments in Robotic Minimally Invasive Surgery | TMI 2018 | Paper |
| Exploiting the potential of unlabeled endoscopic video data with self-supervised learning | IJCARS 2018 | Paper |
| Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery | Arxiv 2018 | Paper |
| Weakly-supervised learning for tool localization in laparoscopic videos | MICCAIW 2018 | |
| Automatic instrument segmentation in robot-assisted surgery using deep learning | ICMLA 2018 | |
| Concurrent segmentation and localization for tracking of surgical instruments | MICCAI 2017 | |
| Simultaneous recognition and pose estimation of instruments in minimally invasive surgery | MICCAI 2017 | Paper |
| Toolnet: holistically-nested real-time segmentation of robotic surgical tools | IROS 2017 | |
| Real-time localization of articulated surgical instruments in retinal microsurgery | MIA 2016 | |
| Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery | IJCARS 2016 | Paper |
| Detecting surgical tools by modelling local appearance and global shape | TMI 2015 | |
| Surgical Tool Tracking and Pose Estimation in Retinal Microsurgery | MICCAI 2015 | Paper |
| Toward Detection and Localization of Instruments in Minimally Invasive Surgery | TBE 2013 | Paper |
| Unified detection and tracking of instruments during retinal microsurgery | TPAMI 2013 | Paper |
| Data-driven visual tracking in retinal microsurgery | MICCAI 2012 | Paper |
Surgical Scene / Anatomy Recognition
| Title | Venue | Links |
|---|---|---|
| Artificial Intelligence for Surgical Safety: Automatic Assessment of the Critical View of Safety in Laparoscopic Cholecystectomy using Deep Learning | AnnS 2020 | Paper |
| EasyLabels: weak labels for scene segmentation in laparoscopic videos | IJCARS 2019 | Paper |
| Kidney edge detection in laparoscopic image data for computer-assisted surgery | IJCARS 2019 | Paper |
| Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy | TBE 2018 | Paper |
| Long Term Safety Area Tracking (LT-SAT) with online failure detection and recovery for robotic minimally invasive surgery | MIA 2018 | Paper |
Human / Activity in Operating Rooms
| Title | Venue | Links |
|---|---|---|
| Self-supervision on Unlabelled OR Data for Multi-person 2D/3D Human Pose Estimation | MICCAI 2020 | Paper |
| Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery | MICCAI 2020 | Paper |
| Privacy-Preserving Human Pose Estimation on Low-Resolution Depth Images | MICCAI 2019 | Paper |
| Face Detection in the Operating Room: Comparison of State-of-the-art Methods and a Self-supervised Approach | IPCAI 2019 | Paper |
| A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms | WACV 2017 | Paper |
| Pictorial Structures on RGB-D Images for Human Pose Estimation in the Operating Room | MICCAI 2015 | Paper |
| Data-driven Spatio-temporal RGBD Feature Encoding for Action Recognition in Operating Rooms | IJCARS 2015 | Paper |
| Video based activity recognition in trauma resuscitation | FG 2013 | Paper |
Others
| Title | Venue | Links |
|---|---|---|
| Surgical Instruction Generation with Transformers | MICCAI 2021 | Paper |
| SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance | MICCAI 2021 | Paper |
| E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception | MICCAI 2021 | Paper |
| Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery | ICRA 2021 | |
| A Computer Vision Platform to Automatically Locate Critical Events in Surgical Videos | AnnS 2021 | Paper |
| Ethical implications of AI in robotic surgical training: A Delphi consensus statement | EUF 2021 | Paper |
| Future Frame Prediction for Robot-assisted Surgery | IPMI 2021 | Paper |
| Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge | Arxiv 2021 | Paper |
| Offline identification of surgical deviations in laparoscopic rectopexy | AIM 2020 | Paper |
| Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets | MICCAI 2020 | Paper |
| Orientation Matters: 6-DoF Autonomous Camera Movement for Minimally Invasive Surgery | Arxiv 2020 | Paper |
| Future-State Predicting LSTM for Early Surgery Type Recognition | TMI 2019 | Paper |
| Dense depth estimation in monocular endoscopy with self-supervised learning methods | TMI 2019 | |
| Real-time identification of blood regions for hemostasis support in laparoscopic surgery | SIVP 2019 | Paper |
| RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations | TMI 2018 | Paper |
| Learning to see forces: Surgical force prediction with rgb-point cloud temporal convolutional networks | MICCAIW 2018 | Paper |
| Deep Neural Networks Predict Remaining Surgery Duration from Cholecystectomy Videos | MICCAI 2017 | Paper |
| Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery | IJCARS 2017 | Paper |
| Surgical Soundtracks: Towards Automatic Musical Augmentation of Surgical Procedures | MICCAI 2017 | Paper |
| Distinguishing surgical behavior by sequential pattern discovery | JBI 2017 | Paper |
| Finding discriminative and interpretable patterns in sequences of surgical activities | AIM 2017 | Paper |
| Automatic matching of surgeries to predict surgeons’ next actions | AIM 2017 | Paper |
| Query-by-example surgical activity detection | IJCARS 2016 | Paper |
| Classification Approach for Automatic Laparoscopic Video Database Organization | IJCARS 2015 | Paper |
| Work domain constraints for modelling surgical performance | IJCARS 2015 | Paper |
| Smoke detection in endoscopic surgery videos: a first step towards retrieval of semantic events | IJMRCAS 2015 | Paper |
| Fisher Kernel Based Task Boundary Retrieval in Laparoscopic Database with Single Video Query | MICCAI 2014 | Paper |
| Non-linear temporal scaling of surgical processes | AIM 2014 | Paper |
| Surgical process modelling: a review | IJCARS 2014 | Paper |
| Real-time dense stereo reconstruction using convex optimisation with a cost-volume for image-guided robotic surgery | MICCAI 2013 | Paper |
| Vision-based proximity detection in retinal surgery | TBE 2012 | Paper |
| Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures | IJCARS 2012 | Paper |
| Similarity metrics for surgical process models | AIM 2012 | Paper |