Systematic Review of Machine Learning Applications Using Nonoptical Motion Tracking in Surgery – npj Digital Medicine
The advance of machine learning (ML) applications in surgery has seen the emergence of nonoptical motion tracking systems (NOMTS), offering insights into skill assessment, tool tracking, and feature detection among other uses. This systematic review examines the breadth of research in this field, assessing a total of 84 reports that meet strict inclusion criteria out of an initial 3,632 unique records. These studies are classified into six primary ML aims: skill assessment, feature detection, combination of skill assessment and feature detection, tool segmentation and/or tracking, undesirable motion filtration, and other innovative uses.
The studies explored diverse experimental setups, employing various sensors and surgical procedures. Most prevalent were robotic procedures, followed by laparoscopic and open surgeries. Bench-top tasks such as suturing dominated the experiments, while clinical models featuring real-life surgeries appeared in 18 experiments. The spectrum of procedures offers a panoramic view of NOMTS applications across different surgical contexts.
Human participant studies revealed variability in dataset sizes, with only 14 out of 40 experiments engaging at least 25 participants. Notably, the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) was a frequently referenced dataset, enabling the comparison of ML algorithms and advancing transfer learning applications. The availability and integration of datasets like the MISTIC-SL and RIOUS highlight ongoing efforts to advance algorithm development.
Diverse sensor types, both standalone and in combination, underpin the experiments detailed in this review. Mechanical and inertial sensors were the most commonly paired types. While NOMTS typically eschews optical inputs for ML analysis, video recordings were employed in 47 of 81 experiments to contextualize NOMTS data. This included an array of optical sensors such as RGB, infrared, and endoscopic cameras. In some setups, the video data served as a manual annotation aid or direct ML training input.
Crucial to the review is understanding the implications of sensor placements and how they influence skill assessment and task recognition. Insights were shared on optimal EMG sensor placements that enhance task measurement accuracy, especially within laparoscopic and robotic environments. The challenges of nonoptical data—including metallic interference and sensor calibration—are highlighted, with proposed solutions ranging from data exclusion to algorithmic corrections.
In discussing machine learning models themselves, the review recognizes the predominance of artificial neural networks (ANNs) in handling nonoptical data. Support vector machines (SVMs) and k-nearest neighbors (kNN) are also notable, though ANNs retain their lead due to versatility in skill and feature assessments. The review underscores the various evaluation metrics across studies, favoring mean accuracy for gauging method effectiveness.
Different domains within the ML applications in surgery, such as skill assessment and feature detection, exhibit distinct methodologies and outcome focuses. For instance, skill assessment—a key focus of the majority of the studies—employs various objective rating systems to benchmark participants and trainees objectively. Some studies reveal the potential for misclassification in skill levels, showcasing the ongoing need for advancements in skill categorization methodologies.
Tool Segmentation and Undesirable Motion Filtration
Tackling tool segmentation, 11 highlighted studies demonstrate the identification and localization of surgical instruments within operative fields, predominantly within robotic environments. Techniques range from ANN-based models to integrate video and EM data, offering new pathways for ML in recognizing surgical gestures and maneuvers.
Undesirable motion filtration, an emerging focus, equips algorithms to address detrimental surgical movements like tremors. The research demonstrates the power and feasibility of these algorithms, proving promising for further developments.
Concluding in the “other” category, several novel ML applications emerge, such as providing robotic surgeons with enhanced force feedback and improving haptic modeling in tissue simulation. These unique approaches continue to push the boundaries of what ML can offer in surgical contexts.
The review further stresses the scoring potential of these studies through the MERSQI scoring framework, noting that many are restricted by their nature as single-institution studies. However, the advances chronicled within these studies underline the transformative potential of ML in surgery, driving continuous innovation.
In summarizing the diverse algorithmic architectures and ML applications reflected in the review, the reliance on a range of NOMTS and ML models underscores the versatile nature of these solutions within the surgical field. This positions NOMTS as an integral part of ongoing efforts to enhance surgical accuracy, skill assessment, and procedural quality across diverse surgical domains.
This systematic review offers a comprehensive insight into the current landscape, potential, and challenges faced by machine learning applications with nonoptical motion tracking in surgery, setting the stage for future innovations and breakthroughs in this exciting field.
