Extendable Artificial Intelligence for Surgical Phase Recognition in Medical Systems

Context

In 2016, surgeons around Feußner et al. discussed the concept of assisting systems for surgeons, analogous to how pilots are supported by avionic systems during flight. To provide such assistance, an assisting system must first understand the current situation in the operating room before determining how to assist the surgeon in the next step. While AI has demonstrated a strong capability to process sensor data, it depends on a known, encapsulated base dataset.

In the medical domain, surgical procedures share common execution steps but differ between medical centers. Furthermore, surgical procedures naturally undergo continuous improvement, leading to changes in the procedure and requiring the AI models to be adapted throughout their lifecycle.

Extending pre-trained AI models is a challenging task across different application domains. When the base dataset is extended by different surgical steps or tools, the model might forget previously learned information or confuse old and new information. In safety-critical applications where reliable performance must be ensured, this leads to the need for re-certification of the AI model, which is a time-consuming and costly process.

By adding guarantees, the recently proposed Prototype Replay with Interference-Suppressed Magnitude-freezing (PRISM) approach from the vehicular domain might also serve as a promising baseline for the medical domain. PRISM ensures that after extending the model with new classes, the likelihood ranking among base classes remains unchanged. This means that a base class is never mistaken for another base class after the extension step. Additionally, PRISM does not require the pre-training data in the extension step, which is a significant advantage in terms of data privacy and storage requirements. This is beneficial in the medical domain, where patient data is sensitive and often cannot be shared freely. However, in some cases, extension classes may still incorrectly supersede the likelihood ordering, causing a base class to be mistaken for an extension class, or vice versa.

Targets

• Analysis of the state of the art in surgical phase detection and classification, including the types of sensor data used 

• Analysis and comparison of existing approaches for establishing guarantees in extendable AI models, in general and in the medical domain 

• Investigation of the applicability of PRISM to surgical phase detection and of the necessary adaptations 

• Development of an extension of PRISM to incorporate temporal dependencies in classification problems, e.g. by adapting the backbone and the classification head 

• Evaluation of the extended approach on surgical phase classification tasks

• Optional: Identification of adaptations to additionally support the recognition of surgical actions within the current phase 

Requirements

• Basic understanding of artificial intelligence and its limitations
• Programming skills in Python and experience with using GPUs (CUDA)
• Familiarity with scientific work, literature research, and understanding of standards
• Willingness to learn about medical regulatory frameworks