AI-driven action recognition during cholecystectomy based on endoscopy image data

Context

Optimized surgery management leads to faster patient care, relief of the surgery team as well as improvement of the hospital's economic situation. The goal of the Kimono project is therefore the automated time estimation of the running surgery by networked components and intelligent algorithms. To this end, various sub-areas are being considered, such as monitoring the instrument table or tracking medical personnel via ceiling cameras. Now, the data collected by the endoscope will be evaluated additionally. The subsystem for evaluating the information from the endoscope enables analyses from inside the body. AI-based image analysis is used to recognize surgical instruments, segment tissue structures and assign specific action processes.

Tasks

The following challenges await you within the project:

• You design and train intelligent algorithms for instrument detection and segmentation of endoscopy image data.
• You design models for action recognition during an ongoing surgery.
• You develop a data management system and create ground truth data for machine learning.

Requirements

• You are motivated, self-initiated and work conscientiously
• Your study focus is on information technology, signal processing or computer vision
• You have experience with Python and Machine Learning