Deep Learning Model for Real‑time Semantic Segmentation During Intraoperative Robotic Prostatectomy

Sung Gon Park; Jeonghyun Park; Hong Rock Choi; Jun Ho Lee; Sung Tae Cho; Young Goo Lee; Hanjong Ahn; Sahyun Pak

doi:10.1016/j.euros.2024.02.005

Deep Learning Model for Real‑time Semantic Segmentation During Intraoperative Robotic Prostatectomy

Eur Urol Open Sci. 2024 Feb 27:62:47-53. doi: 10.1016/j.euros.2024.02.005. eCollection 2024 Apr.

Authors

Sung Gon Park¹, Jeonghyun Park², Hong Rock Choi², Jun Ho Lee², Sung Tae Cho¹, Young Goo Lee¹, Hanjong Ahn³, Sahyun Pak¹

Affiliations

¹ Department of Urology, Hallym University College of Medicine, Kangnam Sacred Heart Hospital, Seoul, South Korea.
² STARLABS Corp., Seoul, South Korea.
³ Department of Urology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea.

Abstract

Background and objective: Recently, deep learning algorithms, including convolutional neural networks (CNNs), have shown remarkable progress in medical imaging analysis. Semantic segmentation, which segments an unknown image into different parts and objects, has potential applications in robotic surgery in areas where artificial intelligence (AI) can be applied, such as in AI-assisted surgery, surgeon training, and skill assessment. We aimed to investigate the performance of a CNN-based deep learning model in real-time segmentation in robot-assisted radical prostatectomy (RALP).

Methods: Intraoperative videos of RALP procedures were obtained. The reinforcement U-Net model was used for segmentation. Segmentation of the images of instruments, bladder, prostate, and seminal vesicle-vas deferens was performed. The dataset was preprocessed and split randomly into training, validation, and test data in a 7:2:1 ratio. Dice coefficient, intersection over union (IoU), and accuracy by class, which are commonly used in medical image segmentation, were calculated to evaluate the performance of the model.

Key findings and limitations: From 120 patient videos, 2400 images were selected for RALP procedures. The mean Dice scores for the identification of the instruments, bladder, prostate, and seminal vesicle-vas deferens were 0.96, 0.74, 0.85, and 0.84, respectively. Overall, when applied to the test data, the model had a mean Dice coefficient value of 0.85, IoU of 0.77, and accuracy of 0.85. Limitations included the sample size, lack of diversity in the methods of surgery, incomplete surgical processes, and lack of external validation.

Conclusions and clinical implications: The CNN-based segmentation provides accurate real-time recognition of surgical instruments and anatomy in RALP. Deep learning algorithms can be used to identify anatomy within the surgical field and could potentially be used to provide real-time guidance in robotic surgery.

Patient summary: We demonstrate the potential effectiveness of deep learning segmentation in robotic prostatectomy procedures. Deep learning algorithms could be used to identify anatomical structures within the surgical field and may provide real-time guidance in robotic surgery.

Keywords: Artificial intelligence; Deep learning; Prostatectomy; Segmentation.