Computer-vision based analysis of the neurosurgical scene - A systematic review

Félix Buyck; Jef Vandemeulebroucke; Jakub Ceranka; Frederick Van Gestel; Jan Frederick Cornelius; Johnny Duerinck; Michaël Bruneau

doi:10.1016/j.bas.2023.102706

Computer-vision based analysis of the neurosurgical scene - A systematic review

Brain Spine. 2023 Nov 7:3:102706. doi: 10.1016/j.bas.2023.102706. eCollection 2023.

Authors

Félix Buyck^{1

2}, Jef Vandemeulebroucke^{3

4

5}, Jakub Ceranka^{3

5}, Frederick Van Gestel^{1

2}, Jan Frederick Cornelius⁶, Johnny Duerinck^{1

2}, Michaël Bruneau^{1

2}

Affiliations

¹ Department of Neurosurgery, Universitair Ziekenhuis Brussel (UZ Brussel), 1090, Brussels, Belgium.
² Vrije Universiteit Brussel (VUB), Research group Center For Neurosciences (C4N-NEUR), 1090, Brussels, Belgium.
³ Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), 1050, Brussels, Belgium.
⁴ Department of Radiology, Universitair Ziekenhuis Brussel (UZ Brussel), 1090, Brussels, Belgium.
⁵ imec, 3001, Leuven, Belgium.
⁶ Department of Neurosurgery, Medical Faculty, Heinrich-Heine-University, 40225, Düsseldorf, Germany.

Abstract

Introduction: With increasing use of robotic surgical adjuncts, artificial intelligence and augmented reality in neurosurgery, the automated analysis of digital images and videos acquired over various procedures becomes a subject of increased interest. While several computer vision (CV) methods have been developed and implemented for analyzing surgical scenes, few studies have been dedicated to neurosurgery.

Research question: In this work, we present a systematic literature review focusing on CV methodologies specifically applied to the analysis of neurosurgical procedures based on intra-operative images and videos. Additionally, we provide recommendations for the future developments of CV models in neurosurgery.

Material and methods: We conducted a systematic literature search in multiple databases until January 17, 2023, including Web of Science, PubMed, IEEE Xplore, Embase, and SpringerLink.

Results: We identified 17 studies employing CV algorithms on neurosurgical videos/images. The most common applications of CV were tool and neuroanatomical structure detection or characterization, and to a lesser extent, surgical workflow analysis. Convolutional neural networks (CNN) were the most frequently utilized architecture for CV models (65%), demonstrating superior performances in tool detection and segmentation. In particular, mask recurrent-CNN manifested most robust performance outcomes across different modalities.

Discussion and conclusion: Our systematic review demonstrates that CV models have been reported that can effectively detect and differentiate tools, surgical phases, neuroanatomical structures, as well as critical events in complex neurosurgical scenes with accuracies above 95%. Automated tool recognition contributes to objective characterization and assessment of surgical performance, with potential applications in neurosurgical training and intra-operative safety management.

Keywords: Automated detection; Neuroanatomy; Surgical instruments; Surgical phase recognition; Surgical videos; computer vision.

Publication types

Review