Use of mixed reality for surgery planning: Assessment and development workflow

Rahul Prasanna Kumar; Egidijus Pelanis; Robin Bugge; Henrik Brun; Rafael Palomar; Davit L Aghayan; Åsmund Avdem Fretland; Bjørn Edwin; Ole Jakob Elle

doi:10.1016/j.yjbinx.2020.100077

Use of mixed reality for surgery planning: Assessment and development workflow

J Biomed Inform. 2020:112S:100077. doi: 10.1016/j.yjbinx.2020.100077. Epub 2020 Sep 3.

Authors

Rahul Prasanna Kumar¹, Egidijus Pelanis², Robin Bugge³, Henrik Brun⁴, Rafael Palomar⁵, Davit L Aghayan⁶, Åsmund Avdem Fretland⁷, Bjørn Edwin⁷, Ole Jakob Elle⁸

Affiliations

¹ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway. Electronic address: rahul.kumar@ous-research.no.
² The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Institute of Clinical Medicine, University of Oslo, Kirkeveien 166, 0450 Oslo, Norway.
³ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Department of Diagnostic Physics, Oslo University Hospital, Sognsvannsvn 20, 0372 Oslo, Norway.
⁴ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Department for Pediatric Cardiology, Oslo University Hospital, Sognsvannsvn 20, 0372 Oslo, Norway.
⁵ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Department of Computer Science, NTNU, Teknologiveien 22, 2815 Gjøvik, Norway.
⁶ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Institute of Clinical Medicine, University of Oslo, Kirkeveien 166, 0450 Oslo, Norway; Department of Surgery N1, Yerevan State Medical University, Yerevan, Armenia.
⁷ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Institute of Clinical Medicine, University of Oslo, Kirkeveien 166, 0450 Oslo, Norway; Department of Hepatopancreatobiliary Surgery, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway.
⁸ The Intervention Centre, Oslo University Hospital, Sognsvannsveien 20, 0372 Oslo, Norway; Department of Informatics, University of Oslo, Gaustadallèen 23 B, 0373 Oslo, Norway.

PMID: 34417006
DOI: 10.1016/j.yjbinx.2020.100077

Abstract

Meticulous preoperative planning is an important part of any surgery to achieve high levels of precision and avoid complications. Conventional medical 2D images and their corresponding three-dimensional (3D) reconstructions are the main components of an efficient planning system. However, these systems still use flat screens for visualisation of 3D information, thus losing depth information which is crucial for 3D spatial understanding. Currently, cutting-edge mixed reality systems have shown to be a worthy alternative to provide 3D information to clinicians. In this work, we describe development details of the different steps in the workflow for the clinical use of mixed reality, including results from a qualitative user evaluation and clinical use-cases in laparoscopic liver surgery and heart surgery. Our findings indicate a very high general acceptance of mixed reality devices with our applications and they were consistently rated high for device, visualisation and interaction areas in our questionnaire. Furthermore, our clinical use-cases demonstrate that the surgeons perceived the HoloLens to be useful, recommendable to other surgeons and also provided a definitive answer at a multi-disciplinary team meeting.

Keywords: Mixed reality; Planning; Segmentation; Surgery.