Telestration with augmented reality improves surgical performance through gaze guidance

Eleni Amelia Felinska; Thomas Ewald Fuchs; Alexandros Kogkas; Zi-Wei Chen; Benjamin Otto; Karl-Friedrich Kowalewski; Jens Petersen; Beat Peter Müller-Stich; George Mylonas; Felix Nickel

doi:10.1007/s00464-022-09859-7

Telestration with augmented reality improves surgical performance through gaze guidance

Surg Endosc. 2023 May;37(5):3557-3566. doi: 10.1007/s00464-022-09859-7. Epub 2023 Jan 6.

Authors

Affiliations

¹ Department of General, Visceral and Transplant Surgery, Heidelberg University Hospital, 69120, Heidelberg, Germany.
² Hamlyn Centre for Robotic Surgery, Imperial College London, London, SW7 2AZ, UK.
³ Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, SW7 2AZ, UK.
⁴ Department of Urology and Urological Surgery, University Medical Center Mannheim, Heidelberg University, 68167, Mannheim, Germany.
⁵ Department of Medical Image Computing, German Cancer Research Center, 69120, Heidelberg, Germany.
⁶ Department of General, Visceral and Transplant Surgery, Heidelberg University Hospital, 69120, Heidelberg, Germany. felix.nickel@med.uni-heidelberg.de.

Abstract

Background: In minimally invasive surgery (MIS), trainees need to learn how to interpret the operative field displayed on the laparoscopic screen. Experts currently guide trainees mainly verbally during laparoscopic procedures. A newly developed telestration system with augmented reality (iSurgeon) allows the instructor to display hand gestures in real-time on the laparoscopic screen in augmented reality to provide visual expert guidance (telestration). This study analysed the effect of telestration guided instructions on gaze behaviour during MIS training.

Methods: In a randomized-controlled crossover study, 40 MIS naive medical students performed 8 laparoscopic tasks with telestration or with verbal instructions only. Pupil Core eye-tracking glasses were used to capture the instructor's and trainees' gazes. Gaze behaviour measures for tasks 1-7 were gaze latency, gaze convergence and collaborative gaze convergence. Performance measures included the number of errors in tasks 1-7 and trainee's ratings in structured and standardized performance scores in task 8 (ex vivo porcine laparoscopic cholecystectomy).

Results: There was a significant improvement 1-7 on gaze latency [F(1,39) = 762.5, p < 0.01, η_p² = 0.95], gaze convergence [F(1,39) = 482.8, p < 0.01, η_p² = 0.93] and collaborative gaze convergence [F(1,39) = 408.4, p < 0.01, η_p² = 0.91] upon instruction with iSurgeon. The number of errors was significantly lower in tasks 1-7 (0.18 ± 0.56 vs. 1.94 ± 1.80, p < 0.01) and the score ratings for laparoscopic cholecystectomy were significantly higher with telestration (global OSATS: 29 ± 2.5 vs. 25 ± 5.5, p < 0.01; task-specific OSATS: 60 ± 3 vs. 50 ± 6, p < 0.01).

Conclusions: Telestration with augmented reality successfully improved surgical performance. The trainee's gaze behaviour was improved by reducing the time from instruction to fixation on targets and leading to a higher convergence of the instructor's and the trainee's gazes. Also, the convergence of trainee's gaze and target areas increased with telestration. This confirms augmented reality-based telestration works by means of gaze guidance in MIS and could be used to improve training outcomes.

Keywords: Augmented reality; Eye tracking; Gaze Guidance; Laparoscopy; Minimally invasive surgery; Patient Safety; Surgical training; Telestration.

Publication types

Randomized Controlled Trial
Research Support, Non-U.S. Gov't

MeSH terms

Animals
Augmented Reality*
Cholecystectomy, Laparoscopic / education
Cholecystectomy, Laparoscopic / methods
Clinical Competence
Cross-Over Studies
Education, Medical* / methods
Humans
Laparoscopy / education
Learning*
Students, Medical
Swine