A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology

Nicole Wake; Andrew B Rosenkrantz; William C Huang; James S Wysock; Samir S Taneja; Daniel K Sodickson; Hersh Chandarana

doi:10.1186/s41205-021-00125-5

A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology

3D Print Med. 2021 Oct 28;7(1):34. doi: 10.1186/s41205-021-00125-5.

Authors

Nicole Wake^{1

2}, Andrew B Rosenkrantz³, William C Huang⁴, James S Wysock⁴, Samir S Taneja⁴, Daniel K Sodickson³, Hersh Chandarana³

Affiliations

¹ Department of Radiology, Montefiore Medical Center, Albert Einstein College of Medicine, 111 East 210th Street, Bronx, NY, 10467, USA. nwake@montefiore.org.
² Center for Advanced Imaging Innovation and Research (CAI2R) and Bernard and Irene Schwartz Center for Biomedical Imaging, Department of Radiology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, USA. nwake@montefiore.org.
³ Center for Advanced Imaging Innovation and Research (CAI2R) and Bernard and Irene Schwartz Center for Biomedical Imaging, Department of Radiology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, USA.
⁴ Department of Urology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY, USA.

Abstract

Augmented reality (AR) and virtual reality (VR) are burgeoning technologies that have the potential to greatly enhance patient care. Visualizing patient-specific three-dimensional (3D) imaging data in these enhanced virtual environments may improve surgeons' understanding of anatomy and surgical pathology, thereby allowing for improved surgical planning, superior intra-operative guidance, and ultimately improved patient care. It is important that radiologists are familiar with these technologies, especially since the number of institutions utilizing VR and AR is increasing. This article gives an overview of AR and VR and describes the workflow required to create anatomical 3D models for use in AR using the Microsoft HoloLens device. Case examples in urologic oncology (prostate cancer and renal cancer) are provided which depict how AR has been used to guide surgery at our institution.

Keywords: Augmented reality; Medical imaging; Surgical planning; Three-dimensional; Urologic oncology.

Abstract

Grants and funding