Application of a Deep Learning System to Detect Papilledema on Nonmydriatic Ocular Fundus Photographs in an Emergency Department

Valérie Biousse; Raymond P Najjar; Zhiqun Tang; Mung Yan Lin; David W Wright; Matthew T Keadey; Tien Y Wong; Beau B Bruce; Dan Milea; Nancy J Newman; BONSAI Study Group

doi:10.1016/j.ajo.2023.10.025

Application of a Deep Learning System to Detect Papilledema on Nonmydriatic Ocular Fundus Photographs in an Emergency Department

Am J Ophthalmol. 2024 May:261:199-207. doi: 10.1016/j.ajo.2023.10.025. Epub 2023 Nov 4.

Authors

Collaborators

BONSAI Study Group:
Clare L Fraser¹⁰, Jonathan A Micieli¹¹, Fiona Costello¹², Étienne Bénard-Séguin¹², Hui Yang¹³, Carmen Kar Mun Chan¹⁴, Carol Y Cheung¹⁴, Noel Cy Chan¹⁴, Steffen Hamann¹⁵, Philippe Gohier¹⁶, Anaïs Vautier¹⁶, Marie-Bénédicte Rougier¹⁷, Christophe Chiquet¹⁸, Catherine Vignal-Clermont¹⁹, Rabih Hage¹⁹, Raoul Kanav Khanna¹⁹, Thi Ha Chau Tran²⁰, Wolf Alexander Lagrèze²¹, Jost B Jonas²², Selvakumar Ambika²³, Masoud Aghsaei Fard²⁴, Chiara La Morgia²⁵, Michele Carbonelli²⁵, Piero Barboni²⁵, Valerio Carelli²⁵, Martina Romagnoli²⁵, Giulia Amore²⁵, Makoto Nakamura²⁶, Takano Fumio²⁶, Axel Petzold²⁷, Maillette de Buy Wenniger Lj²⁷, Richard Kho²⁸, Pedro L Fonseca²⁹, Mukharram M Bikbov³⁰, Dan Milea³¹, Raymond P Najjar³¹, Daniel Ting³¹, Zhiqun Tang³¹, Jing Liang Loo³¹, Sharon Tow³¹, Shweta Singhal³¹, Caroline Vasseneix³¹, Tien Yin Wong³¹, Ecosse Lamoureux³¹, Ching Yu Chen³¹, Tin Aung³¹, Leopold Schmetterer³¹, Nicolae Sanda³², Gabriele Thuman³², Jeong-Min Hwang³³, Kavin Vanikieti³⁴, Yanin Suwan³⁴, Tanyatuth Padungkiatsagul³⁴, Patrick Yu-Wai-Man³⁵, Neringa Jurkute³⁵, Eun Hee Hong³⁵, Valerie Biousse³⁶, Nancy J Newman³⁶, Jason H Peragallo³⁶, Michael Datillo³⁶, Sachin Kedar³⁶, Mung Yan Lin³⁶, Ajay Patil³⁶, Andre Aung³⁶, Matthew Boyko³⁶, Wael Abdulraman Alsakran³⁶, Amani Zayani³⁶, Walid Bouthour³⁶, Ana Banc³⁶, Rasha Mosley³⁶, Fernando Labella³⁶, Neil R Miller³⁷, John J Chen³⁸, Luis J Mejico³⁹, Janvier Ngoy Kilangalanga⁴⁰

Affiliations

¹ From the Department of Ophthalmology (V.B., M.Y.L., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA; Department of Neurology (V.B., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA. Electronic address: vbiouss@emory.edu.
² Singapore Eye Research Institute and Singapore National Eye Centre (R.P.N., Z.T., T.Y.W., D.M.), Singapore; Duke-NUS Medical School (R.P.N., T.Y.W., D.M.), National University of Singapore, Singapore; Eye N' Brain Research Group (R.P.N.), Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; Center for Innovation and Precision Eye Health (R.P.N.), Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
³ Singapore Eye Research Institute and Singapore National Eye Centre (R.P.N., Z.T., T.Y.W., D.M.), Singapore.
⁴ From the Department of Ophthalmology (V.B., M.Y.L., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA.
⁵ Department of Emergency Medicine (D.W.W., M.T.K.), Emory University School of Medicine, Atlanta, Georgia, USA.
⁶ Singapore Eye Research Institute and Singapore National Eye Centre (R.P.N., Z.T., T.Y.W., D.M.), Singapore; Duke-NUS Medical School (R.P.N., T.Y.W., D.M.), National University of Singapore, Singapore; Tsinghua Medicine (T.Y.W.), Tsinghua University, China.
⁷ From the Department of Ophthalmology (V.B., M.Y.L., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA; Department of Neurology (V.B., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA; Rollins School of Public Health (B.B.B.), Emory University School of Medicine, Atlanta, Georgia, USA.
⁸ Singapore Eye Research Institute and Singapore National Eye Centre (R.P.N., Z.T., T.Y.W., D.M.), Singapore; Duke-NUS Medical School (R.P.N., T.Y.W., D.M.), National University of Singapore, Singapore.
⁹ From the Department of Ophthalmology (V.B., M.Y.L., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA; Department of Neurology (V.B., B.B.B., N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA; Department of Neurological Surgery (N.J.N.), Emory University School of Medicine, Atlanta, Georgia, USA.
¹⁰ Save Sight Institute, Faculty of Health and Medicine, The University of Sydney, NSW Australia.
¹¹ Kensington Eye Institute, St. Michael's Hospital, Toronto Western Hospital, Toronto, Canada.
¹² Departments of Clinical Neurosciences and Surgery, University of Calgary, Canada.
¹³ Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, P.R. China.
¹⁴ Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China.
¹⁵ Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark.
¹⁶ Department of Ophthalmology, University Hospital Angers, Angers, France.
¹⁷ Department of Ophthalmology, Hôpital Pellegrin, CHU de Bordeaux, Bordeaux, France.
¹⁸ Department of Ophthalmology, Grenoble Alpes University Hospital, Grenoble, France.
¹⁹ Department of Ophthalmology, Fondation Adolphe de Rothschild, Paris, France.
²⁰ Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University and Inserm U1171, Lille, France.
²¹ Eye Center, Medical Center, University of Freiburg, Freiburg, Germany.
²² Department of Ophthalmology, Medical Faculty Mannheim of the Ruprecht-Karls-University of Heidelberg, Mannheim, Germany.
²³ Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India.
²⁴ Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran.
²⁵ IRCCS Istituto delle Scienze Neurologiche di Bologna, Bologna, Italy.
²⁶ Department of Surgery, Division of Ophthalmology. Kobe University Graduate School of Medicine, Kobe, Japan.
²⁷ Neuro-ophthalmology Expert Centre, Amsterdam University Medical Center, Amsterdam, Netherlands.
²⁸ American Eye Center, Mandaluyong City, Philippines.
²⁹ Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, Coimbra, Portugal.
³⁰ Ufa Eye Research Institute, Ufa, Russia.
³¹ Singapore National Eye Centre; Singapore Eye Research Institute, Singapore; National University of Singapore.
³² The Department of Clinical Neuroscience, Geneva University Hospital, Geneva, Switzerland.
³³ Department of Ophthalmology, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seoul, South Korea.
³⁴ Department of Ophthalmology, Faculty of Medicine Ramathibodi Hospital, Mahidol University. Bangkok, Thailand.
³⁵ Moorfields Eye Hospital NHS Foundation Trust, United Kingdom.
³⁶ Department of Ophthalmology, Emory University School of Medicine, Atlanta GA, USA.
³⁷ Departments of Ophthalmology, Neurology and Neurosurgery. Johns Hopkins University School of Medicine, Baltimore, MD, USA.
³⁸ Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MI, USA.
³⁹ Department of Neurology, SUNY Upstate Medical University, Syracuse, NY, USA; Democratic Republic of Congo.
⁴⁰ Dept of Ophthalmology, Saint Joseph Hospital, Boulevard Lumumba, 15eme rue, Kinshasa, Limete.

PMID: 37926337
DOI: 10.1016/j.ajo.2023.10.025

Abstract

Purpose: The Fundus photography vs Ophthalmoscopy Trial Outcomes in the Emergency Department (FOTO-ED) studies showed that ED providers poorly recognized funduscopic findings in patients in the ED. We tested a modified version of the Brain and Optic Nerve Study Artificial Intelligence (BONSAI) deep learning system on nonmydriatic fundus photographs from the FOTO-ED studies to determine if the deep learning system could have improved the detection of papilledema had it been available to ED providers as a real-time diagnostic aid.

Design: Retrospective secondary analysis of a cohort of patients included in the FOTO-ED studies.

Methods: The testing data set included 1608 photographs obtained from 828 patients in the FOTO-ED studies. Photographs were reclassified according to the optic disc classification system used by the deep learning system ("normal optic discs," "papilledema," and "other optic disc abnormalities"). The system's performance was evaluated by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity using a 1-vs-rest strategy, with reference to expert neuro-ophthalmologists.

Results: The BONSAI deep learning system successfully distinguished normal from abnormal optic discs (AUC 0.92 [95% confidence interval {CI} 0.90-0.93]; sensitivity 75.6% [73.7%-77.5%] and specificity 89.6% [86.3%-92.8%]), and papilledema from normal and others (AUC 0.97 [0.95-0.99]; sensitivity 84.0% [75.0%-92.6%] and specificity 98.9% [98.5%-99.4%]). Six patients with missed papilledema in 1 eye were correctly identified by the deep learning system as having papilledema in the other eye.

Conclusions: The BONSAI deep learning system was able to reliably identify papilledema and normal optic discs on nonmydriatic photographs obtained in the FOTO-ED studies. Our deep learning system has excellent potential as a diagnostic aid in EDs and non-ophthalmology clinics equipped with nonmydriatic fundus cameras. NOTE: Publication of this article is sponsored by the American Ophthalmological Society.

MeSH terms

Artificial Intelligence
Deep Learning*
Emergency Service, Hospital
Eye Diseases* / diagnosis
Fundus Oculi
Humans
Papilledema* / diagnosis
Photography
Retrospective Studies