Automated Identification of Incomplete and Complete Retinal Epithelial Pigment and Outer Retinal Atrophy Using Machine Learning

Abstract

Objective: To assess and validate a deep learning algorithm to automatically detect incomplete retinal pigment epithelial and outer retinal atrophy (iRORA) and complete retinal pigment epithelial and outer retinal atrophy (cRORA) in eyes with age-related macular degeneration.

Design: In a retrospective machine learning analysis, a deep learning model was trained to jointly classify the presence of iRORA and cRORA within a given B-scan. The algorithm was evaluated using 2 separate and independent datasets.

Participants: OCT B-scan volumes from 71 patients with nonneovascular age-related macular degeneration captured at the Doheny-University of California Los Angeles Eye Centers and the following 2 external OCT B-scans testing datasets: (1) University of Pennsylvania, University of Miami, and Case Western Reserve University and (2) Doheny Image Reading Research Laboratory.

Methods: The images were annotated by an experienced grader for the presence of iRORA and cRORA. A Resnet18 model was trained to classify these annotations for each B-scan using OCT volumes collected at the Doheny-University of California Los Angeles Eye Centers. The model was applied to 2 testing datasets to assess out-of-sample model performance.

Main outcomes measures: Model performance was quantified in terms of area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). Sensitivity, specificity, and positive predictive value were also compared against additional clinician annotators.

Results: On an independently collected test set, consisting of 1117 volumes from the general population, the model predicted iRORA and cRORA presence within the entire volume with nearly perfect AUROC performance and AUPRC scores (iRORA, 0.61; 95% confidence interval [CI] [0.45, 0.82]: cRORA, 0.83; 95% CI [0.68, 0.95]). On another independently collected set, consisting of 60 OCT B-scans enriched for iRORA and cRORA lesions, the model performed with AUROC (iRORA: 0.68, 95% CI [0.54, 0.81]; cRORA: 0.84, 95% CI [0.75, 0.94]) and AUPRC (iRORA: 0.70, 95% CI [0.55, 0.86]; cRORA: 0.82, 95% CI [0.70, 0.93]).

Conclusions: A deep learning model can accurately and precisely identify both iRORA and cRORA lesions within the OCT B-scan volume. The model can achieve similar sensitivity compared with human graders, which potentially obviates a laborious and time-consuming annotation process and could be developed into a diagnostic screening tool.

Keywords: Age-related macular degeneration; Deep learning; Geographic atrophy; Incomplete retinal pigment epithelial and outer retinal atrophy; Nonneovascular macular degeneration.