Purpose: To examine the real-world performance of a support vector machine learning software (RetinaLyze) in order to identify the possible presence of diabetic retinopathy (DR) in patients with diabetes via software implementation in clinical practice.
Methods: 1001 eyes from 1001 patients-one eye per patient-participating in the Danish National Screening Programme were included. Three independent ophthalmologists graded all eyes according to the International Clinical Diabetic Retinopathy Disease Severity Scale with the exact level of disease being determined by majority decision. The software detected DR and no DR and was compared to the ophthalmologists' gradings.
Results: At a clinical chosen threshold, the software showed a sensitivity, specificity, positive predictive value and negative predictive value of 84.9% (95% CI: 81.8-87.9), 89.9% (95% CI: 86.8-92.7), 92.1% (95% CI: 89.7-94.4), and 81.0% (95% CI: 77.2-84.7), respectively, when compared to human grading. The results from the routine screening were 87.0% (95% CI: 84.2-89.7), 85.3% (95% CI: 81.8-88.6), 89.2% (95% CI: 86.3-91.7), and 82.5% (95% CI: 78.5-86.0), respectively. AUC was 93.4%. The reference graders Conger's Exact Kappa was 0.827.
Conclusion: The software performed similarly to routine grading with overlapping confidence intervals, indicating comparable performance between the two groups. The intergrader agreement was satisfactory. However, evaluating the updated software alongside updated clinical procedures is crucial. It is therefore recommended that further clinical testing before implementation of the software as a decision support tool is conducted.
Keywords: RetinaLyze; diabetic retinopathy; machine learning; screening.