Diagnostic accuracy of convolutional neural network-based machine learning algorithms in endoscopic severity prediction of ulcerative colitis: a systematic review and meta-analysis

Abstract

Background and aims: Endoscopic assessment of ulcerative colitis (UC) can be performed by using the Mayo Endoscopic Score (MES) or the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). In this meta-analysis, we assessed the pooled diagnostic accuracy parameters of deep machine learning by means of convolutional neural network (CNN) algorithms in predicting UC severity on endoscopic images.

Methods: Databases including MEDLINE, Scopus, and Embase were searched in June 2022. Outcomes of interest were the pooled accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Standard meta-analysis methods used the random-effects model, and heterogeneity was assessed using the I²statistics.

Results: Twelve studies were included in the final analysis. The pooled diagnostic parameters of CNN-based machine learning algorithms in endoscopic severity assessment of UC were as follows: accuracy 91.5% (95% confidence interval [CI], 88.3-93.8; I² = 84%), sensitivity 82.8% (95% CI, 78.3-86.5; I² = 89%), specificity 92.4% (95% CI, 89.4-94.6; I² = 84%), PPV 86.6% (95% CI, 82.3-90; I² = 89%), and NPV 88.6% (95% CI, 85.7-91; I² = 78%). Subgroup analysis revealed significantly better sensitivity and PPV with the UCEIS scoring system compared with the MES (93.6% [95% CI, 87.5-96.8; I² = 77%] vs 82% [95% CI, 75.6-87; I² = 89%], P = .003, and 93.6% [95% CI, 88.7-96.4; I² = 68%] vs 83.6% [95% CI, 76.8-88.8; I² = 77%], P = .007, respectively).

Conclusions: CNN-based machine learning algorithms demonstrated excellent pooled diagnostic accuracy parameters in the endoscopic severity assessment of UC. Using UCEIS scores in CNN training might offer better results than the MES. Further studies are warranted to establish these findings in real clinical settings.