Development and Validation of an Automatic Image-Recognition Endoscopic Report Generation System: A Multicenter Study

Jun-Yan Qu; Zhen Li; Jing-Ran Su; Ming-Jun Ma; Chang-Qin Xu; Ai-Jun Zhang; Cheng-Xia Liu; Hai-Peng Yuan; Yan-Liu Chu; Cui-Cui Lang; Liu-Ye Huang; Lin Lu; Yan-Qing Li; Xiu-Li Zuo

doi:10.14309/ctg.0000000000000282

Development and Validation of an Automatic Image-Recognition Endoscopic Report Generation System: A Multicenter Study

Clin Transl Gastroenterol. 2020 Dec 22;12(1):e00282. doi: 10.14309/ctg.0000000000000282.

Authors

Jun-Yan Qu^{1

2

3}, Zhen Li^{1

2

3}, Jing-Ran Su^{1

2

3}, Ming-Jun Ma^{1

2

3}, Chang-Qin Xu⁴, Ai-Jun Zhang⁵, Cheng-Xia Liu⁶, Hai-Peng Yuan⁷, Yan-Liu Chu⁸, Cui-Cui Lang⁹, Liu-Ye Huang¹⁰, Lin Lu¹¹, Yan-Qing Li^{1

2

3}, Xiu-Li Zuo^{1

2

3}

Affiliations

¹ Department of Gastroenterology, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China.
² Laboratory of Translational Gastroenterology, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China.
³ Robot Engineering Laboratory for Precise Diagnosis and Therapy of GI Tumor, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China.
⁴ Department of Gastroenterology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China.
⁵ Department of Gastroenterology, Qilu Hospital of Shandong University (Qingdao), Qingdao, China.
⁶ Department of Gastroenterology, Binzhou Medical University Hospital, Binzhou, China.
⁷ Department of Gastroenterology, Taian City Central Hospital, Taian, China.
⁸ Department of Gastroenterology, Weihai Municipal Hospital, Weihai, China.
⁹ Department of Gastroenterology, Liaocheng People's Hospital, Liaocheng, China.
¹⁰ Department of Gastroenterology, Yantai Yuhuangding Hospital, Yantai, China.
¹¹ Department of Gastroenterology, Linyi People's Hospital, Linyi, China.

Abstract

Introduction: Conventional gastrointestinal (GI) endoscopy reports written by physicians are time consuming and might have obvious heterogeneity or omissions, impairing the efficiency and multicenter consultation potential. We aimed to develop and validate an image recognition-based structured report generation system (ISRGS) through a multicenter database and to assess its diagnostic performance.

Methods: First, we developed and evaluated an ISRGS combining real-time video capture, site identification, lesion detection, subcharacteristics analysis, and structured report generation. White light and chromoendoscopy images from patients with GI lesions were eligible for study inclusion. A total of 46,987 images from 9 tertiary hospitals were used to train, validate, and multicenter test (6:2:2). Moreover, 5,699 images were prospectively enrolled from Qilu Hospital of Shandong University to further assess the system in a prospective test set. The primary outcome was the diagnosis performance of GI lesions in multicenter and prospective tests.

Results: The overall accuracy in identifying early esophageal cancer, early gastric cancer, early colorectal cancer, esophageal varices, reflux esophagitis, Barrett's esophagus, chronic atrophic gastritis, gastric ulcer, colorectal polyp, and ulcerative colitis was 0.8841 (95% confidence interval, 0.8775-0.8904) and 0.8965 (0.8883-0.9041) in multicenter and prospective tests, respectively. The accuracy of cecum and upper GI site identification were 0.9978 (0.9969-0.9984) and 0.8513 (0.8399-0.8620), respectively. The accuracy of staining discrimination was 0.9489 (0.9396-0.9568). The relative error of size measurement was 4.04% (range 0.75%-7.39%).

Discussion: ISRGS is a reliable computer-aided endoscopic report generation system that might assist endoscopists working at various hospital levels to generate standardized and accurate endoscopy reports (http://links.lww.com/CTG/A485).

Publication types

Multicenter Study
Research Support, Non-U.S. Gov't
Validation Study

MeSH terms

China
Databases as Topic
Endoscopy, Gastrointestinal / methods*
Gastrointestinal Diseases / diagnosis*
Gastrointestinal Diseases / pathology
Gastrointestinal Tract / diagnostic imaging*
Gastrointestinal Tract / pathology
Health Information Exchange*
Humans
Image Interpretation, Computer-Assisted / methods*
Prospective Studies
Reproducibility of Results
Retrospective Studies
Video Recording