Urine cell image recognition using a deep-learning model for an automated slide evaluation system

Abstract

Objectives: To develop a classification system for urine cytology with artificial intelligence (AI) using a convolutional neural network algorithm that classifies urine cell images as negative (benign) or positive (atypical or malignant).

Patients and methods: We collected 195 urine cytology slides from consecutive patients with a histologically confirmed diagnosis of urothelial cancer (between January 2016 and December 2017). Two certified cytotechnologists independently evaluated and labelled each slide; 4637 cell images with concordant diagnoses were selected, including 3128 benign cells (negative), 398 atypical cells, and 1111 cells that were malignant or suspicious for malignancy (positive). This pathologically confirmed labelled dataset was used to represent the ground truth for AI training/validation/testing. Customized CutMix (CircleCut) and Refined Data Augmentation were used for image processing. The model architecture included EfficientNet B6 and Arcface. We used 80% of the data for training and validation (4:1 ratio) and 20% for testing. Model performance was evaluated with fivefold cross-validation. A receiver-operating characteristic (ROC) analysis was used to evaluate the binary classification model. Bayesian posterior probabilities for the AI performance measure (Y) and cytotechnologist performance measure (X) were compared.

Results: The area under the ROC curve was 0.99 (95% confidence interval [CI] 0.98-0.99), the highest accuracy was 95% (95% CI 94-97), sensitivity was 97% (95% CI 95-99), and specificity was 95% (95% CI 93-97). The accuracy of AI surpassed the highest level of cytotechnologists for the binary classification [Pr(Y > X) = 0.95]. AI achieved >90% accuracy for all cell subtypes. In the subgroup analysis based on the clinicopathological characteristics of patients who provided the test cells, the accuracy of AI ranged between 89% and 97%.

Conclusion: Our novel AI classification system for urine cytology successfully classified all cell subtypes with an accuracy of higher than 90%, and achieved diagnostic accuracy of malignancy superior to the highest level achieved by cytotechnologists.

Keywords: artificial intelligence; computer-assisted image recognition; deep learning; urine cytology; urothelial carcinoma.