Classifying Malignancy in Prostate Glandular Structures from Biopsy Scans with Deep Learning

Cancers (Basel). 2023 Apr 17;15(8):2335. doi: 10.3390/cancers15082335.

Abstract

Histopathological classification in prostate cancer remains a challenge with high dependence on the expert practitioner. We develop a deep learning (DL) model to identify the most prominent Gleason pattern in a highly curated data cohort and validate it on an independent dataset. The histology images are partitioned in tiles (14,509) and are curated by an expert to identify individual glandular structures with assigned primary Gleason pattern grades. We use transfer learning and fine-tuning approaches to compare several deep neural network architectures that are trained on a corpus of camera images (ImageNet) and tuned with histology examples to be context appropriate for histopathological discrimination with small samples. In our study, the best DL network is able to discriminate cancer grade (GS3/4) from benign with an accuracy of 91%, F1-score of 0.91 and AUC 0.96 in a baseline test (52 patients), while the cancer grade discrimination of the GS3 from GS4 had an accuracy of 68% and AUC of 0.71 (40 patients).

Keywords: Gleason cancer grading; Gleason score; ISUP grade; convolutional neural network; deep learning; pathology; prostate; transfer learning; uropathology; whole-slide image.

Grants and funding

This research was funded in part by the National Cancer Institute (1R01CA190105, R01CA189295, U01CA200464, P30CA240139 and U01CA239141).