Hagnifinder: Recovering magnification information of digital histological images using deep learning

Hongtai Zhang; Zaiyi Liu; Mingli Song; Cheng Lu

doi:10.1016/j.jpi.2023.100302

Hagnifinder: Recovering magnification information of digital histological images using deep learning

J Pathol Inform. 2023 Feb 16:14:100302. doi: 10.1016/j.jpi.2023.100302. eCollection 2023.

Authors

Hongtai Zhang¹, Zaiyi Liu^{2

3

4}, Mingli Song^{1

5}, Cheng Lu^{2

3

4}

Affiliations

¹ School of Computer and Cyber Sciences, Communication University of China, Beijing 100024, China.
² Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences),Southern Medical University, Guangzhou 510080, China.
³ Medical Research Institute, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou 510080, China.
⁴ Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application, Guangzhou 510080, China.
⁵ State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China.

Abstract

Background and objective: Training a robust cancer diagnostic or prognostic artificial intelligent model using histology images requires a large number of representative cases with labels or annotations, which are difficult to obtain. The histology snapshots available in published papers or case reports can be used to enrich the training dataset. However, the magnifications of these invaluable snapshots are generally unknown, which limits their usage. Therefore, a robust magnification predictor is required for utilizing those diverse snapshot repositories consisting of different diseases. This paper presents a magnification prediction model named Hagnifinder for H&E-stained histological images.

Methods: Hagnifinder is a regression model based on a modified convolutional neural network (CNN) that contains 3 modules: Feature Extraction Module, Regression Module, and Adaptive Scaling Module (ASM). In the training phase, the Feature Extraction Module first extracts the image features. Secondly, the ASM is proposed to address the learned feature values uneven distribution problem. Finally, the Regression Module estimates the mapping between the regularized extracted features and the magnifications. We construct a new dataset for training a robust model, named Hagni40, consisting of 94 643 H&E-stained histology image patches at 40 different magnifications of 13 types of cancer based on The Cancer Genome Atlas. To verify the performance of the Hagnifinder, we measure the accuracy of the predictions by setting the maximum allowable difference values (0.5, 1, and 5) between the predicted magnification and the actual magnification. We compare Hagnifinder with state-of-the-art methods on a public dataset BreakHis and the Hagni40.

Results: The Hagnifinder provides consistent prediction accuracy, with a mean accuracy of 98.9%, across 40 different magnifications and 13 different cancer types when Resnet50 is used as the feature extractor. Compared with the state-of-the-art methods focusing on 4-5 levels of magnification classification, the Hagnifinder achieves the best and most comparable performance in the BreakHis and Hagni40 datasets.

Conclusions: The experimental results suggest that Hagnifinder can be a valuable tool for predicting the associated magnification of any given histology image.

Keywords: Deep learning; Histology images; Magnification prediction; Regression model.