Attention2majority: Weak multiple instance learning for regenerative kidney grading on whole slide images

Abstract

Deep learning consistently demonstrates high performance in classifying and segmenting medical images like CT, PET, and MRI. However, compared to these kinds of images, whole slide images (WSIs) of stained tissue sections are huge and thus much less efficient to process, especially for deep learning algorithms. To overcome these challenges, we present attention2majority, a weak multiple instance learning model to automatically and efficiently process WSIs for classification. Our method initially assigns exhaustively sampled label-free patches with the label of the respective WSIs and trains a convolutional neural network to perform patch-wise classification. Then, an intelligent sampling method is performed in which patches with high confidence are collected to form weak representations of WSIs. Lastly, we apply a multi-head attention-based multiple instance learning model to do slide-level classification based on high-confidence patches (intelligently sampled patches). Attention2majority was trained and tested on classifying the quality of 127 WSIs (of regenerated kidney sections) into three categories. On average, attention2majority resulted in 97.4%±2.4 AUC for the four-fold cross-validation. We demonstrate that the intelligent sampling module within attention2majority is superior to the current state-of-the-art random sampling method. Furthermore, we show that the replacement of random sampling with intelligent sampling in attention2majority results in its performance boost (from 94.9%±3.1 to 97.4%±2.4 average AUC for the four-fold cross-validation). We also tested a variation of attention2majority on the famous Camelyon16 dataset, which resulted in 89.1%±0.8 AUC¹. When compared to random sampling, the attention2majority demonstrated excellent slide-level interpretability. It also provided an efficient framework to arrive at a multi-class slide-level prediction.

Keywords: Deep learning; Digital pathology; Multiple instance learning; Regenerative medicine.