A deep metric learning approach for histopathological image retrieval

Abstract

To distinguish ambiguous images during specimen slides viewing, pathologists usually spend lots of time to seek guidance from confirmed similar images or cases, which is inefficient. Therefore, several histopathological image retrieval methods have been proposed for pathologists to easily obtain images sharing similar content with the query images. However, these methods cannot ensure a reasonable similarity metric, and some of them need lots of annotated images to train a feature extractor to represent images. Motivated by this circumstance, we propose the first deep metric learning-based histopathological image retrieval method in this paper and construct a deep neural network based on the mixed attention mechanism to learn an embedding function under the supervision of image category information. With the learned embedding function, original images are mapped into the predefined metric space where similar images from the same category are close to each other, so that the distance between image pairs in the metric space can be regarded as a reasonable metric for image similarity. We evaluate the proposed method on two histopathological image retrieval datasets: our self-established dataset and a public dataset called Kimia Path24, on which the proposed method achieves recall in top-1 recommendation (Recall@1) of 84.04% and 97.89% respectively. Moreover, further experiments confirm that the proposed method can achieve comparable performance to several published methods with less training data, which hedges the shortage of annotated medical image data to some extent. Code is available at https://github.com/easonyang1996/DML_HistoImgRetrieval.

Keywords: Content base image retrieval; Deep metric learning; Histopathological image analysis.