Cross-domain Nuclei Detection in Histopathology Images using Graph-based Nuclei Feature Alignment

IEEE J Biomed Health Inform. 2023 May 29:PP. doi: 10.1109/JBHI.2023.3280958. Online ahead of print.

Abstract

As powerful tools deep neural networks have been successfully adopted for nuclei detection in histopathology images, whereas require the same probability distribution between training and testing data. However, domain shift among histopathology images widely exists in real-world applications and severely deteriorates the detection performance of deep neural networks. Despite encouraging results of existing domain adaptation methods, there remain challenges for cross-domain nuclei detection task. First, in view of the tiny size of nuclei, it is actually very difficult to obtain sufficient nuclei features, thus leading to a negative influence for feature alignment. Second, due to unavailable annotations in target domain, some extracted features contain background pixels and are thereby indiscriminative, which can largely confuse the alignment procedure. To address these challenges, in this paper, we propose an end-to-end graph-based nuclei feature alignment (GNFA) method for boosting cross-domain nuclei detection. Concretely, sufficient nuclei features are generated from nuclei graph convolutional network (NGCN) by aggregating information of adjacent nuclei upon construction of nuclei graph for successful alignment. In addition, importance learning module (ILM) is designed to further select discriminative nuclei features for mitigating negative influence of background pixels in target domain during alignment. By utilizing sufficient and discriminative node features generated from GNFA, our method can successfully perform feature alignment and effectively alleviate domain shift problem for nuclei detection. Extensive experiments of multiple adaptation scenarios reveal that our method achieves state-of-the-art performance in cross-domain nuclei detection compared with existing domain adaptation methods.