Nasopharyngeal carcinoma segmentation based on enhanced convolutional neural networks using multi-modal metric learning

Phys Med Biol. 2019 Jan 8;64(2):025005. doi: 10.1088/1361-6560/aaf5da.

Abstract

Multi-modality examinations have been extensively applied in current clinical cancer management. Leveraging multi-modality medical images can be highly beneficial for automated tumor segmentation as they provide complementary information that could make the segmentation of tumors more accurate. This paper investigates CNN-based methods for automated nasopharyngeal carcinoma (NPC) segmentation using computed tomography (CT) and magnetic resonance (MR) images. Specially, a multi-modality convolutional neural network (M-CNN) is designed to jointly learn a multi-modal similarity metric and segmentation of paired CT-MR images. By jointly optimizing the similarity learning error and the segmentation error, the feature learning processes of both modalities are mutually guided. In doing so, the segmentation sub-networks are able to take advantage of the other modality's information. Considering that each modality possesses certain distinctive characteristics, we combine the higher-layer features extracted by a single-modality CNN (S-CNN) and M-CNN to form a combined CNN (C-CNN) for each modality, which is able to further utilize the complementary information of different modalities and improve the segmentation performance. The proposed M-CNN and C-CNN were evaluated on 90 CT-MR images of NPC patients. Experimental results demonstrate that our methods achieve improved segmentation performance compared to their counterparts without multi-modal information fusion and the existing CNN-based multi-modality segmentation methods.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Aged
  • Chemoradiotherapy
  • Female
  • Humans
  • Image Processing, Computer-Assisted / methods*
  • Magnetic Resonance Imaging / methods*
  • Male
  • Middle Aged
  • Multimodal Imaging / methods*
  • Nasopharyngeal Carcinoma / diagnostic imaging*
  • Nasopharyngeal Carcinoma / pathology*
  • Nasopharyngeal Carcinoma / therapy
  • Neural Networks, Computer*
  • Tomography, X-Ray Computed / methods*
  • Young Adult