Multimodal Image Fusion Offers Better Spatial Resolution for Mass Spectrometry Imaging

Anal Chem. 2023 Jun 27;95(25):9714-9721. doi: 10.1021/acs.analchem.3c02002. Epub 2023 Jun 9.

Abstract

High-resolution reconstruction has attracted increasing research interest in mass spectrometry imaging (MSI), but it remains a challenging ill-posed problem. In the present study, we proposed a deep learning model to fuse multimodal images to enhance the spatial resolution of MSI data, namely, DeepFERE. Hematoxylin and eosin (H&E) stain microscopy imaging was used to pose constraints in the process of high-resolution reconstruction to alleviate the ill-posedness. A novel model architecture was designed to achieve multi-task optimization by incorporating multi-modal image registration and fusion in a mutually reinforced framework. Experimental results demonstrated that the proposed DeepFERE model is able to produce high-resolution reconstruction images with rich chemical information and a detailed structure on both visual inspection and quantitative evaluation. In addition, our method was found to be able to improve the delimitation of the boundary between cancerous and para-cancerous regions in the MSI image. Furthermore, the reconstruction of low-resolution spatial transcriptomics data demonstrated that the developed DeepFERE model may find wider applications in biomedical fields.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Image Processing, Computer-Assisted* / methods
  • Mass Spectrometry / methods
  • Microscopy*