TransFusion-net for multifocus microscopic biomedical image fusion

Ronghao Pei; Kang Yao; Xiaobin Xu; Xin Zhang; Xiaodong Yang; Weiwei Fu; Yang Zhang

doi:10.1016/j.cmpb.2023.107688

TransFusion-net for multifocus microscopic biomedical image fusion

Comput Methods Programs Biomed. 2023 Oct:240:107688. doi: 10.1016/j.cmpb.2023.107688. Epub 2023 Jun 28.

Authors

Ronghao Pei¹, Kang Yao¹, Xiaobin Xu², Xin Zhang¹, Xiaodong Yang³, Weiwei Fu⁴, Yang Zhang⁵

Affiliations

¹ School of Biomedical Engineering (Suzhou), Division of Life Science and Medicine, University of Science and Technology of China, Hefei 230026, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China.
² College of Mechanical & Electrical Engineering, Hohai University, Changzhou, 213022, China.
³ Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China.
⁴ School of Biomedical Engineering (Suzhou), Division of Life Science and Medicine, University of Science and Technology of China, Hefei 230026, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China. Electronic address: fuww@sibet.ac.cn.
⁵ Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China. Electronic address: zhangyang@sibet.ac.cn.

PMID: 37487310
DOI: 10.1016/j.cmpb.2023.107688

Abstract

Background and objective: Due to the depth of focus (DOF) limitations of the optical systems of microscopes, it is often difficult to achieve full clarity from microscopic biomedical images under high-magnification microscopy. Multifocus microscopic biomedical image fusion (MFBIF) can effectively solve this problem. Considering both information richness and visual authenticity, this paper proposes a transformer network for MFBIF called TransFusion-Net.

Methods: TransFusion-Net consists of two modules. One module is an interlayer cross-attention module, which is used to obtain feature mappings under the long-range dependencies observed among multiple nonfocus source images. The other module is a spatial attention upsampling network (SAU-Net) module, which is used to obtain global semantic information after further spatial attention is applied. Thus, TransFusion-Net can simultaneously receive multiple input images from a nonfull-focus microscope and make full use of the strong correlations between the source images to output accurate fusion results in an end-to-end manner.

Results: The fusion results were quantitatively and qualitatively compared with those of eight state-of-the-art algorithms. In the quantitative experiments, five evaluation metrics, Q^AB/F, Q_MI, Q^AVG, Q_CB, and PSNR, were used to evaluate the performance of each method, and the proposed method achieved values of 0.6574, 8.4572, 5.6305, 0.7341, and 89.5685, respectively, which are higher than those of the current state-of-the-art algorithms. In the qualitative experiments, a differential image was used for further validation, and the near-zero residuals visually verified the adequacy of the proposed method for fusion. Furthermore, we showed some fusion results of multifocused biomedical microscopy images to verify the reliability of the proposed method, which shows high-quality fusion results.

Conclusion: Multifocus biomedical microscopic image fusion can be accurately and effectively achieved by devising a deep convolutional neural network with joint cross-attention and spatial attention mechanisms.

Keywords: Deep learning; End-to-end transformer network; Hybrid attention mechanism; Microscopic image fusion.

MeSH terms

Algorithms*
Benchmarking*
Electric Power Supplies
Image Processing, Computer-Assisted
Microscopy
Reproducibility of Results