A Two-To-One Deep Learning General Framework for Image Fusion

Pan Zhu; Wanqi Ouyang; Yongxing Guo; Xinglin Zhou

doi:10.3389/fbioe.2022.923364

A Two-To-One Deep Learning General Framework for Image Fusion

Front Bioeng Biotechnol. 2022 Jul 14:10:923364. doi: 10.3389/fbioe.2022.923364. eCollection 2022.

Authors

Pan Zhu^{1

2

3}, Wanqi Ouyang^{1

2

3}, Yongxing Guo^{1

2

3}, Xinglin Zhou^{1

2

3}

Affiliations

¹ Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan, China.
² Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China.
³ Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan, China.

Abstract

The image fusion algorithm has great application value in the domain of computer vision, which makes the fused image have a more comprehensive and clearer description of the scene, and is beneficial to human eye recognition and automatic mechanical detection. In recent years, image fusion algorithms have achieved great success in different domains. However, it still has huge challenges in terms of the generalization of multi-modal image fusion. In reaction to this problem, this paper proposes a general image fusion framework based on an improved convolutional neural network. Firstly, the feature information of the input image is captured by the multiple feature extraction layers, and then multiple feature maps are stacked along the number of channels to acquire the feature fusion map. Finally, feature maps, which are derived from multiple feature extraction layers, are stacked in high dimensions by skip connection and convolution filtering for reconstruction to produce the final result. In this paper, multi-modal images are gained from multiple datasets to produce a large sample space to adequately train the network. Compared with the existing convolutional neural networks and traditional fusion algorithms, the proposed model not only has generality and stability but also has some strengths in subjective visualization and objective evaluation, while the average running time is at least 94% faster than the reference algorithm based on neural network.

Keywords: adaptive feature analysis; bionic vision; convolutional neural network; multi-convolution kernel; multi-modal image fusion; y-distribution structure.