Aiming at the problems of missing important features, inconspicuous details and unclear textures in the fusion of multimodal medical images, this paper proposes a method of computed tomography (CT) image and magnetic resonance imaging (MRI) image fusion using generative adversarial network (GAN) and convolutional neural network (CNN) under image enhancement. The generator aimed at high-frequency feature images and used double discriminators to target the fusion images after inverse transform; Then high-frequency feature images were fused by trained GAN model, and low-frequency feature images were fused by CNN pre-training model based on transfer learning. Experimental results showed that, compared with the current advanced fusion algorithm, the proposed method had more abundant texture details and clearer contour edge information in subjective representation. In the evaluation of objective indicators, Q AB/F, information entropy (IE), spatial frequency (SF), structural similarity (SSIM), mutual information (MI) and visual information fidelity for fusion (VIFF) were 2.0%, 6.3%, 7.0%, 5.5%, 9.0% and 3.3% higher than the best test results, respectively. The fused image can be effectively applied to medical diagnosis to further improve the diagnostic efficiency.
针对多模态医学图像融合中的重要特征丢失、细节表现不突出和纹理不清晰等问题，提出一种图像增强下使用生成对抗网络（GAN）和卷积神经网络（CNN）进行电子计算机断层扫描（CT）图像与磁共振成像（MRI）图像融合的方法。生成器针对高频特征图像，双鉴别器针对逆变换后的融合图像；高频特征图像通过GAN模型进行特征融合，低频特征图像通过基于迁移学习的CNN预训练模型进行特征融合。实验结果表明，与当前先进融合算法相比，所提方法在主观表现上纹理细节特征更加丰富，轮廓边缘信息更加清晰突出；在客观指标评估中，融合质量评价指标（Q AB/F）、信息熵（IE）、空间频率（SF）、结构相似性（SSIM）、互信息（MI）和融合视觉信息保真度（VIFF）等关键指标比其他最佳测试结果分别提高了2.0%、6.3%、7.0%、5.5%、9.0%和3.3%。融合后图像可以有效地应用于医学诊断，进一步提高诊断效率。.
Keywords: Deep learning; Generative adversarial network; Image enhancement; Image fusion; Medical image.