A feature refinement and adaptive generative adversarial network for thermal infrared image colorization

Abstract

Colorizing thermal infrared images poses a significant challenge as current methods struggle with issues such as unrealistic color saturation and limited texture. To address these challenges, we propose the Feature Refinement and Adaptive Generative Adversarial Network (FRAGAN). Our approach enhances the detailed, semantic, and contextual capabilities of image coloring by combining multi-level interactions that integrate the lost detailed information from the encoding stage with the semantic information from the decoding stage. Additionally, we introduce the Residual Feature Refinement Module (RFRM) to improve both the accuracy and generalization ability of the model, thereby elevating the quality of colorization results. The Feature Adaptation Module (FAM) is employed to mitigate sub-region information loss during downsampling. Furthermore, we introduce the Trinity Attention Module (TAM) to accurately capture the spatial and channel-wise interaction features of local semantic information. Extensive experimentation on the KAIST dataset and the FLIR dataset demonstrates the superiority of our proposed FRAGAN methodology, surpassing both the performance metrics and visual quality of current state-of-the-art methods. The colorized images generated by our proposed FRAGAN exhibit enhanced clarity and realism. Our code and models are available at GitHub.

Keywords: Attention mechanism; Feature adaptation; Feature refinement; Generative adversarial network; Thermal infrared image colorization.