Deep unsupervised endoscopic image enhancement based on multi-image fusion

Dongjin Huang; Jinhua Liu; Shuhua Zhou; Wen Tang

doi:10.1016/j.cmpb.2022.106800

Deep unsupervised endoscopic image enhancement based on multi-image fusion

Comput Methods Programs Biomed. 2022 Jun:221:106800. doi: 10.1016/j.cmpb.2022.106800. Epub 2022 Apr 26.

Authors

Dongjin Huang¹, Jinhua Liu², Shuhua Zhou², Wen Tang³

Affiliations

¹ Shanghai Film Academy, Shanghai University, Room 304, No.2 Teaching Building, 149 Yanchang Road, Shanghai 200072, China. Electronic address: djhuang@shu.edu.cn.
² Shanghai Film Academy, Shanghai University, Room 304, No.2 Teaching Building, 149 Yanchang Road, Shanghai 200072, China.
³ The Faculty of Science, Design and Technology, University of Bournemouth, Poole, Dorset, UK.

PMID: 35533420
DOI: 10.1016/j.cmpb.2022.106800

Abstract

Background and objective: A deep unsupervised endoscopic image enhancement method is proposed based on multi-image fusion to achieve high quality endoscope images from poorly illuminated, low contrast and color deviated images through an unsupervised mapping and deep learning network without the need for ground truth.

Methods: Firstly, three image enhancement methods are used to process original endoscopic images to obtain three derived images, which are then transformed into HSI color space. Secondly, a deep unsupervised multi-image fusion network (DerivedFuse) is proposed to extract and fuse features of the derived images accurately by utilizing a new no-reference quality metric as loss function. I-channel images of the three derived images are inputted into the DerivedFuse network to enhance the intensity component of the original image. Finally, a saturation adjustment function is proposed to adaptive adjusting the saturation component of HSI color space to enrich the color information of the original input image.

Results: Three evaluation metrics: Entropy, Contrast Improvement Index (CII) and Average Gradient (AG) are used to evaluate the performance of the proposed method. The results are compared with that of fourteen state-of-the-art algorithms. Experiments on endoscopic image enhancement show that the Entropy value of our method is 3.27% higher than the optimal entropy value of comparison algorithms. The CII of our proposed method is 6.19% higher than that of comparison algorithms. The AG of our method is 7.83% higher than the optimal AG of comparison algorithms.

Conclusions: The proposed deep unsupervised multi-image fusion method can obtain image information details, enhance endoscopic images with high contrast, rich and natural color information, visual and image quality. Sixteen doctors and medical students have given their assessments on the proposed method for assisting clinical diagnoses.

Keywords: Derived image; Endoscopic image enhancement; HSI color space; Image fusion; Unsupervised deep learning.

MeSH terms

Algorithms*
Color
Humans
Image Enhancement* / methods
Image Processing, Computer-Assisted / methods