Non-linear perceptual multi-scale network for single image super-resolution

Abstract

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and achieved remarkable progress. However, most of the existing CNN-based SISR networks with a single-stream structure fail to make full use of the multi-scale features of low-resolution (LR) image. While those multi-scale SR models often integrate the information with different receptive fields by means of linear fusion, which leads to the redundant feature extraction and hinders the reconstruction performance of the network. To address both issues, in this paper, we propose a non-linear perceptual multi-scale network (NLPMSNet) to fuse the multi-scale image information in a non-linear manner. Specifically, a novel non-linear perceptual multi-scale module (NLPMSM) is developed to learn more discriminative multi-scale feature correlation by using high-order channel attention mechanism, so as to adaptively extract image features at different scales. Besides, we present a multi-cascade residual nested group (MC-RNG) structure, which uses a global multi-cascade mechanism to organize multiple local residual nested groups (LRNG) to capture sufficient non-local hierarchical context information for reconstructing high-frequency details. LRNG uses a local residual nesting mechanism to stack NLPMSMs, which aims to form a more effective residual learning mechanism and obtain more representative local features. Experimental results show that, compared with the state-of-the-art SISR methods, the proposed NLPMSNet performs well in both quantitative metrics and visual quality with a small number of parameters.

Keywords: Global multi-cascade; Image super-resolution; Local residual nesting; Multi-scale.