Face Hallucination Using Cascaded Super-Resolution and Identity Priors

IEEE Trans Image Process. 2020;29(1):2150-2165. doi: 10.1109/TIP.2019.2945835. Epub 2019 Oct 11.

Abstract

In this paper we address the problem of hallucinating high-resolution facial images from low-resolution inputs at high magnification factors. We approach this task with convolutional neural networks (CNNs) and propose a novel (deep) face hallucination model that incorporates identity priors into the learning procedure. The model consists of two main parts: i) a cascaded super-resolution network that upscales the low-resolution facial images, and ii) an ensemble of face recognition models that act as identity priors for the super-resolution network during training. Different from most competing super-resolution techniques that rely on a single model for upscaling (even with large magnification factors), our network uses a cascade of multiple SR models that progressively upscale the low-resolution images using steps of 2× . This characteristic allows us to apply supervision signals (target appearances) at different resolutions and incorporate identity constraints at multiple-scales. The proposed C-SRIP model (Cascaded Super Resolution with Identity Priors) is able to upscale (tiny) low-resolution images captured in unconstrained conditions and produce visually convincing results for diverse low-resolution inputs. We rigorously evaluate the proposed model on the Labeled Faces in the Wild (LFW), Helen and CelebA datasets and report superior performance compared to the existing state-of-the-art.

MeSH terms

  • Algorithms
  • Databases, Factual
  • Deep Learning*
  • Face / anatomy & histology*
  • Face / diagnostic imaging*
  • Humans
  • Image Processing, Computer-Assisted / methods*
  • Neural Networks, Computer