Source-linked AI summary
Exemplar Guided Face Image Super-Resolution without Facial Landmarks
Berk Dogan, Shuhang Gu, Radu Timofte
TL;DR
Low-resolution face super-resolution seeks to reconstruct facial detail, and another high-resolution image of the same person can provide guidance. GWAInet aligns that guide and fuses its features without requiring facial landmarks during training; the authors report photo-realistic 8× results that outperform state of the art in PSNR and perceptual quality.
Problem
Face super-resolution aims to reconstruct details in low-resolution faces, and a same-person high-resolution image may guide that process.
Source: §1 · p. 1Method
GWAInet warps a same-person high-resolution guide to align it with the low-resolution input and uses the guide during 8× super-resolution without facial landmarks in training.
Source: §1 · p. 2, §3.2 · p. 4Results
GWAInet produces photo-realistic 8× face images and, according to the authors, outperforms state of the art in PSNR and perceptual quality.
Source: §5 · p. 8Takeaways & Limitations
The authors report that GWAInet processes the whole face region uniformly without landmark-based training and that identity loss helps preserve identity-related features.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Nowadays, due to the ubiquitous visual media there are vast amounts of already available high-resolution (HR) face images. Therefore, for super-resolving a given very low-resolution (LR) face image of a person it is very likely to find another HR face image of the same person which can be used to guide the process. In this paper, we propose a convolutional neural network (CNN)-based solution, namely GWAInet, which applies super-resolution (SR) by a factor 8x on face images guided by another unconstrained HR face image of the same person with possible differences in age, expression, pose or size. GWAInet is trained in an adversarial generative manner to produce the desired high quality perceptual image results. The utilization of the HR guiding image is realized via the use of a warper subnetwork that aligns its contents to the input image and the use of a feature fusion chain for the extracted features from the warped guiding image and the input image. In training, the identity loss further helps in preserving the identity related features by minimizing the distance between the embedding vectors of SR and HR ground truth images. Contrary to the current state-of-the-art in face super-resolution, our method does not require facial landmark points for its training, which helps its robustness and allows it to produce fine details also for the surrounding face region in a uniform manner. Our method GWAInet produces photo-realistic images in upscaling factor 8x and outperforms state-of-the-art in quantitative terms and perceptual quality.
1. Introduction
Face super-resolution reconstructs details in low-resolution images, and another high-resolution image of the same person may guide that reconstruction. GWAInet uses such guidance without facial landmarks during training, addressing limitations in prior landmark-guided results.
1. Introduction
- Face super-resolution reconstructs details and high frequencies in low-resolution face images, supporting demand for high-resolution faces in security, surveillance, and recognition applications.Source: §1 · p. 1
- The prior state-of-the-art landmark-guided approach [27] is reported to produce nonuniform, unpredictable facial detail, leaving images only partially sharp.CNN methods trained with reconstruction losses can also produce smooth images lacking fine perceptual details, especially at larger upscaling factors.Source: §1 · p. 1
- GWAInet uses a warper to align a same-person high-resolution guide with the low-resolution input, then uses the guide to improve facial details at 8× upscaling.The guide may differ in expression, pose, or size; the method does not require facial landmarks during training.Source: §1 · pp. 1–2
2. Related Work
Prior work on CNN-based super-resolution generally uses low-resolution images without a high-resolution guide, while face-hallucination research includes methods for handling misalignment and appearance variation. The paper distinguishes its approach through guided feature fusion, landmark-free warping, and adversarial training.
Convolutional Neural Networks (CNNs) and Image Super-Resolution (SR)
- CNN super-resolution methods generally use single or multiple low-resolution images rather than a high-resolution guide, while this work uses another same-person high-resolution face image.Source: §2 · p. 2
Spatial Transformer Networks
Face Hallucination
- Earlier face-hallucination methods address tiny-image misalignment or appearance variation, while the closest guided method [27] concatenates warped guidance and uses landmark losses.GWAInet instead extracts and fuses guiding-image features and does not require landmarks during training.Source: §2 · p. 2
Generative Adversarial Networks (GANs)
- GAN adversarial losses are used to create sharper, more realistic images; this work combines adversarial and L1 reconstruction losses and uses WGAN-GP [14].The passage describes reconstruction loss as matching image contents and adversarial loss as encouraging high-frequency features.Source: §2 · pp. 2–3
3. Proposed Method
GWAInet combines an LR face image with a same-person HR guide warped to align with the input, using WGAN-GP training and identity-aware losses. Its architecture fuses features from the LR and warped-guide pathways to generate an 8× super-resolved image.
3. Proposed Method
- GWAInet maps an LR face image and a same-person HR guide to an SR image, with the LR image derived from ground truth by bicubic downscaling at 8×.The warped guide is denoted IGW.Source: §3 · p. 3
3.1. WGAN-GP [14]
- WGAN-GP trains the model adversarially to make generated SR images as indistinguishable as possible from dataset HR images.Its critic objective adds a gradient penalty rather than using weight clipping.Source: §3.1 · pp. 3–4
3.2. Network Architecture
- Wnet estimates a flow field from the upscaled LR image and HR guide, then bilinear sampling uses it to warp the guide for alignment.The generator pairs an LR-only SRnet with a warped-guide GFEnet, concatenating their features after every fourth residual block.Source: §3.2 · pp. 4–5
3.3. Loss Functions
- The objective combines L1 content loss, adversarial loss, and identity loss to balance image-content matching, perceptual detail, and similarity to the HR identity embedding.The identity loss compares SR and ground-truth embeddings; the content loss supports matching but can produce smooth images lacking high-frequency detail.Source: §3.3 · p. 5
4. Experiments
The experiments evaluate GWAInet on CelebA and a separate dataset, comparing guiding-image variants and state-of-the-art methods. GWAInet improves perceptual details over the no-guide model and reports strong quantitative and qualitative results against prior methods.
4.1. Datasets
- On CelebA, identities are disjoint across the 162,734 training, 19,862 validation, and 19,959 test images; guides are sampled from other images of the same person.The dataset is aligned and cropped, then further cropped to 168 × 144; the separate [27] dataset contains 18,000 training images after validation selection.Source: §4.1 · p. 6
4.2. Experiment Settings
- Training proceeds in stages: the CelebA setup pretrains Wnet, then trains the network without and with adversarial loss, while the [27] dataset setup adds identity loss during its second stage.GWAInet's second stage on the [27] dataset uses λadv = 0.001 and λid = 0.05, with Inet fixed.Source: §4.2 · p. 6
4.3. Results on CelebA
- GWAnet adds missing facial details over BAnet, improving the location and shape of features such as eyes, nose, mouth, hair, and wrinkles.Both models can generate photo-realistic images; the guiding image distinguishes GWAnet by helping complete facial details.Source: Figure 3 · p. 6, §4.3 · p. 7
- Using a different-identity guide can change facial structure, feature locations, and wrinkles compared with using a same-identity guide.The paper reports that in some cases the complete facial structure changes.Source: §4.3 · p. 7, Figure 4 · p. 7
4.4. Comparison with state-of-the-art Methods
- Qualitatively, GWAInet produces sharper, identity-faithful images with details across the image, while GFRNet [27] often sharpens only a small face area and may hallucinate identity or produce blur and artifacts.Figure 5 presents the comparison at 8× upscaling with 32 × 32 LR inputs.Source: §4.4 · pp. 7–8, Figure 5 · p. 8
- GWAInet leads the state of the art by 1.47 dB on VggFace2 and ranks second on WebFace, 0.1 dB behind GFRNet [27].The reported comparisons are at 8× upscaling; the authors note that adversarial and identity losses conflict with maximizing PSNR.Source: §4.4 · p. 7, Table 1 · p. 8
5. Conclusion
GWAInet uses a same-person HR guide and identity loss for 8× face super-resolution without facial landmarks, producing uniform fine detail and photo-realistic results that outperform state of the art in PSNR and perceptual quality.
5. Conclusion
- At 8×, GWAInet produces photo-realistic face images and outperforms state of the art in PSNR and perceptual quality.Source: §5 · p. 8
- GWAInet uses information from an HR guiding image of the same person for face super-resolution.Source: §5 · p. 8
- GWAInet requires no facial landmarks during training and produces fine details uniformly across the face region.Source: §5 · p. 8
- Its identity loss minimizes the distance between embeddings of super-resolved and HR ground-truth images to help preserve identity-related features.Source: §5 · p. 8
A.1. Our Network Architectures
Table 2 describes the Warper, Critic, and Identity Encoder subnetworks used in GWAInet.
A.1. Our Network Architectures
- Table 2 describes the Warper (Wnet), Critic (Cnet), and Identity Encoder (Inet) subnetworks employed in GWAInet.Source: §A.1 · p. 11, Table 2 · p. 11
A.2. Residual Block
Figure 6 shows the residual block introduced in [29].
A.2. Residual Block
- Figure 6 shows the residual block as introduced in [29].Source: Figure 6 · p. 11, §A.2 · p. 11