Deep Cascaded Bi-Network for Face Hallucination

1 Introduction

• Face hallucination offers a way to improve low-resolution facial analysis because down-sampling and blur degrade existing techniques such as alignment and verification.The paper points to applications including surveillance-video identification and facial image enhancement.Source: §1 · p. 1
• Dense pixel-wise correspondence provides semantic face-structure guidance from very low-resolution inputs, extending spatial priors beyond landmarks or face parsing.The paper distinguishes this prior from general image super-resolution, which lacks such global face structure.Source: §1 · pp. 1–2
• The alternating cascade addresses the chicken-and-egg problem by progressively refining dense correspondence as face resolution increases and upscaling images using finer correspondence, without requiring pre-aligned inputs or spatial annotations.The paper describes correspondence estimation and hallucination as complementary tasks refined through successive cascades.Source: §1 · pp. 2–3
• The gated bi-network combines a common branch that recovers visible texture with a high-frequency branch that uses warped correspondence priors to synthesize hidden detail, then fuses their outputs pixel-wise.The gate can favor the common branch when high-frequency synthesis produces occluded facial parts.Source: §1 · pp. 2–3
• The paper reports extensive benchmark experiments against general super-resolution and face-hallucination methods, with high PSNR, superior perceptual quality, and exploration of the minimum input resolution for quality detail recovery.The introduction does not provide quantitative benchmark values.Source: §1 · p. 3

Face hallucination and spatial cues

• Yang et al. assume accurate landmarks, Wang et al. and Kolouri et al. use identical similarity alignment, and Zhou et al. omit spatial cues.The text notes that landmark estimates can fail at 5pxIOD and similarity alignment limits methods to canonical-view faces.Source: §2 · p. 4
• Unlike prior approaches, the framework alternates dense correspondence estimation and face hallucination so the mutual tasks refine spatial cues and reconstruction together.The discussion also notes that low-resolution correspondence may not suffice for high-resolution reconstruction, while estimating high-resolution correspondence from only a low-resolution face is ill-posed.Source: §2 · p. 4

Cascaded prediction

• Prior cascaded methods improve image SR and facial landmark detection, but no existing work integrates both tasks in one framework.The section attributes landmark-detection gains to coarse-to-fine residual prediction and increasing facial resolution across cascades.Source: §2 · p. 4

The bi-network architecture

• Unlike bilinear networks and the two-stream model, this gated bi-network jointly models the low-res face and prior in its high-frequency branch while its common branch models the low-res face.The paper also distinguishes its design from Zhou et al., whose scalar-weighted branch output uses neither spatial cues nor a prior.Source: §2 · p. 4

3.1 Overview

• The dense field is controlled by per-sample deformation coefficients over shared, predefined bases, with four bases for similarity transforms and the remainder for non-rigid deformations.At zero coefficients, the field equals the mean face template.Source: §3.1 · p. 5
• Each of K cascades updates deformation coefficients and the dense field, then performs spatially guided 2× upscaling; the final output is the last hallucinated image.The framework starts from the low-resolution input and zero coefficients for the mean face template.Source: §3.1 · pp. 5–6

3.2 gk - Gated deep bi-network: Face hallucination with spatial cues

• The gated bi-network predicts each cascade’s image residual by blending a common-branch output and a high-frequency-branch output with a pixel-wise soft gate.The common branch uses the interpolated input; the high-frequency branch also uses a warped high-frequency prior, and the gate network predicts the blending map.Source: §3.2 · pp. 6–7
• The high-frequency prior is built by averaging warped training-image residual magnitudes in the mean-face domain and clustering them into 10 continuous contours.The resulting 10-channel map provides spatial guidance for hallucination.Source: §3.2 · p. 7

3.3 fk - Dense field deformation coefficients prediction

• Sparse facial-landmark predictions transfer shared deformation coefficients to the dense field, which is estimated by cascaded regression from landmark-indexed local appearance features.The discriminative coefficient-transfer approach takes 8ms per cascade, versus 1.4s for face flow.Source: §3.3 · p. 8

3.4 Implementation Details

• The first cascade uses 24-layer common and high-frequency sub-nets, while subsequent cascades use 12-layer sub-nets; the architecture has no pooling layers.The depths are chosen as an accuracy-speed trade-off.Source: §3.4 · p. 9, Table 1 · p. 9, Table 2 · p. 9
• Implementation uses concatenated local SIFT descriptors and eye-location-based reference framing before first-cascade field prediction, plus backprojection regularization during hallucination to suppress cascade error accumulation.The regularization effect is specifically significant in subsequent cascades.Source: §3.4 · pp. 9–10

4 Experiments

• Evaluation uses MultiPIE, BioID, PubFig, and Helen, measuring PSNR on facial-region luminance and omitting SSIM for in-the-wild faces.Training alternates between correspondence-field data from 300W and BU4D and hallucination data from CelebA.Source: §4 · p. 10

4.1 Comparison with state-of-the-art methods

• CBN outperforms all general-SR and face-hallucination methods in both conventional-SR scenarios, with especially significant gains from very low-resolution inputs.The authors attribute this to recovering details using both a high-frequency prior and the estimated dense correspondence field; prior hallucination methods struggle with in-the-wild shape and appearance variation.Source: §4.1 · p. 11, Table 3 · p. 12
• CBN significantly outperforms all compared approaches under unknown Gaussian blur, with robustness attributed to spatial guidance from the face high-frequency prior.Evaluation uses approximately 8pxIOD inputs upscaled 4×, with blur σ = 1.6 for PubFig83 and σ = 2.4 for BioID, without blur-specific training.Source: Table 4 · p. 12, §4.1 · pp. 12–14

4.2 An ablation study

• Ablations show that all tested components matter: the gated bi-network, progressively updated dense correspondence field, and cascaded alternating framework.Tests remove either bi-network branch, use correspondence estimated only from the low-resolution input, or replace the cascade with a single network.Source: Table 5 · p. 12, §4.2 · p. 14

4.1 Comparison with state-of-the-art methods

• 3.84s total on a single-core i7-4790 CPU, versus 15–20 min for, 1 min for, and 8 min for.The four hallucination cascades take 0.13s, 0.17s, 0.70s, and 2.76s, respectively; dense-field prediction time is negligible by comparison.Source: §4.1 · p. 14

4.3 Discussion

• Observed failures include over-synthesized occluded parts, ghosting from inaccurate low-resolution spatial prediction, and incorrect details such as gaze direction.Light-colored sunglasses can mislead the gate toward the high-frequency branch; large head poses make localization difficult, and inputs may provide no reliable gaze information.Source: §4.3 · p. 15

5 Input resolution lower bound

• The experiment compares hallucination results from four input face sizes: 3, 5, 8, and 10pxIOD.Source: §5 · p. 15
• 3pxIOD outputs are mostly unrealistic and visually dissimilar to the full-resolution image, suggesting this size is below the recovery lower bound.The dense correspondence field is likely predicted incorrectly, and the low-resolution input provides very little information.Source: §5 · p. 15
• PSNR increases significantly when the input face size is at least 5pxIOD, motivating the use of 5/8pxIOD inputs in Sec. 4.The passage refers to Tab. 5 for the PSNR increase and to Tab. 6 and Fig. 9 for results from 3pxIOD.Source: §5 · p. 15

• The framework alternates face hallucination with adaptive refinement of the dense correspondence field.Source: §6 · p. 16
• The authors report state-of-the-art performance and visually appealing qualitative results.Source: §6 · p. 16
• A high-frequency prior lets the framework leverage spatial cues during hallucination.Source: §6 · p. 16