PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-Resolution Face Recognition

Zhiyi Cheng, Xiatian Zhu, Shaogang Gong

arXiv:1811.08965v2cs.CV

TL;DR

Native unconstrained low-resolution face recognition remains under-studied and lacks a suitable large-scale benchmark. The paper introduces the end-to-end CSRI method and TinyFace dataset, finding a substantial gap between popular high-resolution benchmark results and native low-resolution performance, while CSRI outperforms competing methods.

  • Problem

    Native unconstrained low-resolution face recognition is under-studied and lacks large-scale native low-resolution data for training and testing.

  • Method

    The paper introduces end-to-end CSRI joint learning and the TinyFace benchmark of native low-resolution web face images.

  • Results

    CSRI significantly outperforms competing face-recognition and super-resolution methods, with a 12.7% Rank-1 gain over CentreFace on TinyFace.

  • Takeaways & Limitations

    TinyFace exposes a substantial gap between performance on popular high-resolution benchmarks and native unconstrained low-resolution face recognition.

  • Takeaways & Limitations

    TinyFace is smaller than some existing high-resolution training datasets because labelled native low-resolution images of general-public identities are difficult to collect.

Abstract

from arXiv · show

Whilst recent face-recognition (FR) techniques have made significant progress on recognising constrained high-resolution web images, the same cannot be said on natively unconstrained low-resolution images at large scales. In this work, we examine systematically this under-studied FR problem, and introduce a novel Complement Super-Resolution and Identity (CSRI) joint deep learning method with a unified end-to-end network architecture. We further construct a new large-scale dataset TinyFace of native unconstrained low-resolution face images from selected public datasets, because none benchmark of this nature exists in the literature. With extensive experiments we show there is a significant gap between the reported FR performances on popular benchmarks and the results on TinyFace, and the advantages of the proposed CSRI over a variety of state-of-the-art FR and super-resolution deep models on solving this largely ignored FR scenario. The TinyFace dataset is released publicly at: https://qmul-tinyface.github.io/.

1 Introduction

The paper targets the under-studied problem of recognizing native unconstrained low-resolution faces, introducing CSRI and TinyFace to enable systematic large-scale evaluation.

  • 1 Introduction: The study addresses a major gap because low-resolution face recognition is under-studied and lacks large-scale native low-resolution training and test data.Existing benchmarks predominantly use constrained or posed high-resolution images, while synthetic down-sampling does not provide native low-resolution data.
  • 1 Introduction: The authors propose CSRI, a unified joint Super-Resolution and Identity learning approach designed to improve face-recognition generalization on native low-resolution images.The method jointly optimizes face enhancement and recognition rather than applying super-resolution independently.
  • 1 Introduction: CSRI transfers super-resolution knowledge from good-quality high-resolution web images while enforcing identity-label constraints on native low-resolution faces.This complement learning addresses the absence of paired high-resolution and native low-resolution faces for super-resolution training.
  • 1 Introduction: TinyFace contains 5,139 identities and 169,403 native low-resolution face images averaging 20×16 pixels for large-scale 1:N recognition testing.Images come from public web data captured under uncontrolled pose, illumination, occlusion, and background conditions.
  • 1 Introduction: Current deep-learning FR models suffer significant performance degradation on native low-resolution faces in the TinyFace benchmark.The benchmark evaluates four state-of-the-art FR models and three super-resolution methods.

2 Related Work

Related work has emphasized high-resolution face recognition, synthetic or resolution-invariant low-resolution methods, and independently developed super-resolution models, leaving native low-resolution recognition insufficiently addressed.

  • 2 Related Work: High-resolution face-recognition progress has been driven by large datasets and deep models, but native low-resolution faces have received little attention.State-of-the-art models trained on constrained high-resolution faces generalize poorly to native low-resolution images but better to synthetic low-resolution data.
  • 2 Related Work: Existing low-resolution face-recognition methods mainly use image super-resolution or resolution-invariant learning.These approaches respectively target visual fidelity and identity discrimination or learn features and transformations that reduce resolution dependence.
  • 2 Related Work: Low-resolution recognition deployment includes LR-to-HR matching and the more general LR-to-LR setting adopted by TinyFace.Both settings must synthesize discriminative facial features missing from low-resolution inputs, while LR-to-LR reflects cases without enrolled high-resolution gallery images.
  • 2 Related Work: Super-resolution and face-recognition research have largely advanced independently, and contemporary super-resolution models provide only marginal recognition benefit on native unconstrained low-resolution images.This motivates the paper’s integrated CSRI design.

3 Complement-Super-Resolution and Identity Joint Learning

CSRI jointly learns super-resolution and face recognition through complementary synthetic- and native-LR branches, adapting information from paired auxiliary data to unconstrained native LR imagery.

  • CSRI Overview: The method addresses missing native LR/HR pairs by transferring knowledge from auxiliary HR data, where paired LR/HR examples can be constructed by down-sampling.Native LR face images lack the coupled HR images required to optimize super-resolution directly.
  • CSRI Overview: CSRI uses two parameter-sharing branches: a synthetic LR SR-FR branch trained with down-sampled LR/HR pairs and a native LR SR-FR branch adapted to native LR faces.The synthetic branch improves SR-FR compatibility, while the native branch adapts super-resolving information to the native LR domain without corresponding HR faces.
  • CSRI Overview: The instantiated CSRI combines VDSR for super-resolution with CentreFace for face recognition in an end-to-end architecture.The model is trained using standard stochastic gradient descent, with deployment using the native LR branch for feature extraction and Euclidean-distance matching.
  • Joint Learning of Super-Resolution and Face Recognition: CSRI integrates super-resolution output into face recognition and jointly optimizes both components so the SR representation supports identity discrimination rather than only pixel fidelity.The SR component uses paired LR/HR supervision, while the FR criterion is imposed concurrently during SR optimization.
  • Complement-Super-Resolution Learning: Complement-SR learning jointly uses auxiliary paired data and native LR data to adapt the SR component toward native LR imagery despite synthetic–native appearance differences.The model concurrently optimizes synthetic and native LR branches, imposing native LR face discrimination constraints on SR learning.

4 TinyFace: Low-Resolution Face Recognition Benchmark

TinyFace is constructed as a native low-resolution face benchmark using an explicit ≤32×32-pixel criterion, verified detections, defined partitions, and a 1:N identification protocol. It contains highly challenging, realistic low-resolution imagery and offers broader test coverage than existing benchmarks, although its training set is smaller than some high-resolution datasets.

  • 4.1 Dataset Construction: TinyFace defines low-resolution faces as ≤32×32 pixels, whereas existing face-recognition datasets contain images above 100×100 pixels.This criterion follows the tiny-object definition and establishes the benchmark’s native low-resolution scope.
  • 4.1 Dataset Construction: The benchmark’s face detections were exhaustively manually verified, requiring approximately 280 person-hours of checking.This filtering step was used to remove false face detections and establish a valid benchmark.
  • 4.1 Dataset Construction: TinyFace contains native low-resolution web faces averaging 20 pixels in height, with resolutions ranging from 6 to 32 pixels and realistic imaging variation.Images include uncontrolled variation in illumination, pose, expression, and background clutter.
  • 4.1 Dataset Construction: TinyFace uses 2,570 identities for training, 2,569 for testing, and all unlabelled distractors in the test data.The identities are divided into two halves for training and testing, while unlabelled distractors expand the evaluation gallery.
  • 4.3 Training vs Testing Data Size Comparison: TinyFace has a smaller training set than some high-resolution datasets, including MegaFace2 with 672,057 identities.The paper attributes this difference to the difficulty of collecting natively low-resolution face images with identity labels.
  • 4.3 Training vs Testing Data Size Comparison: TinyFace provides 3–4× more test identities than MegaFace and LFW, plus 15× more distractors than LFW’s 1:N test data.The benchmark contains 2,569 test identities and 154,471 gallery images, compared with smaller identification test sets in existing benchmarks.

5 Experiments

Experiments establish that native low-resolution face recognition is substantially harder than recognition on synthetic low-resolution data and that CSRI outperforms existing recognition and super-resolution approaches on TinyFace.

  • 5.1 Evaluation of Generic Face Resolution Methods: 32.1% Rank-1: CentreFace on TinyFace falls below its 65.2% MegaFace performance, showing a more than half performance drop on native low-resolution images.SphereFace failed to converge during TinyFace fine-tuning and was evaluated using its CelebA-trained model.
  • 5.1 Evaluation of Generic Face Resolution Methods: 18.9%: CentreFace has a higher Rank-20 performance on synthetic SynLR-MF2 than native TinyFace images, 63.4% versus 44.5%.VggFace shows the same pattern with a 6.4% advantage, 46.8% versus 40.4%.
  • 5.2 Evaluation of Low-Resolution Face Resolution Methods: All separately applied SR methods degrade deep-learning FR performance, while RPCN performs worst among the evaluated low-resolution methods.The authors associate this result with SR artifacts and noise, insufficient FR supervision during RPCN fine-tuning, and RPCN's weaker base network.
  • 5.3 Component Analysis of CSRI: 36.1% Rank-1: joint SR-FR learning exceeds independent SR-FR learning by 10.1% and FR-only CentreFace by 4.0%.Adding complement SR learning raises Rank-1 by a further 8.7%, from 36.1% to 44.8%.

6 Conclusions

The paper establishes large-scale native low-resolution face recognition as a distinct challenge, introduces CSRI and TinyFace to study it, and reports that native images are harder than synthetic ones while CSRI performs best among evaluated methods.

  • 6 Conclusions: The study presents CSRI, an end-to-end joint learning method combining Complement Super-Resolution and face Identity, together with the native low-resolution TinyFace benchmark.TinyFace is designed for training and testing under unconstrained native low-resolution conditions.
  • 6 Conclusions: 32.1% Rank-1 versus 65.2% on MegaFace: CentreFace shows more than half performance degradation on TinyFace.The comparison illustrates the greater difficulty of native low-resolution recognition relative to established high-resolution evaluation.
  • 6 Conclusions: TinyFace provides a large-scale benchmark of natively low-resolution, unconstrained faces, distinguishing this setting from artificially down-sampled low-resolution recognition.The benchmark addresses the paper's stated need for realistic native low-resolution evaluation.
Loading 1811.08965v2…