PaperlayerView exact revision on arXiv

Source-linked AI summary

Low Resolution Face Recognition Using a Two-Branch Deep Convolutional Neural Network Architecture

Erfan Zangeneh, Mohammad Rahmati, Yalda Mohsenzadeh

arXiv:1706.06247v1cs.CV

TL;DR

Low-resolution face recognition must match degraded probes with high-resolution gallery images, where traditional recognition performance declines. The paper uses two DCNN branches to map both resolutions into a shared nonlinear space while embedding super-resolution, and reports stronger recognition, especially for very low-resolution probes, alongside high-resolution reconstruction.

  • Problem

    Low-resolution probe faces are difficult to recognize against high-resolution galleries because traditional systems degrade when probe detail is limited.

  • Method

    Two DCNN branches nonlinearly map low- and high-resolution face images into a common space, with a super-resolution network connected to the low-resolution branch.

  • Results

    11.4% improvement in recognition accuracy is reported for very low-resolution 6 × 6 probe images, with performance improvements over competing methods across resolutions.

  • Takeaways & Limitations

    The method combines recognition-oriented coupled mappings with high-resolution reconstruction from low-resolution probes.

Abstract

from arXiv · show

We propose a novel couple mappings method for low resolution face recognition using deep convolutional neural networks (DCNNs). The proposed architecture consists of two branches of DCNNs to map the high and low resolution face images into a common space with nonlinear transformations. The branch corresponding to transformation of high resolution images consists of 14 layers and the other branch which maps the low resolution face images to the common space includes a 5-layer super-resolution network connected to a 14-layer network. The distance between the features of corresponding high and low resolution images are backpropagated to train the networks. Our proposed method is evaluated on FERET data set and compared with state-of-the-art competing methods. Our extensive experimental results show that the proposed method significantly improves the recognition performance especially for very low resolution probe face images (11.4% improvement in recognition accuracy). Furthermore, it can reconstruct a high resolution image from its corresponding low resolution probe image which is comparable with state-of-the-art super-resolution methods in terms of visual quality.

I. INTRODUCTION

Low-resolution probe faces are difficult to recognize against high-resolution galleries because conventional systems degrade when image detail is limited. The paper proposes nonlinear coupled mappings with DCNNs to improve recognition, especially for extremely low-resolution probes, while also reconstructing high-resolution images.

  • I. INTRODUCTION: The approach targets recognition when high-quality gallery images are available but high-resolution probe images are missing.Traditional high-quality face-recognition systems degrade considerably on low-resolution face regions.
  • I. INTRODUCTION: The method uses two DCNN branches to transform low- and high-resolution face images into a shared nonlinear feature space.Training uses distances between transformed low- and high-resolution images as the objective.
  • I. INTRODUCTION: The architecture also reconstructs a high-resolution version of the low-resolution input through an embedded super-resolution CNN.This reconstruction is presented as supporting the recognition-oriented coupled-mapping design.
  • I. INTRODUCTION: The method is designed to require less space than typical deep convolutional face-recognition methods.The supplied passage introduces this contribution but does not provide a numerical space comparison.

II. PREVIOUS WORKS

Previous low-resolution face-recognition work includes super-resolution methods and coupled mappings, with different priorities for visual reconstruction, recognition, and shared feature representation. The paper positions DCNN-based nonlinear mappings as a way to combine recognition-oriented alignment with high-resolution reconstruction.

  • II. PREVIOUS WORKS: Super-resolution methods reconstruct high-resolution probes for recognition, but their visual-quality objective may not optimize recognition performance.The paper contrasts this goal with recognition-focused coupled mappings.
  • II. PREVIOUS WORKS: Coupled mappings learn transformations that bring low- and high-resolution features of corresponding subjects closer in a common space.The reviewed formulations use paired high- and low-resolution training data and commonly employ linear mappings.
  • II. PREVIOUS WORKS: Earlier formulations preserve local relationships using a penalty matrix based on neighborhoods in high-resolution feature space.The neighborhood set contains k nearest neighbors, and the weighting uses a Gaussian-function width parameter.
  • II. PREVIOUS WORKS: Coupled mappings generally achieve better recognition than super-resolution methods, while not aiming to reconstruct high-resolution images.This creates a distinction between recognition performance and visualization-oriented reconstruction.
  • II. PREVIOUS WORKS: The reviewed literature motivates nonlinear DCNN mappings by highlighting their use in computer vision and efficient training infrastructure.The discussion mentions large datasets, modern activation functions, GPUs, and deep-learning frameworks.

III. PROPOSED METHOD

The proposed method uses nonlinear DCNN mappings to project low- and high-resolution face images into a shared feature space, training the branches from paired images under varied conditions.

  • III. PROPOSED METHOD: Nonlinear coupled mappings use two DCNNs to project low- and high-resolution face images into a common space.The method addresses the difficulty of nonlinear optimization that motivated linear transformations in earlier coupled-mapping approaches.
  • III. PROPOSED METHOD: The distance between mapped low- and high-resolution image pairs is minimized by backpropagating error through the network weights.
  • III. PROPOSED METHOD: Training uses paired low- and high-resolution images of the same subjects, which may differ in illumination, pose, and expression.

A. Networks Architecture

The architecture has separate high- and low-resolution branches: a feature-extraction network for high-resolution inputs and a super-resolution network followed by feature extraction for low-resolution inputs.

  • A. Networks Architecture: The high-resolution branch maps 224 × 224 inputs to a 4096-element feature vector after removing VGGnet’s final two fully connected layers.
  • A. Networks Architecture: Increasing the SRnet depth from three to five layers improves recognition performance, although prior work found no visual-quality difference between those depths.
  • A. Networks Architecture: The high-resolution branch uses a 14-layer FECNN, while the low-resolution branch combines a five-layer SRnet with a second FECNN.The SRnet output is fed into the second subnet, giving the bottom branch nineteen layers overall.
  • A. Networks Architecture: Despite having eighteen convolutional layers and one fully connected layer, SRFECNN uses fewer weights than the 141M-weight VGGnet.

B. Common Subspace Learning

Training proceeds in stages, culminating in joint common-space learning where paired inputs train the low-resolution branch to approach fixed high-resolution features.

  • B. Common Subspace Learning: The first stage initializes both branches with VGGnet-derived FECNN weights after removing its task-specific final fully connected layers.
  • B. Common Subspace Learning: The second stage trains the low-resolution branch’s SRnet using paired low- and high-resolution face images.
  • B. Common Subspace Learning: The third training stage merges SRnet and FECNN and feeds same-subject low- and high-resolution pairs into the two branches.The branches are treated as nonlinear functions mapping both inputs into a 4096-dimensional common space.
  • B. Common Subspace Learning: During common-space learning, the high-resolution mapping remains fixed while the low-resolution mapping is trained to minimize pairwise feature distance.The distance error is backpropagated through both the bottom-branch FECNN and SRnet.
  • B. Common Subspace Learning: Learning rates are reduced for fine-tuning, with smaller rates in early FECNN layers than in later layers.The paper attributes this choice to more general features in early layers and more discriminative information in later layers.

C. Reconstruct Input Image

The low-resolution branch can also reconstruct a high-resolution face image from the probe image during testing.

  • C. Reconstruct Input Image: The SRnet produces a high-resolution face image from the low-resolution probe before passing it to FECNN.The reconstructed image can be extracted from the SRnet’s last layer during testing.

D. Test Phase

The test phase maps high-resolution gallery images and a low-resolution probe into a common space, then identifies the probe using gallery comparisons. Experiments assess identification performance against state-of-the-art methods across resolution and appearance variations.

  • Testing maps every high-resolution gallery image through the top branch and the low-resolution probe through the bottom branch before comparison.
  • The experiments evaluate face identification against one super-resolution method and three coupled-mapping approaches.
  • The evaluation examines robustness across probe resolutions and variations in expression, illumination, and age.
  • Additional tests assess the effect of excluding the super-resolution subnet, high-resolution reconstruction, and training convergence.

A. Data Description

The study trains and evaluates the method on FERET face images with variation in pose, expression, illumination, and age. Evaluation uses four probe categories paired with frontal gallery images.

  • Training uses 45,315 face images with variations in pose, expression, illumination, and age, including 10,585 FERET images.
  • FERET contains 14,126 images from 1,199 individuals, with 3,541 images assigned to evaluation.
  • The evaluation set has four probe categories, each paired with a gallery set whose images are frontal.
  • FB contains 1,195 frontal probes with expression differences from its 1,196-image gallery, while duplicateI contains 722 duplicate frontal probes using the same gallery.
  • The fc category contains 194 probes differing in camera and illumination, while duplicateII contains probes acquired at least one year apart from gallery images.

B. Training Phase

Training uses paired FERET data and a two-branch evaluation setup, with cumulative match curves comparing the proposed method against four competing approaches. The method performs best on FB, fc, and duplicateII and is close to the best on duplicateI.

  • At rank 1, recognition reaches 91.8% on FB versus 90.1% for CLPM, a 1.7% advantage.
  • On fc, the proposed method exceeds the best competing method by 5.6% at rank 1 and remains superior across all ranks.The fc probes differ from gallery images in illumination.
  • On duplicateII, the method surpasses the best competing method, MDS, by 5.2% at rank 1.DuplicateII probes differ from gallery images in age condition.
  • The proposed method achieves the best performance on FB, fc, and duplicateII probe images and is close to the best on duplicateI.The comparison uses cumulative match curves against CLPM, MDS, DSR, and NMCF.

D. Evaluation on Different Probe Resolutions

Resolution experiments retrain the super-resolution component for each probe size and compare configurations with different super-resolution modules. The method outperforms competing methods across tested resolutions, with the largest gain at 6 × 6 pixels.

  • The compared configurations vary whether SR and FECNN are jointly trained, separated, replaced by sparse coding, or replaced by bicubic interpolation.
  • 11.4% is the largest reported improvement, achieved at 6 × 6 probe resolution against DSR.The proposed method outperforms all competing methods at 6 × 6, 12 × 12, 24 × 24, and 36 × 36 resolutions.
  • For each probe resolution, SRnet is retrained on reduced-resolution training data, connected to FECNN, and the bottom branch is retrained.
  • Using and training SRnet improves recognition performance, especially for very low-resolution probes.

F. Evaluation on Reconstructed HR Face

The method reconstructs high-resolution faces from low-resolution probes while prioritizing recognition performance over peak visual quality. Its reconstructed images trail the best visual-quality configurations only slightly, while recognition accuracy is substantially better.

  • The method reconstructs high-resolution faces from corresponding low-resolution face images while supporting recognition.
  • Recognition accuracy is much better than competing reconstruction configurations, despite the method ranking third in visual-quality metrics.
  • The visual-quality gap between the proposed method and the two leading reconstruction methods is small.
Loading 1706.06247v1…