PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation

Shiming Ge, Shengwei Zhao, Chenyu Li, Jia Li

arXiv:1811.09998v2cs.CV

TL;DR

The paper addresses how to compress complex face recognition models for accurate low-resolution recognition under severe computational and memory constraints. It uses selective knowledge distillation to transfer informative facial features to a simpler student model, which achieves high recognition accuracy with substantially lower resource requirements.

  • Problem

    Face recognition deployment on mobile and embedded devices requires efficient models that recognize low-resolution faces without remarkable accuracy loss.

  • Method

    Selective knowledge distillation uses graph-based feature selection and a multi-task regression-classification objective to supervise a low-resolution student network with a complex teacher model.

  • Results

    The student model achieves 95.03% accuracy on LFW, with 0.79M parameters versus 138M for VGGFace, while the compressed system uses 0.15MB memory and runs at 418 faces per second on CPU and 9,433 on GPU.

  • Takeaways & Limitations

    The framework jointly addresses model compression and low-resolution recognition, enabling complex face models to be converted into simpler models for deployment in the wild.

  • Takeaways & Limitations

    The authors identify teacher-stream failure cases as a target for future recurrent-mechanism improvements.

Abstract

from arXiv · show

Typically, the deployment of face recognition models in the wild needs to identify low-resolution faces with extremely low computational cost. To address this problem, a feasible solution is compressing a complex face model to achieve higher speed and lower memory at the cost of minimal performance drop. Inspired by that, this paper proposes a learning approach to recognize low-resolution faces via selective knowledge distillation. In this approach, a two-stream convolutional neural network (CNN) is first initialized to recognize high-resolution faces and resolution-degraded faces with a teacher stream and a student stream, respectively. The teacher stream is represented by a complex CNN for high-accuracy recognition, and the student stream is represented by a much simpler CNN for low-complexity recognition. To avoid significant performance drop at the student stream, we then selectively distil the most informative facial features from the teacher stream by solving a sparse graph optimization problem, which are then used to regularize the fine-tuning process of the student stream. In this way, the student stream is actually trained by simultaneously handling two tasks with limited computational resources: approximating the most informative facial cues via feature regression, and recovering the missing facial cues via low-resolution face classification. Experimental results show that the student stream performs impressively in recognizing low-resolution faces and costs only 0.15MB memory and runs at 418 faces per second on CPU and 9,433 faces per second on GPU.

I. INTRODUCTION

The paper addresses deploying face recognition on resource-constrained devices that must recognize low-resolution faces without substantial accuracy loss. It proposes selective knowledge distillation to compress a complex model into a faster, smaller low-resolution face recognizer.

  • Mobile phones and smart cameras require low-resolution face recognition with extremely low computational cost and memory.
  • Low-resolution faces lose visual details, but familiarity with corresponding high-resolution faces can support discriminative feature extraction.
  • Hallucination methods reconstruct high-resolution faces before recognition, but super-resolution adds computational cost and reduces recognition speed.
  • The proposed framework initializes teacher and student streams, selectively distils informative facial features through sparse graph optimization, and uses them to regularize student fine-tuning.
  • The paper contributes model compression, graph-based extraction of discriminative facial features, and experiments targeting high recognition speed with comparable accuracy.

II. RELATED WORKS

Related work covers general deep face recognition, low-resolution recognition, and knowledge transfer. The paper identifies selective transfer of useful high-resolution knowledge as central to low-resolution recognition.

  • A. General Face Recognition Models: Deep face recognition improved performance but commonly uses large models, high-dimensional features, and complex inference functions that hinder deployment in resource-limited settings.
  • B. Low-Resolution Face Recognition: Low-resolution recognition methods mainly either hallucinate high-resolution faces before recognition or embed low-resolution faces into external feature contexts.
  • B. Low-Resolution Face Recognition: Embedding-based approaches transfer knowledge between high- and low-resolution faces, making the choice and use of transferred knowledge key factors in performance.

C. Knowledge Distillation

Knowledge distillation trades model performance against speed by transferring knowledge from complex networks to compact deployable models. Here, the student is co-supervised through selective feature regression and low-resolution classification.

  • Knowledge distillation converts complex face models into simpler models that are easier to deploy in the wild under limited computational resources.
  • The framework uses a complex teacher stream and a compact student stream, with the student learning to imitate the teacher for practical deployment.
  • Its three stages initialize both streams, select informative teacher knowledge while removing unsuitable knowledge, and fine-tune the student jointly with regression and identity classification.

A. Definition

The approach defines a complex high-resolution teacher stream and a much simpler low-resolution student stream. Both are trained using labeled face data, including resolution-degraded versions linked to the same identities.

  • The teacher stream is a complex CNN or ensemble pretrained to recognize high-resolution faces using parameters that encode knowledge from large labeled face sets.
  • The student stream is a much simpler CNN trained to recognize low-resolution faces with its own parameter set.
  • For each high-resolution face, the student dataset contains N resolution-degraded versions sharing the same identity label.

B. Initialization of the Two-stream CNNs

The framework initializes a complex teacher CNN for high-resolution faces and a compact student CNN for low-resolution recognition, then selects informative teacher knowledge through sparse graph optimization.

  • Initialization: The teacher stream uses a pretrained complex face-recognition network, while the student stream recognizes low-resolution faces with a lightweight CNN.The teacher can be initialized from models such as VGGFace, FaceNet, or VGGFace2; the student accepts inputs such as 32 × 32 faces.
  • Initialization: The student architecture has ten convolutional layers, three max pooling layers, and three fully-connected layers.Its parameter count is 0.79M, reported as 0.57% of the 138M-parameter teacher.
  • Initialization: A mimic layer receives teacher knowledge, while an identity layer supports feature compression before the softmax output.The mimic layer has dimensionality D matching the teacher’s learned high-resolution representation.
  • Selection: Selective distillation represents faces and class centroids as feature vectors, using intra-class and inter-class graph edges to identify informative examples.Face and centroid features are extracted from high-resolution images with the teacher model.
  • Selection: The binary graph-labeling objective favors selected faces with high intra-class similarity and low inter-class similarity, while graph cuts solve the optimization.The method can discard outliers likely to reflect teacher errors, and λ controls the amount discarded.

D. Teacher-supervised Student Stream Fine-tuning

Student fine-tuning combines low-resolution face classification with regression toward selected teacher features, allowing the compact stream to learn missing and informative facial cues together.

  • Teacher-supervised fine-tuning: Fine-tuning jointly approximates selected teacher features through regression and recovers missing facial cues through low-resolution face classification.The two tasks are optimized as a multi-task learning problem.
  • Teacher-supervised fine-tuning: The classification loss evaluates the student stream over all low-resolution faces.It is combined with the regression loss with equal importance.
  • Teacher-supervised fine-tuning: The regression loss measures feature mismatch between the student feature-extraction backend and selected teacher features.The student backend extends through the mimic layer, where it produces approximated features matching the teacher.
  • Teacher-supervised fine-tuning: Standard back-propagation with stochastic gradient descent optimizes the combined objectives and improves low-resolution recognition under limited computational cost.The student is supervised by the teacher through feature regression while retaining classification training.

IV. EXPERIMENTS

The experiments examine selective distillation, teacher–student verification performance, and comparisons with state-of-the-art low-resolution face models across recognition tasks.

  • Experimental program: Four experiments analyze selective distillation, compare teacher and student networks in face verification, and evaluate the student against low-resolution face models.The final two experiments address face recognition and face retrieval tasks.

A. Experiment Setting

The evaluation uses four face datasets to assess distillation, verification, identification, and retrieval under standard and surveillance-oriented low-resolution conditions.

  • Datasets and tasks: UMDFaces, LFW, UCCS, and SCface evaluate selective distillation, face verification, face identification, and face retrieval, respectively.The datasets cover both public Internet images and challenging surveillance conditions.
  • Datasets and tasks: UMDFaces contains 367,888 images from 8,419 subjects and trains the student models while verifying selective distillation.Faces are aligned, cropped, and normalized into 224 × 224 high-resolution inputs before degradation or student processing.
  • Datasets and tasks: Teacher features from UMDFaces are used in graph optimization to select informative features before student training with identity labels and student inputs.The selected features are represented by an indicator vector α.
  • Datasets and tasks: LFW evaluates verification on 6,000 pairs, consisting of 3,000 positive and 3,000 negative pairs, with performance reported by AUC.Normalized hidden-layer features are compared using pairwise similarity.
  • Datasets and tasks: UCCS evaluates recognition on 1,732 identities with blurry, occluded, and poorly illuminated images, using exclusive training and testing identities.The dataset targets unconstrained surveillance recognition.
  • Datasets and tasks: SCface evaluates retrieval using 130 subjects and low-resolution surveillance images captured at 4.2m, 2.6m, and 1.0m.Fifty subjects are used for training and 80 for testing.

B. Selective Knowledge Distillation

Selective knowledge distillation chooses informative teacher features and discards noisy faces before supervising the student. The selection becomes more aggressive as λ increases, especially beyond approximately −1024.

  • Parameter influence: Informative-face counts decrease slowly for very small negative λ, then sharply after λ reaches around −1024.The number of selected faces remains zero once λ becomes larger than −32.
  • Face selection: Sparse graph optimization discards faces far from their class centroid or close to other classes, producing more compact feature visualizations.The discarded examples commonly have side postures, heavy occlusions, inconsistent illumination, or blurry appearances.
  • Face selection: Teacher features from challenging cases such as large pose variations may be treated as wrong knowledge and excluded from feature regression.This filtering targets teacher errors associated with low intra-class similarity.

C. Low-Resolution Face Verification on LFW

On LFW, selective distillation and face-class supervision preserve recognition accuracy as the student model compresses the teacher. The gains are especially substantial over direct low-resolution training and direct distillation.

  • Accuracy and compression: 95.03% accuracy is achieved by S-96-sc, only 2.12% below teacher T-224 without metric learning.S-96-sc uses selective distillation and face-class supervision, with 0.79M parameters versus VGGFace’s 138M.
  • Accuracy and compression: Recognition accuracy decreases as the student input resolution becomes lower.The trend is reported across the various student models evaluated on LFW.
  • Supervision effects: 19.49% accuracy improvement raises S-32-sc from 70.23% to 89.72% over S-32-c.The comparison indicates the benefit of jointly using teacher-stream and face-identity supervision.
  • Supervision effects: 2.04% accuracy gain separates S-16-sc from S-16-dc when informative knowledge is selected instead of noisy faces being retained.Face-class supervision also makes S-16-sc more accurate than S-16-s.

E. Low-Resolution Face Retrieval on SCface

SCface retrieval is highly challenging, with accuracy improving as face resolution increases at closer distances. Knowledge transfer methods outperform the PCA baseline, while the reported results compare hallucination- and embedding-based approaches.

  • Overall retrieval results: 4.73% total accuracy is obtained by the PCA baseline on SCface, underscoring the task’s difficulty.All evaluated models achieve below 50% accuracy overall.
  • Overall retrieval results: 41.04% total accuracy is achieved by LRFRW, exceeding DAlign at 24.30%, DCA at 18.72%, and SHSR at 16.50%.The compared methods transfer knowledge across resolutions, domains, or through super-resolution before recognition.
  • Resolution and distance: Recognition accuracy gradually increases as resolution rises and capture distance becomes closer.The reported SCface results identify resolution as an important factor affecting recognition performance.

F. Efficiency Analysis

The approach substantially compresses face-recognition models while preserving recognition accuracy and reducing inference cost. The authors identify teacher-stream failure cases as future work.

  • Memory footprint: 620× memory reduction is achieved for the 16 × 16 low-resolution student model, whose inference memory is only 0.15MB.Memory reductions are 22×, 48×, 182×, and 620× for 96 × 96, 64 × 64, 32 × 32, and 16 × 16 inputs, respectively.
  • Model complexity: 175× lower model complexity results from reducing VGGFace (T-224) from 138 million to about 0.79 million parameters.This reduction incurs a very small drop in recognition accuracy.
  • Inference speed: 0.106ms inference time per face is reached by S-16-sc on a NVIDIA K80 GPU, versus 20.4ms for the VGGFace teacher.The other student models require 0.61ms, 0.32ms, and 0.15ms for S-96-sc, S-64-sc, and S-32-sc, respectively.
  • Efficiency outcome: Selective knowledge transfer produces compact face-recognition models with effectiveness and efficiency while combining feature regression with low-resolution classification.The teacher network transfers informative knowledge to student models through a multi-task loss.
  • Future work: Future work will explore recurrent mechanisms to handle failure cases in the teacher stream.The authors also propose incorporating gender, age, and makeup attributes into the multi-task framework.
Loading 1811.09998v2…