Source-linked AI summary
Deep learning and face recognition: the state of the art
Stephen Balaban
TL;DR
Current face-recognition benchmarks may lack the scale and challenge needed for contemporary deep networks. The paper reviews deep-learning methods and proposes a very large public dataset; it describes LFW as saturated and notes that state-of-the-art pipelines still use face-specific alignment.
Problem
LFW offers 13,233 images, while modern deep networks need much larger training sets and face datasets differ from less constrained mobile-photo settings.
Source: §5.1 · p. 5, §5.2.3 · p. 7, §5.2.2 · p. 7Method
The paper reviews deep-learning techniques and their influence on state-of-the-art facial recognition, alongside a historical overview and dataset proposal.
Source: §1 · p. 1Results
Deep learning is integrated into most state-of-the-art facial-recognition pipelines, and LFW is described as saturated.
Source: §6 · p. 7Takeaways & Limitations
The paper proposes a public, very large-scale face dataset, and identifies companies with mobile photo-sharing applications as potential publishers.
Source: §5.1 · p. 6, §5.3 · p. 7State-of-the-art methods still use face-specific alignment, ranging from rough centering to 3D face-mask estimation and image reprojection.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Deep Neural Networks (DNNs) have established themselves as a dominant technique in machine learning. DNNs have been top performers on a wide variety of tasks including image classification, speech recognition, and face recognition. Convolutional neural networks (CNNs) have been used in nearly all of the top performing methods on the Labeled Faces in the Wild (LFW) dataset. In this talk and accompanying paper, I attempt to provide a review and summary of the deep learning techniques used in the state-of-the-art. In addition, I highlight the need for both larger and more challenging public datasets to benchmark these systems. The high accuracy (99.63% for FaceNet at the time of publishing) and utilization of outside data (hundreds of millions of images in the case of Google's FaceNet) suggest that current face verification benchmarks such as LFW may not be challenging enough, nor provide enough data, for current techniques. There exist a variety of organizations with mobile photo sharing applications that would be capable of releasing a very large scale and highly diverse dataset of facial images captured on mobile devices. Such an "ImageNet for Face Recognition" would likely receive a warm welcome from researchers and practitioners alike.
1. INTRODUCTION
Deep learning and representation learning are driving recent advances in facial recognition. The paper surveys the field and proposes future work on a new benchmark and dataset.
1. INTRODUCTION
- Recent advances in facial recognition are driven by deep learning and representation learning.Source: §1 · p. 1
- The paper reviews face recognition history, representation learning, deep learning, and their influence on current state-of-the-art methods.Source: §1 · p. 1
- It proposes future work to develop a new benchmark and dataset for face recognition research.Source: §1 · p. 1
2. BACKGROUND
Image and face recognition shifted from hand-engineered features toward large-scale learned features. Advances in feature learning and deep networks then carried into facial recognition.
2. BACKGROUND
- Earlier image and facial-recognition pipelines relied on hand-engineered features such as SIFT, LBP, and Fisher vectors [1].Source: §2 · p. 1
- Le et al. demonstrated large-scale feature learning with a sparse autoencoder trained on YouTube data across 1,000 machines and 16,000 cores, then used to initialize a DNN [7].Source: §2 · pp. 1–2
- In 2012, SuperVision achieved a 37% gain over competing hand-engineered features in ILSVRC, helping establish large-scale feature learning without server infrastructure [8].Source: §2 · p. 2
- These advances were applied to facial recognition by Taigman et al. in their 2014 DeepFace paper [6].Source: §2 · p. 2
3. DEEP LEARNING
The section contrasts learned features, derived from training data, with brittle hand-engineered features, then reviews gradient-based learning and neural architectures. It describes autoencoders and CNNs as approaches used in feature learning, with CNNs retaining image structure that flattened inputs lose.
3.1 Feature Engineering vs. Feature Learning
- Learned feature extractors use training-data statistics, while hand-engineered features can be brittle and over-specialized.Feature learning has also been applied across different domains and modalities.Source: §3.1 · p. 2
3.2 A quick introduction and review of gradient based learning
- SGD updates parameters using stochastic gradient approximations from random mini-batches, whose gradient variance decreases as batch size B increases.B=1 is online SGD; 1 < B < |x| is mini-batch SGD, while B=|x| is batch gradient descent.Source: §3.2 · p. 3
3.3 Autoencoders & Deep Neural Networks
- A deep neural network has more layers than traditionally used, and its parameters are optimized to minimize training loss using an algorithm such as SGD or L-BFGS.The network’s activation function is a nonlinearity, such as tanh, sigmoid, or ReLU.Source: §3.3 · p. 3
3.3.1 Unsupervised Learning
- Autoencoders map inputs to a hidden code and decode it into a reconstruction; denoising variants corrupt inputs and reconstruct the original.Autoencoders can initialize DNN weights before supervised fine-tuning, and the reconstruction loss depends on the assumed input-code distribution.Source: §3.3.1 · p. 4
- Unlabeled face data for pretraining can be gathered from photos using a face detector.The paper describes this dataset-building step as relatively straightforward.Source: §3.3.1 · p. 4
3.4 Convolutional Neural Networks
- CNNs use 2D convolutions and preserve image structure, unlike autoencoders and multilayer networks that flatten images into vectors.Nearly every method performing well on LFW utilizes CNNs, and convolution gives some degree of translation-invariance.Source: §3.3.1 · p. 4, §3.4 · p. 4
4. THE STATE-OF-THE-ART IN FACIAL RECOGNITION
Facial-recognition pipelines shifted from hand-engineered to learned features, rougher alignment, and much larger datasets. Even state-of-the-art methods still use face-specific alignment.
4. THE STATE-OF-THE-ART IN FACIAL RECOGNITION
- Pipelines moved from hand-engineered to learned features, face-specific alignment to rough alignment, and datasets of tens of thousands to hundreds of millions of images.The paper presents these as successive changes in the standard pipeline.Source: §4 · p. 4
- Despite the move to deep CNNs, all state-of-the-art methods still use face-specific alignment, from rough centering to 3D face-mask estimation and reprojection [5,31].The methods cited range from Schroff, Florian and Kalenichenko 2015 to Hassner, Tal, et al. 2014.Source: §4 · p. 5
5. DATASETS
The paper argues that face-recognition research needs public datasets that are larger, more diverse, and more challenging than existing resources. It identifies limitations in LFW's scale and evaluation, training-image variety, and access to large datasets.
5.1 Datasets today
- LFW has 13,233 images, top-performing methods use outside data, and private large datasets could restrict progress to researchers with proprietary access.The paper notes an estimate of over 1.8 billion images uploaded and shared daily on mobile photo-sharing networks [35].Source: §5.1 · p. 5
- Public datasets with tens of millions of images could remove a roadblock; the optimal size and structure are outside this paper's scope.Schroff et al. suggest current deep networks benefit from training sets of tens of millions of images and saturate thereafter [5].Source: §5.1 · p. 6
5.2.1 False Accept Rates (FARs) that are too high.
- LFW accuracy may be saturating; BLUFR focuses on lower false accept rates, but the paper also calls for a very large public dataset.The author shares Yi et al.'s view that LFW may be saturating and notes BLUFR as a more challenging alternative [32].Source: §5.2.1 · p. 6
5.2.2 Lack of variety and poor generalization.
- LFW and CACD are largely made from professional, well-lit DSLR photographs, which may generalize poorly to less constrained mobile-photo streams.Earlier datasets such as AT&T and Yale were captured in highly controlled laboratories [20,21].Source: §5.2.2 · p. 7
5.2.3 Not enough data.
- Deep neural networks need large training sets, preferably tens of millions of images; LFW alone is insufficient for modern deep architectures [5].The paper notes that top-performing LFW methods use supplementary datasets.Source: §5.2.3 · p. 7
5.3 Proposal for a new kind of facial recognition dataset
- The author predicts a transition from LFW to a large-scale dataset and challenge, analogous to Caltech 101's transition to ImageNet.The paper identifies companies in Table 3 as potential dataset publishers.Source: §5.3 · p. 7
6. CONCLUSIONS
Deep learning has become part of most state-of-the-art facial-recognition pipelines, and the resulting accuracy gains have left LFW saturated. The paper concludes that research needs a very large public face dataset.
6. CONCLUSIONS
- Deep learning is integrated into most state-of-the-art facial-recognition pipelines and has increased system accuracy, leaving LFW saturated.The conclusion links the benchmark's saturation to the accuracy increase associated with deep learning.Source: §6 · p. 7
- Deep networks' data requirements highlight the need for a public face-recognition dataset containing tens of millions of images.Source: §6 · p. 7