PaperlayerView exact revision on arXiv

Source-linked AI summary

Face Recognition: From Traditional to Deep Learning Methods

Daniel Sáez Trigueros, Li Meng, Margaret Hartnett

arXiv:1811.00116v1cs.CV

TL;DR

Face recognition must contend with substantial real-world variation in unconstrained images. This review synthesizes holistic and hybrid approaches alongside CNN architectures and training losses; it reports that CNN systems have become standard after significant accuracy improvements over other method types.

  • Problem

    Unconstrained face recognition is challenging because real-world images vary in pose, age, occlusion, illumination, and expression.

    Source: §I · p. 1

  • Method

    The review synthesizes holistic and hybrid approaches alongside CNN architectures and training losses for face recognition.

    Source: B. Holistic Methods · p. 4, D. Hybrid Methods · p. 6, E. Deep Learning Methods · pp. 8–9

  • Results

    CNN-based face-recognition systems have become standard after significant accuracy improvement over other method types.

    Source: §III · p. 10

  • Takeaways & Limitations

    CNN accuracy is affected by training data, architecture, and loss, and face datasets need many subjects as well as samples.

    Source: E. Deep Learning Methods · p. 8

    Collecting large amounts of labelled face images is expensive, and very deep CNN architectures are slow to train and deploy.

    Source: §III · p. 10

Abstract

from arXiv · show

Starting in the seventies, face recognition has become one of the most researched topics in computer vision and biometrics. Traditional methods based on hand-crafted features and traditional machine learning techniques have recently been superseded by deep neural networks trained with very large datasets. In this paper we provide a comprehensive and up-to-date literature review of popular face recognition methods including both traditional (geometry-based, holistic, feature-based and hybrid methods) and deep learning methods.

I. INTRODUCTION

Face recognition is challenging in unconstrained settings because real-world faces vary in pose, age, illumination, expression, and occlusion. The introduction traces a shift from hand-crafted methods to CNNs and outlines the stages of a recognition system.

I. INTRODUCTION

  • Face recognition is non-intrusive and can serve applications such as surveillance, access control, fraud detection, identity verification, and social media.Unlike fingerprint, iris, or speaker recognition, it requires users only to be within camera view at a reasonable distance.Source: §I · p. 1
  • Real-world variation in pose, age, occlusion, illumination, and expression makes face recognition especially challenging in unconstrained environments.The paper calls these images faces in-the-wild.Source: §I · p. 1
  • Traditional hand-crafted features and machine-learning methods have been superseded by CNN methods trained on large datasets to learn features robust to real-world variation.The paper notes that web images enabled large-scale face datasets containing real-world variation.Source: §I · p. 1
  • A face-recognition system detects faces, aligns them using reference points, converts pixels into compact feature-vector templates, and matches templates with a similarity score.The system aims to map images of the same subject to similar feature vectors.Source: §I · p. 2

II. LITERATURE REVIEW

The literature review follows face recognition from constrained geometry-based methods through holistic, feature-based, and hybrid approaches to deep learning. It surveys representative works in these method families.

II. LITERATURE REVIEW

  • Early geometry-based methods used image-processing techniques to match simple descriptions of facial geometry.They worked only under very constrained settings but demonstrated automatic face recognition by computers.Source: §II · p. 2
  • Holistic methods use the entire face region, while feature-based methods match local texture features; these approaches were also combined into hybrid methods.Source: §II · p. 2
  • Hybrid methods remained state of the art until deep learning emerged as the leading approach in face recognition.The review summarizes representative work across the method types it describes.Source: §II · p. 2

A. Geometry-based Methods

Early geometry-based face recognition represented faces through facial landmarks and their spatial relationships. These methods were fast and could complement more information-rich approaches, while 3D landmarks add depth information.

A. Geometry-based Methods

  • Early systems used edge and contour detectors to locate facial landmarks and measure their relative positions and distances.The cited theses tested on databases of 10 and 20 subjects, respectively.Source: A. Geometry-based Methods · p. 2
  • Gradient-image comparison achieved better recognition accuracy than a similar geometry-based method, which was faster and required less memory.Source: A. Geometry-based Methods · p. 2
  • Geometry-based methods measured Procrustes distance between landmark sets or ratios of distances between landmarks.The authors argued that these methods could complement more information-rich approaches in hybrid systems.Source: A. Geometry-based Methods · p. 2
  • Depth information in 3D landmarks has made geometry-based methods more effective for 3D face recognition.The review also describes geometry-based methods as a fast alternative to, or component alongside, more advanced methods.Source: A. Geometry-based Methods · p. 3

B. Holistic Methods

Holistic methods represent faces using the whole face region, often projecting images into lower-dimensional spaces. The section reviews PCA, LDA, LPP, sparse representation, and joint Bayesian methods, including their trade-offs and reported results.

Holistic Methods

  • PCA projects whole-face images onto eigenfaces that capture the greatest training-set variance, then compares projected weights with gallery faces.Because variance can reflect irrelevant within-person changes, such as illumination, pose, or expression, the largest-variance directions may hurt recognition.Source: B. Holistic Methods · p. 3
  • LDA uses class labels to maximize between-class variance while minimizing within-class variance, but can overfit when within-class scatter is poorly estimated.This risk arises with high-dimensional inputs and few samples per class; if the within-class scatter matrix is singular, the projection cannot be computed, so PCA is commonly applied first.Source: B. Holistic Methods · p. 3
  • Holistic SVM approaches include training on within- and between-class image differences or training a separate classifier per class; the latter beat Euclidean distance only with PCA projections, not LDA projections.LDA already encodes discriminant information needed for face recognition.Source: B. Holistic Methods · p. 3
  • LPP preserves local image-space structure by mapping images with similar local information to neighboring points, and its approach outperformed PCA and LDA on multiple datasets.The section notes that LPP basis vectors were later made orthogonal for further improvement.Source: B. Holistic Methods · pp. 3–4
  • Sparse representation-based classification represents a test face as a sparse linear combination of training images and classifies it by the lowest class-specific reconstruction error.Adding sparse error coefficients for corrupted pixels can increase robustness to noise or occlusions; variations also target compactness, spatially continuous occlusions, or improved reconstruction.Source: B. Holistic Methods · p. 4
  • Joint Bayesian represents a face as independent Gaussian components for intra-personal and inter-personal variation, reaching 92.4% accuracy on LFW—the highest reported for a holistic method there.It was inspired by probabilistic PCA and models these variations separately rather than using image differences.Source: B. Holistic Methods · p. 4

C. Feature-based Methods

Feature-based methods extract local features from different face-image locations, emphasizing discriminative cues rather than feature geometry. The section surveys regional representations and descriptors that can handle local variations, alongside their trade-offs with holistic methods.

Feature-based Methods

  • Feature-based methods use local face-image features and tend to be more robust than holistic methods to local variations such as expression or illumination.When only the eyes change, feature coefficients around the eyes may differ, whereas all holistic coefficients may change; many descriptors are also designed for scale, rotation, or translation invariance.Source: C. Feature-based Methods · p. 4
  • Modular eigenfaces apply PCA independently to face regions; they matched eigenfaces in accuracy overall but performed better when only a few eigenvectors were used.Binary edge methods used modified Hausdorff distance to require each point to lie near a point in the other set, while line edge maps encode edges compactly as line segments.Source: C. Feature-based Methods · p. 4
  • Elastic bunch graph matching represents faces as graphs whose nodes hold Gabor coefficients at predefined landmarks, then fits test-face nodes by searching a face bunch graph.A HOG-based version outperformed original EBGM, with the passage attributing the result to HOG robustness to illumination, rotation, and small displacements.Source: C. Feature-based Methods · p. 4
  • LBP methods compute descriptor histograms in local regions and concatenate them into a global feature vector; variations include Gabor-based LGBP, rotation-invariant LBP, and high-order LDP.The described figure divides a face into 4 × 4 regions and shows histograms computed from each region.Source: C. Feature-based Methods · p. 5, Figure 5 · p. 5
  • For SIFT matching, dense descriptors on a regular grid gave the best recognition accuracy among three proposed methods, and dense SURF extraction also performed best in its study.Volume-SIFT and partial-descriptor-SIFT both improved accuracy over original SIFT.Source: C. Feature-based Methods · p. 5
  • Learned local descriptors include codes from unsupervised microstructure learning, Gaussian-mixture statistics of differential patterns, and LDA-learned filters and pixel weights.The approaches encode local facial regions using different learned representations.Source: C. Feature-based Methods · p. 5

D. Hybrid Methods

Hybrid methods combine holistic and feature-based approaches, most often extracting local features and projecting them into a lower-dimensional discriminative subspace. The reviewed approaches also combine features and classifiers in varied ways, while feature choice remains a central limitation.

D. Hybrid Methods

  • Local features such as LBP or SIFT are commonly projected into a lower-dimensional discriminative subspace, the most popular hybrid pipeline.The paper illustrates a typical hybrid face representation in Figure 6.Source: D. Hybrid Methods · p. 5, Figure 6 · p. 6
  • Gabor-wavelet hybrids convolve images at multiple orientations and scales, concatenate and downsample outputs, and outperform Gabor features alone.Reviewed projections include enhanced LDA, PCA followed by ICA, and kernel PCA with polynomial kernels.Source: D. Hybrid Methods · pp. 5–6
  • TPLBP and FPLBP combine with LDA and SVMs, encoding similarities between neighboring patches to boost face-recognition accuracy.A dense MLBP representation reduced from 100K to 400 dimensions with PCA and followed by joint Bayesian gained 6–7% accuracy when features increased from 1K to 100K, at high dimensionality-reduction cost.Source: D. Hybrid Methods · p. 6
  • GaussianFace uses multi-task learning, a discriminative Gaussian-process latent-variable model, and efficient kernel LDA to exploit multiple source domains.It achieved 98.52% accuracy on LFW, described as competitive with many deep-learning methods.Source: D. Hybrid Methods · p. 6
  • Other hybrids fuse complementary features, including Gabor and LBP, or combine LBP/LTP and Gabor features with kernel LDA, normalization, and score fusion.Reviewed variants also target blur with multi-scale LPQ and MLBP, age variation with dense SIFT and multi-scale LBP, and shape-texture fusion using facial-landmark distances.Source: D. Hybrid Methods · p. 6
  • Alternative hybrids use attribute and simile classifiers or metric learning, including LDML and MkNN trained on SIFT descriptors at fixed facial points.The attribute and simile classifier outputs for two images are fed to an SVM for comparison.Source: D. Hybrid Methods · pp. 6–7
  • Hybrid methods’ main limitation is finding features that capture enough recognition-relevant information; deep learning extends feature learning end to end.Some hybrids combine feature types or add learning to improve feature discriminability.Source: D. Hybrid Methods · p. 7

E. Deep Learning Methods

Deep learning methods use CNNs to learn face representations from data rather than relying on hand-engineered features, and their performance depends on training data, architecture, and loss. The review covers classification and metric-learning training, CNN designs, and losses that shape feature separation.

E. Deep Learning Methods

  • CNNs learn representations from large datasets that can capture training variations, but need enough varied data to generalise to unseen samples.Face-recognition CNNs may classify training identities and then use bottleneck features for unseen identities, or learn features directly through pairwise or triplet metric losses,, [101],.Source: E. Deep Learning Methods · p. 7
  • DeepFace reached 97.35% accuracy on LFW and reduced the previous state-of-the-art error by 27%, using 4.4 million faces from 4,030 subjects.Its contributions included explicit 3D face modelling for alignment and locally connected layers [108],. DeepID used 60 CNNs on face patches and trained a joint Bayesian classifier on their extracted features,.Source: E. Deep Learning Methods · pp. 7–8
  • With the same number of images, wider training datasets provide better accuracy; the best result in one study used the 10,000 subjects with the most images.Adding more subjects decreased accuracy when few images were available for each additional subject.Source: E. Deep Learning Methods · p. 8
  • GoogleNet-style networks achieved accuracy comparable to other networks with 20 times fewer parameters, while a 100-layer ResNet offered the best accuracy-speed-model-size trade-off in a CNN architecture study [121].ResNet shortcut connections learn residual mappings and facilitate information flow across layers, enabling deeper architectures.Source: E. Deep Learning Methods · p. 8
  • Softmax loss separates training classes but may not reduce within-class variation; triplet loss separates positive from negative pair distances by a margin, while centre and range losses target feature compactness.Triplet training can converge more slowly because it needs many pairs or triplets; centre loss avoids pair formation, and range loss uses extreme within-class distances and closest class centres, requiring combination with softmax to avoid collapse to zero.Source: E. Deep Learning Methods · pp. 8–9
  • Margin-based softmax losses make class decision boundaries more stringent and can increase class separation and intra-class compactness.Variants include multiplicative and additive cosine margins, as well as an additive angular margin; the review describes these approaches as current state-of-the-art in face recognition.Source: E. Deep Learning Methods · p. 9, Figure 8 · p. 9, Table II · p. 9

III. CONCLUSIONS

CNN-based systems have become the standard after replacing traditional hand-engineered methods and improving accuracy. Scaling training data or network capacity can raise accuracy, but labeling costs and deep-model runtime remain concerns.

III. CONCLUSIONS

  • CNN-based face recognition has replaced traditional hand-engineered methods and become the standard following significant accuracy improvements.The review describes GANs as a promising approach to generating additional training images without requiring millions of labeled face images.Source: §III · p. 10
  • Increasing training-set size or network capacity can raise CNN accuracy, while collecting large labeled face datasets is expensive and very deep CNNs are slow to train and deploy.The review points to MobileNets for real-time recognition on devices with limited computational resources.Source: §III · p. 10
  • GAN research on face attributes, expressions, identities, frontalisation, and ageing is expected to support generating additional training images.More efficient architectures such as MobileNets are being developed for real-time face recognition on resource-limited devices.Source: §III · p. 10

Related papers

Loading 1811.00116v1…