Source-linked AI summary
Deep Learning Face Representation by Joint Identification-Verification
Yi Sun, Xiaogang Wang, Xiaoou Tang
TL;DR
Face recognition requires representations that reduce within-identity variation and enlarge between-identity differences. The paper learns DeepID2 features with deep convolutional networks under joint identification and verification supervision. On LFW, the resulting system achieved 99.15% face verification accuracy.
Problem
Face recognition must handle complex variations within the same identity while preserving differences between identities.
Method
DeepID2 features are learned with deep convolutional networks using identification supervision to separate identities and verification supervision to reduce same-identity variation.
Results
99.15% face verification accuracy was achieved on LFW.
Takeaways & Limitations
Combining identification and verification signals produces better face-recognition features than either signal alone.
Abstract
from arXiv · showhide
The key challenge of face recognition is to develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. In this paper, we show that it can be well solved with deep learning and using both face identification and verification signals as supervision. The Deep IDentification-verification features (DeepID2) are learned with carefully designed deep convolutional networks. The face identification task increases the inter-personal variations by drawing DeepID2 extracted from different identities apart, while the face verification task reduces the intra-personal variations by pulling DeepID2 extracted from the same identity together, both of which are essential to face recognition. The learned DeepID2 features can be well generalized to new identities unseen in the training data. On the challenging LFW dataset, 99.15% face verification accuracy is achieved. Compared with the best deep learning result on LFW, the error rate has been significantly reduced by 67%.
1 Introduction
Face recognition must reduce large within-identity variation while enlarging differences between identities, but prior linear or shallow models are limited for these complex nonlinear variations. The paper proposes DeepID2, learned with joint identification and verification supervision, and reports strong LFW performance.
- Variations in pose, illumination, expression, age, and occlusion can make faces of one identity look substantially different.
- The central face-recognition objective is reducing intra-personal variations while enlarging inter-personal differences.
- DeepID2 uses deep convolutional networks and simultaneous face identification and verification signals to learn nonlinear face representations.
- Identification separates features from different identities, whereas verification pulls features from the same identity together.
- 99.15% face verification accuracy was achieved on LFW, improving substantially over prior deep-learning results.
2 Identification-verification guided deep feature learning
DeepID2 features are learned with deep ConvNets under identification and verification supervision, combining identity discrimination with within-identity consistency.
- The deep ConvNet uses four convolutional layers and extracts a 160-dimensional DeepID2 vector from multi-scale features.The third layer uses locally shared weights, while the fourth is locally connected with unshared weights.
- Identification supervision classifies faces into many identity classes, encouraging features with large inter-personal variation.The identification network uses an n-way softmax and minimizes cross-entropy loss.
- Verification supervision encourages DeepID2 vectors from the same identity to be similar and vectors from different identities to exceed a margin.The L2-based loss minimizes same-identity distances and requires different-identity distances to be larger than margin m.
- The training objective weights identification and verification gradients with λ, while only ConvNet parameters are used for feature extraction at test time.The identification and verification parameters propagate supervision during training but are not used for testing features.
- The learning algorithm is summarized in Table 1 and includes adaptive updating of the verification margin during training.The margin is updated using the threshold giving the lowest verification error on recent training samples.
3 Face Verification
The face-verification pipeline aligns faces, extracts selected multi-view patches, compresses their combined DeepID2 representation, and applies Joint Bayesian verification.
- Faces are aligned using 21 detected landmarks and a similarity transformation before feature extraction.The pipeline uses the SDM algorithm for landmark detection and global alignment.
- The system crops 400 face patches varying in position, scale, color channel, and horizontal flipping.
- A greedy selection method retains 25 complementary DeepID2 vectors, concatenating them into a 4000-dimensional vector before PCA compression.The selection reduces redundancy and most feature-extraction time during testing.
- Joint Bayesian models each face feature as inter- and intra-personal variations and verifies pairs using a log-likelihood ratio test.The model represents a face feature as f = µ + ϵ, with both components modeled as Gaussian distributions.
4 Experiments
Experiments evaluate DeepID2 on LFW and analyze how identification and verification supervision, identity-count scale, verification constraints, and feature aggregation affect face verification. Joint supervision with suitable weighting reduces intra-personal variation while preserving discriminative inter-personal variation, and the final system reaches 99.15% accuracy.
- Experimental setup: LFW contains 13,233 images from 5,749 identities; evaluation uses mean accuracy and ROC curves on 6,000 face pairs.Training uses the separate CelebFaces+ dataset, with mutually exclusive identities and 8,192 identities for DeepID2 learning.
- Balancing supervision: Moderate verification weighting preserves diverse inter-personal variation while reducing the magnitude and diversity of intra-personal variation.The reported comparison uses λ = 0.05; identification-only training retains large intra-personal variation, whereas verification-only training collapses both types into a few directions.
- Balancing supervision: When inter-personal variation diversity becomes low, identity clusters overlap and face verification performance degrades significantly.The PCA visualization shows mixed clusters with identification-only training and renewed overlap when verification-only training collapses cluster centers.
- Identity information: 8192 training identities are key to learning effective DeepID2 representations, with more identities potentially improving the result further.Increasing the number of identities supplies richer identity information and helps form more diverse inter-personal variations.
- Verification signals: The L2+ verification signal performs only slightly worse than L2, whereas L2- contributes little beyond identification-only training.These comparisons support the interpretation that verification supervision mainly reduces intra-personal variation; L2 also outperforms the other compared verification metrics.
- Final system: 99.15% face verification accuracy is achieved on LFW by selecting multiple groups of DeepID2 features and fusing seven Joint Bayesian scores with an SVM.The paper reports the best result and a large improvement over previous results in its comparison.
5 Conclusion
Identification and verification supervision jointly shape features by increasing inter-personal variation and reducing intra-personal variation. Their combination produces better features and supports 99.15% face verification accuracy on LFW.
- Identification and verification supervision increase inter-personal variation and reduce intra-personal variation, respectively.
- The combination of both supervisory signals leads to significantly better features than either signal alone.
- 99.15% face verification accuracy was achieved on LFW using the learned features in a traditional face verification pipeline.