Source-linked AI summary
DeepID3: Face Recognition with Very Deep Neural Networks
Yi Sun, Ding Liang, Xiaogang Wang, Xiaoou Tang
TL;DR
DeepID3 investigates whether very deep neural networks can improve face recognition beyond shallower architectures. It proposes two VGG- and GoogLeNet-inspired networks with joint identification-verification supervision, achieving state-of-the-art LFW verification and identification performance, though corrected labels eliminate the verification improvement over DeepID2+.
Problem
DeepID2 and DeepID2+ were much shallower than recently proposed high-performance deep networks such as VGG and GoogLeNet, motivating investigation of deeper architectures for face recognition.
Method
DeepID3 proposes two deeper architectures built from stacked convolution or inception layers, adding joint face identification-verification supervision to final and intermediate feature extraction layers.
Results
DeepID3 achieves state-of-the-art performance on LFW face verification and identification, improving over DeepID2+ from 99.47% to 99.53% verification accuracy and from 95.0% to 96.0% rank-1 identification accuracy.
Takeaways & Limitations
DeepID3 demonstrates strong performance on both LFW face verification and identification tasks, while its verification advantage over DeepID2+ disappears after correcting several wrong labels.
Takeaways & Limitations
After correcting a few wrong LFW labels, DeepID3 no longer improves over DeepID2+ on face verification, leaving the benefit of very deep architectures with larger training data unresolved.
Abstract
from arXiv · showhide
The state-of-the-art of face recognition has been significantly advanced by the emergence of deep learning. Very deep neural networks recently achieved great success on general object recognition because of their superb learning capacity. This motivates us to investigate their effectiveness on face recognition. This paper proposes two very deep neural network architectures, referred to as DeepID3, for face recognition. These two architectures are rebuilt from stacked convolution and inception layers proposed in VGG net and GoogLeNet to make them suitable to face recognition. Joint face identification-verification supervisory signals are added to both intermediate and final feature extraction layers during training. An ensemble of the proposed two architectures achieves 99.53% LFW face verification accuracy and 96.0% LFW rank-1 face identification accuracy, respectively. A further discussion of LFW face verification result is given in the end.
1. Introduction
DeepID3 addresses whether substantially deeper architectures can improve face recognition beyond the shallower DeepID2+ network. It combines VGG- and GoogLeNet-inspired architectures with joint identification-verification supervision.
- Motivation: DeepID2+ remained much shallower than high-performance object-recognition architectures such as VGG and GoogLeNet.DeepID2+ used advanced supervisory signals but had a comparatively shallow architecture.
- Contribution: DeepID3 proposes two significantly deeper neural network architectures for face recognition.The architectures are rebuilt from stacked convolution or inception layers derived from VGG and GoogLeNet.
- Architecture: DeepID3 uses stacked convolution or inception layers from VGG and GoogLeNet as architectural building blocks.
- Training: Joint face identification-verification supervision is applied to final and intermediate feature extraction layers during training.This supervision is used in both proposed architectures.
2. DeepID3 net
DeepID3 extends DeepID2+ with ten to fifteen nonlinear feature-extraction layers organized into two architectures. It combines stacked convolution or inception layers with intermediate supervisory signals and region-specific training for recognition.
- DeepID2+ baseline: DeepID2+ uses three convolutional layers, a locally connected layer, and a fully connected layer, with supervision at final and intermediate branches.
- DeepID3 net: DeepID3 contains ten to fifteen nonlinear feature-extraction layers, compared with five in DeepID2+.
- DeepID3 net1: DeepID3 net1 stacks two convolutional layers before each pooling layer and adds intermediate supervisory branches to ease optimization.
- DeepID3 net2: DeepID3 net2 uses inception layers in later stages, with three before the third pooling layer and two before the fourth.
- Training and evaluation: DeepID3 networks train on the same 25 face regions and approximately 300 thousand samples used for DeepID2+.On average, net1 and net2 reduce verification error by 0.81% and 0.26% relative to DeepID2+, respectively.
3. Experiments
The experiments evaluate DeepID3 on LFW face verification and identification using concatenated region features and Joint Bayesian modeling. The reported results include 99.53% verification accuracy and strong closed- and open-set identification performance.
- Feature extraction: DeepID3 features from original or horizontally flipped regions are concatenated into an approximately 30,000-dimensional vector.
- Feature extraction: PCA reduces the concatenated feature vector to 300 dimensions before Joint Bayesian modeling for face recognition.
- Face verification: 99.53% mean accuracy is achieved on the LFW face verification protocol.The protocol verifies 6,000 given face pairs for whether they depict the same person.
- Face identification: 96.0% closed-set and 81.4% open-set face identification accuracies are achieved on LFW.The open-set figure is reported at a 1% false alarm rate.
4. Discussion
Correcting three mislabeled LFW positive pairs removes DeepID3’s apparent verification gain over DeepID2+, while recurring errors remain associated with age, makeup, similarity, and occlusion.
- 99.52% is the corrected face verification accuracy for both DeepID3 and DeepID2+, eliminating DeepID3’s apparent advantage on LFW.DeepID3 classified two of three mislabeled positive pairs as the same person; DeepID2+ classified all three as different people.
- Three face pairs labeled as the same person were actually different people, exposing label errors in the LFW evaluation set.
- Nine common false positives and three common false negatives were misclassified by all DeepID series algorithms, representing around half of DeepID3’s errors.
- Common false negatives include large age differences, an incorrectly labeled pair, and substantially different makeup.
- Common false positives often involve visually similar faces or interference such as occlusions.
5. Conclusion
DeepID3 introduces two substantially deeper architectures for face recognition and reports state-of-the-art results on LFW verification and identification. However, correcting several LFW labels removes its verification improvement over DeepID2+, motivating evaluation with larger training data.
- DeepID3 proposes two significantly deeper neural network architectures for face recognition.
- DeepID3 achieves state-of-the-art performance on both LFW face verification and identification tasks.
- Correcting a few wrong LFW labels eliminates DeepID3’s improvement over DeepID2+ on face verification.
- The effectiveness of very deep neural networks remains to be investigated with larger-scale training data.