Source-linked AI summary
Unconstrained Face Verification using Deep CNN Features
Jun-Cheng Chen, Vishal M. Patel, Rama Chellappa
TL;DR
The paper addresses face verification under unconstrained pose, illumination, and other difficult conditions. It trains a DCNN on CASIA-WebFace and combines its features with a learned metric for IJB-A evaluation. The proposed approach performs much better than FV-based and commercial matchers on IJB-A and remains competitive on LFW.
Problem
Face-verification performance often degrades on unconstrained faces with large pose, illumination, expression, aging, cosmetics, and occlusion variations.
Method
The approach trains a DCNN on CASIA-WebFace and uses a learned metric to compute similarity between test image sets.
Results
The proposed DCNN performs much better than FV-based methods and commercial off-the-shelf matchers on IJB-A and is competitive on LFW.
Takeaways & Limitations
A DCNN trained on a relatively small face dataset can learn a robust model from varied data and generalize to another dataset.
Takeaways & Limitations
Future work proposes directly training a Siamese network on positive and negative pairs from CASIA-WebFace and IJB-A training data.
Abstract
from arXiv · showhide
In this paper, we present an algorithm for unconstrained face verification based on deep convolutional features and evaluate it on the newly released IARPA Janus Benchmark A (IJB-A) dataset. The IJB-A dataset includes real-world unconstrained faces from 500 subjects with full pose and illumination variations which are much harder than the traditional Labeled Face in the Wild (LFW) and Youtube Face (YTF) datasets. The deep convolutional neural network (DCNN) is trained using the CASIA-WebFace dataset. Extensive experiments on the IJB-A dataset are provided.
1. Introduction
Face verification methods that work in controlled settings often degrade under substantial pose, illumination, expression, aging, cosmetics, and occlusion variations. This paper studies whether a DCNN trained on CASIA-WebFace can generalize to challenging IJB-A faces and evaluates it alongside LFW results.
- Controlled-setting face verification often degrades on faces with large pose, illumination, expression, aging, cosmetics, and occlusion variations.
- Existing approaches learn invariant and discriminative representations, including high-dimensional features followed by learned metrics for similarity computation.
- The paper trains a DCNN on CASIA-WebFace and compares it with commercial face matchers on challenging IJB-A, while also evaluating LFW performance.
- The paper introduces the study through sections covering related work, the proposed representation and metric learning, experiments, and conclusions.
2. Related Work
Related work frames face verification around two learned components: invariant feature representation and similarity measurement. Prior methods use hand-crafted or learned high-dimensional features and learned metrics, but high dimensionality can make training and scaling difficult.
- Feature Learning: Face-verification representation methods divide broadly into hand-crafted features and representations learned from data.
- Feature Learning: LBP, Gabor wavelets, dictionary-based methods, Fisher vectors, and sparse local-patch codes have been used to encode pose- and illumination-robust face information.
- Feature Learning: High-dimensional feature vectors make several representation methods difficult to train and scale to large datasets.
- Metric Learning: Similarity-measure learning is another key component, using labeled face images or pairs to improve verification performance.
- Metric Learning: Prior metric-learning approaches include LMNN, ITML Mahalanobis distance, and joint Bayesian modeling of paired face-image distributions.
3. Method
The method aligns detected faces, trains a deep convolutional representation on CASIA-WebFace, and learns a joint Bayesian metric from IJB-A training data. Its architecture uses deep convolutional layers with PReLU and normalization, while the metric is optimized with large-margin stochastic updates.
- Method Overview: The pipeline detects faces and landmarks, aligns faces, trains the DCNN on CASIA-WebFace, learns a joint Bayesian metric on IJB-A training data, and scores test image sets.
- Face Alignment: Faces are aligned using seven landmarks and a similarity transform before being resized to 100 × 100 pixels.
- Deep Face Feature Representation: The DCNN uses 10 convolutional layers, five pooling layers, one fully connected layer, PReLU activations, and two local normalization layers.
- Joint Bayesian Metric Learning: Joint Bayesian metric learning models paired feature vectors with Gaussian distributions for same-identity and different-identity hypotheses.
- Joint Bayesian Metric Learning: The metric objective is rewritten as a distance involving M and B, then optimized with stochastic gradient updates when pairwise constraints are violated.
- Joint Bayesian Metric Learning: M and B are initialized as positive-semidefinite factorizations, trained on positive and negative pairs with a 1:20 ratio, partly because some IJB-A templates contain one image.
- Feature Maps: Upper-layer feature maps capture more global shape features and are more robust to illumination changes than Conv11 maps.
- Architecture and Training: The DCNN architecture is documented in Table 1, while training uses CASIA-WebFace after excluding overlapping IJB-A subjects and follows specified optimization settings.
4. Experiments
Experiments evaluate the approach on JANUS CS2, IJB-A, and LFW using verification and identification protocols. The DCNN approach outperforms competitive methods on the unconstrained benchmarks, while remaining comparable to other deep learning methods on LFW.
- Datasets and protocols: JANUS CS2 and IJB-A contain 500 subjects, 5,397 images, 2,042 videos, and 20,412 frames, with substantial unconstrained variation.The datasets include variation in pose, illumination, resolution, occlusion, and image quality.
- Datasets and protocols: IJB-A evaluates 1:1 verification and 1:N identification over 10 splits, using multiple-image or video-frame templates.Each verification split contains approximately 11,748 template pairs on average, including positive and negative pairs.
- Evaluation and comparisons: The experiments compare DCNN variants with Fisher vector, COTS1, and GOTS baselines on JANUS CS2, while evaluating component effects through ROC and CMC curves.Variants include finetuning, Joint Bayesian metric learning, RGB inputs, PReLU, and data augmentation.
- Results on JANUS CS2 and IJB-A: DCNNfusion sums similarity scores from grayscale small-region and RGB large-region models, and the DCNN method performs better than other competitive methods.The authors attribute this performance to learning face variations from a large dataset and generalizing to a new small dataset.
- Qualitative results: A query example succeeds at rank 1 for two rows but fails within the top five when the probe has an extreme pose and the gallery contains only near-frontal faces.The authors propose synthesizing varied poses with a generic 3D model as future work.
- Results on LFW: On LFW, the approach performs comparably to other deep learning-based methods despite training on fewer than 500K CASIA-WebFace images.Some compared methods use millions of training samples, whereas this model uses the CASIA dataset.
5. Conclusion
The DCNN method performs strongly on challenging IJB-A faces, outperforming FV-based and commercial matchers while remaining competitive on LFW. Future work targets better performance by using all available positive and negative training pairs directly.
- The DCNN approach learns a robust model from varied face data and generalizes well to another dataset.
- On IJB-A, the proposed DCNN performs much better than the FV-based method and other commercial off-the-shelf matchers.
- The proposed method is competitive on the LFW dataset.
- Future work will directly train a Siamese network on all available positive and negative pairs from CASIA-WebFace and IJB-A training datasets.The goal is to fully use discriminative information for better performance.