Source-linked AI summary
Data Uncertainty Learning in Face Recognition
Jie Chang, Zhonghao Lan, Changmao Cheng, Yichen Wei
TL;DR
Noisy images make face embeddings uncertain, while PFE fixes the feature mean and relies on a costly MLS metric. This work jointly learns feature means and variances with two methods, improving recognition on challenging benchmarks and reducing noisy samples’ adverse effects.
Problem
Data uncertainty is important for noisy face images, but face recognition lacks methods that clarify how jointly learned uncertainty affects feature learning.
Method
The paper applies data uncertainty learning to face recognition by jointly learning embedding means and variances with classification-based and regression-based methods.
Results
The proposed methods improve face-recognition performance over deterministic models and PFE on most public benchmarks, with more remarkable improvement on low-quality images.
Takeaways & Limitations
Learned uncertainty adaptively reduces noisy training samples’ adverse effects on identity embeddings, while the learned feature can use conventional similarity metrics.
Takeaways & Limitations
PFE comparison remains computationally costly because MLS takes 1min28s for 6000 LFW pairs, versus less than 1 second for cosine similarity.
Abstract
from arXiv · showhide
Modeling data uncertainty is important for noisy images, but seldom explored for face recognition. The pioneer work, PFE, considers uncertainty by modeling each face image embedding as a Gaussian distribution. It is quite effective. However, it uses fixed feature (mean of the Gaussian) from an existing model. It only estimates the variance and relies on an ad-hoc and costly metric. Thus, it is not easy to use. It is unclear how uncertainty affects feature learning. This work applies data uncertainty learning to face recognition, such that the feature (mean) and uncertainty (variance) are learnt simultaneously, for the first time. Two learning methods are proposed. They are easy to use and outperform existing deterministic methods as well as PFE on challenging unconstrained scenarios. We also provide insightful analysis on how incorporating uncertainty estimation helps reducing the adverse effects of noisy samples and affects the feature learning.
1. Introduction
Data uncertainty is important for noisy face images, but prior face-recognition methods either ignore it or learn uncertainty without learning the feature itself. This work jointly learns feature means and variances through two methods, improving robustness and recognition performance.
- Motivation: Noisy face images can produce inaccurate embeddings that leave their class cluster and cause mismatches with other identities.Deterministic embeddings represent each image as a point, making noisy samples difficult to place accurately.
- Prior work: PFE models each sample with a Gaussian distribution but fixes the mean from a pretrained face-recognition model and learns only the variance.It estimates smaller variance for high-quality samples and larger variance for noisy ones.
- Proposed approach: The proposed data uncertainty learning jointly learns the feature mean and variance, producing more compact same-class features and better-separated different-class features.The learned feature remains directly usable with conventional similarity metrics, so MLS is unnecessary.
- Proposed approach: The paper proposes classification-based learning from scratch and regression-based learning that improves an existing model.It also analyzes how uncertainty adaptively reduces noisy training samples’ adverse effects on identity-embedding learning.
- Results: Experiments improve recognition over deterministic models and PFE on most public benchmarks, with larger gains on low-quality face-image benchmarks.The authors associate this pattern with suitability for unconstrained face-recognition scenarios.
2. Related Work
Deep uncertainty learning distinguishes noise inherent in data from uncertainty in model parameters, while face-recognition datasets contain both label noise and poor-quality images. Prior work includes PFE and experiments showing the proposed methods are more robust on noisy datasets.
- Uncertainty in Deep Learning: Deep uncertainty learning categorizes uncertainty into model uncertainty from network parameters and data uncertainty from noise inherent in training data.PFE is identified as the first work to consider data uncertainty in face recognition.
- Noisy Data Training: Large online face datasets inevitably contain label noise because examples can receive labels belonging to other identities.Poor-quality images are another reported source of adverse training effects.
- Uncertainty in Face Datasets: Figure 2 illustrates input-dependent noise in regression and visually ambiguous face samples identified as noisy relative to other images of the same identity.The regression view depicts a predicted value together with an estimated noise level.
- Noisy Data Training: The proposed methods experimentally perform more robustly on noisy datasets.
3. Methodology
The methodology learns face embeddings together with input-dependent uncertainty, using classification- and regression-based formulations that represent samples probabilistically. DULcls is trained end-to-end, while DULrgs adapts existing deterministic models and uses uncertainty to attenuate noisy examples.
- 3.1. Preliminaries: Data uncertainty models input-dependent variation in the mapping from face images to embeddings, estimating both the desired embedding and its uncertainty.The formulation represents observations as yi = f(xi) + ϵσ(xi), where σ(xi) captures uncertainty in the predicted value.
- 3.2. Classification-based DUL for FR: DULcls is trained end-to-end with a classification loss plus KL-divergence regularization, which balances variance against representation corruption and optimization stability.The total objective is Lcls = Lsoftmax + λLkl.
- 3.2. Classification-based DUL for FR: DULcls represents each face as a Gaussian distribution whose CNN-predicted mean is the identity feature and variance is its uncertainty.The stochastic representation is sampled with a re-parameterization trick so gradients can propagate during training.
- 3.3. Regression-based DUL for FR: DULrgs converts discrete identity labels into continuous targets using classifier weights from a pretrained deterministic face-recognition model.The convolutional parameters are fixed while the regression model learns distributional outputs around class-center targets.
- 3.3. Regression-based DUL for FR: DULrgs interprets learned variance as confidence and uses likelihood-based loss attenuation, assigning larger variance to ambiguous embeddings far from their class centers.The adaptive weighting mechanism prevents over-fitting to extremely noisy samples.
- 3.4. Discussion of Related Works: The paper relates DULcls’s objective to variational information bottleneck while interpreting it from a data-uncertainty perspective.Both approaches use a stochastic mapping and an objective balancing compactness with predictive information, but the paper emphasizes uncertainty in face embeddings.
4. Experiments
Experiments evaluate DULcls and DULrgs on standard and unconstrained face-recognition benchmarks, then analyze uncertainty, noise robustness, and training effects. The methods generally improve over deterministic baselines and PFE, with strongest gains on challenging, low-quality images.
- Evaluation setup: DULcls and DULrgs are evaluated on LFW, MegaFace, CFP, YTF, and IJB-C using standard protocols.Training uses MS-Celeb-1M; DULcls is trained from scratch, while DULrgs improves pretrained baseline models.
- Benchmark results: DUL methods outperform deterministic baselines on most benchmarks, especially unconstrained benchmarks containing blur or frontal/profile variation.The reported pattern is stronger on CFP, YTF, and IJB-C than on clearer frontal-image benchmarks such as LFW and MegaFace.
- Benchmark results: DULcls achieves higher TPR and lower FPR than the baseline at different IJB-C matching thresholds, resolving more false positives from extreme noise.The improvement is reported on IJB-C, identified as the most challenging benchmark, and becomes stronger as the target FPR decreases.
- Limitations: A documented limitation is that DULrgs combined with L2-Softmax deteriorates on IJB-C and requires further investigation.This caveat is specific to that loss-function and method combination.
- Comparison with PFE: Compared with PFE, DULcls performs better in all reported cases, while DULrgs remains competitive; both DUL and PFE substantially outperform baselines on IJB-C.DUL uses cosine similarity with averaged template-feature pooling, whereas PFE requires MLS-based matching and fusion.
- Uncertainty analysis: Learned uncertainty increases as image quality degrades and can serve as an embedding-quality measure, risk indicator, or image-quality signal.This avoids training a separate quality-assessment model requiring explicit quality labels.
- Uncertainty analysis: DUL alters feature learning by reducing noisy-sample effects: DULrgs pulls easy and semi-hard samples toward class centers while pushing hard samples away.The reported adaptive weighting prevents over-fitting on extremely noisy samples; DUL performance also remains more robust as blur increases.