Source-linked AI summary
Learning Meta Face Recognition in Unseen Domains
Jianzhu Guo, Xiangyu Zhu, Chenxu Zhao, Dong Cao, Zhen Lei, Stan Z. Li
TL;DR
Face recognition models must generalize from source domains to unseen target domains without updating, a setting complicated by distribution shifts and disjoint identities. MFR simulates these shifts with meta-optimization over synthesized source and target domains, combining gradients and meta-gradients. The paper introduces two benchmarks and reports validated generalization on them, while noting that high-order meta-gradients are computationally expensive.
Problem
Generalized face recognition requires models trained on source domains to handle unseen target domains without updating, despite distribution shifts and disjoint identity classes.
Method
MFR uses domain-level sampling and meta-optimization to learn representations on synthesized meta-train and meta-test domains, combining gradients and meta-gradients.
Results
Experiments on two newly defined generalized face recognition benchmarks validate MFR's effectiveness and generalization.
Takeaways & Limitations
A trained MFR model can be directly deployed on target domains without model updating, supporting generalized face recognition evaluation through two proposed benchmarks.
Takeaways & Limitations
High-order meta-gradients require substantial computation; a first-order approximation uses about 82% of the GPU memory and 63% of the time in the reported setting.
Abstract
from arXiv · showhide
Face recognition systems are usually faced with unseen domains in real-world applications and show unsatisfactory performance due to their poor generalization. For example, a well-trained model on webface data cannot deal with the ID vs. Spot task in surveillance scenario. In this paper, we aim to learn a generalized model that can directly handle new unseen domains without any model updating. To this end, we propose a novel face recognition method via meta-learning named Meta Face Recognition (MFR). MFR synthesizes the source/target domain shift with a meta-optimization objective, which requires the model to learn effective representations not only on synthesized source domains but also on synthesized target domains. Specifically, we build domain-shift batches through a domain-level sampling strategy and get back-propagated gradients/meta-gradients on synthesized source/target domains by optimizing multi-domain distributions. The gradients and meta-gradients are further combined to update the model to improve generalization. Besides, we propose two benchmarks for generalized face recognition evaluation. Experiments on our benchmarks validate the generalization of our method compared to several baselines and other state-of-the-arts. The proposed benchmarks will be available at https://github.com/cleardusk/MFR.
1. Introduction
Generalized face recognition targets deployment on unseen domains without updating, unlike domain adaptation settings with accessible target data. MFR addresses this open-set challenge by simulating domain shifts through meta-learning and evaluates it on two new benchmarks.
- Face recognition models often assume similar training and testing distributions, but deployment may involve target domains with different distributions.
- Generalized face recognition requires direct generalization to unseen domains without model updating or fine-tuning.
- Unlike closed-set domain generalization, generalized face recognition combines domain gaps with disjoint source and target identity classes.
- MFR simulates source-target domain shifts and optimizes representations on synthesized source and target domains through meta-learning.
- The paper introduces two generalized face recognition benchmarks and reports extensive experiments validating MFR's efficacy.
2. Related work
Prior domain-generalization methods mainly address closed-set visual recognition, whereas generalized face recognition must handle unseen domains and disjoint identity classes. Meta-learning is therefore positioned as a related strategy, but the paper distinguishes MFR by its task and method.
- Domain Generalization: Existing domain-generalization methods mainly focus on closed-set recognition where source and target domains share the same label space.
- Domain Generalization: Generalized face recognition is more challenging because target classes are disjoint from source classes, making it an open-set problem.
- Meta Learning: MLDG applies meta-learning to domain generalization, while broader meta-learning work also studies fast adaptation and direct classification on new tasks.
- Domain Generalization: The paper states that DIMN differs from MFR in both task and method.
3. Methology
MFR trains face representations for unseen-domain deployment by simulating domain shifts through meta-train/meta-test splits and combining multi-domain losses with meta-optimization. Its training pipeline samples domains, learns domain-invariant and discriminative representations, and updates the model using gradients from both synthesized source and target domains.
- Domain-level sampling: MFR uses domain-level sampling to split source domains into meta-train and meta-test sets, simulating deployment on an unseen domain.Each meta-batch iterates over source domains, holding one out for meta-test while sampling identity pairs from the remaining domains and the held-out domain.
- Multi-domain distribution optimization: Three losses jointly learn hard-pair, batch-level classification, and domain-aligned representations across multiple domains.Hard-pair attention focuses on difficult positive and negative pairs, soft classification models the batch-level distribution, and domain alignment brings domain centers closer.
- Multi-domain distribution optimization: The hard-pair attention loss constructs a similarity matrix from normalized gallery and probe embeddings and filters hard pairs using positive and negative thresholds.The operation has O(B^2 log(B)) complexity, with P indexing hard positives and N indexing hard negatives.
- Multi-domain distribution optimization: The soft-classification loss performs within-batch classification using gallery and probe logits against normalized class weights initialized from their averaged embeddings.The loss complements hard-pair optimization, which can otherwise focus on local optima.
- Meta-optimization: For three source domains, MFR accumulates gradients and meta-gradients from all three leave-one-domain-out divisions before updating the model.The divisions are D2,D3/D1, D1,D3/D2, and D1,D2/D3.
- Meta-optimization: Meta-optimization updates the model so that a meta-train gradient step is followed by strong performance on the meta-test domain.The objective combines meta-train and meta-test losses, with the meta-test term supplying high-order meta-gradients that regularize the update.
4. Experiments
Experiments evaluate MFR on two proposed generalized face-recognition benchmarks spanning racial and facial-variety shifts, with unseen target domains withheld during training. MFR generally outperforms baselines across protocols, while ablations show the importance of meta-gradients and balanced meta-train/meta-test learning.
- Benchmarks: Two benchmarks, GFR-R and GFR-V, evaluate generalization across racial and facial-variety shifts with target domains unseen during training.GFR-R uses racial domains, while GFR-V evaluates CACD-VS, CASIA NIR-VIS 2.0, Multi-PIE, MeGlass, and Public-IvS.
- GFR-R Comparisons: MFR achieves the best result on all four GFR-R protocols among the compared settings and methods.The comparisons include a pretrained base model, domain aggregation baselines, fine-tuning variants, and an adapted MLDG method.
- GFR-V Comparisons: 95.97% (81.92%) improves over 89.89% (69.27%) for Base when FAR=0.1% (0.01%) on CASIA NIR-VIS 2.0.MFR had not seen NIR samples during training yet was comparable to CNN methods fine-tuned on the target NIR-VIS dataset.
- GFR-V Comparisons: 80.86% (66.15%) improves over 71.96% (53.5%) on the hardest MeGlass IV protocol at FAR 0.001%.This result is slightly better than a competitor trained using synthesized eyeglass images across MS-Celeb.
- Ablation Study: Removing the meta-gradient lowers GFR-R I performance from 71.4% to 68.35% when FAR=0.01%, making it the most important of the evaluated components.The other evaluated components are hard-pair attention, soft classification, and domain alignment losses.
- Ablation Study: The first-order approximation uses about 82% of GPU memory and 63% of the time of the high-order implementation while achieving close performance.The authors describe it as a practical substitute for the computationally expensive high-order meta-gradient.
5. Conclusion
The paper frames generalized face recognition as deployment on unseen target domains without model updates and proposes MFR to address it. Experiments on two newly defined benchmarks validate MFR’s effectiveness.
- MFR is designed for direct deployment on target domains after training on source domains, without model updating.
- The paper introduces generalized face recognition as an important problem for practical applications.
- Two newly defined generalized face recognition benchmarks are used to evaluate MFR.
- Extensive experiments on the two benchmarks validate the effectiveness of MFR.