Source-linked AI summary
Learning to Attract and Repel: Dual Quality Margin Learning for Face Recognition (DQM-Face)
El Ouanas Belabbaci, Bhavesh Wani, Philipp Terhörst
TL;DR
Face recognition in unconstrained environments must handle severe variations, while magnitude-only quality modeling can be affected by identity-agnostic noise. DQM-Face fuses magnitude and semantic quality cues with dual attraction–repulsion margin optimization, and it consistently outperforms state-of-the-art methods while providing quality effective for FIQA.
Problem
Severe real-world variations challenge face recognition, while magnitude-based quality modeling alone may introduce identity-agnostic noise that reduces representation reliability and discrimination.
Method
DQM-Face combines magnitude-based and squeeze-and-excitation semantic quality modeling with adaptive attraction and explicit repulsion margins.
Results
DQM-Face consistently achieves the best or second-best face recognition performance across diverse image-to-image and video benchmarks, with the learned quality also achieving high FIQA performance.
Takeaways & Limitations
The learned quality signal is intrinsically aligned with the recognition objective and supports more discriminative, structured face representations.
Abstract
from arXiv · showhide
Face recognition in unconstrained environments remains highly challenging due to diverse and extreme variations encountered in real-world scenarios. To mitigate these effects, existing margin-based approaches model sample quality through feature magnitude. However, magnitude-based modeling alone is susceptible to identity-agnostic noise, which can degrade the reliability and discriminative power of learned representations. In this paper, we propose Dual Quality Margin Learning for Face Recognition (DQM-Face), a novel framework that enables refined attraction and repulsion dynamics during representation learning. Our approach unifies conventional magnitude-based quality estimation with a newly introduced semantic quality learning mechanism, realized via squeeze-and-excitation semantic attention. By jointly leveraging magnitude and semantic cues, we construct enhanced quality-aware margins that adaptively strengthen intra-class compactness through improved attraction during learning. To further enhance inter-class discrimination, we introduce a repulsion margin formulation that explicitly enlarges inter-class separation. The unified integration of semantic quality modeling with dual attraction-repulsion margin optimization results in a more structured and discriminative feature geometry. Extensive experiments on multiple challenging benchmarks demonstrate that DQM-Face consistently outperforms state-of-the-art face recognition methods. Moreover, we show that the quality learned for margin optimization is highly effective for face image quality assessment within the proposed framework, demonstrating that the learned quality signal is intrinsically aligned with the recognition objective. The code is publicly available: https://github.com/RAIB-group/DQM-Face
1 Introduction
Face recognition in unconstrained environments is challenged by severe variations that degrade representations and increase false matches. DQM-Face addresses this with dual quality modeling and adaptive attraction–repulsion margins, achieving consistently strong benchmark performance.
- Unconstrained pose, illumination, occlusion, expression, and age variations can degrade representations and increase false-match likelihood.
- Existing quality-aware margin methods motivate quality-adaptive learning, but the introduction identifies limitations in current approaches.
- DQM-Face combines magnitude-based quality estimation with squeeze-and-excitation semantic attention for identity-aware, context-dependent quality assessment.
- DQM-Face adaptively scales the attractive target margin by sample quality while applying an explicit inter-class repulsion margin.
- DQM-Face consistently achieves state-of-the-art performance across image-based and video-based benchmarks, ranking first or second across major protocols.The reported protocols include LFW, CFP-FP, AgeDB-30, CPLFW, IJB-B, and IJB-C.
- The framework fuses magnitude and semantic quality cues, uses adaptive attraction and curriculum-based repulsion, and reports consistent state-of-the-art results across challenging benchmarks.
2 Related Work
Related work established margin-based and quality-aware approaches for improving face representations, but many methods remained limited by fixed margins or insufficiently explicit quality modeling. DQM-Face combines magnitude and semantic quality cues with joint attraction–repulsion optimization.
- FIQA estimates a face image’s utility for recognition, evolving from predefined image metrics toward predictors aligned with recognition performance.
- SphereFace, CosFace, and ArcFace established normalized-embedding margins that enforce intra-class compactness and inter-class separability.
- Fixed-margin formulations treat samples equally despite differences in sample difficulty and image quality, motivating adaptive margins and optimization curricula.
- Prior methods including CoReFace and QCFace enhanced inter-class separation or content consistency, while explicit image-quality characteristics remained insufficiently addressed.
- MagFace used embedding magnitude as an implicit quality indicator for adaptive margin scaling, whereas DQM-Face adds semantic attention cues and jointly optimizes attraction and repulsion margins.
3 Methodology
DQM-Face extends margin-based face recognition by combining magnitude- and semantic-based quality modeling with adaptive attraction and explicit repulsion margins. This jointly promotes intra-class compactness, inter-class separation, and quality-aware feature geometry.
- Dual quality modeling: DQM-Face combines magnitude- and semantic-based quality estimation to derive an adaptive attraction margin for face representations.Semantic quality uses squeeze-and-excitation attention, while magnitude quality uses the feature norm.
- Geometrical interpretation: Compared with softmax overlap and ArcFace’s fixed margin, DQM-Face produces a structured feature space with greater compactness and a wider clearance zone.The geometry results from combining adaptive attraction with explicit repulsion.
- Quality-aware optimization: The quality-magnitude regularization term makes feature length encode sample quality while bounded normalization prevents extreme norms from dominating estimation.The feature norm is projected into operational bounds before being linearly mapped to a quality score.
- Dual quality modeling: The fused quality score balances semantic and magnitude-based quality cues and adaptively scales the target-class margin m1.The fusion parameter α controls the trade-off between embedding-norm reliability and semantic consistency.
- Dual margin optimization: The DQM loss adds an explicit repulsion margin m2 to push representations away from non-target class centers.Its gradient combines an intra-class pull, an inter-class push, and quality regularization enforcing correlation between feature magnitude and recognizability.
- Dual margin optimization: A curriculum schedules the repulsive margin from 0.05 to 0.10 to 0.15 across training epochs.The staged schedule supports early cluster formation, intermediate separation, and stronger final global separability while preventing gradient instability.
4 Experiments
Experiments evaluate DQM-Face on standard image- and video-based face recognition benchmarks, alongside FIQA evaluation under varied age, pose, quality, and unconstrained conditions. The study uses an iResNet-100 backbone with specified training and quality-margin settings, and Table 1 reports verification accuracy comparisons.
- Face Recognition Evaluation: Recognition experiments use LFW, CFP-FP, AgeDB-30, and CPLFW for predefined face-pair verification, plus IJB-B and IJB-C for larger-scale still-image and video evaluations.The image benchmarks cover pose, age, and image-quality variation, while IJB-B and IJB-C include still images and video frames.
- FIQA Evaluation: FIQA experiments use LFW, Adience, CPLFW, and XQLFW to represent unconstrained, age, pose, and severe image-quality challenges.Methods are compared using Error-vs-Discard curves, with FNMR measured at FMR 10^-3 and partial AUC computed through a 30% rejection rate.
- Model Configuration: The backbone is iResNet-100, producing 512-dimensional embeddings, while semantic attention uses an SE block followed by a two-layer MLP.The SE block uses reduction ratio 16 and is inserted after global average pooling.
- Hyperparameters: DQM-Face interpolates the quality-dependent target margin between 0.35 and 0.8, uses feature-magnitude bounds of 10 and 110, and assigns fusion weight 0.5.Magnitude regularization is fixed at 35.
- Results Presentation: Table 1 compares verification accuracy (%) on single-image benchmarks, highlighting the highest result and underlining the second-best result.The caption states that DQM-Face performs well across the different benchmarks.
5 Results
DQM-Face achieves consistently strong recognition and FIQA performance across challenging benchmarks, while ablations support fused quality cues and curriculum-based repulsion scheduling. Its added semantic branch incurs negligible computational overhead.
- Recognition Performance on Single-Image Benchmarks: DQM-Face ranks first or second across standard image-to-image benchmarks, including cross-pose and cross-age evaluations.
- Recognition Performance on Video-based Benchmarks: DQM-Face achieves the best or second-best performance across evaluated FARs on the IJB-B and IJB-C video benchmarks.The reported metric is True Acceptance Rate (TAR, %) at multiple False Acceptance Rate (FAR) thresholds.
- Face Image Quality Assessment Performance: DQM-Face ranks among the top FIQA methods across four datasets, achieving the best results on LFW, Adience, and CPLFW.The qsem variant achieves the highest performance on XQLFW, linking semantic quality to robustness under severe quality variations.
- Visual Quality Analysis: Grad-CAM analysis shows DQM-Face combines magnitude and semantic cues with stable activation over informative facial regions.The compared variants are qmag (α = 0), qsem (α = 1), and fused DQM-Face (α = 0.5).
- Complexity Analysis: The SE branch adds +0.099M parameters (+0.15%) and less than 0.0001 GFLOPs, keeping complexity comparable to ArcFace.The backbone has 12.149 GFLOPs and 65.156M parameters, while inference is dominated by the backbone.
- Ablation Studies: Recognition accuracy is highest at α = 0.5, supporting fusion of complementary magnitude and semantic quality cues.
- Ablation Studies: The staged curriculum schedule for m2 outperforms fixed, linear, and cosine schedules by stabilizing gradients and progressively increasing inter-class separation.The schedule first supports compact identity clusters and then enforces stronger separation.
6 Conclusion
DQM-Face jointly models magnitude-based and semantic quality through unified attraction–repulsion margin optimization for unconstrained face recognition. Across image-to-image and video benchmarks, it achieves the best or second-best performance, while its learned quality also performs strongly for FIQA.
- DQM-Face jointly models magnitude-based and semantic quality cues through unified attraction–repulsion margin optimization.
- The quality learned for margin optimization achieves high FIQA performance within the proposed framework, aligning with its recognition objective.
- Across diverse image-to-image and video benchmarks, DQM-Face consistently achieves the best or second-best face recognition performance.