Source-linked AI summary
QMagFace: Simple and Accurate Quality-Aware Face Recognition
Philipp Terhörst, Malte Ihlefeld, Marco Huber, Naser Damer, Florian Kirchbuchner, Kiran Raja, Arjan Kuijper
TL;DR
Unconstrained face recognition must handle image variations that can lead to incorrect matches, while prior approaches may omit quality information or use unsuitable estimates. QMagFace combines MagFace-based recognition with model-specific quality-aware comparison. Across reported databases and benchmarks, quality-awareness consistently improves recognition, with particularly strong performance in challenging conditions.
Problem
In unconstrained settings, pose, illumination, and expression variations can lead to incorrect matches, while prior methods either omit face-image quality during comparison or use unsuitable quality estimates.
Source: §1 · p. 1Method
QMagFace combines a MagFace-trained recognition model with comparison scoring that incorporates model-specific face-image quality through a linear quality–score relationship.
Source: §1 · p. 1, §3.1 · p. 3Results
Across four databases and six benchmarks, QMagFace consistently improves over standard comparison scores and achieves state-of-the-art results on three of four image-based benchmarks.
Source: §1 · p. 2Takeaways & Limitations
QMagFace performs especially strongly under cross-pose, cross-age, and cross-quality conditions, and the linear quality weighting generalizes across backbones, training databases, and evaluation benchmarks.
Source: §1 · p. 2, §7 · p. 9Improvements decrease at very low false-match rates such as 10^-7, and video recognition needs more suitable quality aggregation.
Source: §6 · p. 7
Abstract
from arXiv · showhide
Face recognition systems have to deal with large variabilities (such as different poses, illuminations, and expressions) that might lead to incorrect matching decisions. These variabilities can be measured in terms of face image quality which is defined over the utility of a sample for recognition. Previous works on face recognition either do not employ this valuable information or make use of non-inherently fit quality estimates. In this work, we propose a simple and effective face recognition solution (QMagFace) that combines a quality-aware comparison score with a recognition model based on a magnitude-aware angular margin loss. The proposed approach includes model-specific face image qualities in the comparison process to enhance the recognition performance under unconstrained circumstances. Exploiting the linearity between the qualities and their comparison scores induced by the utilized loss, our quality-aware comparison function is simple and highly generalizable. The experiments conducted on several face recognition databases and benchmarks demonstrate that the introduced quality-awareness leads to consistent improvements in the recognition performance. Moreover, the proposed QMagFace approach performs especially well under challenging circumstances, such as cross-pose, cross-age, or cross-quality. Consequently, it leads to state-of-the-art performances on several face recognition benchmarks, such as 98.50% on AgeDB, 83.95% on XQLFQ, and 98.74% on CFP-FP. The code for QMagFace is publicly available
1. Introduction
Unconstrained face-recognition conditions can make matching unreliable, while existing approaches either omit image quality or use estimates not inherently suited to comparison. QMagFace combines MagFace-trained recognition with model-specific quality-aware scoring, improving separation and recognition performance, especially in challenging settings.
1. Introduction
- Face-image quality measures a sample’s utility for recognition, but prior approaches either omit it during comparison or use estimates not inherently suited to that task.In Figure 1, quality-aware scoring increases the separability of genuine and imposter distributions despite similar quality distributions.Source: §1 · pp. 1–2, Figure 1 · p. 1
- QMagFace combines a quality-aware comparison function with a face-recognition model trained using the magnitude-aware angular margin (MagFace) loss.The scoring function exploits the linear relationship between image qualities and comparison scores induced by the loss.Source: §1 · pp. 1–2
- Across four databases and six benchmarks, QMagFace consistently improves over standard comparison scores and achieves state-of-the-art results on three of four image-based benchmarks.It is especially effective under cross-pose, cross-age, and cross-quality conditions.Source: §1 · p. 2
2. Related Works
The related work progresses from general and model-specific quality assessment to recognition losses that encode quality, then reviews quality-aware comparison methods. QMagFace’s distinction is its end-to-end use of model-specific quality estimates through a linear comparison weighting.
2.1. Face Image Quality Assessment
- Third-generation FIQA methods use recognition networks to produce model-specific quality; MagFace encodes quality in embedding magnitude and yields a linear quality–comparison-score relationship.SER-FIQ instead measures representation robustness to dropout and has a complex relationship between quality and comparison scores.Source: §2.1 · p. 2
2.2. Face Recognition
- MagFace uses a magnitude-aware angular margin and regularization to encode sample utility in embedding magnitudes, unlike fixed-margin losses that assume equally distributed samples.ElasticFace samples random margins, while CurricularFace adapts margins over training to address easy and hard samples.Source: §2.2 · pp. 2–3
2.3. Quality-Aware Face Recognition
- Prior quality-aware recognition methods include quality-weighted video aggregation, computationally expensive non-end-to-end EQFace training, and PFE’s separately trained uncertainty estimates.QMagFace instead linearly incorporates model-specific quality estimates end-to-end.Source: §2.3 · p. 3
3. Methodology
QMagFace combines model-specific face quality with MagFace-based recognition, adjusting comparison scores to account for quality. Its two-parameter linear weighting function is trained across a target false-match-rate range.
3. Methodology
- QMagFace combines a MagFace-trained recognition model with a quality-aware comparison function using model-specific face image qualities.The approach targets consistent improvement, especially for cross-pose, cross-age, and cross-quality comparisons.Source: §3 · p. 3
3.1. Quality-Aware Comparison Scoring
- The comparison function assumes a linear relationship between image quality and comparison scores, with adjustments dependent on the pair’s lower quality.It assumes only comparisons involving at least one low-quality sample need adjustment; the model-specific qualities are not readily interchangeable with other quality estimates.Source: §3.1 · pp. 3–4
- Comparisons with similar scores but different minimum qualities receive different adjustments, with the higher-quality pair receiving the stronger adjustment and lower quality-aware score.The approach assumes high similarity scores reflect two high-quality samples and need no quality-based adjustment.Source: §3.1 · p. 4
3.2. Training the Quality-Weighting Function
- Training separates genuine and imposter comparisons with each pair’s minimum quality, then defines an optimal quality weight for each threshold to minimize FNMR.The quality-aware score is sigmoid-scaled to [0, 1].Source: §3.2 · p. 4
- The training thresholds cover FMRs from 10−2 to 10−5, and the quality weight is modeled as a linear function with two learned parameters, α and β.The threshold range is chosen to cover varied applications while accounting for the available training data.Source: §3.2 · p. 4
4. Experimental Setup
The experiments use six benchmarks and four face-recognition datasets, evaluate performance with verification metrics, and test QMagFace across MagFace models and multiple investigations.
4.1. Databases and Benchmarks
- Six benchmarks cover standard verification and cross-age, cross-pose, cross-quality, and video-based recognition settings, while four datasets broaden threshold coverage.The benchmarks include LFW, AgeDB-30, CFP-FP, XQLFW, IJB-B, and IJB-C; the datasets are Morph, LFW, ColorFeret, and Adience.Source: §4.1 · pp. 4–5
4.2. Evaluation Metrics
- Evaluation reports FNMR at fixed FMR, EER, and ROC AUC, plus benchmark verification accuracy for comparison with prior work.Performance is reported over a wide range of FMRs to cover potential applications.Source: §4.2 · p. 5
4.3. Face Recognition Models
- QMagFace uses three pretrained MagFace models trained on MS1MV2, with iResNet-18, iResNet-50, and iResNet-100 backbones.The quality-weighting parameters are trained on Adience; the authors state that training is robust and database choice affects performance minimally.Source: §4.3 · p. 5
4.4. Investigations
- The investigations compare six-benchmark performance with ten recent methods, examine improvements across FMRs, and analyze quality-weight robustness and generalizability.The benchmark comparison includes image- and video-based recognition tasks.Source: §4.4 · pp. 5–6
5. Results
QMagFace improves recognition across image and video benchmarks, with particularly strong results on challenging cross-age, cross-pose, and cross-quality tasks. The full analysis supports broad gains across backbones, while video-level quality estimates and a few challenging comparisons remain limitations.
5.1. Performance on Single-Image Benchmarks
- 98.50% on AgeDB, 98.74% on CFP-FP, and 83.95% on XQLFW are state-of-the-art results for QMagFace-100 on challenging image benchmarks.On LFW, QMagFace-100 reaches 99.83%, close to the 99.85% state of the art.Source: §5.1 · p. 6, Table 2 · p. 6
5.3. Full Performance Analysis
- QMagFace-18 improves recognition in 19 of 21 scenarios and QMagFace-50 in 20 of 21; QMagFace-100 improves performance in all investigated cases.The three decreases occur on ColorFeret, which includes many challenging frontal-to-profile comparisons; lower-performing backbones also estimate quality less accurately.Source: §5.3 · pp. 6–7
5.2. Performance on Video-Based Benchmarks
- For video recognition, QMagFace generally reduces MagFace error and performs best at FMRs up to 10^-3, although aggregated video quality poorly reflects embedding utility.Video embeddings aggregate frame-level unitized embeddings and qualities by a quality-weighted sum.Source: §5.2 · p. 6, Table 3 · p. 7
5.4. Robustness Analysis
- Optimal quality weights follow a linear relationship with decision thresholds across the tested models and databases, and are generally similar across databases.The reported exception is MagFace-100 on LFW, where the database is too easy to train the model effectively; elsewhere, the similar functions support generalizability across databases.Source: §5.4 · p. 7
6. Limitations and Ethical Considerations
QMagFace has limits at very low false-match rates and in video quality aggregation. The authors also warn that biased quality estimates may produce discriminatory matching decisions.
Limitations
- At very low FMRs such as 10−7, QMagFace’s performance improvements decrease because quality information is most beneficial for lower-quality images.Video recognition also needs more suitable quality aggregation than the approach currently provides.Source: §6 · p. 7
Ethical Considerations
- Biased quality estimates might lead to unfair, discriminatory matching decisions depending on users’ demographic and non-demographic factors.The authors emphasize the importance of unbiased quality estimates for fair face recognition.Source: §6 · pp. 7–8
7. Conclusion
QMagFace integrates model-specific face image quality into face comparison to address challenging recognition conditions. Across ten databases and benchmarks, quality-awareness consistently improves performance, with strong results in cross-pose, cross-age, and cross-quality settings.
Conclusion
- QMagFace integrates model-specific face image quality into the comparison process to improve recognition under challenging conditions such as cross-pose and cross-age.The method combines a quality-aware comparison function with a face recognition model trained using MagFace loss.Source: §7 · p. 8
- Across ten face recognition databases and benchmarks, quality-awareness consistently increases performance, and QMagFace constantly beats state-of-the-art approaches in cross-pose, cross-age, or cross-quality settings.The approach also reaches competitive state-of-the-art recognition results overall.Source: §7 · p. 9
- Additional experiments support the suitability of linear quality weighting and show high generalizability.The conclusion reports that the approach consistently improves recognition performance.Source: §7 · p. 9
Supplementary
The supplementary material adds database context, examines quality distributions, and tests how training data affects QMagFace. These analyses report stable gains across training sources, with Adience particularly useful at low FMRs.
Supplementary
- The supplementary material covers database quality distributions, licensing and creation processes, and additional experiments using different training databases.These additions aim to clarify database selection and support the paper’s generalizability analysis.Source: Supplementary · p. 12
Quality Distributions of MagFace
- LFW and Morph have similarly high quality distributions, while ColorFeret spans a wider range for MagFace-50 and MagFace-100, partly because of pose variation.MagFace-18’s weaker quality estimation leads to similar-looking distributions and lower QMagFace-18 performance.Source: Quality Distributions of MagFace · p. 12, Figure 3 · p. 13
Additional Information on the Utilized Databases
- The supplement identifies dataset-specific provenance and licensing, including LFW’s CC-BY-4.0 basis, CFP-FP’s manually collected frontal and profile images, and XQLFW’s MIT license.IJB-B and IJB-C are described as having different Creative Commons license variants.Source: Additional Information on the Utilized… · p. 12
- The database notes document origins and usage conditions, including research-only access for AgeDB and Morph and controlled, multi-session collection for ColorFeret.The supplement also describes licensing and creation information for the utilized databases.Source: Additional Information on the Utilized… · p. 12
Additional Experiments on Various Training Data
- At low FMRs such as 10−5, Adience training yields stable improvements across evaluations, while different training datasets generally produce similar performance.The cited evaluations cover Adience, ColorFeret, LFW, and Morph.Source: Additional Experiments on Various… · p. 13, Table 6 · p. 14, Table 7 · p. 15
- QMagFace-100 improves recognition independent of training data and achieves state-of-the-art performance especially on cross-age, cross-pose, and cross-quality benchmarks.Adience training yields the weakest performance on the listed single-image benchmarks, although its quality-factor variety motivates its selection.Source: Additional Experiments on Various… · p. 13, Table 5 · p. 14