Source-linked AI summary
SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness
Philipp Terhörst, Jan Niklas Kolf, Naser Damer, Florian Kirchbuchner, Arjan Kuijper
TL;DR
Face-quality assessment needs a recognition-relevant measure without unreliable human or comparison-score labels. SER-FIQ estimates quality from embedding robustness across dropout subnetworks, and it generally outperforms competing methods with stable performance, especially when using the deployed model.
Problem
Existing face-quality methods rely on artificial or human labels that may be poorly defined, biased, or mismatched to the deployed recognition system.
Method
SER-FIQ generates stochastic embeddings from random subnetworks of a dropout-trained face-recognition model and derives quality from their variation.
Results
Across three databases and two embeddings, SER-FIQ outperformed competing approaches in most scenarios and showed consistently stable performance; using the deployed model produced a large-margin advantage.
Takeaways & Limitations
Model-specific, unsupervised quality estimation can avoid quality-label training while capturing the deployed face-recognition model’s decision patterns and bias.
Takeaways & Limitations
The method estimates recognition-task utility rather than generic image quality, and assumes a dropout-trained recognition network for its stochastic subnetworks.
Abstract
from arXiv · showhide
Face image quality is an important factor to enable high performance face recognition systems. Face quality assessment aims at estimating the suitability of a face image for recognition. Previous work proposed supervised solutions that require artificially or human labelled quality values. However, both labelling mechanisms are error-prone as they do not rely on a clear definition of quality and may not know the best characteristics for the utilized face recognition system. Avoiding the use of inaccurate quality labels, we proposed a novel concept to measure face quality based on an arbitrary face recognition model. By determining the embedding variations generated from random subnetworks of a face model, the robustness of a sample representation and thus, its quality is estimated. The experiments are conducted in a cross-database evaluation setting on three publicly available databases. We compare our proposed solution on two face embeddings against six state-of-the-art approaches from academia and industry. The results show that our unsupervised solution outperforms all other approaches in the majority of the investigated scenarios. In contrast to previous works, the proposed solution shows a stable performance over all scenarios. Utilizing the deployed face recognition model for our face quality assessment methodology avoids the training phase completely and further outperforms all baseline approaches by a large margin. Our solution can be easily integrated into current face recognition systems and can be modified to other tasks beyond face recognition.
1. INTRODUCTION
Unconstrained face recognition suffers from acquisition, facial, and system-related variability, making sample quality important for reliable recognition. The paper proposes unsupervised quality estimation from stochastic embedding robustness and reports stable, generally superior performance without quality-label training.
- Unconstrained conditions introduce illumination, background, blur, resolution, pose, occlusion, expression, and system-bias variability that degrades recognition accuracy.
- Sample quality measures a biometric image’s utility for recognition and supports robust enrolment, attack prevention, and quality-based fusion.
- Existing quality-assessment methods use human or comparison-score labels that are poorly defined, system-mismatched, or vulnerable to low-quality-sample effects.
- SER-FIQ estimates quality without supervision by measuring the robustness of embeddings produced by random subnetworks of the deployed face-recognition model.
- Across three databases and two face-recognition systems, the approach outperformed six state-of-the-art solutions in most investigated scenarios.
- Using the deployed recognition model, the approach outperformed all baselines by a large margin and supported model-specific quality estimation.
2. Related work
Prior face-quality assessment spans standards, analytic metrics, machine-learning combinations, and end-to-end supervised approaches. SER-FIQ instead motivates representation robustness as a single-sample, model-aware quality measure that avoids misleading quality labels.
- Standards divide face quality into image-based factors such as pose and illumination and subject-based factors such as accessories.
- Analytic approaches manually model individual factors, motivating learning-based methods that combine metrics or learn quality functions.
- End-to-end methods have used multidimensional scaling, patch-based probabilistic models, ranking losses, and objective or relative quality measures.
- Prior systems include human-assessment and similarity-score labels, support-vector machines, and regression fine-tuning of face-recognition networks.
- All previous approaches require artificial or manual quality labels, which can transfer human bias, miss system-specific factors, or be biased by low-quality samples.
- The paper hypothesizes that representation robustness better measures individual-sample quality while incorporating the deployed embedding system’s biases and decision patterns.
3. Our approach
SER-FIQ estimates face quality from the stability of embeddings generated by dropout-induced subnetworks of a face-recognition model. It converts pairwise embedding variation into a bounded quality score, with stability controlled by the number of stochastic passes.
- 3.1. Sample-quality estimation: The model generates m = 100 stochastic embeddings through forward passes using different dropout patterns.
- 3.1. Sample-quality estimation: For image I, random subnetworks of model M produce stochastic embeddings whose pairwise variations define image quality.
- 3.1. Sample-quality estimation: Quality is the sigmoid of the negative mean Euclidean distance across stochastic embedding pairs, yielding q in [0, 1].
- 3.1. Sample-quality estimation: Greater embedding variation indicates lower representation robustness and quality, whereas lower variation indicates higher robustness and quality.
- 3.2. Properties: SER-FIQ targets recognition utility rather than generic image quality, so stable identity-related embeddings indicate high task-specific usefulness.
- 3.2. Properties: The sole parameter m controls estimate stability, but computation grows as O(m^2), creating a time–stability trade-off.
- 3.2. Properties: When the deployed recognition system uses dropout, the same network can estimate quality without a separate training phase while reflecting its decision patterns and bias.
4. Experimental setup
The experiments test SER-FIQ across three databases and two face-recognition embeddings in a cross-database setting, comparing it with six baseline quality-assessment solutions. Evaluation uses error-versus-reject curves at multiple verification operating points.
- Databases: ColorFeret, Adience, and LFW provide databases with varied quality for cross-database evaluation.ColorFeret is used for training, while unconstrained Adience and LFW are used for testing.
- Evaluation metrics: Performance is measured with error-versus-reject curves, reporting verification error using FNMR at fixed FMR and EER.The study reports EER, FNMR at 0.01 FMR, and FNMR at 0.001 FMR across three operating points.
- Face recognition models: The evaluation uses FaceNet and ArcFace face-recognition models, producing embeddings of 128 and 512 dimensions, respectively.Identity verification compares pairs of embeddings using cosine similarity.
- SER-FIQ settings: SER-FIQ is evaluated as both an on-top model and a same-model approach using the deployed recognition network for quality assessment.The same-model setting applies stochastic forward passes between the last two network layers.
- Baselines: The comparison includes three no-reference image-quality metrics, two academic face-quality methods, and the commercial COTS product.The six baselines are Brisque, Niqe, Piqe, Best-Rowden and Jain, FaceQnet, and COTS.
- Figures: Figure 3 compares quality distributions across databases, while Figure 4 evaluates rejection of low-quality images at 0.001 FMR.Figure 4 reports FNMR for FaceNet and ArcFace embeddings on Adience and LFW.
5. Results
Across the evaluated databases, embeddings, and operating points, SER-FIQ was generally more stable and competitive than the tested baselines. The same-model variant showed the strongest advantage, particularly when quality assessment used the deployed recognition model.
- No-reference image quality approaches: No-reference image quality metrics generally increased verification error when low-quality images were rejected, indicating that image quality alone was insufficient for generalized face quality estimation.These metrics do not cover face factors such as pose, occlusions, and expressions or biases of the recognition model.
- Best-Rowden: Best-Rowden and Jain performed well in most scenarios but produced unstable and sometimes highly incorrect quality predictions.The instability was particularly visible on Adience with FaceNet embeddings.
- FaceQnet: FaceQnet showed suitable behavior in most cases but only mediocre performance and difficulty distinguishing medium- from high-quality samples.This occurred even though its training labels were derived from the same FaceNet embeddings used in the evaluation.
- COTS: COTS performed well with FaceNet embeddings but its ArcFace quality predictions had no significant effect on recognition performance.Rejecting images with low predicted quality did not improve recognition performance for ArcFace.
- SER-FIQ (on-top model): SER-FIQ (on-top model) remained stable across all investigated scenarios and outperformed baseline approaches in most scenarios.The method was especially effective with recent ArcFace embeddings.
- SER-FIQ (same model): SER-FIQ (same model) outperformed all baseline approaches by a large margin in nearly all evaluated cases.The advantage was even stronger at the Frontex-recommended operating point of 0.001 FMR.
6. Conclusion
The paper addresses limitations of supervised face quality assessment by introducing an unsupervised method based on stochastic embedding robustness. Across cross-database experiments, it generally outperformed competing approaches, remained consistently stable, and achieved a large-margin advantage when using the deployed recognition model.
- 6. Conclusion: The proposed methodology estimates face quality from embedding variations generated by random subnetworks of a dropout-trained face recognition model.Representation robustness is used as the basis for determining sample quality.
- 6. Conclusion: Across three databases and two face embeddings, the approach outperformed other methods in most investigated scenarios and was the only consistently stable solution.The evaluation used a cross-database setting against six academic and industry state-of-the-art approaches.
- 6. Conclusion: Using the deployed face recognition model avoided the training phase completely and outperformed all baseline approaches by a large margin.This result supports estimating quality with respect to the deployed recognition model.