Source-linked AI summary
FaceQnet: Quality Assessment for Face Recognition based on Deep Learning
Javier Hernandez-Ortega, Javier Galbally, Julian Fierrez, Rudolf Haraksim, Laurent Beslay
TL;DR
Face recognition quality assessment needs to estimate how suitable an individual image is for recognition despite variable capture conditions and image-pair dependence. The paper trains FaceQnet using automatically generated, performance-based quality labels and evaluates it with an independent commercial comparator, finding that its quality scores track verification performance. The authors also identify dependence on ICAO-based definitions and particular recognition systems as limitations and future-improvement targets.
Problem
Face quality must predict recognition usefulness from one image even though comparison performance depends on both the gallery and probe images.
Method
FaceQnet fine-tunes a pretrained face-recognition CNN using ICAO-based image selection and FaceNet comparison scores to generate performance-based quality labels.
Results
FaceQnet quality measures reliably estimate recognition accuracy, with higher mean-quality verification samples producing lower FAR, FRR, and EER on a different dataset and comparator.
Takeaways & Limitations
FaceQnet provides a numerical, reference-free quality estimate that can support selecting higher-quality images for face recognition.
Takeaways & Limitations
The solution defines quality through ICAO conformance and remains dependent on the systems used to generate groundtruth and recognition scores.
Abstract
from arXiv · showhide
In this paper we develop a Quality Assessment approach for face recognition based on deep learning. The method consists of a Convolutional Neural Network, FaceQnet, that is used to predict the suitability of a specific input image for face recognition purposes. The training of FaceQnet is done using the VGGFace2 database. We employ the BioLab-ICAO framework for labeling the VGGFace2 images with quality information related to their ICAO compliance level. The groundtruth quality labels are obtained using FaceNet to generate comparison scores. We employ the groundtruth data to fine-tune a ResNet-based CNN, making it capable of returning a numerical quality measure for each input image. Finally, we verify if the FaceQnet scores are suitable to predict the expected performance when employing a specific image for face recognition with a COTS face recognition system. Several conclusions can be drawn from this work, most notably: 1) we managed to employ an existing ICAO compliance framework and a pretrained CNN to automatically label data with quality information, 2) we trained FaceQnet for quality estimation by fine-tuning a pre-trained face recognition network (ResNet-50), and 3) we have shown that the predictions from FaceQnet are highly correlated with the face recognition accuracy of a state-of-the-art commercial system not used during development. FaceQnet is publicly available in GitHub.
1. Introduction
Face recognition must handle variable, often unconstrained capture conditions that affect accuracy. This paper develops a deep-learning quality-assessment tool to estimate face-image suitability and support more robust enrollment.
- Face recognition can be acquired remotely, continuously, and non-intrusively, but its real-world viability depends strongly on accuracy.
- Unconstrained applications expose systems to variability that favorable, collaborative acquisition settings do not address.Face images can vary in illumination, location, background, focus, sharpness, pose, occlusion, and expression.
- A face-image quality measure estimates the expected recognition performance associated with a sample.
- A quality-monitoring tool could make enrollment more robust and predict expected biometric recognition accuracy across acquisition locations.If an acquired image is insufficiently qualified, the system can support a re-acquisition strategy.
- The paper develops a face Quality Assessment tool based on deep learning.
2. Introduction to Face Quality Measures
Face quality assessment treats image quality as a predictor of recognition utility and can be organized by groundtruth, input references, extracted features, and output. The proposed system produces a numerical, performance-related quality measure from a single input image.
- A biometric quality metric maps a sample to an estimated quality level related to its expected recognition accuracy.
- Quality may describe biometric character, acquisition fidelity, or expected recognition performance; this paper focuses on the performance-prediction interpretation.
- Performance-based groundtruth defines quality through the relationship between an input image and expected automatic face-recognition performance.
- Quality-assessment systems differ by whether they use full, reduced, or no reference information, with no-reference systems using no reference image.
- Face-quality features can describe the face, acquisition sensor, or environment, using handcrafted image-processing features or learned representations.
- Outputs may be quality ranges, compliance decisions, or numerical scores that predict expected recognition performance.
- The proposed system predicts a numerical quality measure from 0 to 1 for one input image without another reference image.Its performance-based groundtruth relates the score to expected recognition accuracy.
3. Related Works
Earlier face-quality approaches used handcrafted features, ICAO compliance tests, or CNN-derived features to estimate recognition-related quality, but important labeling burdens remained. FaceQnet builds on performance-based quality estimation while avoiding human quality labels and manual gallery selection.
- Prior work estimated face quality using handcrafted image features, ICAO compliance tests, or CNN features paired with learned output models.Approaches included DIP features, automated ICAO tests, and VGGFace features combined with an SVM.
- The BioLab-ICAO framework produced 30 individual compliance scores from 0 to 100 but did not combine them into one quality metric.
- The strongest prior approach correlated machine accuracy and human-perceived quality, with human-perceived quality reported as the more accurate predictor.
- That approach required substantial human effort for perceptual labeling and manual selection of a high-quality reference image, introducing potential human bias.
- FaceQnet instead uses automated ICAO-based gallery selection and performance-derived labels, then fine-tunes a pretrained face-recognition CNN for quality prediction.
4. Datasets
The study uses VGGFace2 for development and evaluation, alongside BioSecure for cross-dataset evaluation. These datasets provide varied acquisition conditions and separate training from testing.
- The authors split VGGFace2 into disjoint subsets for FaceQnet fine-tuning and evaluation with a commercial verification system.
- BioSecure includes samples acquired remotely, in controlled mugshot conditions, and in uncontrolled indoor and outdoor mobile-camera settings.
- The evaluation protocol applies FaceQnet across test subsets and compares quality-stratified verification performance using DET curves.
5. FaceQnet: Development
FaceQnet converts comparison performance against an ICAO-compliant reference into image-level quality labels, then learns quality regression by adapting a pretrained ResNet-50 while freezing its original weights.
- The method targets correlation between an image’s quality score and its expected face-recognition comparison accuracy.
- 5.1. Generation of the Groundtruth: Because comparison scores depend on both gallery and probe images, the method assumes a perfectly ICAO-compliant gallery image represents perfect quality.
- 5.1. Generation of the Groundtruth: For 300 VGGFace2 subjects, the highest-ICAO-score image becomes the gallery, and FaceNet comparisons with same-subject images generate normalized quality labels.
- 5.2. Regression Model and Training: FaceQnet extends a pretrained ResNet-50 by replacing classification with two fully connected regression layers producing a scalar quality output.
- 5.2. Regression Model and Training: Only the new regression layers are trained on the generated labels, while the pretrained network weights remain frozen.
- 5.2. Regression Model and Training: After training, FaceQnet outputs a 0-to-1 quality measure related to recognition accuracy and interpretable as proximity to an ICAO-compliant sample.
6. FaceQnet: Evaluation
Evaluation stratifies images by predicted quality and tests them with an unseen commercial verifier. Recognition errors decrease as mean FaceQnet quality increases across both evaluation datasets.
- The authors evaluated FaceQnet on separate VGGFace2 and BioSecure test subsets, including 100 and 140 subjects respectively.
- Each test database was divided into LowQ, MediumQ, and HighQ groups, each containing one third of the images ranked by predicted quality.
- Face++ provided mated and non-mated comparison scores, with verification reported using FAR, FRR, and EER.
- FAR, FRR, and EER decrease as the mean quality of verification samples increases.
- The quality–accuracy correlation remains clear on datasets and a comparator not used during FaceQnet training.
7. Conclusion and Future Work
The paper presents a face-quality assessment system that fine-tunes a pretrained face-recognition CNN using automatically generated quality groundtruth. Its quality measures reliably estimate recognition accuracy, while future work targets system dependence, alternative quality definitions, outliers, and more labeled data.
- The solution fine-tunes a CNN pretrained for face recognition using automatically generated quality groundtruth from comparison scores and ICAO-compliant samples.
- Quality measures extracted from VGGFace2 and BioSecure reliably estimated face-recognition accuracy.
- Using only high-quality images achieved better recognition accuracy than using lower-quality images.
- Future work should reduce dependence on particular networks and algorithms used for groundtruth generation or recognition scoring.
- The approach defines quality through ICAO-parameter conformance, motivating investigation of other quality definitions and larger labeled datasets.