Face Quality Estimation and Its Correlation to Demographic and Non-Demographic Bias in Face Recognition

1. Introduction

• Biometric sample quality is the utility of a sample for recognition, which makes quality assessment important to recognition applications.Recognition performance is driven by capture quality.Source: §1 · p. 1
• Adapting quality assessment to a deployed recognition model can improve accuracy and robustness but may also introduce bias into quality estimates.Source: §1 · p. 1
• Transferred demographic bias in quality estimates can increase discriminatory decisions in enrolment and quality-based fusion, while non-demographic bias need not have discriminatory consequences.Source: §1 · p. 1
• Face quality assessment scores lower quality for classes affected by recognition bias, revealing a strong correlation between bias in recognition and quality estimation.The analysis covers two recognition systems, four quality assessment algorithms, and pose, ethnicity, and age classes.Source: §1 · p. 2
• The study frames low quality in controlled captures as a possible warning of faulty recognition decisions that may warrant manual inspection.Source: §1 · p. 2

2.1. Bias in face recognition

• Face recognition performance varies with demographic attributes, motivating efforts to reduce differential performance across groups.Prior work reports these patterns in commercial and open-source systems.Source: §2.1 · p. 2
• Bias mitigation approaches include cost-sensitive learning, range loss, adversarial learning, margin-based methods, and data augmentation.Prior methods have reduced decision bias, but further reduction remains an active research topic [28].Source: §2.1 · p. 2
• Liang et al. learn disentangled representations via one-vs-rest and additive adversarial learning; Wang et al. use reinforcement learning to choose margins minimizing ethnic bias, while Huang et al. address unbalanced training data.Source: §2.1 · p. 2

2.2. Face quality assessment

• Face image quality standards distinguish image-based factors such as illumination from subject-based factors such as pose and expression.Source: §2.2 · p. 2
• Early quality estimators used manually specified image factors, while later work turned to learning-based approaches because recognition-relevant characteristics may be difficult to specify manually.Source: §2.2 · p. 2
• Learned quality methods include score-based labels, relative quality, and FaceQnet, while SER-FIQ estimates quality from embedding variation in random subnetworks.SER-FIQ can use the deployed recognition model without training.Source: §2.2 · p. 3
• Recognition-adapted quality estimation can improve quality estimates, but may transfer recognition bias, motivating this study of their relationship and practical implications.Source: §2.2 · p. 3

3.1. COTS

• COTS is a 2019 off-the-shelf Neurotechnology product whose working principles are not disclosed.Its predicted quality synchronizes more with FaceNet performance than with ArcFace performance.Source: §3.1 · p. 3

3.2. Best-Rowden

• Best-Rowden estimates quality with a support vector regressor trained on deployed-model embeddings and matcher-dependent comparison-score labels.The experiments train this approach on MORPH.Source: §3.2 · p. 3

3.3. FaceQnet

• FaceQnet adapts comparison-score quality labels to a ResNet-based network trained on FaceNet embeddings and VGGFace2.Prior work found better synchronization with ArcFace performance despite FaceNet training, indicating some overfitting to FaceNet embeddings.Source: §3.3 · p. 3

3.4. SER-FIQ

• SER-FIQ estimates quality from embedding variation across random subnetworks of the deployed recognition model, avoiding quality labels and training.Greater variation indicates lower robustness and lower sample quality; the experiments repeat dropout on the network’s last layer.Source: §3.4 · pp. 3–4

4. Experimental setup

• ColorFERET provides 14,000 images of 1,200 people under controlled conditions, while Adience contains over 26,500 images of over 2,200 people in unconstrained settings.Both include identity, gender, and age information; ColorFERET also includes ethnicity and pose labels.Source: §4 · p. 4
• The quality estimators were trained or developed using MORPH, VGGFace2, and MS1M, datasets with differing demographic, age, and pose distributions.Source: §4 · p. 4
• Quality estimation is evaluated with error-versus-reject curves, which track verification error as the lowest-quality images are excluded.Consistently decreasing verification error as more images are rejected indicates good quality estimation.Source: §4 · p. 4
• Recognition bias is assessed by subgroup FNMR at a fixed 0.1% FMR, and quality-bias correlation by subgroup proportions across quality thresholds and subgroup quality distributions.Source: §4 · p. 4
• The study extracts FaceNet and ArcFace embeddings after image alignment, scaling, and cropping, then compares embeddings using cosine similarity.Both recognition models were trained on MS1M.Source: §4 · p. 4
• The analysis first evaluates quality estimation, then identifies recognition bias across pose, ethnicity, and age, and finally examines quality-score separation between affected and unaffected classes.Source: §4 · p. 4

5.1. Face quality assessment performance

• SER-FIQ has the best quality-estimation performance in all scenarios; COTS performs better under constrained capture, and FaceQnet is solid across cases.Performance is reported as FNMR at FMR 0.1%.Source: §5.1 · p. 5, Figure 1 · p. 6

5.2. Identifying biases in pose, ethnicity, and age

• Profile-view errors exceed the next-highest pose class by more than 10 times; Asian and Black faces and children under 7 also show higher errors, while ArcFace has lower errors overall.Both FaceNet and ArcFace nevertheless show high bias toward certain classes.Source: §5.2 · p. 5, Table 1 · p. 5

5.3. The correlation study - bias versus quality

• The subgroup-proportion analysis treats stable proportions across rising quality thresholds as evidence of an unbiased estimator; affected classes shrink when assigned lower qualities.The analysis covers pose, ethnicity, and age for FaceNet and ArcFace embeddings.Source: §5.3 · p. 5, Figure 2 · p. 6, Figure 3 · p. 7
• COTS quality assignments reflect pose and age bias: profile images and young age classes receive lower qualities, while ethnicity proportions largely stagnate.The profile pattern appears with both FaceNet and ArcFace embeddings.Source: §5.3 · p. 5
• ArcFace embeddings show a strong bias toward Black faces, while age classes under 12 years receive lower quality estimates; despite MORPH being 80.4% Black, utilized embeddings have the major influence.Source: §5.3 · p. 6
• FaceQnet reduces the proportions of pose- and age-affected classes at higher thresholds; its quality distributions also indicate ethnicity bias, with Asian scores among the highest.The authors trace its bias more to the utilized embeddings than to its VGGFace2 training data.Source: §5.3 · pp. 6–7
• SER-FIQ's quality estimates reflect the deployed model's decision patterns, reducing the proportions of classes affected by recognition bias as thresholds rise.The authors distinguish non-demographic bias transfer from demographic transfer, which they characterize as discriminatory.Source: §5.3 · p. 7
• Across evaluated quality estimators, classes affected by recognition bias receive lower quality assignments; the authors attribute this bias transfer to the utility-based definition of face quality.They identify developing estimators that do not adapt demographic bias, or mitigating bias in recognition models, as possible directions.Source: §5.3 · p. 7

• The investigation used two public databases, four state-of-the-art quality-assessment algorithms, and two face-recognition systems.Source: §6 · p. 8
• Face-image quality correlates with demographic and non-demographic bias, and current quality-assessment methods already adapt that bias.Source: §6 · p. 8
• Quality-based enrollment and fusion also carry the transferred bias.Source: §6 · p. 8
• Possible responses are quality assessment designed to prevent demographic bias transfer or bias-mitigating recognition models.Source: §6 · p. 9