ATTN-FIQA: Interpretable Attention-based Face Image Quality Assessment with Vision Transformers

Guray Ozgur; Tahar Chettaoui; Eduarda Caldeira; Jan Niklas Kolf; Marco Huber; Andrea Atzori; Naser Damer; Fadi Boutros

Source-linked AI summary

ATTN-FIQA: Interpretable Attention-based Face Image Quality Assessment with Vision Transformers

Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Jan Niklas Kolf, Marco Huber, Andrea Atzori, Naser Damer, Fadi Boutros

arXiv:2604.22841v1Apr 21, 2026cs.CVeess.IV

TL;DR

FIQA needs reliable measures of face-image recognition utility, but existing methods can require costly computation and offer limited spatial interpretability. ATTN-FIQA instead derives quality from pre-softmax attention in pretrained ViT face-recognition models with one forward pass, and evaluations report effective quality correlation, interpretable regions, and competitive benchmark behavior.

Problem
Existing FIQA methods can require multiple forward passes, backpropagation, or additional training, while their scalar scores often lack spatial interpretability.
Method
ATTN-FIQA extracts final-block pre-softmax attention from pretrained ViT-based face-recognition models, aggregates heads and patches, and computes quality without training or architectural changes.
Results
Across eight benchmarks, attention magnitudes correlate with face-image quality, provide spatial interpretability, and yield competitive performance across four face-recognition models.
Takeaways & Limitations
Pre-softmax attention can function as a single-pass face-quality proxy while indicating which facial regions contribute to quality determination.

Abstract

from arXiv · show

Face Image Quality Assessment (FIQA) aims to assess the recognition utility of face samples and is essential for reliable face recognition (FR) systems. Existing approaches require computationally expensive procedures such as multiple forward passes, backpropagation, or additional training, and only recent work has focused on the use of Vision Transformers. Recent studies highlighted that these architectures inherently function as saliency learners with attention patterns naturally encoding spatial importance. This work proposes ATTN-FIQA, a novel training-free approach that investigates whether pre-softmax attention scores from pre-trained Vision Transformer-based face recognition models can serve as quality indicators. We hypothesize that attention magnitudes intrinsically encode quality: high-quality images with discriminative facial features enable strong query-key alignments producing focused, high-magnitude attention patterns, while degraded images generate diffuse, low-magnitude patterns. ATTN-FIQA extracts pre-softmax attention matrices from the final transformer block, aggregate multi-head attention information across all patches, and compute image-level quality scores through simple averaging, requiring only a single forward pass through pre-trained models without architectural modifications, backpropagation, or additional training. Through comprehensive evaluation across eight benchmark datasets and four FR models, this work demonstrates that attention-based quality scores effectively correlate with face image quality and provide spatial interpretability, revealing which facial regions contribute most to quality determination.

I. INTRODUCTION

ATTN-FIQA addresses FIQA methods that often require extra computation or training and produce opaque scores by using pre-softmax ViT attention as a training-free, spatially interpretable quality signal.

FIQA measures how effectively a face image supports automated identity verification.
Existing approaches may require additional training or architectural modifications, while many produce scalar scores without spatial explanations.
In controlled visualizations, high-quality frontal images show focused high-magnitude attention, whereas degraded conditions show diffuse low-magnitude patterns.
The method aggregates attention across heads and patches to compute image-level quality scores, preserving magnitude differences that post-softmax normalization removes.
ATTN-FIQA uses pre-softmax attention scores from pre-trained ViT-based face-recognition models as quality indicators in a single forward pass.
Attention-based assessment provides spatial indications of which facial regions contribute most to quality determination and degradation.

II. RELATED WORK

Prior FIQA and attention-interpretability research uses supervised, unsupervised, self-supervised, visualization, and utility-proxy approaches, but existing methods often retain computational or interpretability limitations that motivate ATTN-FIQA.

Attention interpretability methods include rollout, gradient-based attribution, relevance propagation, and causal masking or perturbation analyses.
Attention distributions have also been studied as quantitative signals related to prediction reliability, overfitting, and bias in specific settings.
FIQA research spans supervised quality estimation, unsupervised approaches, and self-supervised or FR-integrated methods.
ViT-FIQA adds a learnable quality token to standard ViT backbones to predict utility scores.
Existing approaches often require multiple forward passes, backpropagation, or separate training, while their scalar predictions provide limited regional interpretability.

III. METHODOLOGY

The methodology represents face images as patch embeddings processed through transformer blocks, where multi-head self-attention computes relationships among patch tokens before subsequent feed-forward processing.

A face image is divided into non-overlapping P × P patches, producing N = HW/P^2 patch tokens.
Each flattened patch is linearly projected into a D-dimensional embedding and combined with learnable positional embeddings.
The embedded patches pass through L transformer blocks, each applying multi-head self-attention and a multilayer perceptron with residual connections.
At each block, the multi-head self-attention mechanism computes query, key, and value matrices from the input representations.
Attention heads independently process these representations before their outputs are concatenated and projected through the multi-head attention module.

B. Pre-Softmax Attention for Quality Assessment

ATTN-FIQA uses pre-softmax patch affinities because they preserve relationship magnitudes that normalization obscures, treating these attention statistics as proxies for face image quality.

Post-softmax attention normalizes patch relationships into probability distributions, potentially obscuring their raw strengths.
Pre-softmax attention preserves the magnitudes of scaled query-key relationships before normalization.
The scaled dot-product scores represent unnormalized affinities between facial patches, with larger absolute values indicating stronger positive or negative relationships.
The method is grounded in the hypothesis that pre-softmax attention reflects the input-model interaction and can serve as a face image quality proxy.

C. Attention-Based Quality Score Computation

ATTN-FIQA forms an image-level quality score by aggregating pre-softmax attention from every head and patch pair in the final transformer block. Averaging these values captures a global quality signal without additional training or architectural changes.

The method collects pre-softmax attention matrices from all heads in the final transformer block.The final block is selected because it contains refined, high-level semantic relationships relevant to face recognition.
These matrices are flattened into one vector containing all attention scores across heads and patch pairs.Flattening preserves the complete attention distribution while enabling efficient aggregation.
The image-level quality score is computed by averaging all flattened attention values.The score aggregates information across all attention heads and patch interactions.
High-quality faces produce higher average attention scores, whereas blur or occlusions produce weaker alignments and lower scores.The proposed interpretation links stronger query-key alignments with discriminative features and degraded alignments with reduced quality.

D. Interpretability Through Attention Visualization

ATTN-FIQA retains spatial interpretability despite producing a globally aggregated quality score. Inspecting raw attention matrices can identify facial regions associated with quality assessment and degradation.

Raw attention matrices can be analyzed to determine which facial regions receive strong attention.Practitioners can visualize individual heads or compute patch-level statistics from the matrices.
Occlusions may appear as reduced attention involving affected regions, while blur may produce diffuse attention patterns.These patterns provide region-level cues about how degradation affects quality assessment.

IV. IMPLEMENTATION DETAILS

The implementation evaluates ATTN-FIQA across pretrained ViT-based face-recognition models and eight benchmark datasets. It uses final-block attention to compute scores with a single-pass, low-overhead procedure.

Evaluation Benchmarks and Metrics: Figure 3 analyzes 40 images across eight datasets, with each row sorted from highest to lowest quality.Each image includes an original, pre-softmax attention heatmap, and normalized-score overlay using shared global normalization.
Pre-trained Models: ATTN-FIQA is evaluated with ViT-B and ViT-S models trained on WebFace4M using AdaFace or ArcFace.The models use 512-dimensional embeddings and process aligned 112 × 112 face images as 144 patches.
Implementation: The final score requires no additional training, fine-tuning, or architectural modification.The procedure enables deployment on pretrained ViT-based face-recognition models with minimal computational overhead.
Evaluation Benchmarks and Metrics: The eight evaluation datasets are LFW, AgeDB-30, CFP-FP, CALFW, Adience, CPLFW, XQLFW, and IJB-C.Performance is assessed with EDC curves and FNMR at fixed FMR thresholds.
Empirical Validation on SynFIQA: SynFIQA provides approximately 550,000 quality-labeled synthetic images for empirical validation of attention-quality correlation.The dataset includes controlled occlusions, blur, and downsampling generated through a two-stage pipeline.
Empirical Validation on SynFIQA: Mean attention-based scores increase across SynFIQA quality groups.This monotonic trend indicates that the scores distinguish the dataset’s quality groups.

V. RESULTS

Qualitative analyses show that attention magnitudes and spatial patterns track face-image quality across controlled degradations and diverse datasets. Ablations further examine architecture, loss, head aggregation, and score aggregation choices.

A. Qualitative Analysis: Attention Visualization: High-quality frontal faces show focused, high-magnitude attention on discriminative regions, while degraded faces show diffuse, low-magnitude patterns.The degraded conditions include occlusions, extreme poses, and face covers.
A. Qualitative Analysis: Attention Visualization: Across eight datasets, high-quality samples receive high-magnitude attention on identity-relevant features and challenging samples receive low-magnitude attention.The cross-dataset visualization uses 40 images with shared colormap and quality-score normalization.
A. Qualitative Analysis: Attention Visualization: Pre-softmax attention magnitudes encode quality-relevant information, while spatial patterns explain which facial regions contribute to predictions.Together, these findings support both quantitative quality scoring and spatial interpretation.
B. Ablation and Design Choice Analysis: The ablations vary architecture depth, training loss, head aggregation, and aggregation metric across seven benchmark datasets.They use pAUC-EDC and AUC-EDC at FMR thresholds of 1e^-3 and 1e^-4.
B. Ablation and Design Choice Analysis: Table II compares ATTN-FIQA with state-of-the-art methods using pAUC-EDC under fixed FMR settings.Lower pAUC-EDC is better, and values are scaled by a 10^3 factor for readability.
B. Ablation and Design Choice Analysis: ViT-S achieves lower pAUC-EDC error rates than ViT-B on most individual benchmarks.For example, AgeDB-30 reports 7.82/11.13 versus 9.95/15.26, while CPLFW reports 30.17/44.60 versus 50.10/68.27.
B. Ablation and Design Choice Analysis: AdaFace achieves lower mean pAUC-EDC scores than ArcFace: 32.40/43.26 versus 35.45/48.11.The reported gap is modest across most benchmarks, indicating quality information is captured under both margin-based losses.

C. Comparison to State-of-the-Art

ATTN-FIQA is competitive across benchmark conditions, with particularly strong performance on unconstrained IJB-C, while constrained distributions can limit fine-grained quality discrimination. The method also provides interpretable spatial quality information and raises demographic-bias risks requiring careful evaluation.

Benchmark comparison: ATTN-FIQA achieves particularly strong effectiveness on unconstrained IJB-C, with competitive pAUC-EDC results across multiple FR models.On IJB-C, it obtains 6.74/10.28 for ArcFace, 6.49/10.00 for ElasticFace, and 6.46/9.44 for CurricularFace at FMR 1e−3/1e−4.
Benchmark comparison: ATTN-FIQA remains competitive on constrained benchmarks involving age, pose, or quality variations but may struggle with fine-grained differences within controlled distributions.The limitation is attributed to uniform age ranges and controlled pose angles, whereas heterogeneous real-world degradations are captured more effectively.
Interpretability: Spatial interpretability reveals which facial regions contribute most to quality determination, unlike methods that provide only opaque scalar scores.This supports understanding the causes of quality degradation in practical biometric systems.
Ethical considerations: Bias in quality assessments may disadvantage demographic groups and enable discriminatory service access or more effective mass surveillance.The authors recommend diverse demographic evaluation, regular bias audits, legal oversight, consent, and human review in high-stakes decisions.

APPENDIX

The appendix supplies additional quantitative and visual evidence, including ablations, full SOTA metrics, attention visualizations, cross-dataset analyses, and EDC curves across models and benchmarks.

Ablation studies: Table III evaluates architecture depth, training loss, head aggregation, and aggregation metrics in ATTN-FIQA ablations.Concatenating all heads with mean aggregation achieves optimal performance.
Controlled degradation analysis: Figures 5 and 6 visualize original images, attention heatmaps, and quality scores under controlled conditions across architectures and losses.High-quality frontal poses show focused high-magnitude facial attention, while degraded conditions show diffuse patterns.
Error-versus-discard analysis: Figure 9 compares ATTN-FIQA with SOTA methods using EDC curves across eight benchmarks and four FR models.The curves evaluate FNMR@FMR=1e−4 and show progressive error reduction as low-quality samples are discarded.

Quality Score Distributions

The appendix compares normalized quality-score distributions and attention behavior across datasets, architectures, objectives, and aggregation configurations. These materials emphasize focused attention for high-quality faces, diffuse attention for degraded faces, and broad EDC evaluation against SOTA methods.

Quality score distributions: Quality-score distributions compare ATTN-FIQA with SOTA methods across evaluation benchmarks using normalized scores in [0, 1].The distributions show how different methods allocate quality predictions across datasets.
Training objectives: Attention-quality behavior remains similar across ArcFace and AdaFace training objectives.The visual analyses describe training-objective-agnostic behavior.
Attention visualizations: High-quality frontal faces produce focused red, high-magnitude attention, whereas degraded conditions produce diffuse blue, low-magnitude attention.This pattern appears in both ViT-S and deeper ViT-B visualizations.
Architecture depth: Attention-quality relationships remain similar across ViT-S and deeper 24-block ViT-B architectures.The appendix characterizes this as architecture-agnostic behavior applicable to ViT-based face recognition models.
EDC comparison: EDC curves compare ATTN-FIQA with SOTA methods across eight datasets and four FR models at FNMR@FMR=1e−4.The proposed method is represented by solid red lines.