ViTNT-FIQA: Training-Free Face Image Quality Assessment with Vision Transformers

1. Introduction

• Prior FIQA methods mainly use final-layer representations, while training-free alternatives typically require multiple forward passes or backpropagation.The paper identifies the potential quality relevance of intermediate-block feature stability as unexplored for FIQA.Source: §1 · p. 1
• Across most transformer-block transitions, SynFIQA distances decrease as ground-truth face-image quality increases.The authors report this pattern on a quality-labeled synthetic dataset with controlled degradation levels.Source: §1 · p. 2
• The approach needs one forward pass and no quality labels, training, custom loss, backpropagation, or architectural modifications.The paper reports competitive performance across eight benchmark datasets.Source: §1 · p. 2
• ViTNT-FIQA measures patch-level embedding changes across consecutive intermediate ViT blocks to assess face-image utility for recognition.The method compares L2-normalized embeddings and aggregates patch-level distances into quality scores.Source: §1 · p. 2, Figure 2 · p. 2

2.1. Vision Transformer Internals

• ViT blocks maintain highly similar representations and refine features incrementally, with residual connections carrying information forward through slight refinements.The cited analysis contrasts ViT’s uniform layer-wise similarity with the stage boundaries observed in ResNets.Source: §2.1 · p. 3
• Early-exit research uses intermediate blocks for inference, supporting the relevance of representations beyond their role as steps toward final outputs.Different network depths capture distinct levels of feature abstraction.Source: §2.1 · p. 3
• ViTNT-FIQA tests whether cross-block patch-embedding changes can distinguish high-utility from low-utility face samples.Its motivation draws on ViT’s smooth feature-refinement trajectory.Source: §2.1 · p. 3

2.2. Face Image Quality Assessment

• FIQA approaches include label-generation, non-FR-model, pre-trained FR-analysis, and FR-integrated methods, with different training and computational trade-offs.FR-integrated approaches incorporate quality assessment into recognition training, while pre-trained FR-analysis methods use fixed FR models.Source: §2.2 · p. 3
• Among training-free FIQA methods, ViTNT-FIQA uses a single forward pass without backpropagation, unlike approaches requiring multiple passes or backpropagation.It analyzes patch-representation stability across intermediate transformer blocks in pre-trained models.Source: §2.2 · p. 3

3. Methodology

• The method extracts patch embeddings from selected consecutive ViT blocks and L2-normalizes them to focus on directional changes rather than magnitude.The selected block indices are consecutive, and normalization makes the measurement more robust to scale variations across blocks.Source: §3 · p. 4
• For each patch, ViTNT-FIQA averages Euclidean distances between normalized embeddings from consecutive blocks to measure change across the refinement trajectory.The average cross-block distance reflects how much a patch embedding changes as it propagates through the blocks.Source: §3 · p. 4
• A monotonic transformation maps smaller patch distances to quality scores approaching 1 and larger distances to scores approaching 0.The patch score is bounded in (0, 1], with α > 0 as a scaling parameter.Source: §3 · p. 4
• Image-level quality is aggregated either by treating all patches equally or by weighting patch scores with normalized importance from last-block attention.Attention weights reflect how much each patch is attended to during recognition.Source: §3 · pp. 4–5
• The complete method runs in one forward pass, with no additional training, backpropagation, or architectural modifications.It extracts intermediate patch representations from a pre-trained ViT and aggregates their scores into an image-level estimate.Source: §3 · p. 5
• Across most block transitions, lower consecutive-block patch distances corresponded to higher ground-truth quality across SynFIQA’s 11 quality groups.The analysis covered all 550,000 images in the quality-controlled dataset.Source: §3 · p. 5

• Four pre-trained ViT face-recognition models, plus CLIP ViT-B/16, were evaluated across LFW, AgeDB-30, CFP-FP, CALFW, Adience, CPLFW, XQLFW, and IJB-C.The FR models were ViT-S/WebFace4M/Adaface [28], ViT-B/WebFace4M/Adaface [28], ViT-B/WebFace12M/Adaface [28], and FRoundation ViT-B/16; CLIP ViT-B/16 [44] was included to demonstrate applicability to a pre-trained ViT model.Source: §4 · p. 5
• EDC curves measured verification-error changes as low-quality images were discarded, using FNMR at FMR thresholds of 1e-3 and 1e-4, AUC, and pAUC up to 25% rejection.The 1e-3 threshold is described as recommended for border control, while 1e-4 is for higher-security applications.Source: §4 · p. 5
• FIQA was evaluated in cross-model settings using ArcFace [14], ElasticFace [9], MagFace [36], and CurricularFace [25], distinct from the models used to extract face features.Source: §4 · p. 5

5.1. Ablation Studies

• WebFace4M and WebFace12M models scored 0.0279/0.0351 and 0.0280/0.0368 mean pAUC-EDC, while CLIP scored 0.0363/0.0456 at FMR=1e−3/1e−4.The FR-specific models performed nearly identically; CLIP was not trained for face recognition.Source: §5.1 · pp. 5–6
• ViT-S and ViT-B scored 0.0273/0.0359 and 0.0279/0.0351, respectively, indicating minimal performance differences across network depths.The models were trained on the same WebFace4M dataset; ViT-S was better on some datasets and ViT-B on others.Source: §5.1 · p. 6, Table 2 · p. 6
• Using 16 or 20 blocks yielded 0.0262/0.0336 and 0.0266/0.0338, while all 24 blocks yielded 0.0279/0.0351 at FMR=1e−3/1e−4.With uniform aggregation, 12–20 blocks balance efficiency and performance; using blocks 0–15 can reduce computational costs while retaining near-optimal performance.Source: §5.1 · p. 6, Table 2 · p. 6
• Last-block attention at 12 blocks scored 0.0260/0.0334 versus uniform aggregation’s 0.0279/0.0351, and attention weighting aligns with quality-relevant regions without additional supervision.Averaging attention weights across blocks scored 0.0273/0.0342, comparably to last-block weighting.Source: §5.1 · pp. 6–8

5.2. Comparison with State-of-the-Art

• ViTNT-FIQA is competitive with state-of-the-art FIQA methods across four face-recognition models and eight benchmarks.Table 3 compares the method with three IQA and twelve FIQA approaches using EDC pAUCs.Source: Table 3 · p. 7, §5.2 · p. 8
• ViTNT-FIQA matches or outperforms SER-FIQ and GraFIQs across multiple datasets and models using one forward pass without backpropagation.SER-FIQ uses 100 stochastic-dropout forward passes; GraFIQs computes gradient magnitudes through backpropagation.Source: §5.2 · p. 8
• Across eight datasets and four recognition models, the EDC curves track state-of-the-art approaches, particularly on Adience, as discarding low-quality images reduces verification errors.ViTNT-FIQA is the red curve; the plotted metric is FNMR at FMR=1e−3.Source: Figure 3 · p. 6, §5.2 · p. 8

6. Conclusion

• ViTNT-FIQA assesses face-recognition utility by aggregating attention-weighted L2 distances between normalized patch embeddings from consecutive ViT blocks in one forward pass.It requires no backpropagation, architectural modifications, or quality-specific training.Source: §6 · p. 8
• Ablations show generalization across pretrained models and network depths, with subset block selection offering efficiency–performance trade-offs and attention-based patch weighting improving assessment.Attention weighting consistently improves quality assessment; uniform aggregation performs best with a subset of encoder blocks.Source: §6 · p. 8
• Intermediate ViT representations contain quality-relevant information beyond their role in producing final embeddings, supporting an efficient, immediately applicable approach to face-image quality assessment.The reported evaluations span eight benchmarks and four face-recognition models.Source: §6 · p. 8

Tables: Quantitative Evidence

• In ViT-B’s comparison of consecutive six-block windows (0–5, 6–17, 18–23), early blocks 0–5 achieve superior AUC-EDC and pAUC-EDC performance, suggesting quality-discriminative information emerges in initial processing stages.Source: Tables: Quantitative Evidence · p. 12
• The ablations identify 12–20 blocks with last-block attention weighting as the optimal configuration across four design choices.The choices cover dataset, architecture, block depth, and aggregation strategy.Source: Tables: Quantitative Evidence · p. 12
• The state-of-the-art comparison covers 15 competing methods—three IQA and twelve FIQA approaches—across four face-recognition models.It reports AUC-EDC in addition to the main paper’s pAUC-EDC comparison.Source: Tables: Quantitative Evidence · p. 12

Figures: Visual Evidence and Insights

• The early ViT-B window, blocks 0–5, has the strongest AUC-EDC and pAUC-EDC results across seven benchmarks.The comparison reports mean metrics at FMR=1e−3 and 1e−4.Source: Table 4 · p. 13
• The AUC-EDC table compares the method with three IQA and twelve FIQA approaches; XQLFW uses SER-FIQ as its labeling method.The table covers fixed FMR values of 1e−3 and 1e−4.Source: Table 6 · p. 14
• Across most ViT-S block transitions, higher-quality groups show lower distances between consecutive patch embeddings.The boxplots summarize mean distances across 11 quality groups drawn from 5.5 million SynFIQA images.Source: Figures: Visual Evidence and Insights · p. 12, Figure 4 · p. 15
• Ablations report that FR-trained models outperform foundation models, ViT-S and ViT-B differ minimally, and 12–20 blocks balance efficiency and performance.Attention weighting improves results, particularly at 12–20 blocks; gains diminish beyond 16 blocks.Source: Figures: Visual Evidence and Insights · p. 12, Figure 5 · p. 16
• EDC comparisons cover eight benchmarks and four face-recognition models at FNMR@FMR=1e−2 and 1e−4.ViTNT-FIQA is marked in red; the supplementary figures report that competitiveness holds across multiple security requirements.Source: Figures: Visual Evidence and Insights · p. 12, Figure 8 · p. 19, Figure 9 · p. 20
• Figure 10 compares normalized quality-score distributions for ViTNT-FIQA and state-of-the-art methods across evaluation benchmarks.The distributions are scaled to [0, 1] and can indicate calibration or range compression affecting quality ranking.Source: Figures: Visual Evidence and Insights · p. 12, Figure 10 · p. 21