PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-Quality Face Recognition using Center Aligned Representations and Local Margin Constraints

Vedat Can Dilaver, Benjamin S. Riggan

arXiv:2609.01014v1cs.CV

TL;DR

Low-quality face recognition is challenged by degraded imagery, scarce low-quality training data, and the tension between low-quality adaptation and high-quality generalization. The paper combines Local Probability Margin, Nested Attention Module, and Quality Gating Protocol, achieving competitive performance across high-quality, mixed-quality, and surveillance benchmarks. The framework therefore provides a unified quality-adaptive approach, while its margin and residual-scale choices remain empirical and its gating has limited utility on uniformly low-quality datasets.

  • Problem

    Low-quality face recognition suffers from degraded imagery, scarce low-quality data, and domain shift that can make HQ-pretrained models over-adapt when fine-tuned.

  • Method

    The framework combines sample-adaptive Local Probability Margin, self-attention within a low-rank Nested Attention Module, and quality-gated adapter residuals.

  • Results

    The framework achieves competitive performance across high-quality, mixed-quality, and surveillance benchmarks, with +0.77% and +0.52% TPR@FPR=10^-4 gains on IJB-B and IJB-C.

  • Takeaways & Limitations

    A single parameter-efficient model can adapt across a broad quality spectrum while preserving the frozen pretrained representation for high-quality inputs.

  • Takeaways & Limitations

    LPM does not consistently improve all benchmarks, its margin offset and residual scale are selected empirically, and QGP provides little signal on uniformly low-quality datasets.

Abstract

from arXiv · show

Low-quality face recognition (LQFR) remains challenging due to the difficulty of matching degraded query (probe) images against low-quality (LQ) enrollment (gallery) imagery and the scarcity of training data for large-scale models. While recent face recognition (FR) models perform well on high-quality (HQ) imagery, their accuracy drops significantly on LQ images with extremely low signal-to-noise ratio (SNR). Moreover, fine-tuning HQ-pretrained models on LQ data often improves LQ recognition at the expense of HQ generalization. This trade-off becomes more pronounced in modern evaluation settings spanning multiple datasets with varying image quality levels. To address these limitations, we propose a unified framework that combines three main components: (1) Local Probability Margin (LPM), which estimates per-sample difficulty directly from the model's discriminative landscape; (2) Nested Attention Module (NAM), a new low-rank adapter module that embeds a self-attention mechanism within selected transformer layers; and (3) Quality Gating Protocol (QGP), where an off-the-shelf image quality estimator modulates the adapter contribution at test time, enabling a single model to handle the full quality spectrum without sacrificing HQ performance. Experiments on surveillance (TinyFace, SurvFace) and standard (IJB-B, IJB-C) face recognition benchmarks demonstrate consistent gains in both identification and verification. Code and models will be released at github.com/candllq/nam.

1. Introduction

Low-quality face recognition is hindered by degraded imagery, scarce low-quality training data, and domain mismatch between standard and surveillance datasets. The proposed framework combines adaptive margins, attention-based parameter-efficient adaptation, and quality-aware inference control.

  • Motivation: Surveillance face recognition remains difficult because low-quality images have reduced interocular resolution and limited training examples.Surveillance protocols use low-resolution gallery and probe imagery, while large-scale datasets primarily contain higher-quality images.
  • Motivation: 25–50% lower Rank-1 identification accuracies are reported for TinyFace and SurvFace than for standard benchmarks.The comparison concerns recent methods evaluated on surveillance versus standard face-recognition datasets.
  • Motivation: Fine-tuning HQ-pretrained models on scarce LQ data can over-adapt to degraded image characteristics and risk weaker generalization.The paper frames this as a consequence of domain shift and the imbalance between available low- and high-quality training data.
  • Proposed framework: Local Probability Margin estimates sample difficulty from the probability distribution defined by the actual and nearest negative class centers.The method uses local angular relationships in the discriminative landscape to adjust each sample's margin.
  • Proposed framework: Nested Attention Module embeds lightweight self-attention inside a low-rank adapter for token-aware, context-dependent adaptation of frozen pretrained models.The attention mechanism enables token interactions while retaining a compact parameter budget.
  • Proposed framework: Quality Gating Protocol uses an off-the-shelf quality estimator to multiplicatively gate adapter residuals, strengthening adaptation for low-quality inputs.The protocol is designed to let one model adapt across image-quality conditions at inference time.

2. Related Work

Prior LQFR work centers on super-resolution, embedding alignment, adaptive margins, and parameter-efficient adaptation. The proposed framework combines token-aware attention, local class-center margins, and quality-gated residual adaptation to address limitations in these approaches.

  • 2.1. Low Quality Face Recognition: Super-resolution reconstructs HQ faces but can suffer from distribution shift and identity loss because LQ-to-HQ reconstruction is ill-posed.A single LQ image may correspond to multiple valid HQ representations, requiring additional identity-preserving constraints.
  • 2.1. Low Quality Face Recognition: Embedding alignment brings HQ and LQ representations into a shared latent space using soft predictions, intermediate features, or parameter weights.These methods commonly use transfer learning, knowledge distillation, or domain adaptation between source and target networks.
  • 2.2. Margin-based Classifiers for Face Recognition: Adaptive margin methods estimate sample quality from training progress or embedding norms, while local-margin methods instead use angular relationships among nearby class centers.The proposed local formulation defines margin terms from distances between the actual and nearest-negative class centers and uses a broader local neighborhood.
  • 2.3. Parameter-Efficient Fine-Tuning.: PEFT updates few parameters to reduce catastrophic forgetting and overfitting, but conventional adapters and LoRA variants lack inter-token information exchange.Existing low-rank methods apply fixed linear corrections independently to each token, while standard adapter modules treat spatial tokens identically.
  • 2.3. Parameter-Efficient Fine-Tuning.: NAM embeds lightweight self-attention within a low-rank adapter, redistributing capacity across tokens with a parameter budget comparable to standard LoRA.Shared projections produce query, key, and value representations, enabling context-dependent adaptation based on inter-token affinity.
  • 2.4. Image Quality Assessment Methods.: QGP uses an estimated image-quality score at inference to strengthen adapter residuals for LQ images while preserving the pretrained representation for HQ images.This inference-time quality control is presented as a way for one model to cover a broad quality spectrum without retraining or model switching.

3. Methodology

The framework adapts a ViT at both the loss and representation levels, using local probability structure for sample-adaptive margins and token-aware residuals for low-quality inputs. At inference, quality scores gate the adapter so one model can address varied image quality.

  • The framework combines Local Probability Margin, Nested Attention Module, and Quality Gating Protocol modifications across training and inference.LPM changes per-sample margins, NAM augments selected transformer projections, and QGP scales NAM using image quality.
  • Local Probability Margin: LPM estimates embedding quality from the local probability distribution formed by the positive and nearest negative class centers.The local neighborhood contains the k negative classes with largest cosine similarities, and its size is selected by ablation.
  • Local Probability Margin: Approximately 0.5 versus 0.10: local probability correlates more strongly with SER-FIQ scores than embedding norm on 1000 CASIA-WebFace samples.The local probability is therefore used as a local discriminability metric and training-time proxy for image quality.
  • Local Probability Margin: During LQ fine-tuning, the additive-only margin preserves the HQ-learned angular structure and weakens updates from highly ambiguous samples.The monotonic additive margin increases penalties for locally reliable LQ samples while reducing them for highly ambiguous ones.
  • Nested Attention Module: NAM embeds inner self-attention in a shared low-rank subspace to produce input-dependent, context-aware corrections for query and value projections.It redistributes adapter capacity across tokens, addressing nonuniform degradation of local facial regions.

4. Experimental Results

The experiments evaluate LPM, NAM, and QGP across high-, mixed-, and low-quality benchmarks, using frozen-backbone adaptation and quality-aware testing. Results show strong surveillance performance with NAM+LPM and additional gains from QGP on mixed-quality benchmarks.

  • Evaluation Protocol: LPM, NAM, and QGP are evaluated under distinct settings spanning training from scratch, low-quality fine-tuning, and full-spectrum testing.LPM is evaluated for high-quality training and low-quality fine-tuning, whereas NAM and QGP are evaluated during low-quality fine-tuning across quality levels.
  • Evaluation Protocol: The low-quality fine-tuning setup freezes the pretrained backbone and updates only adapter modules and the final classifier head.The backbone is pretrained on high-quality images, while remaining parameters stay frozen during low-quality training.
  • Evaluation Protocol: The benchmarks include HQ datasets, mixed-quality IJB-B and IJB-C, and surveillance datasets TinyFace and SurvFace, with verification and identification metrics reported.Reported measures include verification accuracy, TAR@FAR, Rank-1, Rank-5, and SurvFace TPIR@FPIR.
  • Benchmark Results: 75.95% TinyFace Rank-1 is achieved by NAM+LPM with under 200k trainable parameters and a fully frozen ViT-B.NAM+LPM reaches this result while matching or exceeding full-network LQFR methods; wider ARoFace and DArFace backbones remain at or below 75.7%.
  • Benchmark Results: 2.95%/3.44%/2.38% relative gains over DoRA are reported by NAM+LPM on SurvFace at the 30%/20%/10% open-set operating points.NAM+LPM achieves the best results across all five reported SurvFace metrics.
  • Benchmark Results: +0.77% and +0.52% improvements in IJB-B and IJB-C TPR@FPR=10^-4 demonstrate the strongest QGP effect on mixed-quality benchmarks.TinyFace and SurvFace are unchanged because QGP is disabled on uniformly low-quality data.

5. Conclusion

The paper concludes that LPM, NAM, and QGP form a unified approach to low-quality face recognition, achieving competitive performance across high-quality, mixed-quality, and surveillance benchmarks. It also identifies empirical design choices and dataset- or operating-condition dependence as limitations and future-work directions.

  • Conclusion: The framework combines LPM for per-sample difficulty estimation, NAM for attention-based low-rank adaptation, and QGP for inference-time residual modulation.LPM uses class-center angular relationships, NAM embeds lightweight self-attention, and QGP modulates adapter contribution at inference.
  • Conclusion: Competitive performance is reported across HQ, mixed-quality, and surveillance benchmarks.The conclusion presents this as the framework’s overall supported outcome.
  • Limitations and Future Work: LPM does not consistently improve all benchmarks, while its margin offset and NAM’s residual scale are selected empirically.The authors state that LPM is most beneficial in uniformly degraded domains and at strict-FAR operating points.
  • Limitations and Future Work: Future work includes testing QGP with broader FIQA estimators and extending comparisons to large-scale multimodal low-quality databases.CAS-AIR-3D is identified as an example of a target database.
Loading 2609.01014v1…