PaperlayerView exact revision on arXiv

Source-linked AI summary

MagFace: A Universal Representation for Face Recognition and Quality Assessment

Qiang Meng, Shichao Zhao, Zhida Huang, Feng Zhou

arXiv:2103.06627v4cs.CV

TL;DR

Face recognition degrades as unconstrained face variability makes image quality and feature values uncertain, while existing quality and uncertainty methods have labeling, architectural, or computational limitations. MagFace learns a universal face representation whose magnitude measures quality and whose adaptive margins structure within-class features. The paper reports unified benefits for face recognition, quality assessment, and clustering, with a general framework potentially extendable to other classification tasks.

  • Problem

    Unconstrained face variability produces ambiguous features, while existing quality measures and uncertainty models may rely on error-prone labels, extra network blocks, or added computation.

  • Method

    MagFace learns a universal quality-aware embedding using feature magnitude as a label-free quality indicator and adaptive margins that organize within-class features by recognition difficulty.

  • Results

    MagFace simultaneously provides face quality for input images and unified features for face recognition and quality assessment, with experiments also demonstrating benefits for clustering.

  • Takeaways & Limitations

    MagFace bundles quality assessment with conventional face-feature computation and its magnitude-based principle may extend to other classification and object-quality tasks.

  • Takeaways & Limitations

    Prior uncertainty-based alternatives require additional computational costs or network blocks, complicating their use in conventional face systems.

Abstract

from arXiv · show

The performance of face recognition system degrades when the variability of the acquired faces increases. Prior work alleviates this issue by either monitoring the face quality in pre-processing or predicting the data uncertainty along with the face feature. This paper proposes MagFace, a category of losses that learn a universal feature embedding whose magnitude can measure the quality of the given face. Under the new loss, it can be proven that the magnitude of the feature embedding monotonically increases if the subject is more likely to be recognized. In addition, MagFace introduces an adaptive mechanism to learn a wellstructured within-class feature distributions by pulling easy samples to class centers while pushing hard samples away. This prevents models from overfitting on noisy low-quality samples and improves face recognition in the wild. Extensive experiments conducted on face recognition, quality assessments as well as clustering demonstrate its superiority over state-of-the-arts. The code is available at https://github.com/IrvingMeng/MagFace.

1. Introduction

Face recognition in unconstrained settings is challenged by variable image conditions and ambiguous low-quality features. MagFace addresses these issues by learning a quality-aware embedding whose magnitude reflects recognizability and whose within-class structure separates easy from hard samples.

  • Unconstrained face images vary in illumination, background, blur, resolution, pose, occlusion, expression, and system bias.
  • Human-designed and learned face-quality scores may not directly support feature learning because quality lacks a clear definition and labels can be error-prone.
  • Deterministic embeddings inadequately represent ambiguous low-quality or large-pose faces, increasing the risk of false recognition.
  • MagFace learns a cone-like within-class structure by keeping high-quality samples near class centers while pushing ambiguous samples toward the origin and away from centers.
  • MagFace uses feature magnitude, independent of direction, as a quality indicator that requires no quality labels and can accompany conventional face features.
  • Adaptive angular margins and sample weighting prevent overfitting to noisy low-quality faces while producing distributions suited to recognition and clustering.

2. Related Works

Prior face-recognition and quality-assessment methods use metric or classification losses, image- or subject-based quality criteria, learned labels, or uncertainty modeling. MagFace instead combines quality assessment with recognition features while avoiding the additional computational costs or network blocks required by some alternatives.

  • Deep face-recognition systems have advanced through metric-learning and classification losses, including contrastive, triplet, n-pair, angular, and margin-based objectives.
  • Traditional and learning-based quality methods estimate quality from image factors, subject attributes, or human- and similarity-assessed labels.
  • Uncertainty-based methods model face features as distributions or use dropout variation, but some require additional computational costs or network blocks.
  • ArcFace uses a fixed, quality-agnostic angular margin, which can produce unstable within-class structure when unconstrained face variability is large.

3. Methodology

MagFace extends cosine-based face recognition by optimizing feature magnitude alongside direction, using magnitude-aware margins and regularization to encode face quality. Under stated assumptions, its magnitude has a unique optimum that increases with recognition-friendly geometry, while experiments contrast this structured behavior with Softmax and ArcFace.

  • 3.1. ArcFace Revisited: ArcFace trains embeddings on a hypersphere with an additive angular margin m and scale s, but its margin is quality-agnostic.The angle θj is defined between an embedding and class center wj; the loss uses m > 0 and scaling parameter s.
  • 3.2. MagFace: MagFace optimizes unnormalized feature magnitude ai = ∥fi∥ alongside direction, preserving cosine-based inference while encoding face quality.It extends cosine-similarity losses with a magnitude-aware angular margin and regularizer, avoiding an additional uncertainty term.
  • 3.2. MagFace: MagFace assigns adaptive angular margins and regularization so high-quality samples approach class centers while lower-quality samples remain nearer class boundaries.The design uses magnitude as a proxy for quality and dynamically changes the margin with sample magnitude.
  • 3.2. MagFace: Under bounded-magnitude and convexity assumptions with sufficiently large λg, MagFace loss is strictly convex in ai and has a unique optimum.The convergence guarantee applies when m(ai) is increasing and convex, g(ai) is decreasing and convex, and ai lies in [la, ua].
  • 3.2. MagFace: The optimal magnitude increases as similarity to the class center rises and similarity to other classes falls, making magnitude a recognition-quality indicator.The paper states that convergence gives a unique solution, while monotonicity links magnitude to recognition difficulty and face quality.
  • 3.3. Analysis on Feature Magnitude: Unlike Softmax and ArcFace, MagFace produces a strong magnitude–class-center-similarity correlation, with upper-right samples representing the highest-quality faces.Softmax magnitude is nearly independent of negative loss, while ArcFace shows unstable variation for easy samples and a decreasing lower bound for quality.

4. Experiments

Experiments evaluate MagFace on face recognition, quality assessment, and clustering. MagFace improves recognition on difficult benchmarks, provides a generalizable quality signal, and consistently improves clustering over ArcFace.

  • 4.1. Face Recognition: MagFace achieves the overall best verification accuracy on LFW, CFP-FP, AgeDB-30, CALFW, and CPLFW.It surpasses ArcFace by 0.02%, 0.06%, 0.12%, 0.19%, and 0.15%, respectively.
  • 4.1. Face Recognition: MagFace remains top at nearly all FAR criteria on IJB-B and IJB-C, with larger gains over ArcFace on these challenging benchmarks.Compared with ArcFace, improvements reach 2.23%, 1.38%, and 0.24% on IJB-B and 3.61%, 0.98%, and 0.07% on IJB-C at FAR values 1e-6, 1e-5, and 1e-4.
  • 4.1. Face Recognition: Magnitude-weighted feature aggregation further improves MagFace verification when identities contain multiple images.MagFace+ improves over MagFace by up to 1.41%/0.98% at FAR=1e-6 on IJB-B/IJB-C.
  • 4.2. Face Quality Assessment: MagFace magnitude ranks face images by quality: increasing magnitudes correspond to mean faces with more detail and a shift toward frontal, distinctive appearances.Images with magnitudes around 15 are absent or very noisy, while magnitudes from 20 to 40 show progressively better facial quality.
  • 4.2. Face Quality Assessment: MagFace magnitude produces the lowest error-versus-reject curves across benchmarks for both ArcFace and MagFace evaluation features.This indicates that the magnitude-based quality measure generalizes across datasets and face features.
  • 4.3. Face Clustering: MagFace outperforms ArcFace in both F-score and NMI for every evaluated clustering-method and protocol combination.The authors attribute the improvement to a better within-class distribution, with high-quality samples nearer class centers.

5. Conclusion

MagFace learns unified features for face recognition and quality assessment by structuring within-class distributions around face quality. The paper reports theoretical and experimental support for using feature magnitude as a quality signal and suggests extensions beyond face recognition.

  • 5. Conclusion: MagFace learns unified features for face recognition and quality assessment by pushing ambiguous samples away from class centers.The authors report that this improves within-class feature distributions and enables simultaneous quality assessment.
  • 5. Conclusion: The paper suggests extending MagFace to fine-grained object recognition, person re-identification, and quality estimation for other objects.Examples include person bodies in re-identification and action snippets in activity classification.

A. Proofs for MagFace

The proof section establishes conditions under which the target angular term is well behaved and specifies requirements on the adaptive margin and regularization functions. A large identity count makes the key probability approach one.

  • Lemma 1: Under top-k correct classification, m(ai) ∈ [0, π/2], and n ≫ k, the probability that θyi + m(ai) lies in [0, π/2] approaches 1.Lemma 1 assumes uniformly distributed angles to non-target class centers.
  • Proof setup: The lemma is intended to support subsequent proofs and is especially applicable when the number of identities is very large.The paper gives 3.8M identities for MS1Mv2 as a real-world example.
  • Function requirements: MagFace requires m(ai) to be increasing and convex with derivative bounded in (0, K].These conditions define how the adaptive angular margin changes with feature magnitude.
  • Function requirements: The regularization coefficient must satisfy λg ≥ sK − g′(la).This inequality is one of the stated requirements for the MagFace functions.

A.2. Proof for Property of Convergence

The proof establishes that the loss is strictly convex in feature magnitude and therefore has a unique optimum. This optimum increases as the sample approaches its class center or becomes more separated from other classes.

  • A.2. Proof for Property of Convergence: Li is strictly convex in ai over [la, ua].The second derivative is strictly positive under the stated constraints and convexity assumptions.
  • A.2. Proof for Property of Convergence: A unique optimal feature magnitude a∗i exists within [la, ua].Strict convexity makes the optimum unique once existence is established.
  • A.2. Proof for Property of Convergence: a∗i increases monotonically as cosine-distance to its class center decreases.The result formalizes larger magnitudes for features closer to their own class centers.
  • A.2. Proof for Property of Convergence: a∗i increases monotonically as cosine-distances to other classes increase.Equivalently, increasing inter-class separation raises the optimal feature magnitude.

B.1. Training settings for Figure 3

The Figure 3 experiments use a ResNet50 backbone trained on MS1Mv2 for 20 epochs with scheduled learning-rate drops.

  • B.1. Training settings for Figure 3: ResNet50 is trained on MS1Mv2 for 20 epochs with batch size 512.The initial learning rate is 0.1 and is reduced by 0.1 every 5 epochs.
  • B.1. Training settings for Figure 3: The final training iteration contributes 512 samples for visualization.
  • B.1. Training settings for Figure 3: The magnitude-aware margin m(ai) is linear on [la, ua], while g(ai) is defined as a reciprocal-quadratic regularizer.The functions satisfy m(la)=lm and m(ua)=um.

C. Ablation Study on Margin Distributions

The ablation shows that margin distributions must balance feature separation with the treatment of hard or noisy samples. The setting (lm, um)=(0.40, 0.80) achieves the strongest reported benchmark improvement.

  • C. Ablation Study on Margin Distributions: Larger maximum margins and smaller minimum margins produce more separated feature distributions.The comparison fixes (λg, la, ua)=(35, 10, 110) and aligns the training mean margin to ArcFace's 0.5.
  • C. Ablation Study on Margin Distributions: (lm, um)=(0.40, 0.80) balances feature distribution and margins for hard/noisy samples and significantly improves benchmarks.The alternative settings (0.45, 0.65) and (0.25, 1.60) degrade recognition relative to the baseline.
  • C. Ablation Study on Margin Distributions: The setting (lm, um)=(0.25, 1.60) performs worse than baseline because hard samples receive small margins and are downweighted.

D. Extended Visualization of Figure 6

The extended visualization links feature magnitude with face quality across IJB-C examples and supports using magnitude for filtering or quality labeling. The Mag-CosFace appendix states analogous convergence properties for its loss.

  • D. Extended Visualization of Figure 6: IJB-C faces with magnitudes around 28 are mostly profile faces, whereas magnitudes around 35 are high-quality frontal faces.Similar magnitude patterns across benchmarks are consistent with profile and frontal peaks in CFP-FP.
  • D. Extended Visualization of Figure 6: A magnitude threshold can filter similar low-quality faces in real applications.
  • D. Extended Visualization of Figure 6: Feature magnitudes can serve as quality labels without human annotation and can support other quality models.These labels are described as more suitable for recognition.
  • D. Extended Visualization of Figure 6: Mag-CosFace is presented as theoretically achieving the same effects as the ArcFace-based MagFace loss.
  • D. Extended Visualization of Figure 6: For Mag-CosFace, Li is strictly convex in ai and has a unique optimum within [la, ua].The appendix labels these as Properties 5 and 6.

E.2. Property of Monotonicity for Mag-CosFace

The appendix establishes monotonicity properties for the optimal feature magnitude in Mag-CosFace. The magnitude increases as samples become closer to their class centers and as inter-class separation increases.

  • With fixed competing weights, the optimal feature magnitude a_i* increases monotonically as the cosine distance to class center W_yi decreases.This is stated as Property 7.
  • The proofs proceed by analyzing the first derivative of the loss and follow the same structure as the original MagFace proofs.
  • With other quantities fixed, the optimal feature magnitude a_i* increases monotonically as B decreases, corresponding to increasing inter-class distance.This is stated as Property 8.
Loading 2103.06627v4…