PaperlayerView exact revision on arXiv

Source-linked AI summary

KappaFace: Adaptive Additive Angular Margin Loss for Deep Face Recognition

Chingis Oinar, Binh M. Le, Simon S. Woo

arXiv:2201.07394v2cs.CV

TL;DR

Fixed-margin softmax losses do not account for class imbalance and learning difficulty, both practical challenges in deep face recognition. KappaFace uses the von Mises-Fisher distribution to adapt class margins using concentration and population. The paper reports that it consistently outperforms state-of-the-art counterparts across facial benchmarks.

  • Problem

    Fixed-margin methods do not account for class imbalance or class learning difficulty, despite imbalance being a practical challenge in deep face recognition.

    Source: §I · p. 2, A. PRELIMINARY INFORMATION ON LOSS… · p. 5

  • Method

    KappaFace uses the von Mises-Fisher distribution to adapt class margins based on class concentration and population.

    Source: §I · p. 2, D. KappaFace · p. 7

  • Results

    KappaFace consistently outperforms state-of-the-art counterparts across facial benchmarks.

    Source: §I · p. 3

  • Takeaways & Limitations

    KappaFace relaxes margins for easily learned, high-concentration classes and increases them for difficult or underrepresented classes.

    Source: §I · p. 2, §V · p. 12

Abstract

from arXiv · show

Feature learning is a widely used method employed for large-scale face recognition. Recently, large-margin softmax loss methods have demonstrated significant enhancements on deep face recognition. These methods propose fixed positive margins in order to enforce intra-class compactness and inter-class diversity. However, the majority of the proposed methods do not consider the class imbalance issue, which is a major challenge in practice for developing deep face recognition models. We hypothesize that it significantly affects the generalization ability of the deep face models. Inspired by this observation, we introduce a novel adaptive strategy, called KappaFace, to modulate the relative importance based on class difficultness and imbalance. With the support of the von Mises-Fisher distribution, our proposed KappaFace loss can intensify the margin's magnitude for hard learning or low concentration classes while relaxing it for counter classes. Experiments conducted on popular facial benchmarks demonstrate that our proposed method achieves superior performance to the state-of-the-art.

I. INTRODUCTION

KappaFace addresses the limitations of fixed margins by adapting class importance to learning difficulty and class imbalance. It uses the von Mises-Fisher distribution to adjust margins, and the paper reports consistent gains over state-of-the-art methods across facial benchmarks.

I. INTRODUCTION

  • Fixed-margin methods do not account for class imbalance or learning difficulty, which can bias decision boundaries and leave minority classes poorly represented.Larger margins may also overfit, while smaller margins may fail to distinguish positive and negative samples.Source: §I · p. 2
  • KappaFace adapts class margins by increasing them for difficult, low-concentration or underrepresented classes and relaxing them for well-learned classes.The strategy treats class difficulty and class size as distinct factors.Source: §I · p. 2, Figure 2 · p. 3
  • Identities with minimal facial-angle and age variation are described as more likely to yield invariant representations and easier to learn.Source: §I · p. 2
  • KappaFace uses the von Mises-Fisher distribution to model identity embeddings and estimate class concentration, which reflects learning difficulty.It combines concentration and population parameters to adapt margins across classes.Source: §I · p. 2
  • KappaFace consistently outperforms state-of-the-art counterparts across nine facial benchmarks.The evaluation also includes additional ablation studies.Source: §I · p. 3

II. RELATED WORKS

Related work covers metric learning, hyperspherical feature modeling, margin-based losses, and class-imbalance methods. The proposed-method overview then outlines how KappaFace estimates class concentrations and uses them to adjust margins.

II. RELATED WORKS

  • Metric-learning methods produce discriminative face representations, but methods requiring positive and negative sample mining can become computationally burdensome on large datasets.Source: §II · p. 3
  • Face-recognition methods commonly normalize features onto a unit hypersphere, where the softmax inner product becomes cosine similarity.Prior work also models facial-feature distributions with the von Mises-Fisher distribution; increasing κ corresponds to greater within-class compactness.Source: §II · p. 3, Figure 3 · p. 4
  • KappaFace manages concentrations across classes with either a memory buffer or a momentum encoder to adjust loss according to class difficulty and imbalance.Source: §II · pp. 3–4
  • Margin-based methods address limited feature discrimination, but class imbalance and sample difficulty remain challenges; CurricularFace addresses sample difficulty without accounting for class imbalance.Source: §II · p. 4
  • LDAM sets margins according to class population, whereas KappaFace also uses class concentration to reflect learning difficulty.Fair Loss uses an additional reinforcement-learning network, while KappaFace does not rely on additional trainable networks.Source: §II · p. 4

III. PROPOSED METHOD

  • The proposed-method section proceeds from conventional margin objectives to vMF modeling and concentration estimation, then to margin adjustment and gradient analysis.Source: §III · p. 4

A. PRELIMINARY INFORMATION ON LOSS FUNCTIONS

Margin-based softmax losses modify the target logit to encourage compact, diverse embeddings, but fixed margins do not account for class imbalance. KappaFace uses vMF concentration to adapt margins to class learning difficulty and population.

A. PRELIMINARY INFORMATION ON LOSS FUNCTIONS

  • Margin-based losses modify the target-class logit, while normalized weights and features and a zero bias form the stated setup.ArcFace uses an additive angular margin; CosFace and SphereFace use additive cosine and multiplicative angular margins, respectively.Source: A. PRELIMINARY INFORMATION ON LOSS… · pp. 4–5
  • Fixed margins ignore class imbalance, motivating dynamic adjustment of relative importance by class difficulty and imbalance.The paper identifies class imbalance as a practical challenge for deep face recognition.Source: A. PRELIMINARY INFORMATION ON LOSS… · p. 5

B. THE VON MISES-FISHER DISTRIBUTION

  • The vMF distribution models data on a unit sphere, with concentration κ controlling how tightly samples cluster around the mean direction.At κ = 0 the distribution is uniform; increasing κ concentrates samples more closely around the mean direction.Source: B. THE VON MISES-FISHER DISTRIBUTION · p. 5
  • KappaFace uses estimated class concentration κ̂ to modulate the angular margin, treating lower-concentration classes as harder to learn.The paper associates low concentration with noisy or high-variance facial samples and high concentration with easier-to-learn classes.Source: B. THE VON MISES-FISHER DISTRIBUTION · p. 5

C. CALCULATION OF κ

The training algorithm uses either a memory buffer or momentum encoder while normalizing embeddings, computing the KappaFace loss, and updating model parameters and class-weight scalers.

C. CALCULATION OF κ

  • Algorithm 1 takes the dataset, connected-layer parameters, initial margin, temperature, learning rate, and approach-specific state as training inputs.The memory-buffer variant uses M; the momentum-encoder variant uses a momentum parameter and class-sum calculation head Zacc.Source: C. CALCULATION OF κ · p. 6
  • Each iteration embeds and normalizes online and momentum features, then normalizes the weight matrix before updating the memory buffer.The algorithm lists these operations as the forward-sample and embedding steps.Source: C. CALCULATION OF κ · p. 6
  • The algorithm accumulates momentum features by class, computes the KappaFace loss, updates model parameters, and updates the class-weight scalers.The encoder parameters are momentum-updated, and the scalers are refreshed using the memory buffer's update_weights procedure.Source: C. CALCULATION OF κ · p. 6

1) Memory buffer

The memory-buffer and momentum-encoder approaches provide alternative ways to collect class representations for concentration estimation. The momentum encoder is discussed as a potentially desirable choice as dataset size increases.

1) Memory buffer

  • The memory buffer updates each sample representation with an exponential moving average and l2-normalizes the stored vectors.Class concentration is then computed from the class-wise sums of stored vectors.Source: 1) Memory buffer · p. 6

2) Momentum encoder

  • The momentum encoder is initialized from the online encoder and updated with a momentum rule to reduce representation inconsistency.The momentum encoder's representations are used in the same way as those from the memory-buffer approach to form class sums.Source: 2) Momentum encoder · p. 6
  • The momentum encoder may be preferable as dataset size grows because it is updated every iteration, whereas memory-bank representations are updated when samples are last seen.The discussion attributes the comparison of representation consistency to He et al..Source: 2) Momentum encoder · p. 6

D. KappaFace

KappaFace combines concentration and population weights to calibrate class-specific additive margins. These calibrated margins are applied through the ArcFace-style classification loss.

D. KappaFace

  • Concentration values are recalculated each epoch and converted into concentration weights through normalization and a sigmoid controlled by temperature T.A higher-than-mean concentration yields a smaller concentration weight; the paper uses relative concentration across the training set to represent learning difficulty.Source: D. KappaFace · pp. 6–7
  • Population weights encode class representation, prioritize underrepresented classes, and remain fixed because class sample counts do not change during training.The class sample count n_c is compared with K, the maximum class count.Source: D. KappaFace · p. 7
  • The class-specific margin calibration balances concentration and population weights with γ, then applies the resulting additive margin in the ArcFace formulation.m0 is the fixed initial margin value.Source: D. KappaFace · p. 7
  • The resulting loss dynamically adjusts angular margins according to class learning difficulty and population.The method uses the concentration and population parameters together to set the class margin.Source: D. KappaFace · p. 7

E. GRADIENT ANALYSIS OF KAPPAFACE

The gradient analysis compares KappaFace with ArcFace and shows that its correction depends on class concentration: it is smaller for high-concentration classes and larger for low-concentration classes.

  • High-concentration classes receive a smaller correction on W under KappaFace than under ArcFace, with the reverse relationship for low-concentration classes.The theorem states this comparison for the softmax loss in Eq. 14.Source: E. GRADIENT ANALYSIS OF KAPPAFACE · p. 7
  • For a high-concentration class, the scaler ψ_l makes its margin smaller than ArcFace’s fixed margin, and its true-class gradient norm is also smaller.The analysis relates this result to the probability comparison in Eq. 17.Source: E. GRADIENT ANALYSIS OF KAPPAFACE · p. 7
  • For non-target classes, KappaFace has the same gradient property as ArcFace; the analysis inversely deduces the corresponding outcomes for low-concentration classes.Source: E. GRADIENT ANALYSIS OF KAPPAFACE · p. 7

IV. EXPERIMENTAL RESULTS

The experimental section describes validation on nine facial benchmark datasets, comparisons with state-of-the-art baselines, and ablations of the proposed method’s hyperparameters.

  • The method is validated using nine facial benchmark datasets.Source: §IV · p. 7
  • The experiments compare KappaFace performance with state-of-the-art baselines across the datasets.Source: §IV · p. 7
  • Ablation studies examine the individual contributions of the proposed method’s hyperparameters.Source: §IV · p. 7

A. IMPLEMENTATION DETAILS

The implementation uses MS1MV2 training data and a ResNet100 backbone, evaluates face verification and identification on facial benchmarks, and reports training convergence through loss and concentration curves.

A. IMPLEMENTATION DETAILS

  • MS1MV2 provides approximately 5.8M training images from 85,742 identities, and the embedding network uses ResNet100.Training uses 112 × 112 normalized face crops, batch size 512, and 24 epochs.Source: A. IMPLEMENTATION DETAILS · pp. 7–8
  • Table 1 reports verification accuracy across LFW, YTF, pose benchmarks CFP-FP and CPLFW, and age benchmarks AgeDB and CALFW.Source: Table 1 · p. 8
  • As training loss decreases, average concentration increases, which the authors describe as demonstrating the algorithm’s convergence capabilities.Figure 5 plots margin loss below and average concentration above across training epochs.Source: A. IMPLEMENTATION DETAILS · pp. 8–9, Figure 5 · p. 8
  • Verification uses accuracy on LFW, CFP-FP, CPLFW, AgeDB, CALFW, and YTF, and TAR at specified FAR levels on IJB-B/C and MegaFace.IJB-B/C use TAR@FAR=1e-4; MegaFace uses TAR@FAR=1e-6, while MegaFace identification uses rank-1 accuracy.Source: A. IMPLEMENTATION DETAILS · p. 9

Results on LFW, CFP-FP, CPLFW, AgeDB, CALFW,

KappaFace outperforms ArcFace or other comparison methods across the reported face benchmarks, including IJB verification and MegaFace identification and verification. Its qualitative analysis relates class concentration to learning difficulty, while ablations identify effective settings and a dependency on auxiliary concentration estimation.

Results on IJB-B and IJB-C

Results on LFW, CFP-FP, CPLFW, AgeDB, CALFW

  • KappaFace outperforms comparison methods on CFP-FP, CPLFW, and AgeDB, while improving on LFW, YTF, and CALFW, where performance is nearly saturated.The authors describe LFW as relatively easy because it has less variation in facial angles and ages.Source: Results on LFW, CFP-FP, CPLFW, AgeDB,… · p. 9

Results on MegaFace

  • 98.77 on MegaFace identification and 98.91 on MegaFace verification; memory-buffer KappaFace outperforms other state-of-the-art methods on both tasks.The evaluation uses the refined MegaFace under a large-training-set protocol with ResNet100 trained on MS1MV2.Source: Results on LFW, CFP-FP, CPLFW, AgeDB,… · p. 10

Probabilistic models with smaller backbone

  • With ResNet34 + Momentum Encoder, KappaFace consistently outperforms probabilistic counterparts by a decent margin and achieves 99.82% accuracy on LFW, comparable to ResNet100 results.All three probabilistic models improve considerably over the ArcFace baseline across all experimental datasets.Source: Results on LFW, CFP-FP, CPLFW, AgeDB,… · p. 10, Table 4 · p. 10

Qualitative analysis

Hyper-parameter fine-tuning and ablations

  • The reported settings favor m0 = 0.8 and T = 0.4 overall, while γ = 0.7 is optimal in the concentration-and-population ablation; both class weights contribute to results.m0 = 0.7 and T = 0.4 performs better on CFP-FP and AgeDB but drops on IJB-B and IJB-C; the authors say the population weight may encourage learning underrepresented categories when concentrations are equal.Source: Results on LFW, CFP-FP, CPLFW, AgeDB,… · pp. 10–11, Table 5 · p. 10, Table 6 · p. 10

Discussion & Limitations

  • Class concentration estimation depends on an auxiliary model or memory buffer, which the authors say adds insignificant computational overhead.The authors leave adaptation to other margin-based losses for future investigation, contingent on an appropriate design.Source: Results on LFW, CFP-FP, CPLFW, AgeDB,… · pp. 11–12

V. CONCLUSION

KappaFace adapts positive additive margins to class imbalance and learning difficulty, using the von Mises-Fisher distribution to lower margins for easy, concentrated classes and raise them for counter-classes. It reports consistent outperformance across facial benchmarks, while retaining training-time auxiliary-module dependence and an angular-margin configuration primarily tailored to ArcFace.

V. CONCLUSION

  • Up to 0.5% improvement on IJB-B/C accompanies consistent outperformance of eight contemporary baselines across nine facial benchmarks.The reported improvement is up to 0.5% on the IJB-B/C datasets.Source: §V · p. 12
  • KappaFace dynamically adjusts positive additive margins by class imbalance and learning difficulty, lowering them for easy, high-concentration classes and raising them for counter-classes via the von Mises-Fisher distribution.Source: §V · p. 12
  • Momentum encoders and memory buffers model hyperspherical latent distributions without additional computational or memory costs during inference.Source: §V · p. 12
  • Training still requires an auxiliary model or memory module, and the current adaptive angular-margin configuration is primarily tailored to ArcFace.The stated next steps are more efficient difficulty estimation and compatibility with a wider range of similar algorithms.Source: §V · p. 12

Related papers

Loading 2201.07394v2…