Source-linked AI summary
Mis-classified Vector Guided Softmax Loss for Face Recognition
Xiaobo Wang, Shifeng Zhang, Shuo Wang, Tianyu Fu, Hailin Shi, Tao Mei
TL;DR
Existing face-recognition softmax losses overlook informative-feature mining, non-ground-truth discriminability, or adaptive class margins. MV-Softmax addresses these issues by emphasizing mis-classified feature vectors within a unified margin-and-mining loss. Experiments report superiority over baseline, mining-based, margin-based, and naively fused alternatives, with larger gains on challenging protocols.
Problem
Existing softmax losses may ignore informative-feature mining, use only ground-truth-class discriminability, and impose the same fixed margin across classes.
Method
MV-Softmax explicitly identifies mis-classified feature vectors and adaptively emphasizes them, combining feature mining with feature margins.
Results
MV-Softmax outperforms baseline, mining-based, margin-based, and naively fused Softmax losses across the reported benchmarks, with at least 3% improvements in both identification and verification on Trillion-Pairs Challenge.
Takeaways & Limitations
The unified loss provides more discriminative face features and shows strong generalization for face recognition, especially at very low false positive rates.
Takeaways & Limitations
On several common benchmarks, improvements are not large because the test protocols are relatively easy and performance is near saturation.
Abstract
from arXiv · showhide
Face recognition has witnessed significant progress due to the advances of deep convolutional neural networks (CNNs), the central task of which is how to improve the feature discrimination. To this end, several margin-based (\textit{e.g.}, angular, additive and additive angular margins) softmax loss functions have been proposed to increase the feature margin between different classes. However, despite great achievements have been made, they mainly suffer from three issues: 1) Obviously, they ignore the importance of informative features mining for discriminative learning; 2) They encourage the feature margin only from the ground truth class, without realizing the discriminability from other non-ground truth classes; 3) The feature margin between different classes is set to be same and fixed, which may not adapt the situations very well. To cope with these issues, this paper develops a novel loss function, which adaptively emphasizes the mis-classified feature vectors to guide the discriminative feature learning. Thus we can address all the above issues and achieve more discriminative face features. To the best of our knowledge, this is the first attempt to inherit the advantages of feature margin and feature mining into a unified loss function. Experimental results on several benchmarks have demonstrated the effectiveness of our method over state-of-the-art alternatives.
Introduction
Face recognition relies on discriminative features learned by deep CNNs, but existing softmax variants overlook informative-feature mining, non-ground-truth discriminability, or adaptive margins. MV-Softmax unifies feature mining and feature margins, and experiments across common benchmarks report superiority over several alternatives.
- Motivation: Deep CNN face-recognition models depend on learned discriminative features, while Softmax-based objectives have limited feature-discrimination power.Discriminative features require intra-class compactness and inter-class separability.
- Limitations of prior work: Existing approaches may ignore informative-feature mining, use only ground-truth-class discriminability, and apply the same fixed margin across classes.These shortcomings motivate a loss that handles feature mining, non-ground-truth classes, and class-specific margins.
- Proposed approach: MV-Softmax explicitly identifies hard examples as mis-classified vectors and adaptively emphasizes them to guide discriminative feature learning.The loss also incorporates discriminability from non-ground-truth classes and adaptive margins.
- Contribution: The paper presents MV-Softmax as a unified loss combining the advantages of feature-margin and feature-mining techniques.The authors describe this as the first such attempt to their knowledge.
- Evaluation: Extensive experiments on LFW, CALFW, CPLFW, AgeDB, CFP, RFW, MegaFace, and Trillion-Pairs report superiority over baseline, mining-based, margin-based, and naively fused Softmax losses.The comparison covers common face-recognition benchmarks and multiple classes of alternatives.
Preliminary Knowledge
Softmax loss combines a fully connected layer, softmax, and cross-entropy, using normalized class weights and features with a scale parameter. Mining-based variants reweight examples, while margin-based variants modify the ground-truth logit with angular or additive margins.
- Softmax: Softmax loss combines the final fully connected layer, softmax function, and cross-entropy loss.For face recognition, class weights and features are typically normalized, with their magnitudes replaced by scale parameter s.
- Softmax: The normalized feature x and class weights w_k produce cosine similarities, with θ_{w_k,x} denoting the angle between w_k and x.The ground-truth label is y, and K denotes the number of classes.
- Mining-based Softmax: Mining-based Softmax methods use an indicator or weighting function g(p_y) to emphasize informative examples according to ground-truth probability.Focal loss uses (1 − p_y)^γ, whereas HM-Softmax gives zero weight to easy samples and unit weight to hard samples.
- Margin-based Softmax: Margin-based Softmax methods replace the ground-truth angular term with a margin function f(m, θ_{w_y,x}).A-Softmax uses cos(m_1θ_{w_y,x}), AM-Softmax subtracts m_2 from cosine similarity, and Arc-Softmax adds m_3 to the angle.
Problem Formulation
The paper identifies limits in existing margin- and mining-based Softmax losses and proposes MV-Softmax, which adaptively emphasizes mis-classified vectors while incorporating class-specific margins.
- Problem Formulation: Margin-based Softmax losses neglect informative-feature mining, use only ground-truth discriminability, and impose the same fixed margin across classes.These limitations motivate a loss that mines informative vectors and accounts for non-ground-truth classes.
- Naive Mining-Margin Softmax Loss: Naive mining-margin fusion combines hard-example weighting with margin enlargement, but its gains are limited because existing mining strategies either discard easy examples or obscure which examples are hard.HM-Softmax discards easy examples, whereas F-Softmax re-weights all examples without an intuitive hard-example interpretation.
- Mis-classified Vector Guided Softmax Loss: MV-Softmax defines an indicator I_k that identifies whether a feature is mis-classified by a specific non-ground-truth classifier.The indicator is evaluated from the current decision boundary and targets genuinely informative features.
- Mis-classified Vector Guided Softmax Loss: Figure 1 illustrates the method using two class-1 samples, where red dots denote vectors mis-classified by a specific classifier such as w2.The figure’s geometry motivates selectively emphasizing the mis-classified vector rather than treating both samples identically.
- Mis-classified Vector Guided Softmax Loss: The loss re-weights indicated mis-classified vectors through h(t, θ_wk,x, I_k), with fixed-weight and adaptive formulations controlled by t ≥ 0.When t = 0, MV-Softmax reduces to the original margin-based Softmax losses.
- Mis-classified Vector Guided Softmax Loss: Compared with mining-based losses, MV-Softmax identifies hard examples semantically from the local relationship between a feature and classifier rather than a global feature-to-feature relationship.HM-Softmax discards easy samples, while F-Softmax re-weights all samples without explicitly identifying hard examples.
- Mis-classified Vector Guided Softmax Loss: MV-Softmax enlarges margins from non-ground-truth classes for mis-classified features while avoiding extra enforcement for well-classified classes.This design addresses the missing non-ground-truth discriminability and fixed-margin limitations.
- Optimization: The proposed loss remains trainable with standard SGD by applying margin-based computation to the ground-truth class and conditional re-computation to non-ground-truth classes.Well-classified non-ground-truth features retain the original Softmax value; mis-classified ones receive fixed or adaptive re-weighting.
Experiments
Experiments evaluate MV-Softmax across standard, bias-focused, and difficult face-recognition benchmarks, using refined training data and single-model testing. The method generally improves over competing losses, with larger gains under challenging MegaFace and Trillion-Pairs protocols.
- Experimental Setup: Experiments use the refined MS-Celeb-1M-v1c-R training set and eight face-recognition benchmarks, including LFW, RFW, MegaFace, and Trillion-Pairs.The refined training set removes 14,186 identities, and testing uses cosine similarity with standard and challenge-specific protocols.
- Experimental Setup: All reported test results use a single model without model ensembling or other fusion strategies.Only original image features are used to compose the face representation at test time.
- Optimization: The MV-Softmax loss shows good convergence behavior as the number of training epochs increases.The convergence assessment is based on the loss curves in Figure 2.
- Standard Benchmarks: MV-Softmax is reported as better than state-of-the-art alternatives in most cases on LFW, CALFW, CPLFW, AgeDB, and CFP.Improvements are limited on these relatively easy, near-saturated test protocols.
- Challenging Benchmarks: On MegaFace, MV-AM-Softmax-a exceeds AM-Softmax by about 0.3% in identification and 0.5% in verification.The adaptive variant also outperforms naive mining-and-margin fusions on the reported difficult protocols.
- Challenging Benchmarks: On Trillion-Pairs, MV-AM-Softmax-a achieves at least 3% improvements in both identification and verification.The gains are reported under very low false-positive-rate evaluation and the same trend is observed on MegaFace.
Conclusion
The paper proposes MV-Softmax for face recognition by concentrating optimization on mis-classified feature vectors. It unifies feature-margin and feature-mining motivations and outperforms several alternative loss families across benchmarks.
- MV-Softmax concentrates optimization on mis-classified feature vectors for face recognition.
- The loss unifies the motivations of feature margin and feature mining.
- MV-Softmax achieves higher performance than baseline, mining-based, margin-based, and naively fused losses.