CosFace: Large Margin Cosine Loss for Deep Face Recognition

Hao Wang; Yitong Wang; Zheng Zhou; Xing Ji; Dihong Gong; Jingchao Zhou; Zhifeng Li; Wei Liu

Source-linked AI summary

CosFace: Large Margin Cosine Loss for Deep Face Recognition

Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, Wei Liu

arXiv:1801.09414v2Jan 29, 2018cs.CV

TL;DR

Traditional softmax loss often lacks sufficient discriminative power for face recognition. CosFace reformulates it with normalized features and weights plus a cosine margin, achieving state-of-the-art performance across several face benchmarks.

Problem
Traditional softmax loss is insufficiently discriminative for face-recognition feature classification.
Method
LMCL normalizes features and weights to remove radial variations, then maximizes an inter-class cosine margin in angular space.
Results
CosFace consistently achieves state-of-the-art performance across popular face benchmarks, including 99.73% on LFW and 97.6% on YTF.
Takeaways & Limitations
LMCL provides a geometrically and theoretically interpreted approach for learning highly discriminative face features.
Takeaways & Limitations
LMCL can fail to converge when its cosine-margin parameter is too large, increasing sensitivity to noisy data and degrading performance.

Abstract

from arXiv · show

Face recognition has made extraordinary progress owing to the advancement of deep convolutional neural networks (CNNs). The central task of face recognition, including face verification and identification, involves face feature discrimination. However, the traditional softmax loss of deep CNNs usually lacks the power of discrimination. To address this problem, recently several loss functions such as center loss, large margin softmax loss, and angular softmax loss have been proposed. All these improved losses share the same idea: maximizing inter-class variance and minimizing intra-class variance. In this paper, we propose a novel loss function, namely large margin cosine loss (LMCL), to realize this idea from a different perspective. More specifically, we reformulate the softmax loss as a cosine loss by $L_2$ normalizing both features and weight vectors to remove radial variations, based on which a cosine margin term is introduced to further maximize the decision margin in the angular space. As a result, minimum intra-class variance and maximum inter-class variance are achieved by virtue of normalization and cosine decision margin maximization. We refer to our model trained with LMCL as CosFace. Extensive experimental evaluations are conducted on the most popular public-domain face recognition datasets such as MegaFace Challenge, Youtube Faces (YTF) and Labeled Face in the Wild (LFW). We achieve the state-of-the-art performance on these benchmarks, which confirms the effectiveness of our proposed approach.

1. Introduction

The introduction identifies traditional softmax loss as insufficiently discriminative for face recognition and motivates an angular-space approach. CosFace reformulates softmax with normalized features and weights, adds a cosine margin, and reports improved benchmark performance.

Traditional softmax loss is insufficient to acquire discriminating power for classification despite deep CNNs’ strong feature extraction capabilities.
Prior methods seek maximum discrimination by maximizing inter-class variance and minimizing intra-class variance, but often use multi-loss learning.
Angular margins are preferred because cosine similarity is intrinsically consistent with softmax and commonly measures face-recognition similarity.
LMCL L2-normalizes features and weight vectors to remove radial variations, then introduces cosine margin m to maximize inter-class angular separation.
Unlike A-Softmax, LMCL defines the decision margin in cosine space, avoiding nonmonotonic optimization and θ-dependent margins.
CosFace applies LMCL during training and advances state-of-the-art performance on LFW, YTF, and Megaface benchmarks.

2. Related Work

Related work advances deep face recognition through CNN-based feature learning, specialized metric and classification losses, and normalization methods that reduce intra-class variation while optimizing inter-identity angles.

Deep Face Recognition: DeepFace and DeepID framed face recognition as multiclass classification and used deep CNNs to learn features from large multi-identity datasets.DeepID2 additionally used identification and verification signals for improved feature embedding.
Loss Functions: Contrastive, triplet, center, large margin softmax, and angular softmax losses improve feature discrimination by enlarging margins or reducing intra-class variance.Contrastive and triplet losses increase Euclidean margins, while center loss learns identity-specific feature centers.
Normalization Approaches: Normalization-based approaches replace inner products with cosine similarity or constrain features with L2 normalization to embed faces in normalized spaces.Weight normalization and feature normalization are presented as complementary approaches in prior work.
Normalization Approaches: Normalizing feature or weight vectors reduces intra-class angular variability, allowing angles between identities to be better optimized during training.The passage also identifies vMF-based methods and A-Softmax as normalization-based approaches.

3. Proposed Approach

LMCL reformulates softmax through L2 normalization of features and weights, then adds a cosine-space margin to produce more discriminative angular features. Compared with Softmax, NSL, and A-Softmax, its margin is consistent across samples and improves robustness to boundary perturbations, while theory characterizes normalization and feasible margin choices.

3.1 LMCL formulation: L2-normalizing weights and fixing feature norm ∥x∥ = s removes radial variation, making posterior probabilities depend on angular information and yielding the Normalized Softmax Loss.NSL learns features separable in angular space, consistent with cosine similarity used for face-recognition scoring.
3.1 LMCL formulation: LMCL strengthens NSL by requiring a cosine gap m between the ground-truth and competing classes, where m ≥ 0 controls the decision margin.For binary classes, the constraints are cos(θ1) − m > cos(θ2) and cos(θ2) − m > cos(θ1).
3.2 Comparison with other losses: Softmax can produce an overlapping cosine-space decision area with margin < 0, whereas NSL perfectly classifies cosine-space samples with margin = 0 but is sensitive to perturbations.Softmax boundaries depend on weight magnitudes and angular cosines; NSL removes radial variations but has no decision margin.
3.2 Comparison with other losses: A-Softmax’s angular margin shrinks as θ decreases and vanishes at θ = 0, while LMCL applies its cosine margin consistently regardless of weight-vector angles.The paper identifies smaller margins for visually similar classes as a potential issue of A-Softmax.
3.4 Theoretical analysis: The theoretical analysis states that increasing the number of classes decreases the upper bound on the cosine margin, and excessively large m can prevent convergence.In experiments, m = 0, m = 0.1, and m = 0.2 are compared; increasing m amplifies angular margins between classes.

4. Experiments

Experiments evaluate CosFace under a controlled 64-layer CNN setup, ablate feature normalization and the margin parameter, and compare against state-of-the-art losses on LFW, YTF, and MegaFace benchmarks. CosFace consistently benefits from feature normalization and achieves leading face-verification results, including 99.73% on LFW and 97.6% on YTF.

Experimental Setup: Experiments use a 64-layer residual CNN, SGD with batch size 64 on 8 GPUs, and scaling parameter s=64 for fair comparison.Training uses weight decay 0.0005, with dataset-dependent learning-rate schedules and termination at 30k or 240k iterations.
Experimental Setup: Testing concatenates original and horizontally flipped image features, computes cosine similarity, and evaluates verification and identification on LFW, YTF, and MegaFace.Faces are detected and aligned with MTCNN, cropped to 112 × 96, and pixel-normalized before training and testing.
Effect of m: The margin parameter m is evaluated from 0 to 0.45 on CASIA-WebFace, with m=0 producing the worst performance and larger values than 0.45 failing to converge.Performance is measured on LFW and YTF using the small training dataset.
Effect of Feature Normalization: Feature normalization consistently outperforms its absence across LFW, YTF, and MegaFace Challenge 1 when m=0.35.The unnormalized model is initialized with softmax loss before supervision by LMCL.
Comparison with State-of-the-Art Methods: CosFace is compared with state-of-the-art loss functions using the same training data and 64-layer CNN architecture on LFW, YTF, and MF1.The study also evaluates large-scale face identification and verification on MegaFace, whose gallery contains more than 1 million images.
Comparison with State-of-the-Art Methods: 99.73% on LFW and 97.6% on YTF are achieved by CosFace, establishing state-of-the-art face-verification results in the reported comparison.The evaluation follows unrestricted-with-labeled-outside-data protocols using 6,000 LFW pairs and 5,000 YTF video pairs.

5. Conclusion

The paper proposes LMCL to guide deep CNNs toward highly discriminative face features, supported by geometric and theoretical analysis. The approach consistently achieves state-of-the-art results on several face benchmarks.

Conclusion: LMCL guides deep CNNs to learn highly discriminative face features.The authors present it as an innovative approach for face recognition.
Conclusion: The paper provides a geometric and theoretical interpretation supporting LMCL’s effectiveness.
Conclusion: LMCL consistently achieves state-of-the-art results on several face benchmarks.

Proposition of the Scaling Parameter s

The proposition derives a lower bound for the scaling parameter s under normalized features and unit weight vectors, using the expected minimum posterior probability of the class center. Equality is possible only when the class count satisfies C ≤ K + 1.

Proposition setup: The proposition assumes normalized learned features lie on a hypersphere around their corresponding unit weight vectors and defines C > 1 and Pw as the expected minimum class-center posterior.These assumptions establish the setting for formulating the lower bound of s.
Proof: The proof applies Jensen’s inequality because f(x) = e^(s·x) is convex.This convexity provides the inequality used in the derivation.
Equality condition: Equality requires every pair of unit weight vectors to satisfy W_i^T W_i = 0.The supplied passage states this as the necessary and sufficient equality condition.
Equality condition: C ≤ K + 1 is required for equality, because at most K + 1 unit vectors can satisfy the condition in a K-dimensional hyperspace.Here, K denotes the dimension of the learned features.

Proposition of the Cosine Margin m

The cosine margin m is bounded by C/(C−1) when the number of classes does not exceed feature dimension plus one, but is much smaller when classes are more numerous. Here, C denotes training classes and K denotes learned-feature dimension.

Margin range: 0 ≤ m ≤ C/(C−1) when K > 2 and C ≤ K + 1.The bound applies under uniformly distributed weight vectors on a unit hypersphere.
Margin range: 0 ≤ m ≪ C/(C−1) when K > 2 and C > K + 1.The strict upper-bound equality cannot be satisfied in this regime.
Definitions and condition: C is the total number of training classes, while K is the dimension of the learned features.The equality condition can hold only if C ≤ K + 1.