Circle Loss: A Unified Perspective of Pair Similarity Optimization

Yifan Sun; Changmao Cheng; Yuhan Zhang; Chi Zhang; Liang Zheng; Zhongdao Wang; Yichen Wei

Source-linked AI summary

Circle Loss: A Unified Perspective of Pair Similarity Optimization

Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, Yichen Wei

arXiv:2002.10857v2Feb 25, 2020cs.CV

TL;DR

Existing deep feature-learning losses often reduce (s_n-s_p), which restricts penalty strengths and permits ambiguous convergence. The paper introduces Circle loss, which adaptively re-weights similarity scores in a unified formulation for class-level and pair-wise learning. Analyses and experiments report more flexible optimization, a more definite convergence target, and performance on par with the state of the art across several tasks.

Problem
Many popular losses optimize s_n and s_p through (s_n-s_p), enforcing equal penalty strengths and allowing ambiguous convergence that can compromise feature-space separability.
Method
Circle loss re-weights each similarity according to its distance from the optimum and provides a unified formulation for class-level and pair-wise learning.
Results
Across face recognition, person re-identification, and other deep feature-learning tasks, Circle loss achieves performance on par with the state of the art.
Takeaways & Limitations
Circle loss supports flexible optimization with similarity scores learning at different paces and favors a more definite convergence target.
Takeaways & Limitations
The paper’s limitation is that conventional losses use equal gradients before convergence and do not prefer among points sharing the same similarity margin.

Abstract

from arXiv · show

This paper provides a pair similarity optimization viewpoint on deep feature learning, aiming to maximize the within-class similarity $s_p$ and minimize the between-class similarity $s_n$. We find a majority of loss functions, including the triplet loss and the softmax plus cross-entropy loss, embed $s_n$ and $s_p$ into similarity pairs and seek to reduce $(s_n-s_p)$. Such an optimization manner is inflexible, because the penalty strength on every single similarity score is restricted to be equal. Our intuition is that if a similarity score deviates far from the optimum, it should be emphasized. To this end, we simply re-weight each similarity to highlight the less-optimized similarity scores. It results in a Circle loss, which is named due to its circular decision boundary. The Circle loss has a unified formula for two elemental deep feature learning approaches, i.e. learning with class-level labels and pair-wise labels. Analytically, we show that the Circle loss offers a more flexible optimization approach towards a more definite convergence target, compared with the loss functions optimizing $(s_n-s_p)$. Experimentally, we demonstrate the superiority of the Circle loss on a variety of deep feature learning tasks. On face recognition, person re-identification, as well as several fine-grained image retrieval datasets, the achieved performance is on par with the state of the art.

1. Introduction

The paper reframes deep feature learning as similarity optimization and identifies inflexibility and ambiguous convergence in losses that reduce (s_n-s_p). Circle loss re-weights individual similarities, unifies class-level and pair-wise learning, and provides flexible optimization toward a definite convergence target.

Motivation: Class-level and pair-wise deep feature learning both optimize between-class similarity s_n and within-class similarity s_p through similarity relationships.The paper interprets classification losses as optimizing sample–weight similarities and metric losses as optimizing sample–sample similarities.
Problems with Existing Optimization: Popular losses embed s_n and s_p into pairs and reduce (s_n-s_p), imposing equal penalty strength on both similarity scores.This symmetric optimization can continue strongly penalizing s_n even when s_n is near 0 while s_p remains small.
Problems with Existing Optimization: Reducing (s_n-s_p) permits multiple convergence points with the same margin, creating ambiguous convergence that compromises feature-space separability.The paper contrasts T and T′, which share margin m = 0.3 despite differing similarity gaps.
Circle Loss: Circle loss generalizes the objective to (α_n s_n-α_p s_p), with adaptive weights that increase for similarity scores farther from their optima.Its decision boundary α_n s_n-α_p s_p = m forms a circle in the (s_n, s_p) space.
Contributions: Circle loss offers a unified loss for class-level and pair-wise labels while dynamically adjusting gradients and favoring a specified convergence point.The proposed optimization emphasizes increasing s_p or reducing s_n according to each pair’s optimization status, and experiments cover recognition, re-identification, and retrieval tasks.

2. A Unified Perspective

The paper unifies class-level and pair-wise deep feature learning as optimization of within-class and between-class similarities. It shows that conventional losses have inflexible gradients and ambiguous convergence, motivating a unified similarity-pair formulation.

Deep feature learning seeks to maximize within-class similarity sp and minimize between-class similarity sn, with cosine targets sp → 1 and sn → 0.
Class-level and pair-wise learning are treated as two elemental paradigms that can be viewed through the same similarity-optimization framework.Class-level methods optimize sample-to-proxy similarities, whereas pair-wise methods optimize similarities between samples.
The unified loss iterates over within-class and between-class similarity pairs to reduce their weighted difference and can degenerate into classification or hard-mined triplet losses.For class-level labels, similarities are computed against target and non-target weight vectors; for pair-wise labels, they are computed against positive and negative samples.
Circle loss instead assigns adaptive weighting factors so less-optimized similarities receive larger gradients and convergence favors a specified status.Its circular decision boundary is intended to establish a more definite optimization target and benefit feature separability.
Conventional losses optimize (sn − sp) with equal gradients, so poorly and well-optimized similarity pairs can receive similar penalties.At a status with sn = sp = 0.8, the loss still applies a large gradient to sp; penalties also remain approximately equal across statuses before convergence.

3. A New Loss Function

Circle loss adaptively reweights individual similarity scores, giving less-optimized scores stronger gradients and more-optimized scores milder updates. Its circular decision boundary establishes a more definite convergence target than losses that optimize s_n−s_p.

3.1. Self-paced Weighting: Circle loss lets each similarity score learn at its own pace by generalizing the objective to independently weighted positive and negative similarities.The weighting factors are implemented as adaptive linear functions of the similarities and are constrained to be non-negative.
3.1. Self-paced Weighting: Similarity scores farther from their optima receive larger weighting factors and consequently larger back-propagated gradients.This self-paced weighting is intended to provide effective updates for poorly optimized scores.
3.1. Self-paced Weighting: Unlike conventional equal re-scaling, Circle loss multiplies each similarity by an independent weighting factor before applying the common scale factor γ.This removes the equal re-scaling constraint and enables more flexible optimization.
3.2. Within-class and Between-class Margins: Circle loss uses separate within-class and between-class margins because s_n and s_p occupy asymmetric positions in its objective.The parameters Δ_n and Δ_p represent the between-class and within-class margins, respectively.
3.2. Within-class and Between-class Margins: Its decision boundary is circular, centered at s_n=(O_n+Δ_n)/2 and s_p=(O_p+Δ_p)/2, with a radius determined by the boundary equation.After reducing the hyperparameters, Circle loss aims for s_p→1 and s_n→0, while m controls the boundary radius as a relaxation factor.
3.3. Optimization Properties: Circle loss balances optimization by assigning the larger gradient to the less-optimized member of a similarity pair.Its gradients also attenuate as training approaches convergence, supporting mild optimization near the target.
3.3. Optimization Properties: Circle loss favors a more definite convergence status because its circular boundary prefers T over T′, unlike the homogeneous boundary of losses minimizing s_n−s_p.The paper attributes this preference to T having the smallest similarity gap among points on the boundary and reports a more concentrated similarity distribution.

4. Experiments

Circle loss is evaluated across class-level and pair-wise deep feature learning tasks, including face recognition, re-identification, and fine-grained retrieval. It generally matches or exceeds competing methods, while showing robustness to key hyper-parameter settings and improved similarity optimization.

Experimental Settings: Circle loss is evaluated on face recognition, person re-identification, and fine-grained image retrieval under class-level and pair-wise learning settings.The experiments use face datasets, Market-1501 and MSMT17 for re-identification, and CUB-200-2011, Cars196, and Stanford Online Products for retrieval.
Face Recognition: +0.13% rank-1 accuracy over ArcFace with ResNet34 and +0.14% with ResNet100 on MFC1 show Circle loss marginally outperforms competing face-recognition losses.With ResNet100, ArcFace reaches 98.36% rank-1 accuracy before Circle loss exceeds it by +0.14%; verification shows the same pattern.
Face Recognition: +0.05%, +0.06% and +0.18% over ArcFace on LFW, YTF and CFP-FP, respectively, make Circle loss the best among the compared methods on these verification datasets.These datasets are described as near saturation, so the reported differences are small but consistent.
Face Recognition: +1.16% and +2.55% over ArcFace on TAR@FAR=1e-4 and TAR@FAR=1e-5 with ResNet34 demonstrate considerable gains on IJB-C 1:1 verification.The comparison is made against AM-Softmax and ArcFace under both ResNet34 and ResNet100 backbones.
Person Re-identification: Circle loss achieves competitive re-identification accuracy, exceeds AM-Softmax, and gains incremental improvement when combined with MGN part-level features.JDGL is slightly higher than MGN + Circle loss on MSMT17, while MGN + Circle loss improves over ResNet50 + Circle loss.
Fine-grained Image Retrieval: Circle loss reaches state-of-the-art-level performance on CUB-200-2011, Cars196, and Stanford Online Products without hard mining strategies or model ensembles.This evaluation targets pair-wise labeled data and compares against methods using elaborate mining or ensemble techniques.
Hyper-parameter Analysis: Circle loss remains robust across γ from 32 to 1024 for AM-Softmax and Circle loss, and surpasses ArcFace and AM-Softmax for m from −0.05 to 0.25.ArcFace becomes unstable with larger γ in the reported implementation, whereas Circle loss shows high robustness to γ.
Optimization Analysis: Circle loss gives less-optimized similarities larger gradients, producing balanced optimization, stronger within-class compactness, and greater between-class discrepancy than AM-Softmax on training data.The reported dynamics show larger early weights for within-class similarities, followed by improved compactness and discrepancy at training’s end.

5. Conclusion

Circle loss provides a unified, flexible optimization framework for class-level and pair-wise deep feature learning, with a more definite convergence target. Across face recognition, person re-identification, and fine-grained image retrieval, it achieves performance on par with the state of the art.

Circle loss assigns different penalty strengths to similarity scores, enabling them to learn at different paces.This yields higher flexibility in optimization than treating similarity scores symmetrically.
Circle loss uses one unified formula for learning with class-level labels and pair-wise labels.
Circle loss offers a more definite convergence target than optimization methods based on similarity differences.
Circle loss achieves performance on par with the state of the art across face recognition, person re-identification, and fine-grained image retrieval.