Source-linked AI summary
Balanced Contrastive Learning for Long-Tailed Visual Recognition
Jianggang Zhu, Zheng Wang, Jingjing Chen, Yi-Ping Phoebe Chen, Yu-Gang Jiang
TL;DR
Long-tailed data expose a representation-learning gap because head classes dominate supervised contrastive learning and prevent an ideal regular-simplex geometry. The paper proposes balanced contrastive learning with class averaging and class complement in a two-branch framework, achieving competitive results on long-tailed benchmarks. Its analysis and guarantees rely on stated assumptions about batch composition and normalized representations.
Problem
Representation-learning methods for imbalanced data are under-explored, while cross-entropy struggles to represent and classify tail classes.
Method
BCL combines class-averaging and class-complement with a classification branch using logit-compensated cross-entropy.
Results
The two-branch BCL framework achieves competitive results on long-tailed CIFAR, ImageNet-LT, and iNaturalist 2018 benchmarks.
Takeaways & Limitations
BCL produces a balanced feature space by optimizing classes toward a regular simplex while addressing classifier bias with logit compensation.
Takeaways & Limitations
The theoretical analysis assumes K ≥ h + 1 and normalized representations, and includes a stated condition on class-specific loss values.
Abstract
from arXiv · showhide
Real-world data typically follow a long-tailed distribution, where a few majority categories occupy most of the data while most minority categories contain a limited number of samples. Classification models minimizing cross-entropy struggle to represent and classify the tail classes. Although the problem of learning unbiased classifiers has been well studied, methods for representing imbalanced data are under-explored. In this paper, we focus on representation learning for imbalanced data. Recently, supervised contrastive learning has shown promising performance on balanced data recently. However, through our theoretical analysis, we find that for long-tailed data, it fails to form a regular simplex which is an ideal geometric configuration for representation learning. To correct the optimization behavior of SCL and further improve the performance of long-tailed visual recognition, we propose a novel loss for balanced contrastive learning (BCL). Compared with SCL, we have two improvements in BCL: class-averaging, which balances the gradient contribution of negative classes; class-complement, which allows all classes to appear in every mini-batch. The proposed balanced contrastive learning (BCL) method satisfies the condition of forming a regular simplex and assists the optimization of cross-entropy. Equipped with BCL, the proposed two-branch framework can obtain a stronger feature representation and achieve competitive performance on long-tailed benchmark datasets such as CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist2018. Our code is available at https://github.com/FlamieZhu/BCL .
1. Introduction
Long-tailed data make representation learning difficult because head classes dominate optimization and compress tail-class features. The paper analyzes this imbalance and proposes BCL with a two-branch framework to form balanced representations and achieve competitive benchmark results.
- Long-tailed datasets contain few high-frequency classes and many low-frequency classes, causing deep models to generalize poorly, especially for rare classes.
- BCL modifies SCL with class-complement prototypes and class-averaging of negative-class gradients to address representation imbalance.
- BCL is combined with logit-compensated cross-entropy in a two-branch framework containing classification and contrastive learning branches.
- Supervised contrastive learning forms asymmetric geometry on long-tailed data because head classes numerically dominate its optimization.
- The proposed framework achieves competitive results on several popular long-tailed datasets.
2. Related Work
Prior long-tailed recognition methods rebalance data or losses, while contrastive approaches target representation quality with specialized mechanisms. BCL differs by addressing majority-class domination through its balanced contrastive loss and class prototypes.
- Early long-tailed methods primarily rebalance training through resampling or reweighting.
- Contrastive approaches use branches, prototypes, centers, or equalized positives to improve representations or reduce imbalance-related degradation.
- BCL differs from Hybrid-SC by introducing a balanced contrastive loss specifically to address majority-class domination.
- Unlike TSC, which learns targets without class semantics, BCL uses class prototypes as extra contrastive samples.
3. Method
The method analyzes why supervised contrastive learning becomes geometrically asymmetric on long-tailed data and introduces balanced contrastive learning to restore class-independent optimization. BCL combines class-averaging and class-complement within a two-branch framework that uses classifier-derived prototypes.
- Drawbacks of SCL: SCL collapses within-class representations but its repulsion term is dominated by high-frequency classes, reducing inter-class separability on long-tailed data.The attraction term promotes intra-class collapse regardless of class frequency, whereas imbalance primarily affects inter-class repulsion.
- Balanced contrastive learning: Class-averaging equalizes gradient contributions across negative classes by averaging the negative samples from each class.This reduces the gradients contributed by head-class negatives, although unequal class sampling can still destabilize optimization.
- Balanced contrastive learning: Class-complement adds class-center prototypes so every class appears in each mini-batch, making the BCL lower bound class-independent.In practice, class-complement is applied before class-averaging.
- Balanced contrastive learning: At its lower bound, BCL gives every negative class equal gradient contribution and produces a regular-simplex structure with less head-class bias.The loss of each sample is consistent and class-independent at the lower bound.
- Framework: The framework jointly trains classification and balanced contrastive branches sharing an encoder, while transformed classifier weights serve as class prototypes.Representations used for the balanced contrastive loss are ℓ2-normalized.
4. Experiment
Experiments evaluate BCL through ablations and long-tailed benchmarks, showing that its two components jointly improve performance and that it consistently performs strongly across datasets and class-frequency groups.
- Experimental setup: The experiments cover long-tailed CIFAR-10, CIFAR-100, ImageNet-LT, and iNaturalist 2018 using the reported backbone and training configurations.CIFAR experiments use ResNet-32; ImageNet-LT and iNaturalist experiments use ResNet-50 variants or ResNet-50.
- Ablation study: L1 achieves the best performance among class-averaging implementations, while prototype-based L3 outperforms L2.The authors attribute L3’s advantage over L2 to prototypes’ well-represented characteristics.
- Ablation study: Using class-complement or class-averaging alone does not improve overall accuracy, whereas applying both yields a significant performance boost.The ablation compares the primary BCL components against LC and SC baselines.
- Main results: BCL consistently outperforms other methods on long-tailed CIFAR, with a larger advantage at higher imbalance factors.The accuracy gap between BCL and Hybrid-SC decreases as imbalance becomes less severe.
- Main results: 4.6 and 1.9 percentage-point gains over DRO-LT and PaCo are reported at 200 and 400 training epochs, respectively, on CIFAR-100-LT.BCL also improves head-class performance while simultaneously improving tail classes.
- Main results: BCL surpasses Balanced Softmax across Many-shot, Medium-shot, and Few-shot groups on ImageNet-LT.The result supports the reported benefit of learned representations for overall performance.
5. Conclusion
The paper frames long-tailed recognition as a representation-learning problem and develops BCL to produce a balanced feature space through regular-simplex optimization.
- Conclusion: The analysis shows that supervised contrastive learning forms an undesired asymmetric geometry for long-tailed data.BCL addresses this representation-learning problem by optimizing all classes toward a regular simplex configuration.
- Conclusion: BCL is designed so that all classes are optimized for a regular simplex configuration yielding a balanced feature space.This conclusion summarizes the proposed balanced contrastive loss.
1. Proof of Theorem 2 and Theorem 3
The proof decomposes the contrastive objective into attraction and repulsion terms, then derives conditions under which balanced contrastive learning reaches a regular-simplex geometry.
- 1. Proof of Theorem 2 and Theorem 3: The proof separates class-specific batch-wise loss into attraction and repulsion terms before analyzing their minimization.The attraction term exhibits variability collapse, after which the solution satisfies a simplex configuration.
- 1. Proof of Theorem 2 and Theorem 3: Jensen-based equality conditions require equal within-class and cross-class inner products across the relevant batch classes.These are expressed through conditions Q1, Q2, and Q3.
- 1. Proof of Theorem 2 and Theorem 3: Under Q5, all within-class representations collapse to their class means, making the label configuration balanced for every batch and class.The proof then uses this balanced configuration to derive the simplex solution.
- 1. Proof of Theorem 2 and Theorem 3: Q6 states that the class-center representations z_c1, . . . , z_cK form a regular simplex.This condition is the geometric endpoint of the proof.
- 1. Proof of Theorem 2 and Theorem 3: The resulting lower bound is |D| log(1 + (K-1)exp(-K/(K-1))).The bound follows after combining the equality conditions and attraction-term analysis.
2. Gradient Analysis
The gradient analysis identifies head-class domination in supervised contrastive learning and shows how class averaging in balanced contrastive learning equalizes negative-class contributions.
- 2. Gradient Analysis: Balanced contrastive loss averages each negative class by its batch size, avoiding a tremendous gradient update from negative head-class samples.Its gradient retains harder-negative emphasis while reducing the influence of easy negatives.
- 2. Gradient Analysis: In supervised contrastive learning, a shared denominator makes each negative-class gradient proportional to its sample count.Because head classes contribute more samples in long-tailed mini-batches, they dominate the gradient.
- 2. Gradient Analysis: This domination pushes all classes away from the head category as far as possible, producing an unbalanced feature space.The effect is especially pronounced because head classes substantially outnumber tail classes within mini-batches.
3. More Results
Additional experiments compare prototype implementations and view augmentations, while qualitative analyses show improved confusion patterns and more uniform, separable BCL features.
- 3. More Results: Our prototype implementation achieves the best results on CIFAR-100-LT, while exponential-moving-average and learnable-parameter prototypes perform similarly.The comparison uses an imbalance factor of 100 and a common 400-epoch training scheme.
- 3. More Results: Stronger augmentation yields better performance on ImageNet-LT across the compared view configurations.The configurations use SimAug for both views, RandAug for one view, or RandAug for both views.
- 3. More Results: On CIFAR-10-LT, BCL more accurately predicts low-frequency classes while also improving high-frequency-class accuracy compared with vanilla cross-entropy and logit compensation.The confusion matrix shows reduced low-frequency-class misclassification and improved high-frequency predictions.
- 3. More Results: BCL features distribute more uniformly on the sphere and are more separable than SCL features on CIFAR-10-LT.The comparison visualizes two-dimensional MLP outputs learned by the two methods.