Source-linked AI summary
Targeted Supervised Contrastive Learning for Long-Tailed Recognition
Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio Feris, Piotr Indyk, Dina Katabi
TL;DR
Long-tailed imbalance can produce poor feature uniformity and unclear minority-class boundaries under supervised contrastive learning. TSC assigns classes uniformly distributed hyperspherical targets during training, and experiments report state-of-the-art performance across long-tailed benchmarks. The method requires the number of classes in advance and uses approximate target locations because the analytical Thomson-problem solution remains open.
Problem
Long-tailed imbalance can cause supervised contrastive losses to learn reduced uniformity and unclear class boundaries, degrading performance on minority classes.
Method
TSC generates uniformly distributed targets on a hypersphere and adaptively matches samples from each class to its target during contrastive training.
Results
TSC achieves state-of-the-art single-model performance on CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist.
Takeaways & Limitations
TSC learns a more uniform and balanced feature space for long-tailed recognition, improving class boundaries and performance across benchmark datasets.
Takeaways & Limitations
TSC requires knowing the number of classes in advance, and its hyperspherical targets are based on an approximate solution because the analytical Thomson-problem solution remains open.
Abstract
from arXiv · showhide
Real-world data often exhibits long tail distributions with heavy class imbalance, where the majority classes can dominate the training process and alter the decision boundaries of the minority classes. Recently, researchers have investigated the potential of supervised contrastive learning for long-tailed recognition, and demonstrated that it provides a strong performance gain. In this paper, we show that while supervised contrastive learning can help improve performance, past baselines suffer from poor uniformity brought in by imbalanced data distribution. This poor uniformity manifests in samples from the minority class having poor separability in the feature space. To address this problem, we propose targeted supervised contrastive learning (TSC), which improves the uniformity of the feature distribution on the hypersphere. TSC first generates a set of targets uniformly distributed on a hypersphere. It then makes the features of different classes converge to these distinct and uniformly distributed targets during training. This forces all classes, including minority classes, to maintain a uniform distribution in the feature space, improves class boundaries, and provides better generalization even in the presence of long-tail data. Experiments on multiple datasets show that TSC achieves state-of-the-art performance on long-tailed recognition tasks.
1. Introduction
Long-tailed recognition is difficult because head classes dominate learning, harming minority-class representation and decision boundaries. TSC addresses this by matching classes to uniformly distributed hyperspherical targets and achieves state-of-the-art performance across benchmarks.
- Long-tailed data contains many head-class instances but few tail-class instances, making minority classes especially important in applications such as diagnosis, autonomous driving, and fairness.
- Supervised contrastive learning can provide strong gains, but applying contrastive losses to imbalanced data yields poor feature uniformity and hampers performance.Ideally, class embeddings are uniformly distributed on a hypersphere, maximizing inter-class margins and improving generalizability.
- Imbalance weights majority-class losses more heavily, causing class centers to become non-uniform and minority classes to lose separability in the feature space.KCL class centers are uniform when classes are balanced but exhibit poor uniformity under high imbalance.
- Oversampling and loss re-weighting can improve tail-class performance while harming head classes, motivating instance-balanced sampling that still learns a uniform feature space.
- TSC generates uniformly distributed class targets offline and adaptively matches samples to targets during contrastive training, producing a class-balanced feature space across imbalance ratios.Semantically close classes are matched to nearby targets, while semantically distant classes are matched to relatively distant targets.
- TSC improves state-of-the-art performance on CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist.
2. Related Works
Prior long-tailed recognition methods rebalance data or losses, while contrastive learning offers a representation-learning alternative. Related work also motivates separating representation learning from classifier learning and studying contrastive-learning properties.
- Conventional long-tailed methods use data re-sampling or class-dependent loss weighting to address head-class bias.These approaches can improve tail-class performance at the expense of head-class performance.
- Decoupling representation learning from classifier learning can produce good features, motivating feature-extractor pretraining for long-tailed recognition.Self-supervised initialization and self-supervised learning have also been studied for reducing imbalance bias and improving robustness.
- Contrastive learning aligns positive sample pairs and repels negative sample pairs, supporting successful applications across multiple tasks.
3. Method
TSC learns a class-balanced feature space by assigning classes to uniformly distributed hyperspherical targets and pulling their samples toward those targets during contrastive training. Its matching-training scheme preserves semantic structure while accommodating computational and geometric constraints.
- Target generation and assignment: TSC pre-computes uniformly distributed class targets on a hypersphere and adaptively assigns classes to them during training.The targets are fixed before training, while class-target assignments are updated online.
- Target generation: The target positions are optimized to be as far apart as possible, ideally forming the vertices of a regular simplex.When the hypersphere dimension is too small, the minimum cannot generally be computed analytically.
- Matching-training scheme: Random class-target matching can produce poor semantics, so TSC seeks nearby targets for semantically close classes and relatively distant targets for semantically distant classes.The paper uses a heuristic, adaptive matching procedure based on class centers and the Hungarian Algorithm.
- Targeted loss: The targeted supervised contrastive loss combines standard contrastive learning with a target-based term that pulls samples toward their class target and away from other targets.This aligns each class with its assigned target while retaining the standard contrastive component.
4. Experiments
Experiments evaluate TSC across standard and large-scale long-tailed benchmarks using two-stage training, showing consistent improvements over established baselines and compatibility with ensembling.
- Experiment Setup: TSC is evaluated on CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist using established long-tailed recognition protocols.CIFAR experiments use ResNet-32 with imbalance ratios 10, 50, and 100; ImageNet-LT and iNaturalist use ResNet-50.
- Experiment Setup: TSC uses two-stage training: TSC-loss representation learning followed by linear classifier training with dataset-specific losses and sampling.CIFAR classifiers use LDAM with class re-weighting, while ImageNet-LT and iNaturalist classifiers use cross-entropy with class-balanced sampling.
- CIFAR-10-LT & CIFAR-100-LT: TSC consistently improves over all baselines across CIFAR-10-LT and CIFAR-100-LT imbalance ratios.Its design does not require prior knowledge of the dataset imbalance ratio.
- ImageNet-LT: On ImageNet-LT, TSC outperforms τ-norm by 5.7%, cRT by 5.1%, and LWS by 4.7%, while improving over KCL across all class splits.The gains over KCL are 1.1% for many, 0.7% for medium, and 0.9% for few classes.
- iNaturalist: TSC achieves the best performance among all baselines on every iNaturalist class split.The benchmark contains 8142 classes and represents a challenging large-scale long-tailed setting.
- Combination with Ensembling: Combining TSC with the ensemble method RIDE further improves performance on ImageNet-LT.This combination replaces RIDE’s original stage-1 training with TSC while retaining its stage-2 procedure.
5. Analysis
The analysis evaluates representation quality through alignment, uniformity, neighborhood uniformity, and semantic reasonability, then compares TSC with KCL and ablates its sampling and matching components. Across these analyses, TSC produces more balanced feature spaces, clearer class separation, and better generalization while preserving semantic structure.
- 5.1. Understanding the Learned Representations: Representation quality is evaluated using intra-class alignment, inter-class uniformity, neighborhood uniformity, and semantic reasonability.Alignment measures distances within classes; inter-class uniformity measures distances between class centers, while neighborhood uniformity and reasonability assess nearby classes and semantic structure.
- 5.1. Understanding the Learned Representations: Neighborhood uniformity captures whether each class is too close to its nearest class centers, which affects the clarity of decision boundaries.Unlike overall alignment and uniformity, it focuses on the top-k closest class centers because those neighbors determine potentially unclear boundaries.
- 5.1. Understanding the Learned Representations: TSC achieves better uniformity, neighborhood uniformity, and reasonability than KCL across ImageNet-LT class splits while maintaining almost the same alignment.The comparison uses k=10 for neighborhood uniformity and reasonability.
- 5.1. Understanding the Learned Representations: On CIFAR-10-LT with imbalance ratio 100, KCL exhibits collapsed class pairs and empty feature-space regions, whereas TSC yields clearer separation and better classification performance.The visualization reports class-center collapse for KCL and good uniformity with clear class separation for TSC.
- 5.2. Ablations: Class-balanced sampling performs worse than standard KCL on CIFAR-10 and CIFAR-100, while TSC uses instance-balanced sampling and obtains good uniformity through pre-computed targets.The result supports preserving instance-balanced sampling while controlling the feature distribution with targets.
- 5.2. Ablations: Balanced positive sampling improves KCL by 1.7% over FCL but improves TSC with FCL by only 0.7%.The smaller gain for TSC is attributed to its already balanced feature space and naturally balanced within-class alignment.
- 5.2. Ablations: Removing online matching substantially worsens TSC’s reasonability and produces much poorer generalization performance.Random target assignment can preserve training uniformity while placing semantically close classes far apart, such as deer and horse.
- 5.2. Ablations: The final target-generation objective remains stable across random seeds, supporting the stability of the numerical target-generation procedure.The targets are generated by stochastic-gradient-descent approximation.
6. Conclusion & Limitations
The paper concludes that TSC addresses reduced uniformity and unclear class boundaries in long-tailed recognition by assigning uniformly distributed targets to classes. It reports state-of-the-art single-model performance across benchmark datasets, while requiring approximate target generation and advance knowledge of the class count.
- Conclusion: TSC assigns uniformly distributed targets to classes during training, producing a more uniform and balanced feature space than traditional supervised contrastive losses on unbalanced data.The paper links traditional losses under imbalance with reduced uniformity, unclear class boundaries, and poorer performance.
- Conclusion: TSC achieves state-of-the-art single-model performance on all benchmark datasets evaluated for long-tailed recognition.The conclusion bases this claim on extensive experiments across multiple datasets.
- Limitations: TSC’s targets are obtained by stochastic-gradient-descent approximation because the analytical Thomson-problem solution remains open.The paper reports consistent and significant empirical gains despite this approximation.
- Limitations: TSC requires the number of classes to be known in advance, limiting applicability when the class count is unknown.This requirement arises because the targets must be computed for the class set.