Source-linked AI summary
Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets
Tong Wu, Qingqiu Huang, Ziwei Liu, Yu Wang, Dahua Lin
TL;DR
Multi-label long-tailed recognition must handle both label co-occurrence and the dominance of negative labels in binary classification. Distribution-Balanced Loss modifies binary cross-entropy with co-occurrence-aware re-balanced weighting and negative-tolerant regularization. Experiments on Pascal VOC and COCO show significant improvement over previous methods.
Problem
Multi-label long-tailed recognition is challenging because label co-occurrence complicates balancing and most classes are negative for each image.
Method
Distribution-Balanced Loss combines re-balanced weighting that accounts for co-occurring labels with negative-tolerant regularization using a margin and re-scaling factor.
Results
The proposed loss achieves significant improvement over previous methods on the Pascal VOC and MS COCO multi-label recognition benchmarks.
Takeaways & Limitations
The experiments validate Distribution-Balanced Loss as an effective approach for multi-label long-tailed visual recognition.
Takeaways & Limitations
The re-balanced weighting is applied to both positive and negative labels although its derivation originally concerns only positive labels.
Abstract
from arXiv · showhide
We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions. Compared to conventional single-label classification problem, multi-label recognition problems are often more challenging due to two significant issues, namely the co-occurrence of labels and the dominance of negative labels (when treated as multiple binary classification problems). The Distribution-Balanced Loss tackles these issues through two key modifications to the standard binary cross-entropy loss: 1) a new way to re-balance the weights that takes into account the impact caused by label co-occurrence, and 2) a negative tolerant regularization to mitigate the over-suppression of negative labels. Experiments on both Pascal VOC and COCO show that the models trained with this new loss function achieve significant performance gains over existing methods. Code and models are available at: https://github.com/wutong16/DistributionBalancedLoss .
1 Introduction
Multi-label long-tailed recognition combines skewed class frequencies with label co-occurrence and a dominance of negative labels. Distribution-Balanced Loss addresses these challenges through co-occurrence-aware re-balancing and negative-tolerant regularization.
- Real-world visual recognition commonly involves long-tailed object-category distributions and images associated with multiple semantic labels.
- Binary cross-entropy with inverse-frequency class re-weighting often provides limited improvement because it ignores label co-occurrence and negative-label dominance.
- Re-sampling co-occurring images may fail to balance classes because selecting rare concepts also selects frequently occurring labels.
- Distribution-Balanced Loss combines co-occurrence-aware re-balanced weighting with negative-tolerant regularization to address the two multi-label challenges.The regularization uses a margin and a re-scaling factor to reduce over-suppression of negative labels.
- Experiments on Pascal VOC and MS COCO report remarkable improvement over previous methods.
2 Related Work
Prior long-tailed recognition methods mainly use re-sampling or cost-sensitive learning. These approaches balance data or training costs but involve distinct trade-offs and weighting strategies.
- Long-tailed recognition research mainly follows re-sampling and cost-sensitive learning.
- Re-sampling: Over-sampling minority classes can overfit duplicated samples, whereas under-sampling majority classes can weaken feature learning by omitting valuable instances.
- Cost-sensitive Learning: Class-level cost-sensitive methods commonly weight classes by inverse frequency or its square root, with later methods using effective sample numbers or larger rare-class margins.
3 Distribution-Balanced Loss
Distribution-Balanced Loss addresses long-tailed multi-label training with re-balanced weighting that accounts for label co-occurrence and negative-tolerant regularization. These components are integrated into a unified loss that smooths classifier outputs, especially for tail classes.
- Re-balanced Weighting after Re-sampling: Class-aware sampling can create inner-class imbalance because co-occurring labels make class selections non-independent.Sampling instances for one class influences the sample counts of other classes that occur in the same instances.
- Re-balanced Weighting after Re-sampling: The re-balanced weighting strategy estimates sampling frequencies from label co-occurrence and compensates for the gap between expected and actual instance sampling times.The method uses conditional co-occurrence probabilities and defines a weight to close this sampling-frequency gap.
- Re-balanced Weighting after Re-sampling: A smoothing function maps re-balancing weights into a proper range because weights near zero can make optimization difficult.Its parameters provide an overall weight lift and control a mapping that rises rapidly near zero and flattens near one.
- Negative-Tolerant Regularization: BCE can over-suppress negative labels because sigmoid treats class logits independently, continuously pushing negative logits lower.This can make tail-class classifiers over-fit to limited positives and produce poor generalization.
- Negative-Tolerant Regularization: Negative-tolerant regularization uses non-zero bias thresholds, linear scaling of negative logits, and gradient constraints to slow optimization after negative logits become sufficiently low.The class-specific bias accounts for model bias toward passive predictions on tail classes.
- Distribution-Balanced Loss: The final Distribution-Balanced Loss integrates re-balanced weighting with negative-tolerant regularization for end-to-end training.The loss is reported to smooth classifier-output distributions, particularly for tail classes, and to achieve superior performance on long-tailed multi-label datasets.
4 Experiments
Experiments evaluate Distribution-Balanced Loss on artificially long-tailed VOC-MLT and COCO-MLT using mAP across head, medium, and tail classes. Step-wise and combination studies show that re-balanced weighting and negative-tolerant regularization complement re-sampling, improving performance across class groups.
- Experimental Settings: The experiments use VOC-MLT and COCO-MLT, artificially constructed long-tailed subsets evaluated with overall and group-wise mAP.Classes are divided into head, medium, and tail groups by training-example counts.
- Benchmarking Results: Class-aware re-sampling sharply raises tail-class mAP but lowers head-class performance by about 0.9%.The VOC-MLT results show this trade-off directly.
- Benchmarking Results: Re-balanced weighting fixes the head-class drop and improves mAP for head, medium, and tail classes.It is evaluated as an enhancement to re-sampling rather than a redundant replacement.
- Benchmarking Results: DB-Loss adds about 0.8% average improvement and raises tail-class mAP by about 1.0%.The reported comparison follows re-sampling, re-balanced weighting, and focal-loss combinations.
- Ablation Study: Step-wise evaluation finds that negative-tolerant regularization benefits many classes and produces a marked tail-class improvement.Re-balanced weighting generally improves performance, while regular re-sampling gives little or negative increment for head and medium classes.
- Further Analysis: The smoothing-function and regularization analyses identify λ ∈[5, 10] as best for VOC, improving tail classes by about 2.5%.At this range, head and medium classes improve by about 1%; the effect of ν is relatively small.
5 Conclusion
Distribution-Balanced Loss addresses multi-label long-tailed recognition through co-occurrence-aware re-balanced weighting and negative-tolerant regularization. Experiments on Pascal VOC and COCO report significant gains over existing methods.
- Conclusion: Distribution-Balanced Loss targets label co-occurrence and negative-label dominance in multi-label long-tailed recognition.Its two ingredients are co-occurrence-aware re-balanced weighting and negative-tolerant regularization.
- Conclusion: Experiments on Pascal VOC and COCO validate the loss and report significant performance gains over existing methods.The authors position the method as a strong baseline for future research.
A.1 Dataset Construction
VOC-MLT and COCO-MLT are constructed by reshaping VOC and COCO class frequencies with a Pareto reference distribution. Classes are processed from head to tail so tail-class sample counts remain constrained.
- Dataset Construction: The datasets are built from VOC and COCO subsets using a Pareto distribution whose α controls distribution shape.The probability distribution is truncated at CDF 0.99 and rescaled to the maximum per-class sample count.
- Dataset Construction: Classes are ranked by original-data statistics and processed head-to-tail, with instances randomly added or eliminated toward the reference distribution.This incremental construction constrains tail classes to relatively small sample counts.
A.2 Implementation Details of Comparing Methods
The comparison methods are adapted for multi-label long-tailed experiments with method-specific settings. Adjustments include effective-number averaging for class-balanced loss, long-tailed adjacency matrices for ML-GCN, and class-level margins for LDAM.
- Comparing Methods: The long-tailed test distributions are assessed on the whole class set and on head, medium, and tail subsets using mAP.The comparison includes multiple adapted multi-label and long-tailed recognition methods.
- Comparing Methods: Class-balanced loss uses β = 0.99 for VOC-MLT and β = 0.9 for COCO-MLT, plus an extra loss weight of 10.Effective numbers are averaged across positive classes in a label set.
- Comparing Methods: ML-GCN uses a 256-dimensional hidden layer and an adjacency matrix generated from the long-tailed datasets.Using adjacency matrices from the original datasets produces little difference.
- Comparing Methods: LDAM uses a class-level margin with C tuned so the largest margin equals 0.5.This follows the cited class-level margin adaptation for the multi-label setting.
A.3 The Effect of µ in Smoothing Function.
The study finds that µ has relatively little influence on results, so separate values are selected for COCO-MLT and VOC-MLT main experiments.
- µ has a relatively small influence on the results.The passage reports this effect for the smoothing function.
- The main COCO-MLT experiments use µ = 0.2.This value was selected from an insignificant peak in the observed influence.
- The main VOC-MLT experiments use µ = 0.3.This value was selected from an insignificant peak in the observed influence.
A.4 The Effect of ν in Negative-tolerant Regularization
The study varies ν through κ while fixing λ, finding good results at κ = 0 and little sensitivity across a small κ range.
- ν is varied by changing κ while λ is fixed at 2 for COCO-MLT and 5 for VOC-MLT.The independent effect of ν is examined under these fixed regularization settings.
- κ = 0 produces a relatively good result.This indicates that the regularization thresholds can be fixed at zero.
- Changing κ within a small range has little effect.The reported result suggests limited sensitivity to κ near the tested setting.