Source-linked AI summary
Imbalanced Deep Learning by Minority Class Incremental Rectification
Qi Dong, Shaogang Gong, Xiatian Zhu
TL;DR
Severely imbalanced training data can bias deep models toward majority classes, especially in large-scale multi-label visual learning. The paper introduces batch-wise incremental minority-class rectification through hard-sample mining and CRL, achieving reported advantages across benchmark settings, including faster overall training than LMLE.
Problem
Deep learning methods largely assume balanced or moderately imbalanced data, leaving severely imbalanced large-scale multi-label learning insufficiently addressed.
Method
CRL performs batch-wise incremental rectification of minority classes by mining hard samples from majority-class boundary regions during end-to-end training.
Results
CRL surpasses the best alternative LMLE by 3% on CelebA and 5% on X-Domain, while providing over 7× faster model training.
Takeaways & Limitations
The framework is reported to scale to significantly imbalanced multi-label visual data while preserving stochastic-gradient-descent optimization convergence characteristics.
Takeaways & Limitations
The paper frames conventional cross-entropy as suboptimal under highly imbalanced data and motivates explicit minority-class structural discrimination.
Abstract
from arXiv · showhide
Model learning from class imbalanced training data is a long-standing and significant challenge for machine learning. In particular, existing deep learning methods consider mostly either class balanced data or moderately imbalanced data in model training, and ignore the challenge of learning from significantly imbalanced training data. To address this problem, we formulate a class imbalanced deep learning model based on batch-wise incremental minority (sparsely sampled) class rectification by hard sample mining in majority (frequently sampled) classes during model training. This model is designed to minimise the dominant effect of majority classes by discovering sparsely sampled boundaries of minority classes in an iterative batch-wise learning process. To that end, we introduce a Class Rectification Loss (CRL) function that can be deployed readily in deep network architectures. Extensive experimental evaluations are conducted on three imbalanced person attribute benchmark datasets (CelebA, X-Domain, DeepFashion) and one balanced object category benchmark dataset (CIFAR-100). These experimental results demonstrate the performance advantages and model scalability of the proposed batch-wise incremental minority class rectification model over the existing state-of-the-art models for addressing the problem of imbalanced data learning.
1 INTRODUCTION
Large-scale multi-label visual data are often severely class imbalanced, while existing methods are poorly suited to this setting. The paper introduces batch-wise minority class rectification with hard-sample mining and reports advantages over state-of-the-art models.
- Class imbalance biases learning toward frequent classes, harming minority-class recognition despite its importance for rare event discovery.
- Existing imbalanced-learning studies mainly target small-scale, single-label binary problems with imbalance ratios within 1:100.
- Large-scale person-attribute recognition is challenging because images carry multiple labels, attribute classes can be fine-grained, and balanced data are difficult to obtain.
- Conventional methods are poorly suited to severely imbalanced visual data because hand-crafted features, deep learning itself, and direct algorithm integration remain inadequate.
- The paper addresses multi-label learning with imbalance ratios greater than 1:1000 using batch-wise hard mining for incremental minority-class rectification and a Class Rectification Loss.
- The approach learns all attribute labels concurrently without additional single-label sampling assumptions and is independent of overall training size, label count, and global clustering.
- Evaluations on CelebA, X-Domain, DeepFashion, and CIFAR-100 report clear advantages over 12 state-of-the-art models and test scalability across controlled imbalance ratios.
2 RELATED WORK
Prior imbalance methods rebalance data or modify learning algorithms, but generally focus on small-scale, non-extreme, single-label settings. The paper distinguishes CRL through end-to-end, batch-wise minority hard-sample mining for scalable deep learning.
- Data-level methods down-sample majority classes or over-sample minority classes, but can discard information or overfit duplicated samples.
- Algorithm-level methods emphasize minority classes through costs or thresholds, yet cost matrices are often expert-defined, problem-specific, and nonscalable.
- LMLE requires computationally expensive clustering and quintuplet construction, including quadratic O(n^2) search, for each attribute label.
- The related comparison uses feature distributions from balanced, imbalanced, and CRL-trained ResNet32 models, highlighting minority-class separation under imbalance.
- Hybrid and classical techniques generally target limited data, non-extreme imbalance ratios, single-label classification, and hand-crafted low-dimensional features.
- Earlier neural methods address small-scale imbalance or use neural networks as nonlinear classifiers without end-to-end learning.
- Unlike deep metric learning and most attribute methods, CRL explicitly addresses imbalance through batch-wise incremental rectification of minority-class structure.
3 SCALABLE IMBALANCED DEEP LEARNING
The paper formulates scalable multi-label deep learning for extremely imbalanced person-attribute data by adding batch-wise minority-class rectification to cross-entropy. CRL combines minority profiling, hard-sample mining, and imbalance-adaptive boundary regularisation.
- Problem: The target problem is end-to-end multi-label, multi-class person-attribute recognition with weakly labelled regions and highly unequal class frequencies.The setting covers facial and clothing attributes in large training collections.
- Class Rectification Loss: CRL adds batch-wise class balancing to cross-entropy, incrementally promoting under-represented minority classes during model learning.The regularisation is designed to rectify the learning bias caused by majority-class over-representation.
- Cross-Entropy Limitation: Cross-entropy alone learns decision boundaries biased toward majority classes when training data are highly imbalanced, neglecting minority-class structure.The proposed remedy explicitly models inter-class geometry around minority decision boundaries.
- Minority Class Hard Sample Mining: Each mini-batch profiles per-label class counts, defines minority classes as the smallest collectively accounting for at most 50% of samples, and then mines hard samples at class and instance levels.Classes with fewer than two samples in a batch are ignored for minority processing.
- Minority Class Hard Sample Mining: Hard-positives expose sparsely sampled minority boundaries, while hard-negatives improve margins against visually similar distracting classes.Hardness is measured by prediction scores for class-level mining and feature distances for instance-level mining.
- Minority Class Neighbourhood Rectification: CRL uses learning-to-rank and contrastive constraints to reinforce minority margins, with regularisation weight scaled linearly by an overall class-imbalance measure.The imbalance-adaptive weighting assigns more weight to more imbalanced labels.
4 EXPERIMENTS
Experiments evaluate CRL on imbalanced facial and clothing attributes, balanced and simulated-imbalanced CIFAR-100, and controlled design choices. CRL consistently outperforms comparison models while reducing overall CNN training time relative to LMLE.
- Datasets and evaluation: CelebA, X-Domain, DeepFashion, and CIFAR-100 provide imbalanced, balanced, and simulated-imbalance settings for evaluating CRL.CelebA includes facial attributes, X-Domain and DeepFashion include clothing attributes, and CIFAR-100 supports controlled class-distribution experiments.
- Facial attribute recognition: CRL improves CelebA mean accuracy by 6% over DeepID2 and 3% over LMLE on class-balanced accuracy.Classic resampling and cost-sensitive methods provide smaller gains, while down-sampling and threshold adjustment reduce performance.
- Clothing attribute recognition: CRL surpasses prior state-of-the-art models on all X-Domain attribute labels, exceeding LMLE by 4.65% in mean accuracy.X-Domain reaches a maximum imbalance ratio of 4,162, compared with 43 for CelebA attributes.
- Object recognition: CRL improves CifarNet, ResNet32, and DenseNet accuracy by 3.6%, 1.2%, and 0.8%, respectively, even on balanced CIFAR-100.The evaluation also constructs CIFAR-100imb(γ) using a power-law class distribution and a corresponding balanced dataset.
- Further evaluations and discussions: Hard mining adds 1.53% mean accuracy, joint learning adds 3.12% over LMLE, and η=0.01 is the best tested CRL loss weight.Applying CRL to all classes reaches 81.30% mean accuracy but costs 28.5% more training than minority-only scope; κ=25 is selected because larger values do not improve learning.
5 CONCLUSION
The paper presents CRL as an end-to-end framework for learning from severely imbalanced, multi-label visual data. Experiments report advantages over alternative imbalanced-learning and deep-recognition methods, including accuracy gains and faster training.
- CRL performs batch-wise incremental minority class rectification using scalable hard sample mining during end-to-end deep learning.It regularises bias from extremely imbalanced training data while preserving stochastic-gradient-descent convergence characteristics.
- CRL surpasses LMLE by 3% on CelebA and 5% on the extremely imbalanced X-Domain clothing attribute benchmark.
- CRL provides over 7× faster model training than LMLE while achieving the reported benchmark improvements.
- Experiments also show benefits from CRL when learning standard deep models with class-balanced training data.