PaperlayerView exact revision on arXiv

Source-linked AI summary

M2m: Imbalanced Classification via Major-to-minor Translation

Jaehyung Kim, Jongheon Jeong, Jinwoo Shin

arXiv:2004.00431v2cs.CVcs.LGstat.ML

TL;DR

Class-imbalanced datasets make deep networks harder to generalize to balanced test criteria. M2m translates majority samples into minority examples, improving balanced test accuracy across tested datasets over re-sampling and re-weighting methods and surpassing LDAM.

  • Problem

    Class-imbalanced datasets make standard deep-network training harder to generalize, particularly when performance is judged by a class-balanced metric.

    Source: §1 · p. 1

  • Method

    M2m augments minority classes by translating majority-class samples into synthetic minority examples using an independently trained classifier.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    Across all tested datasets, M2m significantly improves balanced test accuracy over re-sampling or re-weighting methods and surpasses LDAM; on Reuters' most severe-imbalance case, balanced accuracy rises relatively 17.1% over standard training.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Majority-sample diversity could help class-imbalanced training even with a simple translation method using a pre-trained classifier.

    Source: §4 · p. 8

    Imperfect translation classifiers can leave source-class features in synthetic samples, a risk that worsens when the source class has few samples.

    Source: §2.3 · p. 3

Abstract

from arXiv · show

In most real-world scenarios, labeled training datasets are highly class-imbalanced, where deep neural networks suffer from generalizing to a balanced testing criterion. In this paper, we explore a novel yet simple way to alleviate this issue by augmenting less-frequent classes via translating samples (e.g., images) from more-frequent classes. This simple approach enables a classifier to learn more generalizable features of minority classes, by transferring and leveraging the diversity of the majority information. Our experimental results on a variety of class-imbalanced datasets show that the proposed method improves the generalization on minority classes significantly compared to other existing re-sampling or re-weighting methods. The performance of our method even surpasses those of previous state-of-the-art methods for the imbalanced classification.

1. Introduction

M2m translates majority examples into synthetic minority samples instead of reusing existing minority samples. The paper reports improved balanced test accuracy over prior re-sampling and re-weighting methods across tested datasets, with results surpassing LDAM.

Introduction

  • Long-tailed data from costly collection and labeling impairs balanced-test generalization; naive rebalancing overfits, and SMOTE struggles under extreme imbalance because it uses same-class minority samples.SMOTE-based methods synthesize minority samples from existing samples of that same class.Source: §1 · p. 1
  • Prior strategies include inverse-frequency loss reweighting and over- or under-sampling, while other work explores margin-based regularization, active learning, and meta-learning.The paper notes that margin-based approaches can serve as data-dependent regularizers for penalizing minority classes more strongly.Source: §1 · pp. 1–2

Contribution

  • M2m generates minority samples by translating majority samples with an independently trained classifier, avoiding existing minority samples while leveraging richer majority information.The paper reports that this approach helps learn more generalizable minority-class features.Source: §1 · p. 2
  • M2m combines an objective that preserves majority-class performance, a rejection criterion favoring samples from more majority classes, and an optimal distribution for selecting majority seeds.The optimization translates a majority sample into a synthetic minority sample without affecting majority-class performance, even when the sample is labeled as minority.Source: §1 · p. 2

Results

  • 17.1% and 9.2% relative balanced-accuracy improvements over standard training and LDAM [4], respectively, are reported for the most severely imbalanced Reuters case.Across the tested datasets, M2m also improves balanced test accuracy over previous re-sampling or re-weighting methods and surpasses LDAM [4]; evaluation covers synthetic CIFAR-10/100 and ImageNet imbalances and real-world CelebA, SUN397, Twitter, and Reuters datasets.Source: §1 · p. 2

2. M2m: Major-to-minor translation

M2m balances imbalanced training by augmenting minority classes with synthetic examples translated from majority samples, using majority diversity to reduce minority overfitting. The method trains for a balanced test criterion and includes safeguards for unreliable translations.

2.1. Overview of M2m

  • M2m constructs a balanced training set by translating majority-class samples into synthetic minority examples, rather than augmenting minority samples directly.It oversamples each class to the largest class size; generated samples are added during training.Source: §2.1 · p. 2, §2 · p. 2
  • A pre-trained classifier g guides each translation toward a target minority class, while a logit penalty suppresses confidence in the seed’s original class.The generated example is labeled as the target class and used to train f; f is not forced to predict that class, allowing it to learn minority features and extend its decision boundary.Source: §2.1 · p. 3, Figure 2 · p. 3

2.2. Underlying intuition on M2m

  • M2m can produce samples close to their majority seeds, yet the authors report that it still improves minority-class generalization.They hypothesize that majority-sample diversity helps prevent minority overfitting and that g captures useful information from the small minority dataset; augmentation supplements rather than replaces the original data.Source: §2.2 · p. 3

2.3. Detailed components of M2m

  • When g may retain seed-class features, M2m rejects translations probabilistically and replaces rejected samples with existing minority examples.The rejection probability depends on the gap between source- and target-class sample counts and β, which controls the assumed reliability of g.Source: §2.3 · pp. 3–4
  • M2m chooses seed classes to balance translation acceptance with source-class diversity, and implements generation through batch-wise resampling during training.For a target class, it samples a seed class using Q and then a seed uniformly within that class; in a balanced mini-batch, an accepted translation replaces the target-class sample.Source: §2.3 · p. 4

3. Experiments

Across the tested datasets, M2m improves minority-class generalization over re-sampling baselines. The ablations support the roles of majority-sample diversity, correct synthetic labels, and adversarial perturbations in its performance.

3.1. Experimental setup

  • Experiments assess balanced-test performance using bACC and GM, report means and standard deviations over three trials, and compare re-sampling, re-weighting, and loss-function baselines.M2m uses an ERM-pretrained auxiliary classifier and deferred scheduling; the CIFAR-LT datasets vary the imbalance ratio while retaining balanced test sets.Source: §3 · p. 4, §3.1 · p. 5, §3.2 · p. 5

3.2. Long-tailed CIFAR datasets

  • Table 1 reports ERM M2m at 78.3±0.16 and LDAM DRW at 77.1±0.49 among classification results; the authors say M2m consistently improves bACC across tested baselines, surpassing LDAM+DRW.The authors also report that applying M2m on top of LDAM can further improve bACC in most cases.Source: §3.2 · pp. 5–6, Table 1 · p. 6

3.3. Real-world imbalanced datasets

  • M2m performs best among the baselines on CelebA-5, SUN397, Twitter, and Reuters; its Reuters result suggests greater effectiveness under more extreme imbalance.Twitter and Reuters have imbalance ratios of about 150 and 710, respectively; Reuters evaluation excludes classes with fewer than five test samples.Source: Table 2 · p. 6, §3.3 · pp. 6–7

3.4. Ablation study

  • Minority-class accuracy rises as the seed-sample pools become more diverse, supporting M2m’s use of majority-class diversity to prevent minority overfitting.The ablation limits each class’s candidate seed samples to a fixed subset of the training set.Source: §3.4 · p. 7, Table 3 · p. 7
  • Removing the λ regularizer degrades balanced test accuracy, while replacing target labels with random labels worsens minority-class generalization.The regularizer discourages synthetic samples from retaining features of their original class; the random-label comparison is made against M2m-RS.Source: §3.4 · p. 7, Table 4 · p. 7
  • M2m-RS, which omits deferred scheduling, still outperforms baselines except DRS and DRW, supporting the method’s effectiveness beyond deferred training.The default method begins after a fixed period of standard ERM training.Source: §3.1 · p. 5, §3.4 · p. 7, Table 4 · p. 7
  • M2m produces more separable penultimate features, and its cumulative false-positive curve is lower and more linear than those of the other methods.The curve indicates fewer false positives that are more evenly distributed across classes.Source: §3.4 · pp. 7–8, Figure 4 · p. 8, Figure 5 · p. 8
  • Using clean seed samples instead of translated samples significantly reduces balanced accuracy, showing that M2m’s adversarial perturbations are crucial to its performance.The paper observes that generated minority samples can remain close to their original majority samples despite the small perturbation.Source: Table 4 · p. 7, §3.4 · p. 8, Figure 6 · p. 8

4. Conclusion

M2m uses majority-sample diversity to augment minority classes through translation, and the authors report that even a simple method using a pre-trained classifier can help imbalanced training. They suggest adversarial perturbations may provide useful features for imbalanced learning and identify more powerful translation methods as future work.

4. Conclusion

  • M2m is an over-sampling method that translates majority samples to help imbalanced classification, and even a simple method using a pre-trained classifier can leverage majority-sample diversity.The authors propose exploring more powerful translation methods, such as CycleGAN.Source: §4 · p. 8
  • The authors suggest adversarial perturbations could be useful features for imbalanced learning, where minority classes can overfit because of insufficient data.They frame this as a possible application of adversarial perturbations, not as a general claim about feature quality.Source: §4 · p. 8
  • The authors present M2m as a possible new research direction for both imbalanced learning and adversarial examples.They also identify more powerful Major-to-minor translation methods as an interesting future research direction.Source: §4 · p. 8

Supplementary Material · A. Details on the datasets

The supplementary dataset details describe how five image and text benchmarks were constructed, split, and paired with dataset-specific model and training settings.

A. Details on the datasets

  • CIFAR-10/100 contain 60,000 images across 10/100 classes; synthetic long-tailed variants [4] use 10% of the test set for validation and the remainder for testing.Each dataset has 50,000 training and 10,000 test images.Source: A. Details on the datasets · p. 11
  • CIFAR-LT uses ResNet-32 [19] for 200 epochs, with batch size 128 and learning rate 0.1 decayed 100-fold at epochs 160 and 180; deferred scheduling begins after epoch 160.Weight decay is 2 × 10^-4; the schedule follows [7].Source: A. Details on the datasets · p. 11
  • CelebA-5 adapts CelebA [30] to five hair-color classes, selecting 50 validation and 100 test samples per class; it uses ResNet-32 [19] for 90 epochs.The five labels are blonde, black, bald, brown, and gray; deferred scheduling begins after 60 standard-training epochs.Source: A. Details on the datasets · p. 11
  • SUN397 contains 108,754 images labeled across 397 scene classes; center patches are resized to 32 × 32, with 10 validation and 40 test samples per class.The experiments use pre-activation ResNet-18 [19] for 90 epochs, with deferred scheduling after 60 standard-training epochs.Source: A. Details on the datasets · p. 11
  • Twitter is a 25-class POS-tagging dataset; after removing two classes with no test samples, it has 14,614 training samples across 23 classes and uses 50-dimensional token embeddings.A two-layer fully connected network with 256 hidden units is trained for 15 epochs; deferred scheduling begins after 10 standard-training epochs.Source: A. Details on the datasets · p. 11
  • Reuters uses 1000-dimensional bag-of-words inputs; removing classes with fewer than five test samples yields 36 classes and 6,436 training samples, with 10% of training data held out for validation.A two-layer fully connected network with 256 hidden units is trained for 15 epochs; deferred scheduling begins after 10 standard-training epochs.Source: A. Details on the datasets · p. 11

B. More results from ablation study

The ablations show that rejection criteria and the γ threshold improve M2m’s generation quality, while using the training classifier for generation weakens its gains; an ensemble generator offers a slight improvement.

Rejection criteria

  • M2m-No-Reject performs significantly worse than M2m, confirming the benefit of rejecting potentially unreliable generated samples.Without the rejection policy, all generated samples are used to train f; the policy addresses risk from weak generalization of g.Source: Table 5 · p. 12, B. More results from ablation study · p. 12

The effect of γ

  • Removing γ causes performance degradation, indicating that g’s confidence level affects the final quality of generation.The threshold filters synthetic samples whose generation objective was not sufficiently minimized, mainly due to a limited budget.Source: Table 5 · p. 12, B. More results from ablation study · p. 12

Generation from another classifier g

  • M2m-Self, which uses f rather than g to generate minority samples, yields only marginal improvement over DRS and performs much worse than M2m.Using f could make the training process redundant because generated samples may already be confident in the target minority class to f.Source: B. More results from ablation study · p. 11, Table 5 · p. 12

Using multiple classifiers for generation

C. Results on ImageNet-LT

On ImageNet-LT, M2m significantly outperforms the baselines, with relative gains over DRS under ERM loss and a larger improvement with LDAM. The evaluation uses a synthetic Pareto-imbalanced training set and the roughly balanced original ImageNet validation set for testing.

C. Results on ImageNet-LT

  • 3.43 % bACC and 4.75 % GM relative gains over DRS: M2m significantly outperforms the baselines under ERM loss on ImageNet-LT.With LDAM, the improvement is much enlarged, though no numerical gain is supplied.Source: C. Results on ImageNet-LT · p. 12
  • With LDAM, M2m’s improvement over the baselines is much enlarged, but the passage gives no numerical gain.The compared LDAM baseline is LDAM-DRW.Source: C. Results on ImageNet-LT · p. 12
  • ImageNet-LT has 115,846 training samples across 1,000 categories, with 1,280 images in its largest class and 5 in its smallest.Its synthetic class imbalance follows a Pareto distribution with power α = 6; Figure 7 presents the class distribution.Source: C. Results on ImageNet-LT · p. 12, Figure 7 · p. 12
  • The evaluation uses ResNet-50 and reports bACC and GM, testing on the roughly balanced original ImageNet validation set.A random holdout of 20 samples per class from the original ImageNet training set forms the validation set; deferred scheduling is applied after 160 epochs of standard training.Source: C. Results on ImageNet-LT · p. 12

Related papers

Loading 2004.00431v2…