PaperlayerView exact revision on arXiv

Source-linked AI summary

Remix: Rebalanced Mixup

Hsin-Ping Chou, Shih-Chieh Chang, Jia-Yu Pan, Wei Wei, Da-Cheng Juan

arXiv:2007.03943v3cs.CVcs.LG

TL;DR

Class-imbalanced training can bias deep classifiers toward majority classes, while Mixup-based regularization is mainly designed and evaluated for balanced data. Remix separates feature and label mixing factors to favor minority labels, consistently improving performance over prior methods across imbalanced benchmarks and iNaturalist 2018.

  • Problem

    Class imbalance can bias deep classifiers toward majority classes, while common regularization techniques have limited evaluation in real-world long-tailed settings.

  • Method

    Remix relaxes Mixup by disentangling feature and label mixing factors, assigning disproportionately higher label weight to the minority class.

  • Results

    Remix consistently and significantly improves performance over previous methods across imbalanced CIFAR, CINIC, and iNaturalist 2018 evaluations.

  • Takeaways & Limitations

    Remix provides a computationally cheap, end-to-end trainable regularizer that can complement re-weighting and re-sampling for imbalanced training.

  • Takeaways & Limitations

    The current analysis is intuitive, and Remix requires selecting two hyperparameters.

Abstract

from arXiv · show

Deep image classifiers often perform poorly when training data are heavily class-imbalanced. In this work, we propose a new regularization technique, Remix, that relaxes Mixup's formulation and enables the mixing factors of features and labels to be disentangled. Specifically, when mixing two samples, while features are mixed in the same fashion as Mixup, Remix assigns the label in favor of the minority class by providing a disproportionately higher weight to the minority class. By doing so, the classifier learns to push the decision boundaries towards the majority classes and balance the generalization error between majority and minority classes. We have studied the state-of-the art regularization techniques such as Mixup, Manifold Mixup and CutMix under class-imbalanced regime, and shown that the proposed Remix significantly outperforms these state-of-the-arts and several re-weighting and re-sampling techniques, on the imbalanced datasets constructed by CIFAR-10, CIFAR-100, and CINIC-10. We have also evaluated Remix on a real-world large-scale imbalanced dataset, iNaturalist 2018. The experimental results confirmed that Remix provides consistent and significant improvements over the previous methods.

1 Introduction

The introduction motivates improved training for imbalanced data by highlighting the limitations of re-weighting, re-sampling, and balanced-data regularization. It presents Remix as a computationally cheap relaxation of Mixup designed to improve generalization under class imbalance.

  • Training data critically affects neural-model performance, and class distributions generally need to be balanced for effective learning.
  • Existing imbalance methods primarily use re-weighting or re-sampling, but each presents practical limitations for deep neural networks.Re-weighting can make optimization difficult under extreme imbalance and may be ineffective without regularization, while re-sampling can be difficult to integrate into model training.
  • Many advanced regularization techniques target balanced data, despite real-world datasets often having long-tailed label distributions.
  • Remix relaxes Mixup’s assumption that features and labels should use the same mixing factor, selecting label mixing to better trade off majority and minority classes.Mixup creates virtual samples by convexly combining paired features and labels with a shared factor λ.
  • The work proposes Remix as a cheap, broadly applicable regularizer that extends Mixup-based methods, combines with existing imbalance solutions, and is evaluated across varied imbalanced settings.

2 Related Works

Prior work addresses class imbalance through re-weighting, re-sampling, alternative objectives, representation or classifier learning, and interpolation-based regularization. These approaches modify class importance, data composition, decision margins, model components, or training samples in different ways.

  • Re-weighting and re-sampling: Re-weighting changes class or sample importance through cost or loss adjustments, commonly using inverse class frequency or inverse square-root frequency.Re-sampling instead over-samples minority classes or under-samples majority classes, risking minority overfitting or discarding data and information.
  • Novel objectives: Alternative objectives include Focal Loss, which adds a modulating term to cross entropy to focus on hard negative examples.Label-Distribution-Aware Margin Loss encourages larger margins for minority classes and smaller margins for majority classes.
  • Representation and classifier learning: Representation- and classifier-learning methods include adjusting only the classifier for long-tailed recognition and using a Bilateral-Branch Network for both components.
  • Interpolation-based regularization: Mixup trains on interpolated samples and inspired Manifold Mixup, RICAP, and CutMix, which reported significant improvements over Mixup.Mixup has also influenced semi-supervised learning, adversarial defense, and neural-network calibration.

3 Preliminaries

This section introduces Mixup and two related interpolation-based regularization methods: Manifold Mixup, which mixes representations in embedding space, and CutMix, which synthesizes samples by masking image patches.

  • Mixup: Mixup generates synthesized samples by linearly combining arbitrary sample pairs with a mixing factor λ sampled from a beta distribution.The method mixes both inputs and labels.
  • Manifold Mixup: Manifold Mixup performs the linear combination in embedding space by randomly selecting an eligible layer k and mixing (g_k(x_i), y_i) with (g_k(x_j), y_j).The resulting mixed representations are forwarded from layer k to the output layer as mixed samples.
  • CutMix: CutMix masks an image patch B rather than mixing the entire input feature space when generating synthesized samples.The masking box has width r_w = W.
  • CutMix: CutMix generates an image-level mask M with zeros inside the blocked patch and ones elsewhere, using W, H, and λ through a random mapping f(·).The masked-image proportion is defined by r_w r_h / WH = 1 − λ, and M is applied through element-wise multiplication.

4 Remix

Remix disentangles feature and label mixing factors, allowing synthesized labels to favor minority classes in imbalanced classification. This shifts decision boundaries toward majority classes while supporting integration with re-weighting and re-sampling techniques.

  • Disentangled mixing: Remix relaxes Mixup’s shared mixing factor by using separate factors for feature mixing and label mixing.Mixup, Manifold Mixup, and CutMix are recovered when λy = λx.
  • Label mixing rule: λx is sampled from a beta distribution, while λy is determined from λx, class-count ratios, and hyper-parameters κ and τ.κ compares the sample counts of the two classes, and τ controls the conditions for label reassignment.
  • Label mixing rule: When κ-majority conditions are met, Remix can set λy to 0 or 1, making the synthesized label entirely the minority class.The selected endpoint depends on which sample is the κ-majority and the remaining condition in Eq. 10.
  • Decision-boundary effect: Favoring minority labels for mixed majority–minority samples moves the decision boundary toward the majority class.The method avoids assigning every mixed sample to the minority class because pushing the boundary too far can hurt majority-class performance.
  • Implementation and integration: Remix can interpolate any two samples, explicitly labels the resulting mixed data, and can be combined with re-weighting or re-sampling techniques.Algorithm 1 samples pairs, draws λx, constructs Remix images and labels, and updates the model with the resulting loss.

5 Experiments … Imbalanced CINIC

The experiments compare Remix with state-of-the-art methods for class imbalance on artificially imbalanced CIFAR-10, CIFAR-100, and CINIC-10 datasets, as well as real-world iNaturalist 2018. The datasets include controlled long-tailed and step imbalance settings, balanced validation designs, and large-scale class-imbalanced benchmarks.

  • 5 Experiments: Remix is evaluated against state-of-the-art class-imbalance methods on imbalanced CIFAR-10, CIFAR-100, CINIC-10, and iNaturalist 2018.The experiments cover artificially created imbalanced datasets and a real-world, large-scale imbalanced dataset.
  • 5.1 Datasets: The original CIFAR-10 and CIFAR-100 datasets contain 50,000 training images and 10,000 validation images, with 10 and 100 classes, respectively.Both datasets consist of 32×32 images.
  • Imbalanced CIFAR: Imbalanced CIFAR-10 and CIFAR-100 datasets use long-tailed and step imbalance constructions, while retaining the original balanced validation set.Long-tailed imbalance follows exponential class-size decay, whereas step imbalance uses a parameter µ.
  • Imbalanced CINIC: CINIC-10 combines CIFAR-10 images with images downsampled from ImageNet and contains 270,000 images split equally among training, validation, and test sets.Only the official training and validation splits are used.
  • Imbalanced CINIC: CINIC-10 provides 9,000 training images per class, enabling extensive experiments across imbalance ratios while preserving sufficient data for each class.This dataset is used to compare different methods more effectively.
  • Imbalanced CINIC: iNaturalist 2018 contains 437,513 training images across 8,142 classes and has many visually similar species that are difficult to classify without expert knowledge.The official training and validation splits are used; training is long-tailed and validation is balanced.

CIFAR and CINIC-10 · iNaturalist 2018

The paper evaluates methods on imbalanced CIFAR-10, CIFAR-100, CINIC-10, and iNaturalist 2018 using specified ResNet backbones and training protocols. Comparisons include standard training, focal loss, re-weighting, and re-sampling approaches.

  • CIFAR and CINIC-10: CIFAR and CINIC-10 experiments use ResNet-32 for CIFAR and ResNet-18 for CINIC-10.The reported setup trains for 300 epochs with SGD, momentum 0.9, and weight decay 0.0002.
  • CIFAR and CINIC-10: CIFAR and CINIC-10 training uses learning-rate decay during the 300-epoch schedule.One stated schedule decays the learning rate at epochs 150 and 225, while another decays it at epochs 150 and 250.
  • CIFAR and CINIC-10: CIFAR and CINIC-10 non-LDAM experiments use mini-batches of 128 examples.The passage specifies this batch size for non-LDAM methods.
  • iNaturalist 2018: iNaturalist 2018 experiments use ResNet-50 as the backbone network.Images are resized to 256 × 256 before a random 224 × 224 crop is taken from the image or its horizontal flip.
  • iNaturalist 2018: iNaturalist 2018 preprocessing applies color jittering and lighting augmentation after random cropping.The crop is sampled from either the image or its horizontal flip.
  • iNaturalist 2018: The comparison includes ERM, focal loss, effective-number re-weighting, and inverse-effective-number re-sampling.ERM is standard training without anti-imbalance techniques; focal replaces cross entropy, while re-weighting and re-sampling use the effective number.

Baseline Methods for Comparison · 5.2 Results on Imbalanced CIFAR and CINIC

The paper compares Remix and Remix-integrated methods with prior state-of-the-art, re-weighting, re-sampling, and LDAM approaches on imbalanced CIFAR and CINIC benchmarks. Remix performs particularly well with DRS and DRW, while LDAM provides no further improvement.

  • Baseline Methods for Comparison: The reported benchmark tables include top-1 accuracy on imbalanced CINIC-10 with ResNet-18 and top-1 accuracy on imbalanced CIFAR-10 at imbalance ratio ρ=100.These tables form part of the comparison of methods for imbalanced image classification.
  • Baseline Methods for Comparison: The comparison includes Mixup, BBN, and LWS among the baseline methods, with BBN and LWS identified as state-of-the-art methods.The BBN and LWS results are copied directly from their original paper.
  • 5.2 Results on Imbalanced CIFAR and CINIC: The evaluation compares prior state-of-the-art methods with pure Remix and Remix-integrated re-weighting, re-sampling, and LDAM variants.The experiments use effective-number-based techniques and their deferred versions.
  • 5.2 Results on Imbalanced CIFAR and CINIC: Remix works particularly well when combined with re-weighting and re-sampling, with Remix-DRS and Remix-DRW achieving the best performance.The authors therefore focus subsequent experiments on DRS- and DRW-integrated Remix methods.
  • 5.2 Results on Imbalanced CIFAR and CINIC: Remix-DRS and Remix-DRW are attributed to imbalance-aware label mixing, especially when λ_y is set to 0 or 1.These settings give the learning algorithm more opportunities to adjust for data imbalance.
  • 5.2 Results on Imbalanced CIFAR and CINIC: Re-sampling increases minority-class samples in training batches, making qualifying majority–minority pairs more likely and improving Remix’s trade-off across classes.The mechanism depends on increasing the probability of forming pairs that satisfy Remix’s conditions.
  • 5.2 Results on Imbalanced CIFAR and CINIC: Re-weighting does not increase qualifying-pair probability directly, but minority-class weights amplify Remix’s effect when majority–minority pairs occur.This combination guides the classifier toward a better trade-off across imbalanced classes.
  • 5.2 Results on Imbalanced CIFAR and CINIC: LDAM does not further improve performance, possibly because its intended trade-off competes with the trade-off induced by Remix.The authors consequently emphasize DRS and DRW integrations in the remaining experiments.

5.3 Results on iNaturalist 2018 · 5.4 Ablation Studies

On iNaturalist 2018, Remix follows the trends observed on CIFAR and CINIC, outperforming Mixup and prior methods while substantially exceeding ERM. Ablations show that Remix improves complementary Mixup-based regularizers, with larger gains under step or extreme imbalance.

  • 5.3 Results on iNaturalist 2018: On iNaturalist 2018, Remix outperforms original Mixup, previous state-of-the-art methods, and the ERM baseline by a large margin.The passage describes the improvement as significant and consistent with results on CIFAR and CINIC.
  • 5.4 Ablation Studies: When combined with DRS or DRW, Remix-based methods still outperform corresponding Mixup-based methods.Table 4 compares Mixup and Remix integrated with re-sampling or re-weighting.
  • 5.4 Ablation Studies: Remix’s improvement is more significant on step-imbalanced datasets than on long-tailed datasets.Long-tailed class sizes make Eq.10 conditions for λ_y to become 0 or 1 less likely, whereas step imbalance makes them relatively more likely.
  • 5.4 Ablation Studies: Remix regularization improves both Manifold Mixup and CutMix, significantly outperforming their vanilla versions.The method is presented as general and applicable to other Mixup-based regularizers.
  • 5.4 Ablation Studies: When ρ=10, Mixup and Remix produce little difference because the imbalance ratio is not very extreme.The passage contrasts this condition with the substantially larger gains under extreme imbalance.
  • 5.4 Ablation Studies: When ρ=100, Remix is significantly better than vanilla Manifold Mixup or CutMix.The advantage appears when the imbalance is extreme.
  • 5.4 Ablation Studies: Remix creates a tighter margin for the majority class and compensates for the effect of data imbalance.This summarizes the decision-boundary visualization in Fig. 4.

5.5 Qualitative Analysis

The qualitative analysis evaluates Remix on imbalanced two-blobs, two-moons, and two-circles datasets, comparing them with their original balanced versions. The experiments use imbalance ratio ρ=10 and visualize majority and minority classes separately.

  • Dataset setup: Remix is qualitatively evaluated on scikit-learn’s “two blobs,” “two moons,” and “two circles” datasets.The original balanced datasets appear alongside artificially imbalanced versions in Fig. 4.
  • Visualization: The created imbalanced datasets use imbalance ratio ρ=10, with majority samples plotted in black and minority samples in white.The figure places the original balanced datasets in the rightmost column and the imbalanced datasets in the three columns to the left.

6 Conclusions and Future Work

Remix redesigns Mixup-based regularization for imbalanced data by relaxing the mixing factor to push decision boundaries toward majority classes. It is easy to implement, end-to-end trainable, computationally efficient, and compatible with existing techniques.

  • Remix redesigns Mixup-based regularization for imbalanced data by relaxing the mixing factor.
  • Relaxing the mixing factor pushes decision boundaries toward majority classes.
  • Remix is easy to implement, end-to-end trainable, and computationally efficient for large-scale imbalanced datasets.
  • Remix can be combined with existing techniques to achieve superior performance.

Supplementary Materials

The supplementary materials discuss Remix hyperparameter selection, Mixup’s α setting, and why the frequency of non-degenerate Remix mixing conditions may matter. They also use confusion matrices to examine whether overall-accuracy gains arise from minority-class improvements.

  • Hyperparameter selection: Simple grid search tunes Remix’s κ and τ, fixing τ before searching κ and then fixing κ while searching τ.This procedure was applied to both step and long-tail imbalance; κ is easier to tune for binary step imbalance when it separates minority from majority samples.
  • Mixup α: Remix uses Mixup’s α values when Mixup first improves performance: 1.0 for CIFAR and CINIC, and 0.4 for iNaturalist 2018.The settings are reused because these datasets were also evaluated with Mixup.
  • Non-degenerate mixing: The frequency of Remix’s first and second conditions may matter because the third condition degenerates to Mixup.With re-sampling, majority–minority pairings become much more likely, increasing opportunities for non-degenerate Remix mixing.
  • Minority-class analysis: Confusion matrices on CIFAR-10 are used to show whether overall-accuracy improvement comes from improved minority-class performance.The figure is presented specifically to examine the source of the overall-accuracy gain.
Loading 2007.03943v3…