PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data

Damien Dablain, Bartosz Krawczyk, Nitesh V. Chawla

arXiv:2105.02340v1cs.CVcs.LG

TL;DR

Imbalanced data biases deep-learning classifiers, while existing pixel-based and GAN-based oversampling methods have limitations. DeepSMOTE combines an encoder/decoder, SMOTE-based oversampling, and a penalty-enhanced loss, and the authors report high-quality images, robustness, and strong performance against reference methods.

  • Problem

    Deep-learning models remain vulnerable to imbalanced data, while existing pixel-based and GAN-based oversampling methods have important limitations.

  • Method

    DeepSMOTE combines an encoder/decoder framework, SMOTE-based oversampling, and a reconstruction loss enhanced with a penalty term.

  • Results

    DeepSMOTE is reported to outperform state-of-the-art resampling approaches while generating high-quality, information-rich artificial images and remaining robust to extreme imbalance ratios.

  • Takeaways & Limitations

    DeepSMOTE provides a simple, transparent oversampling approach for training deep classifiers on imbalanced raw-image data.

Abstract

from arXiv · show

Despite over two decades of progress, imbalanced data is still considered a significant challenge for contemporary machine learning models. Modern advances in deep learning have magnified the importance of the imbalanced data problem. The two main approaches to address this issue are based on loss function modifications and instance resampling. Instance sampling is typically based on Generative Adversarial Networks (GANs), which may suffer from mode collapse. Therefore, there is a need for an oversampling method that is specifically tailored to deep learning models, can work on raw images while preserving their properties, and is capable of generating high quality, artificial images that can enhance minority classes and balance the training set. We propose DeepSMOTE - a novel oversampling algorithm for deep learning models. It is simple, yet effective in its design. It consists of three major components: (i) an encoder/decoder framework; (ii) SMOTE-based oversampling; and (iii) a dedicated loss function that is enhanced with a penalty term. An important advantage of DeepSMOTE over GAN-based oversampling is that DeepSMOTE does not require a discriminator, and it generates high-quality artificial images that are both information-rich and suitable for visual inspection. DeepSMOTE code is publicly available at: https://github.com/dd1github/DeepSMOTE

I. INTRODUCTION

Imbalanced data biases classifiers toward majority classes, while deep models face additional representation challenges. DeepSMOTE addresses this gap with an encoder/decoder, SMOTE-based oversampling, and a penalty-enhanced loss.

  • Imbalanced class distributions bias classifiers toward majority classes and can cause high error or omission of minority classes.
  • Deep architectures remain vulnerable to imbalance while learning complex data representations and embeddings.
  • Existing pixel-based methods may miss image properties, while GAN-based methods require substantial data, difficult tuning, and may suffer mode collapse.
  • DeepSMOTE combines an encoder/decoder framework, SMOTE-based oversampling, and a penalty-enhanced loss function.
  • The method is designed as an end-to-end architecture that learns lower-dimensional representations and generates visually inspectable outputs.
  • DeepSMOTE does not require a discriminator during artificial instance generation and uses a penalty function to prime its generator.

II. LEARNING FROM IMBALANCED DATA

Imbalanced class distributions conflict with standard uniform-loss training, motivating data-level methods that rebalance classes before classifier training. These methods include undersampling, oversampling, and hybrid approaches, while ensemble techniques manage diversity and classifier competence.

  • Standard 0-1 loss applies uniform penalties and can bias learning toward the majority class.
  • Data-level approaches rebalance datasets before classifier training through undersampling, oversampling, or hybrid combinations.
  • Random under- and oversampling has low complexity but may remove important instances or amplify noisy ones.
  • Ensemble methods can manage classifier diversity, while dynamic selection methods retain the most competent classifiers for final decisions.

III. DEEP LEARNING FROM IMBALANCED DATA

Deep-learning approaches to imbalance include instance generation and loss-function modification, but existing generative and resampling methods have important limitations. Related methods span encoder/decoder models, VAEs, WAEs, GANs, and specialized losses for long-tailed recognition.

  • Traditional resampling and shallow oversampling do not efficiently augment training sets for deep models, motivating generative approaches.
  • Encoder/decoder combinations can introduce artificial instances into an embedding space.
  • VAEs combine reconstruction loss with KL divergence, whose penalty encourages varied reconstructions based on a latent distribution.
  • WAEs combine reconstruction loss with a penalty term expressed through a discriminator network.
  • GANs formulate image generation as a min-max game between generator and discriminator networks.
  • Loss modifications penalize minority-class errors more strongly, while long-tailed recognition addresses many rare classes under extreme imbalance.

IV. DEEPSMOTE

DeepSMOTE is designed to extend oversampling to deep models by operating on raw inputs, learning lower-dimensional representations, and producing inspectable outputs. Its implementation trains an encoder/decoder with reconstruction and penalty losses before applying SMOTE to generate oversampled data.

  • A. Motivation: DeepSMOTE targets deep-learning imbalance while accepting raw images, learning lower-dimensional embeddings, and generating visually inspectable outputs.
  • A. Motivation: The method combines an encoder/decoder, SMOTE-based oversampling, and reconstruction-plus-penalty loss.
  • A. Motivation: During training, same-class samples are encoded and decoded in permuted order to introduce variance into the encoding/decoding process.
  • A. Motivation: DeepSMOTE generates sharp, information-rich images without requiring a discriminator network.

B. Deep SMOTE Description

DeepSMOTE combines an encoder/decoder framework with SMOTE-based oversampling and a penalty-enhanced loss. Training introduces embedding-space variance through class-specific reconstruction permutations, while generation applies SMOTE directly.

  • Architecture and training: DeepSMOTE combines an encoder/decoder framework, SMOTE-based oversampling, and a reconstruction loss enhanced with a penalty term.The encoder/decoder provides the representation and reconstruction backbone, while SMOTE supplies synthetic-instance generation.
  • Architecture and training: During training, all classes contribute to end-to-end reconstruction so the model learns to reconstruct both majority and minority images.The imbalanced dataset is processed in batches, with reconstruction loss computed on the sampled data.
  • Penalty loss: The penalty mechanism samples same-class images, encodes them, and decodes them in a permuted order to introduce variance.The penalty compares reconstructed embeddings using mean squared error, approximating SMOTE-like interpolation during training.
  • SMOTE-based oversampling: DeepSMOTE simulates SMOTE by measuring distances between an embedded class sample and its neighbors, using the training-time distance as an implicit reconstruction penalty.The distance is computed in the embedding or feature space with mean squared error.
  • SMOTE-based oversampling: During generation, SMOTE replaces the training-time permutation step to introduce variance, and it requires no training because it is non-parametric.An input is encoded, SMOTEd, and decoded during inference.

V. EXPERIMENTAL STUDY

The experimental study evaluates whether DeepSMOTE improves on pixel-based and GAN-based oversampling, how test-set distribution affects performance, and whether image quality and robustness persist under severe imbalance.

  • Research questions: RQ1 asks whether DeepSMOTE can outperform state-of-the-art pixel-based oversampling algorithms.The question targets comparative performance against pixel-level resampling methods.
  • Research questions: RQ2 asks whether DeepSMOTE can outperform GAN-based resampling algorithms designed for complex and imbalanced data representations.The comparison focuses on deep generative oversampling methods.
  • Research questions: RQ3 examines the impact of test-set distribution on DeepSMOTE performance.The study considers how evaluation changes under different class-distribution protocols.
  • Research questions: RQ4 evaluates the visual quality of artificial images generated by DeepSMOTE.The study treats image quality as a distinct experimental question.
  • Research questions: RQ5 and RQ6 assess robustness to increasing imbalance ratios and model stability under extreme imbalance.These questions test performance as class skew becomes more severe.

A. Setup

The study benchmarks oversampling methods across five image datasets with varied content and class distributions. It evaluates models using majority-insensitive metrics and includes an imbalanced training/test protocol.

  • Datasets: The benchmark uses MNIST, Fashion-MNIST, CIFAR-10, SVHN, and CelebA.These datasets cover handwritten digits, clothing, natural images, house-number digits, and facial attributes.
  • Datasets: MNIST and Fashion-MNIST each contain 60,000 training images, 10,000 test images, grayscale 1 X 28 X 28 images, and 10 classes.The two datasets provide matched-size grayscale benchmarks.
  • Class distributions: Table I reports the class distributions used for the five benchmark datasets.The table is the study’s distribution summary for experimental evaluation.
  • Class distributions: For MNIST and Fashion-MNIST, the imbalanced training classes contain [4000, 2000, 1000, 750, 500, 350, 200, 100, 60, 40] examples.Imbalance is introduced by randomly selecting samples from each class.
  • Evaluation: All resampling methods use the same Resnet-18 classifier and are evaluated with ACSA, macro-averaged GM, and macro-averaged FM.The study uses a 5-fold cross-validation testing procedure and metrics described as not prejudiced toward the majority class.

B. Experiment 1: Comparison with state-of-the-art

DeepSMOTE is compared with pixel-based and GAN-based oversampling under imbalanced and balanced test distributions. It generally achieves stronger classification results while producing high-quality artificial images and handling both evaluation scenarios.

  • Placement of artificial instances: Artificial instances should target class borders, overlapping areas, and small disjuncts rather than being positioned randomly.The objective is to preserve minority-class properties while reducing learning difficulty.
  • Pixel-based oversampling: DeepSMOTE outperforms all pixel-based approaches with statistically significant results while offering an intuitive and easy-to-tune architecture.The comparison uses results from Tables II–IV across the evaluated metrics and test distributions.
  • Pixel-based oversampling: Pixel-based oversampling is inferior to GAN-based methods and DeepSMOTE, while standard SMOTE performs worst among the evaluated algorithms.MC-CCR and MC-RBO perform best within the four pixel-based methods but incur high computational complexity and difficult parameter tuning.
  • GAN-based oversampling: DeepSMOTE outperforms baseline GAN methods on all but two cases, where BAGAN has slightly higher F1 on CelebA and GAMO has higher F1 on CIFAR.DeepSMOTE still reports significantly higher ACSA and GM values on those benchmarks.
  • GAN-based oversampling: DeepSMOTE outperforms GAN-based approaches statistically significantly and generates higher-quality artificial images without sharing GAN mode collapse limitations.The authors attribute performance to artificial-instance placement and penalized-loss-driven oversampling.
  • Test-set distribution: DeepSMOTE performs well under both imbalanced-test and balanced-test protocols, supporting its use for imbalanced and long-tailed recognition scenarios.For the long-tailed setup, it achieves slightly better F1 on CIFAR-10 and CelebA, while other metrics and datasets show similar trends across protocols.

C. Experiment 2: Quality of artificially generated images

Figures 3 to 7 present artificially generated minority-class images from BAGAN, GAMO, and DeepSMOTE across five benchmark datasets. The passage attributes DeepSMOTE’s image quality to its encoding/decoding architecture, enhanced loss, and metric-based preservation of class topology.

  • Figures 3 to 7 compare artificially generated minority-class images from BAGAN, GAMO, and DeepSMOTE across five benchmark datasets.
  • DeepSMOTE-generated images are described as high quality.
  • The reported quality is attributed to encoding/decoding, an enhanced loss function, and metric-based preservation of class topology.

D. Experiment 3: Robustness and stability under varied imbalance ratios

The experiments evaluate DeepSMOTE under increasing imbalance ratios and training-data perturbations. They measure robustness through performance degradation and stability through metric spread across repeated cross-validation.

  • Robustness to varying imbalance ratios: Robust resampling methods should show stable or small performance degradation as class disproportions increase.
  • Robustness to varying imbalance ratios: Sharp performance declines indicate breaking points where a method no longer generates useful instances or counters class imbalance.
  • Model stability under varying imbalance ratios: Model stability is evaluated by measuring performance-metric spread under small perturbations in the data.
  • Model stability under varying imbalance ratios: The evaluation uses 20 repetitions of 5-fold cross-validation with randomly created minority classes.

VI. DISCUSSION

The discussion presents DeepSMOTE as a simple, transparent oversampling method combining deep representations with SMOTE-based generation. It reports strong image quality, robustness, stability, and performance relative to pixel-based and GAN-based approaches.

  • DeepSMOTE combines an encoder/decoder, a dedicated loss function, and SMOTE-based resampling for imbalanced deep learning.
  • A two-phase design learns an embedding before applying SMOTE to enrich minority-class training data.
  • Artificial instances follow minority-class geometry and are placed on class borders to improve discriminative-model training.
  • DeepSMOTE outperforms state-of-the-art pixel-based and GAN-based resampling approaches while working on raw images.
  • Generated images are described as visually comparable to real images while remaining sharp and information-rich.
  • DeepSMOTE is reported to handle extreme imbalance ratios and remain less sensitive to training-data variation than reference methods.

VII. CONCLUSION

The conclusion describes DeepSMOTE as a data-level solution that balances training sets with artificial instances for deep classifiers. It reports superiority over pixel-based and GAN-based methods, alongside robustness, stability, and high image quality.

  • DeepSMOTE combines an encoder/decoder framework, SMOTE-based oversampling, and an enhanced loss function.
  • The method operates on raw images, creates low-dimensional embeddings, and generates high-quality artificial images.
  • Extensive experiments report better performance than state-of-the-art pixel-based and GAN-based oversampling algorithms.
  • The reported evaluation also finds robustness to varying imbalance ratios, high model stability, and excellent artificial-image quality.
  • Future work targets class-level and instance-level difficulty information for challenging feature-space regions.
  • Planned extensions include continual and lifelong learning, dynamic class ratios, catastrophic forgetting, and graph data.
Loading 2105.02340v1…