DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data

Motivation

• Imbalanced distributions can bias classifiers toward majority classes, causing high error or minority-class omission; deep models also face challenges from complex representations and extracted embeddings,,,,.The introduction also identifies difficult or borderline instances, small disjuncts, small sample sizes, and streaming-data drift as additional challenges.Source: §I · p. 1
• Pixel-based methods may fail to capture image properties, while GAN-based generation can require substantial data, be difficult to tune, and suffer mode collapse [10], [11], [12],.The authors motivate an oversampler that works on raw images while preserving their properties and generating visually high-quality, discriminative artificial images.Source: §I · p. 1

Summary

• DeepSMOTE combines an encoder/decoder, SMOTE-based oversampling, and a penalty-enhanced loss in an end-to-end process spanning low-dimensional embeddings, artificial-image generation, and multiclass classification.The method bridges metric-based resampling using data characteristics with a deep architecture for complex, high-dimensional data.Source: §I · p. 1

Main contributions

• DeepSMOTE is presented as a self-contained, three-component architecture for oversampling complex, imbalanced, high-dimensional data such as images.The authors' criteria are end-to-end operation, learned lower-dimensional representations of raw data, and readily inspectable outputs; the framework's components handle representation, resampling, and classification.Source: §I · p. 2
• DeepSMOTE needs no discriminator for artificial-instance generation and uses a penalty function to make efficient use of training data [10], [11], [12], [13].The generated images are described as high-quality, visually inspectable, and information-rich, supporting class balancing.Source: §I · p. 2
• The planned evaluation compares DeepSMOTE with state-of-the-art oversampling and GAN-based methods on five image benchmarks, using three skew-insensitive metrics and two testing protocols.The authors report robustness to increasing imbalance ratios, including extremely skewed problems.Source: §I · p. 2

• Data-level methods balance datasets through undersampling, oversampling, or hybrid preprocessing; SMOTE-inspired oversampling is popular, but can struggle with multimodal data, overlap, and noise.Random sampling is simple but may remove important instances or amplify noisy ones; alternatives include guided preprocessing and methods that do not rely on k-nearest neighbors.Source: §II · p. 2
• Algorithm-level methods alter classifier training to reduce majority-class bias, including skew-insensitive tree splits, SVM instance weighting, deep-learning layer modifications, cost-sensitive learning, and one-class classification.Unlike data-level preprocessing, they work within the training procedure, trading flexibility for a more direct intervention that requires understanding how the procedure creates bias.Source: §II · p. 2
• Ensemble approaches combine Bagging or Boosting with resampling or cost-sensitive learning, or build skew-insensitive ensembles; data-level techniques can also manage diversity and dynamic selection can retain competent classifiers.Dynamic classifier or ensemble selection addresses individual accuracy by excluding weaker learners from the final decision.Source: §II · pp. 2–3

Instance generation with deep neural networks

• Encoder-decoder models and GANs, VAEs, and WAEs can generate instances for oversampling, extending deep generative modeling to tasks such as medical imaging,,.GANs use a generator-discriminator min-max game but require two networks, can be difficult to train, and may suffer mode collapse [67], [10], [11], [12],.Source: §III · p. 3
• VAEs combine reconstruction loss with KL divergence, while WAEs combine reconstruction loss with a discriminator-output penalty [63], [64].The VAE penalty encourages variation in reconstructions, enabling generation from the input's latent distribution.Source: §III · p. 3

Loss function adaptation

• Loss-function adaptation penalizes minority-class errors more strongly, with Mean False Error balancing class impacts and Focal Loss reducing the influence of easy instances,,.Other proposed losses include Log Bilinear Loss, Cross Entropy Loss, and Class-Balanced Loss.Source: §III · p. 3

Long-tailed recognition

• Long-tailed recognition addresses hundreds of classes with extreme imbalance, where resampling risks shrinking majority classes or requiring too many synthetic instances, while tail classes have few examples.Most works assume a balanced test set; approaches include equalization, hubless, and range losses,,, balanced-set decomposition, cumulative learning, small classifier ensembles [77], and meta-learning for tail-distribution estimation.Source: §III · p. 3

A. Motivation

• DeepSMOTE is proposed as an oversampling algorithm to counter learning bias caused by imbalanced classes in deep learning.The paper notes that oversampling has traditionally been used with classical machine-learning models.Source: A. Motivation · p. 3
• Prior attempts to extend methods such as SMOTE to deep learning have produced mixed results,,.Source: A. Motivation · p. 3
• Metric-based oversampling can be computationally expensive because it requires full-dataset access during training and inference.This access can be challenging for image or speech data in systems that also need substantial memory for gradients.Source: A. Motivation · p. 3
• A suitable deep-learning oversampler should operate end-to-end on raw inputs such as images.Source: A. Motivation · p. 3
• DeepSMOTE’s encoder-decoder is trained on imbalanced data with reconstruction and penalty loss; sampling, encoding, and permutation precede decoding, after which SMOTE produces oversampled data.Source: Figure 1 · p. 4
• It should learn a representation of raw data in a lower-dimensional feature space that can be used for oversampling.Source: A. Motivation · p. 4
• It should readily generate outputs, such as images, that can be visually inspected without extensive manipulation.Source: A. Motivation · p. 4
• The authors state that DeepSMOTE meets the criteria and generates high-quality, sharp, information-rich images without a discriminator network.Source: A. Motivation · p. 4

Encoder/decoder framework

• The DCGAN-based encoder/decoder [82] trains end-to-end on all classes, using majority examples to learn shared reconstruction patterns that can support scarce minority classes.The approach assumes classes share characteristics, such as basic digit contours.Source: B. Deep SMOTE Description · p. 4

Enhanced loss function

• A same-class batch is encoded and decoded in permuted order, and pairwise MSE differences form a penalty that inserts variance without a discriminator.Permutation replaces SMOTE during training because it is more memory- and computationally efficient.Source: B. Deep SMOTE Description · pp. 4–5

Artificial image generation

• At inference, DeepSMOTE encodes images, applies SMOTE to features, and decodes them into synthetic images for classifier-training augmentation.Unlike training, generation uses SMOTE instead of order permutation to introduce variance; SMOTE is non-parametric and requires no training.Source: B. Deep SMOTE Description · p. 5

V. EXPERIMENTAL STUDY

• The study asks whether DeepSMOTE outperforms state-of-the-art pixel-based oversampling and GAN-based resampling algorithms for complex, imbalanced data.These are RQ1 and RQ2; the passages state research questions, not their answers.Source: §V · p. 5
• The study asks how test-set distribution affects DeepSMOTE performance and what visual quality its artificial images have.These are RQ3 and RQ4; the supplied passages do not report findings.Source: §V · p. 5
• The study asks whether DeepSMOTE remains robust as class-imbalance ratios increase and produces stable models under extreme imbalance.These are RQ5 and RQ6; the supplied passages do not report findings.Source: §V · p. 5

Datasets

• The benchmark spans MNIST, Fashion-MNIST, CIFAR-10, SVHN, and CelebA, using 10 classes for the first four and five selected hair-color classes for resized CelebA.MNIST and Fashion-MNIST use grayscale 28 × 28 images; CIFAR-10 and SVHN use color 32 × 32 images. CelebA contains 200,000 images with 40 attribute annotations, resized to 32 × 32 for this study.Source: A. Setup · p. 5

Imbalance and reference methods

• Random class subsampling created 100:1 imbalance for MNIST and Fashion-MNIST and approximately 56:1 for CIFAR-10, SVHN, and CelebA; six resamplers served as references.The pixel-based references were SMOTE, AMDO, MC-CCR, and MC-RBO; the GAN-based references were BAGAN [91] and GAMO.Source: A. Setup · p. 5

Evaluation protocol

• All methods used ResNet-18 and five-fold cross-validation, with imbalanced and approximately balanced test sets evaluated using ACSA, GM, FM, and significance tests at 0.05.Statistical comparisons used the Shaffer post-hoc test and Bayesian Wilcoxon signed-rank test; ACSA, GM, and FM were selected as measures not prejudiced toward the majority class.Source: A. Setup · p. 6

DeepSMOTE implementation

• DeepSMOTE used a modified DCGAN encoder-decoder with latent dimensions of 300 for MNIST and Fashion-MNIST and 600 for the other datasets, trained for 50–350 epochs using Adam at 0.0002.The encoder has four convolutional layers with batch normalization and LeakyReLU; the mirrored decoder uses transposed convolutional layers, batch normalization, ReLU, and a final Tanh layer.Source: A. Setup · p. 6

Placement of artificial instances

• DeepSMOTE uses class geometry and a penalized loss to place artificial instances in difficult minority regions, unlike BAGAN and GAMO’s concentration within class distributions.The targeted regions include class borders, overlaps, and small disjuncts, aiming to reduce minority-class error and improve discrimination.Source: B. Experiment 1: Comparison with… · p. 6

Comparison with pixel-based oversampling

• DeepSMOTE significantly outperforms all tested pixel-based approaches, which are inferior to GAN-based methods and DeepSMOTE on complex imbalanced images.MC-CCR and MC-RBO perform best among the four pixel-based methods, but require high computational complexity and challenging parameter tuning.Source: B. Experiment 1: Comparison with… · pp. 6–7, Table II · p. 7, Table III · p. 7

Comparison with GAN-based oversampling

• DeepSMOTE outperforms baseline GAN methods on all but two F1 cases, while achieving significantly higher ACSA and GM and statistically significant overall gains.The two F1 exceptions are BAGAN on CelebA and GAMO on CIFAR; DeepSMOTE’s SMOTE-based placement also avoids GAN mode collapse and enriches minority classes with diverse, discriminative data.Source: Table II · p. 7, Table III · p. 7, Table IV · p. 7

Effects of test set distribution

• DeepSMOTE performs well with both imbalanced test sets that follow training distributions and balanced test sets used in long-tailed recognition.On balanced tests, it returns slightly better F1 on CIFAR-10 and CelebA; other metrics and datasets show similar trends across the two test distributions.Source: B. Experiment 1: Comparison with… · pp. 7–8, Table II · p. 7, Table III · p. 7

C. Experiment 2: Quality of artificially generated images

• DeepSMOTE generated high-quality images across five benchmark datasets, attributed to its encoder/decoder architecture, enhanced loss, and metric-based instance imputation preserving class topology.Figures 3–7 show images from BAGAN, GAMO, and DeepSMOTE.Source: C. Experiment 2: Quality of… · p. 8
• The experiments’ outcomes indicate DeepSMOTE images are information-rich, improving deep classifiers’ discriminative ability and countering majority bias.The authors state that the outcomes of both experiments also demonstrate high visual quality and answer RQ4.Source: C. Experiment 2: Quality of… · p. 8
• For GAMO, the displayed images were those used for classification, not images generated by GAMO2PIX, to enable direct comparison with BAGAN and DeepSMOTE training images.Source: C. Experiment 2: Quality of… · p. 8

Robustness to varying imbalance ratios

• At extreme imbalance ratios, pixel-based methods—especially MC-CCR and MC-RBO—became more robust, while the two GAN-based methods declined more rapidly.The authors explain that pixel-based methods use more sophisticated instance-placement mechanisms, whereas GAN-based methods face simpler placement mechanisms and increasingly small minority samples.Source: D. Experiment 3: Robustness and… · p. 8
• Across imbalance ratios of 20–400, DeepSMOTE had the lowest performance decline on all evaluated metrics, outperforming both pixel- and GAN-based approaches in robustness.The authors attribute this to SMOTE generating instances according to class geometry using nearest neighbors, which makes DeepSMOTE less affected by small minority sample sizes and the need for precise instance placement.Source: D. Experiment 3: Robustness and… · p. 8

Model stability under varying imbalance ratios

• DeepSMOTE returned the lowest variance, while GAN-based approaches showed increasing variance at higher imbalance ratios across 20 repetitions of 5-fold cross-validation.Minority classes were randomly created from the balanced benchmarks during each repetition, testing stability across both training-data permutations and minority classes of varying difficulty.Source: D. Experiment 3: Robustness and… · p. 10

VI. DISCUSSION

• A dedicated two-phase encoder/decoder learns high-quality embeddings for SMOTE-based oversampling to enrich minority-class training data.The architecture is trained first, then used to obtain embeddings for oversampling.Source: §VI · p. 10
• DeepSMOTE places artificial instances along minority-class borders, following class geometry to improve discriminative-model training on balanced datasets.Source: §VI · p. 10
• DeepSMOTE outperforms state-of-the-art resampling approaches and better enriches minority classes than GAN-based solutions under varying imbalance ratios.It extracts features from raw images to generate more meaningful instances than pixel-based approaches, despite using simpler generation rules.Source: §VI · p. 12
• DeepSMOTE is presented as an accurate, easy-to-tune, off-the-shelf method for use on varied data or as a starting point for new deep-learning architectures.It can be used as a black-box solution or as a steppingstone for developing architectures.Source: §VI · p. 12
• Generated artificial images are sharp, information-rich, and visually indistinguishable from real images under inspection.The paper presents this image quality as making DeepSMOTE an all-around approach.Source: §VI · p. 12
• DeepSMOTE handles extreme imbalance ratios and is less prone to training-data variation than reference methods.It is described as robust to small sample size and within-data variance, and suitable for real-world deep-learning applications.Source: §VI · p. 12

Summary

• DeepSMOTE combines an encoder/decoder, SMOTE-based oversampling, and an enhanced loss to generate high-quality artificial images from raw images and balance training data.The resulting instances can train any deep classifier without suffering from bias.Source: §VII · p. 12
• Extensive experiments show DeepSMOTE outperforms state-of-the-art pixel-based and GAN-based oversampling algorithms, with robustness to varying imbalance ratios and high model stability.The conclusion also describes the generated artificial images as excellent in quality.Source: §VII · p. 12

Future work

• Future work will address class- and instance-level difficulties by penalizing borderline or overlapping instances while discarding outliers and noisy instances.The planned compound skew-insensitive loss is intended to bridge data-level and algorithm-level approaches to imbalanced learning.Source: §VII · p. 12
• The authors also plan to extend DeepSMOTE to continual and lifelong learning with dynamic class ratios, new artificial instances, and additional modalities such as graphs.They envision these extensions could help counter online class imbalance and improve robustness to catastrophic forgetting.Source: §VII · p. 12