Source-linked AI summary
Generative Adversarial Minority Oversampling
Sankha Subhra Mullick, Shounak Datta, Swagatam Das
TL;DR
Class imbalance harms minority-class performance, while classical oversampling is difficult to apply within end-to-end deep learning systems. GAMO introduces a three-player adversarial oversampling framework with a convex generator, classifier, and discriminator; experiments on imbalanced image datasets report better performance than competing methods, with limitations motivating future work on image quality and overfitting.
Problem
Class imbalance biases learning toward majority classes, and oversampling methods effective in classical systems are difficult to incorporate into end-to-end deep learning.
Method
GAMO trains a convex generator, multiclass classifier, and real/fake discriminator in a three-player adversarial game to oversample minority classes within an end-to-end framework.
Results
GAMO retains better overall performance than competing methods on CIFAR10, SVHN, and a 50-class SUN397 experiment.
Takeaways & Limitations
GAMO provides an effective end-to-end oversampling technique for handling class imbalance in deep learning frameworks.
Takeaways & Limitations
Future work should improve GAMO2pix image quality and reduce GAMO’s tendency to overfit, potentially using alternative or improved GAN architectures.
Abstract
from arXiv · showhide
Class imbalance is a long-standing problem relevant to a number of real-world applications of deep learning. Oversampling techniques, which are effective for handling class imbalance in classical learning systems, can not be directly applied to end-to-end deep learning systems. We propose a three-player adversarial game between a convex generator, a multi-class classifier network, and a real/fake discriminator to perform oversampling in deep learning systems. The convex generator generates new samples from the minority classes as convex combinations of existing instances, aiming to fool both the discriminator as well as the classifier into misclassifying the generated samples. Consequently, the artificial samples are generated at critical locations near the peripheries of the classes. This, in turn, adjusts the classifier induced boundaries in a way which is more likely to reduce misclassification from the minority classes. Extensive experiments on multiple class imbalanced image datasets establish the efficacy of our proposal.
1. Introduction
GAMO addresses class imbalance in end-to-end deep learning by adversarially oversampling minority classes while constraining generated samples to remain faithful to their distributions. Its three-player design and locality-sensitive oversampling are evaluated through ablation and comparisons on imbalanced image datasets.
- Motivation: Class imbalance biases learning algorithms toward majority classes, causing poor minority-class performance in applications including fraud detection and medical diagnosis.In medical diagnosis, misclassifying unfit patients as fit can have catastrophic implications.
- Motivation: Existing oversampling methods such as SMOTE are difficult to incorporate into end-to-end deep learning because oversampling typically follows feature extraction.Only a few class-imbalance techniques have been extended to deep learning, where imbalance affects both feature extraction and classification.
- Proposal: GAMO is an end-to-end feature-extraction-classification framework that uses adversarial oversampling of minority classes to mitigate class imbalance.The framework is proposed as a novel approach for deep learning systems.
- Proposal: GAMO uses a three-player adversarial game among a convex generator G, classifier M, and discriminator D.The generator attempts to fool both the classifier and discriminator, unlike adversarial classification schemes where the generator works in harmony with the classifier to fool the discriminator.
- Mechanism: Adversarial contention with M pushes learned conditional distributions toward class peripheries, while D preserves fidelity to minority-class distributions and limits bleeding into majority-class territory.The generator can oversample different localities to different extents based on gradients obtained from M.
- Extensions and evaluation: GAMO2pix generates realistic images from GAMO’s synthetic instances in distributed representation space, and the paper evaluates the method through ablation and state-of-the-art comparisons.The authors report these experiments in Sections 4 and 5.
2. Related Works
Related work includes selective and asymmetric oversampling methods for classical classifiers and adversarial schemes that combine classifiers with discriminators. The paper positions GAMO as addressing limitations in extending oversampling to deep learning and images.
- Classical oversampling: Selective and asymmetric oversampling methods generate more synthetic minority points near class boundaries or difficult-to-classify instances.These approaches achieved improvement on classical classifiers.
- Classical oversampling: Classical oversampling methods cannot be extended to deep learning or applied to images because of end-to-end deep learning structure and modality-specific constraints.The passage distinguishes limitations for deep learning and image application.
- Adversarial methods: Related adversarial approaches incorporate classifiers into GAN training through extra outputs, multiclass discriminators, or shared discriminator-classifier networks.These methods are related to GAMO because they combine classification with adversarial learning.
3. Proposed Method
GAMO integrates feature extraction, classification, adversarial discrimination, and convex minority oversampling in an end-to-end framework. Its generator creates difficult minority examples within class-specific convex hulls, while classifier and discriminator training guides their placement toward useful boundary regions.
- Convex Generator: G is trained to produce minority points that the classifier is likely to misclassify, concentrating synthetic samples near minority-class peripheries.This adversarial placement is intended to help M learn class boundaries that are more robust to imbalance.
- Additional Discriminator: Convex-hull constraints prevent arbitrary drift from the intended minority classes, while the discriminator addresses generated points that may not match non-convex class distributions.The additional discriminator distinguishes real and generated points conditionally on class.
- Convex Generator: The generator maps latent samples to convex combinations of instances from each minority class.Its cTMU produces class-conditioned intermediate representations, while class-specific IGUs generate nonnegative weights summing to one.
- Framework: GAMO trains a classifier M, convex generator G, and conditional discriminator D in an adversarial oversampling framework.M classifies data, D distinguishes real from generated instances, and G generates minority samples designed to challenge both networks.
- Optimization Behavior: The optimization can move generated distributions toward real distributions of other classes, producing points near class peripheries without generally collapsing class-specific distributions.The convex-hull constraint keeps generated distributions within the respective classes.
- Training Procedure: Training compensates for imbalance by sampling artificial examples with probability proportional to Pc − Pi, while G receives classes with equal probability.The framework can use either cross-entropy or a least-square loss variant for adversarial training.
4. Experiments
Experiments across imbalanced image datasets evaluate GAMO against classifier, oversampling, augmentation, and adversarial baselines. GAMO generally performs best, while its discriminator and feature-extraction components matter differently across datasets.
- Experimental setup: Experiments use seven image datasets, repeated 10 times, with ACSA and GM as class-unbiased evaluation measures.Most datasets are converted into imbalanced variants by sampling unequal class sizes; SUN397 evaluates many-class performance.
- Ablation study: Figure 4 shows that complete GAMO produces realistic, class-consistent samples, unlike ablated generators that sacrifice realism or class identity.The ablation contrasts SMOTE, cGAN, cG+CN, cG+CN+D, and GAMO\D before the complete model.
- Experimental setup: MNIST and Fashion-MNIST experiments compare GAMO with classifier, SMOTE, cGAN, augmentation, cDCGAN, DOS, and GAMO variants.MNIST uses flattened images for visualization, whereas Fashion-MNIST uses convolutional features.
- MNIST and Fashion-MNIST: GAMO performs better than all other methods on MNIST and Fashion-MNIST, while GAMO\D behaves much worse on MNIST but improves significantly on Fashion-MNIST.The authors attribute the Fashion-MNIST difference to distributed convolutional representations whose classes are nearly convex with little overlap.
- MNIST and Fashion-MNIST: Least-squares classifier variants mostly outperform cross-entropy counterparts, so subsequent experiments use the least-squares formulation.The paper links this choice to a more stable and better decision boundary.
- CIFAR10 and SVHN: GAMO performs better than competing methods on both CIFAR10 and SVHN, closely followed by GAMO\D.The result further supports the advantage of convolutional feature extraction within GAMO.
- CelebA and LSUN: CelebA and LSUN experiments vary image resolution, class-instance counts, and imbalance ratios to test higher-resolution and larger-sample settings.Both datasets use five classes and include smaller and larger variants with different imbalance ratios.
- SUN397: SUN397 evaluates GAMO across 50 classes, where all methods are affected by class complexity and scarce data, but GAMO retains better overall performance.The dataset uses 64×64 images and an imbalance ratio of 14.21.
5. GAMO2pix
GAMO2pix converts GAMO-generated feature vectors into images for applications needing a balanced image dataset. Its class-specific VAE-inspired decoder produces more realistic and diverse images than cDCGAN, with lower FID.
- Motivation: GAMO2pix addresses the convolutional GAMO variant’s limitation that it generates distributed-representation vectors rather than images.The mechanism maps those vectors back into image space.
- Method: GAMO2pix replaces a VAE encoder with GAMO’s fixed convolutional feature extractor and trains a separate decoder for each class.The decoder learns an inverse map from the feature space to the original image space.
- Results: GAMO2pix generates more realistic and diverse images than cDCGAN on CIFAR10, Fashion-MNIST, SVHN, and CelebA-Small.The comparison is shown in Figures 5(b)–5(c), where cDCGAN also exhibits minority-class mode collapse.
- Results: GAMO2pix achieves lower Fréchet Inception Distance than cDCGAN when real and artificial images are compared within each class and averaged.The FID comparison is reported in Table 5.
6. Conclusions and Future Work
GAMO is reported as effective for end-to-end minority oversampling in deep learning, with convolutional and LS-loss variants performing especially well. Future work targets image quality and reducing overfitting through improved GAN architectures.
- GAMO provides an effective end-to-end oversampling technique for class-imbalanced deep learning frameworks.
- The convolutional GAMO variant is more effective, attributed to distributed representations learned by convolutional layers.
- The LS loss variant generally performs better than the CE loss variant.
- Future work includes improving GAMO2pix image quality with architectures such as BEGAN.
- Hybridization with improved GAN variants may reduce GAMO’s tendency to overfit and potentially improve performance with fewer training samples.