PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Reliability of Generative Augmentation: A Wasserstein-Based Theoretical and Empirical Study

Chathurika S Abeykoon, Mathias Nthiani Muia, Mallory Goldstein

arXiv:2609.01410v1stat.MLcs.LG

TL;DR

Class imbalance motivates generative augmentation, but its effects on downstream generalization and the role of distributional fidelity remain insufficiently understood. The paper develops a Wasserstein-based framework with risk bounds and evaluates conditional GAN augmentation, finding that CWGAN-GP has better fidelity without necessarily outperforming classical oversampling. Its conclusions are limited by the small dataset and model scope and by theoretical assumptions that may not capture highly complex, high-dimensional settings.

  • Problem

    Theoretical connections between discrepancies in real and generated distributions and downstream classification risk remain insufficiently developed.

  • Method

    The paper models conditional generative augmentation as distribution mixing and derives Wasserstein-based risk bounds, including capacity-dependent guarantees, for CGAN and CWGAN-GP augmentation.

  • Results

    CWGAN-GP consistently achieves lower Wasserstein discrepancies than CGAN, but improved distributional fidelity does not necessarily produce superior classification performance, with SMOTE often competitive or better.

  • Takeaways & Limitations

    Wasserstein discrepancy provides a measure of augmentation reliability, while distributional fidelity and predictive utility should be evaluated as distinct aspects of synthetic data quality.

  • Takeaways & Limitations

    The evaluation covers only two tabular datasets and two generative architectures, while the theory relies on Lipschitz and Wasserstein assumptions that may not capture highly complex high-dimensional models.

Abstract

from arXiv · show

Generative data augmentation is widely used to mitigate class imbalance, yet its theoretical effect on downstream generalization remains poorly understood. In this work, we develop a statistical framework for conditional generative augmentation and analyze its impact on classification risk. We formalize augmentation as a distribution-mixing process and show that the resulting risk distortion is controlled by both the augmentation strength and the class-conditional Wasserstein discrepancy between real and generated distributions. We further derive a capacity-dependent generalization bound based on Rademacher complexity, revealing an explicit trade-off between hypothesis complexity, augmentation intensity, and generative fidelity. Empirically, we evaluate the framework on binary and multiclass imbalanced classification tasks using Conditional GAN and Conditional WGAN-GP augmentation. Across datasets, CWGAN-GP consistently achieves lower Wasserstein discrepancies than CGAN, indicating improved distributional fidelity. However, improved fidelity does not necessarily translate into superior classification performance, with classical oversampling methods often remaining competitive. These findings support the central theoretical prediction that augmentation reliability is governed by distributional approximation error rather than predictive performance alone. Overall, this work establishes generative augmentation as a distributional perturbation process whose reliability can be quantified through Wasserstein-based measures and supported by finite-sample generalization guarantees. The proposed framework provides a principled foundation for evaluating synthetic data quality beyond classification accuracy alone.

1 Introduction

Class imbalance can bias learning toward majority classes, motivating conditional generative augmentation and a reliability framework that links distributional fidelity to classification risk. Experiments compare CGAN and CWGAN-GP on imbalanced binary and multiclass tasks, finding higher fidelity for CWGAN-GP but no guaranteed predictive advantage.

  • Imbalanced class distributions distort learning, bias decision boundaries, reduce minority-class recall, and degrade generalization.
  • Generative augmentation learns class-conditional distributions to create minority samples rather than only replicating observations or changing loss weights.
  • CGAN conditions generation and discrimination on class labels, enabling targeted synthesis for underrepresented classes.
  • WGANs use Wasserstein-1 distance and a Lipschitz-constrained critic to provide more informative gradients and smoother optimization when distributions have non-overlapping supports.
  • The paper addresses limited theoretical understanding by deriving Wasserstein-based risk bounds and evaluating CGAN and CWGAN-GP on binary and multiclass imbalanced tasks.
  • CWGAN-GP consistently achieves lower Wasserstein discrepancies than CGAN, but improved fidelity does not necessarily yield superior classification performance over classical oversampling.

2 Related Work

Related work addresses imbalance through frequency modification and generative modeling, but commonly evaluates synthetic augmentation through prediction rather than distributional fidelity. This paper connects class-conditional Wasserstein error to risk and evaluates fidelity alongside predictive performance.

  • Class Imbalance and Data Augmentation: SMOTE generates minority samples by interpolating between neighboring observations, while traditional methods primarily modify sample frequencies without modeling class-conditional distributions.
  • Generative Models for Augmentation: Generative augmentation learns the data-generating process and samples new observations instead of replicating or interpolating existing ones.
  • Generative Models for Augmentation: CGANs condition both generator and discriminator on class labels, enabling class-specific generation and targeted minority-class augmentation.
  • Wasserstein-Based Generative Models: WGAN-GP replaces divergence-based objectives with Wasserstein-1 distance and uses gradient regularization to enforce the Lipschitz constraint.
  • Evaluation Gaps: Most studies emphasize accuracy, F1 score, or recall, leaving the relationship between generative fidelity and classification performance poorly understood.
  • This Work: The paper derives explicit risk bounds linking class-conditional Wasserstein approximation error to classification risk and treats augmentation as distribution mixing.
  • This Work: The framework adds class-wise and weighted Wasserstein fidelity measures and capacity-dependent guarantees based on Rademacher complexity.
  • Empirical Validation: Across binary and multiclass imbalanced datasets, CWGAN-GP consistently achieves lower class-conditional Wasserstein discrepancies than CGAN.

3 Theoretical Framework for Generative Augmentation Reliability

The framework treats conditional generative augmentation as controlled distributional mixing and quantifies its reliability through class-conditional Wasserstein errors, risk bounds, and capacity-dependent generalization analysis.

  • Conditional Generative Modeling: Conditional generators learn class-conditional feature distributions and produce additional samples through measurable mappings from latent variables.The generator-induced distributions are trained to approximate the true class-conditional distributions.
  • Wasserstein Approximation Error: Class-conditional Wasserstein error εy measures the discrepancy between real and generated distributions, with smaller values indicating higher generative fidelity.The Wasserstein-1 distance is defined through the minimum expected transport cost over couplings, and εy is used as the class-specific approximation error.
  • Synthetic Augmentation as Distribution Mixing: Augmentation mixes the real distribution P with the synthetic distribution PG, while augmented risk evaluates loss under the resulting mixture.This formulation makes synthetic augmentation a distributional perturbation rather than merely a sample-replication procedure.
  • Risk Control Under Conditional Wasserstein Approximation: The risk-stability theorem shows that augmentation-induced risk distortion scales linearly with augmentation strength α and class-conditional approximation errors.Under the stated Lipschitz-loss, bounded-loss, and approximation assumptions, the Wasserstein discrepancies control the difference between original and augmented risks.
  • Asymptotic Consistency: As maximum class-conditional Wasserstein error vanishes, augmented classification risk converges to risk under the true distribution.The corollary characterizes conditional generative augmentation as asymptotically unbiased under increasingly accurate generators.
  • Capacity-Dependent Generalization: Finite-sample generalization balances hypothesis-class capacity, augmentation bias, augmentation strength, and generative fidelity, so better fidelity need not yield proportional accuracy gains.Rademacher complexity controls estimation error, while Wasserstein approximation errors control augmentation bias.

4 Experimental Design

The experiments evaluate conditional generative augmentation across binary and multiclass imbalanced datasets, comparing generative models with classical oversampling and real-data training. They vary augmentation strength, assess distributional fidelity and classification performance, and use Random Forest and MLP classifiers.

  • Datasets: Experiments use the imbalanced Credit Card Default and Forest Cover Type datasets, covering binary and seven-class multiclass settings.
  • Augmentation models: CGAN and CWGAN-GP learn class-conditional feature distributions and mix generated samples with real training data according to augmentation strength α.
  • Baselines: The comparison includes real-data training, Random Oversampling, SMOTE, CGAN, and CWGAN-GP.
  • Classifiers: Downstream effects are evaluated with a 100-tree Random Forest and a two-hidden-layer ReLU MLP.
  • Evaluation: Augmentation strengths are α ∈ {0.25, 0.50, 0.75, 1.00}, with Macro-F1, Macro Recall, and class-wise Wasserstein distances as evaluation metrics.
  • Implementation: Reported results average performance over three independent random seeds, using dataset-specific training configurations for CGAN and CWGAN-GP.

5 Results and Discussion

CWGAN-GP consistently produces lower Wasserstein discrepancies than CGAN, but this fidelity advantage does not reliably improve classification performance. Classical oversampling remains competitive, and greater generative reliance can reduce Macro-F1.

  • Distributional fidelity: CWGAN-GP achieves lower Wasserstein discrepancies than CGAN across both datasets, indicating closer approximation of class-conditional distributions.
  • Distributional fidelity: Credit Card Wasserstein distance decreases from approximately 27,408 to 3,715 when comparing CGAN with CWGAN-GP.
  • Distributional fidelity: For Credit Card, CWGAN-GP discrepancies range from approximately 6,340 to 11,744, versus 89,924 to 166,518 for CGAN as augmentation strength varies.
  • Classification performance: On Credit Card, Random Oversampling has the highest Macro-F1 and Macro Recall, while CGAN and CWGAN-GP perform comparably despite CWGAN-GP’s lower Wasserstein discrepancy.
  • Classification performance: On Forest Cover, SMOTE achieves Macro-F1 0.7901 and Macro Recall 0.8183, while CGAN outperforms CWGAN-GP across augmentation levels.
  • Statistical testing: No statistically significant Macro-F1 differences between CGAN and CWGAN-GP were observed on either dataset, with p > 0.05.
  • Classification performance: Increasing synthetic-sample proportions generally reduces Macro-F1 for CGAN and CWGAN-GP, whereas SMOTE and Random Oversampling remain relatively stable.
  • Minority-class analysis: Forest Cover minority-class recall remains often similar for CGAN and CWGAN-GP, whereas SMOTE consistently achieves the highest recall across the four least frequent classes.

6 Discussion

The study links generative fidelity to classification risk through Wasserstein-based analysis, while showing that fidelity and predictive performance can diverge. Results therefore support evaluating augmentation with both distributional and predictive criteria, accounting for dataset-dependent augmentation effects and stated scope limitations.

  • The framework connects generative fidelity and classification risk through class-conditional Wasserstein approximation error.This provides a distributional basis for assessing augmentation reliability beyond predictive performance alone.
  • CWGAN-GP consistently achieved lower Wasserstein discrepancies than CGAN across the Credit Card and Forest Cover datasets.The results are interpreted as closer approximation of the underlying class-conditional distributions.
  • SMOTE sometimes achieved higher Macro-F1 scores and minority-class recall than generative augmentation despite lower distributional fidelity.Classification performance also depends on class separability, decision-boundary complexity, sample size, and the downstream model’s inductive bias.
  • Augmentation strength produced contrasting outcomes: stronger augmentation reduced predictive performance on Forest Cover but modestly improved Credit Card results under CWGAN-GP.These differences indicate that effective augmentation depends on dataset characteristics, imbalance structure, and generative fidelity.
  • Distributional fidelity and predictive performance should be treated as complementary rather than interchangeable evaluation criteria.Wasserstein-based fidelity measures how closely synthetic data approximates class-conditional distributions, whereas predictive utility reflects additional downstream factors.
  • The empirical evaluation covers two structured tabular datasets, and the theory relies on Lipschitz continuity and Wasserstein comparisons that may not capture highly complex high-dimensional models.The study also evaluates only CGAN and CWGAN-GP and guarantees fidelity and risk distortion rather than direct classification improvement.

7 Conclusion

The paper develops a statistical framework that models conditional generative augmentation as distribution mixing and connects Wasserstein fidelity, augmentation strength, and classification risk. Experiments support the framework while showing that generative fidelity and predictive utility are distinct, motivating evaluation beyond accuracy alone.

  • The theory models augmentation as distribution mixing and derives Wasserstein-based risk bounds linking generative fidelity, augmentation strength, and classification risk.
  • CWGAN-GP achieved lower Wasserstein discrepancies than CGAN across the Credit Card Default and Forest Cover datasets.SMOTE often remained competitive with or outperformed generative approaches on classification performance.
  • Distributional reliability and predictive utility are related but distinct aspects of synthetic data quality.The framework therefore extends augmentation evaluation beyond predictive accuracy alone.

8 Future Work

Future work proposes adaptive augmentation strategies that select augmentation strength by class using imbalance, generative fidelity, and structural data characteristics. The appendix provides classifier-specific performance details for Random Forest and Multi-Layer Perceptron models.

  • 8 Future Work: Future work will investigate class-specific augmentation strengths rather than relying on a fixed augmentation level.The proposed strategies would incorporate class imbalance, generative fidelity, and structural characteristics of the data.
  • Appendix: The appendix reports classifier-specific results for Random Forest and Multi-Layer Perceptron models alongside aggregated performance summaries.

A.1 Credit Card Dataset

Table 1 reports Credit Card dataset Macro-F1 scores by classifier, summarized as mean ± standard deviation over three random seeds. Random Forest generally benefits more from conventional oversampling, while CGAN and CWGAN-GP differ little in performance despite CWGAN-GP’s lower Wasserstein discrepancy.

  • Table 1 reports classifier-specific Credit Card Macro-F1 scores as mean ± standard deviation over three random seeds.
  • Random Forest consistently achieved higher performance than MLP across most augmentation settings.
  • Random Oversampling and SMOTE produced particularly pronounced improvements for Random Forest.The result suggests that this classifier benefits from increased minority-class representation.
  • CGAN and CWGAN-GP showed relatively small performance differences despite substantial Wasserstein-discrepancy reductions for CWGAN-GP.

A.2 Forest Cover Type Dataset

On Forest Cover, Random Forest outperforms MLP across augmentation strategies, while conventional oversampling methods outperform generative augmentation on Macro-F1.

  • Random Forest consistently outperforms MLP across all augmentation strategies on the Forest Cover dataset.
  • Random Oversampling and SMOTE achieve the strongest Macro-F1 performance for both RF and MLP.
  • CGAN and CWGAN-GP generally remain below the real-data baseline for both classifiers.
  • Lower Wasserstein discrepancies from CWGAN-GP do not translate into superior classifier performance for either RF or MLP.
  • The classifier-specific results support the distinction between distributional fidelity and predictive utility and show robustness across learning algorithms.

B Experimental Hyperparameters

The experiments use documented hyperparameter settings for Credit Card and Forest Cover, with identical classifier settings across augmentation methods unless otherwise stated.

  • Table 3 lists the hyperparameters used for the Credit Card and Forest Cover experiments.
  • Identical classifier settings were used across augmentation methods unless otherwise stated to ensure fair comparisons.
Loading 2609.01410v1…