PaperlayerView exact revision on arXiv

Source-linked AI summary

CTAB-GAN+: Enhancing Tabular Data Synthesis

Zilong Zhao, Aditya Kunar, Robert Birke, Lydia Y. Chen

arXiv:2204.00401v1cs.LG

TL;DR

Existing tabular GANs struggle with mixed, skewed, and multimodal variables, while privacy-aware training requires controlling utility across privacy budgets. CTAB-GAN+ addresses these challenges with specialized encoders, conditional and auxiliary supervision, Was+GP training, and differentially private discriminator training, outperforming the evaluated baselines on key utility measures.

  • Problem

    Existing GAN-based tabular generators struggle to represent mixed variables and skewed multi-mode continuous distributions accurately.

  • Method

    CTAB-GAN+ combines conditional GAN training, specialized encoders, auxiliary classifier or regressor supervision, Was+GP loss, and differentially private SGD for the discriminator.

  • Results

    CTAB-GAN+ outperforms the evaluated baselines by 41.2% on accuracy and 56.4% on AUC on average in the non-private setup, while also performing better on regression datasets.

  • Takeaways & Limitations

    The approach provides a tabular data generator designed to improve utility and statistical similarity while controlling performance under different privacy budgets.

Abstract

from arXiv · show

While data sharing is crucial for knowledge development, privacy concerns and strict regulation (e.g., European General Data Protection Regulation (GDPR)) limit its full effectiveness. Synthetic tabular data emerges as alternative to enable data sharing while fulfilling regulatory and privacy constraints. State-of-the-art tabular data synthesizers draw methodologies from Generative Adversarial Networks (GAN). As GANs improve the synthesized data increasingly resemble the real data risking to leak privacy. Differential privacy (DP) provides theoretical guarantees on privacy loss but degrades data utility. Striking the best trade-off remains yet a challenging research question. We propose CTAB-GAN+ a novel conditional tabular GAN. CTAB-GAN+ improves upon state-of-the-art by (i) adding downstream losses to conditional GANs for higher utility synthetic data in both classification and regression domains; (ii) using Wasserstein loss with gradient penalty for better training convergence; (iii) introducing novel encoders targeting mixed continuous-categorical variables and variables with unbalanced or skewed data; and (iv) training with DP stochastic gradient descent to impose strict privacy guarantees. We extensively evaluate CTAB-GAN+ on data similarity and analysis utility against state-of-the-art tabular GANs. The results show that CTAB-GAN+ synthesizes privacy-preserving data with at least 48.16% higher utility across multiple datasets and learning tasks under different privacy budgets.

I. INTRODUCTION

CTAB-GAN+ addresses privacy-utility trade-offs and representation gaps in tabular GANs by combining private training with specialized modeling and downstream supervision. Evaluations across seven datasets and multiple tasks report higher utility than prior methods.

  • Motivation: Synthetic data can support data sharing under privacy constraints, but GAN realism can expose training records and differential privacy can reduce utility.The paper frames privacy-preserving tabular synthesis as a balance between useful data and protection against membership inference.
  • Motivation: Existing tabular GANs overlook mixed variables and may handle highly imbalanced or skewed variables inefficiently.The paper also notes that DP GAN efficacy on tabular data and the preferred DP training framework remain unclear.
  • Contributions: CTAB-GAN+ extends CTAB-GAN with new encoders, Was+GP training, an auxiliary classifier or regressor, and DP-SGD using one discriminator.It also reduces privacy cost through accounting for subsampling smaller discriminator-training subsets.
  • Evaluation: 41.2% higher accuracy and 56.4% higher AUC are reported against non-private baselines, while private comparisons show at least 48.16% higher accuracy and 38.05% higher F1-Score.The evaluation uses seven datasets and compares CTAB-GAN+ with seven state-of-the-art GAN-based generators across two privacy setups.
  • Motivation: Industrial tabular data challenges include single-Gaussian, mixed-type, long-tail, and skewed multimodal distributions.Examples include mortgage values combining zero and positive amounts, credit transactions with rare large values, and working hours with several peaks.

II. RELATED WORK

Related work spans GAN, conditional GAN, and differentially private GAN approaches. CTAB-GAN+ combines prior techniques while adding coverage for mixed, long-tail, single-Gaussian, and imbalanced tabular variables.

  • Conditional GAN-based generator: Conditional GANs use condition vectors to target classes and can oversample minority classes in highly skewed tabular data.Prior methods include CW-GAN and CTGAN, which combine conditional generation with Wasserstein or PacGAN-based designs.
  • CTAB-GAN+: CTAB-GAN+ combines Was+GP, classifier, information, and generator losses with encodings for mixed variables and strategies for single-Gaussian and long-tail distributions.Its conditional-vector structure also targets imbalanced datasets.
  • Differential Private Tabular GANs: Prior DP GANs use PATE or DP-SGD, with RDP accounting providing tighter privacy-cost bounds in relevant approaches.PATE-GAN trains a student discriminator from perturbed teacher outputs, whereas DP-WGAN, GS-WGAN, and RDP-GAN use DP-SGD with Wasserstein loss.
  • Differential Private Tabular GANs: CTAB-GAN+ uses one discriminator, RDP accounting, and Was+GP instead of PATE-style multiple discriminators or DP-WGAN weight clipping.The paper states that this design results in more stable DP GAN training than the state-of-the-art algorithms.

A. Technical Background

The technical foundation combines adversarial tabular generation, conditional sampling for imbalance, Wasserstein gradient-penalty training, specialized normalization, and differential privacy accounting.

  • Tabular GAN: Conditional generation represents categorical classes in a conditional vector and resamples real records to give minority classes higher training probability.This adapts CTGAN-style conditional generation and training-by-sampling to dataset imbalance.
  • Tabular GAN: Was+GP replaces weight clipping with a discriminator gradient-norm constraint, improving training stability and reducing hyper-parameter tuning.The discriminator is updated five times per minibatch, compared with one generator update, affecting the privacy budget.
  • Tabular GAN: Mode-Specific Normalization encodes continuous values as value-mode pairs derived from a Gaussian mixture model.This representation targets complex continuous distributions.
  • Differential Privacy: RDP tracks cumulative privacy loss across composed mechanisms and supports tighter accounting for subsampled private mechanisms.The paper applies this accounting to a tabular GAN model trained under a privacy budget.
  • Differential Privacy: Post-processing allows one DP-trained network to provide differential privacy guarantees for the overall GAN architecture.The paper uses this property when applying private training to the discriminator.

B. Architecture of CTAB-GAN+

CTAB-GAN+ uses a generator, discriminator, and auxiliary classifier or regressor, with feature encoders preceding downstream processing. Its generator combines adversarial, statistical, downstream, and conditional losses.

  • Architecture: The architecture contains generator G, discriminator D, and auxiliary component C, while the generator receives noise and a conditional vector.Variable-specific encoders transform data before it reaches D and C.
  • Training objectives: Information, downstream, and generator losses supplement Was+GP to guide feature statistics, semantic integrity, and conditional-class consistency.Information loss matches feature means and standard deviations; downstream loss compares target values with auxiliary predictions.
  • Training objectives: The downstream loss penalizes discrepancies between synthesized targets and classifier or regressor predictions from synthesized features.The auxiliary component therefore supplies task-oriented supervision for classification or regression.
  • Training objectives: Was+GP provides the default adversarial objective, with discriminator loss based on the difference between generated and real-sample scores plus a gradient penalty.The generator objective adds the auxiliary losses to this adversarial term.

C. Mixed-type Encoder

CTAB-GAN+ introduces a mixed-type encoder that represents variables containing categorical and continuous regions through value-mode pairs and Gaussian-mixture-based normalization.

  • Mixed-type representation: The Mixed-type Encoder represents each mixed-variable value as a concatenated value-mode pair.Mixed variables include categorical-plus-continuous values or continuous values with missing values.
  • Continuous-region modeling: A variational Gaussian mixture estimates the continuous region’s modes and fits a Gaussian mixture to model them.In the example, the model estimates two continuous modes.
  • Mode assignment: Each continuous value is assigned to the mode with the highest probability before normalization.The encoder compares the probability densities of candidate modes for the value being encoded.
  • Final encoding: The final encoding concatenates the normalized value with a one-hot mode indicator.For example, the mode indicator can be β = [0, 1, 0, 0].
  • Conditional representation: The conditional-vector example selects class 2 from the third variable among three variables.The figure illustrates selecting a categorical class through the conditional representation.

D. Counter Imbalanced Training Datasets

CTAB-GAN+ counters imbalanced training through conditional sampling that covers continuous and mixed-variable modes as well as categorical classes, while using specialized encoding choices for different distributions.

  • Conditional sampling: Conditional training-by-sampling is extended to include modes of continuous and mixed columns, not only categorical classes.The conditional vector specifies one selected mode or class.
  • Conditional vector: The conditional vector concatenates one-hot encodings for modes and classes across all variables.It is a zero vector with one active entry corresponding to the selected variable and mode or class.
  • Rebalancing procedure: Variables are selected uniformly, while modes or classes are sampled using logarithmic frequency probabilities.Using log probability instead of raw frequency gives minority modes or classes higher sampling chances.
  • General Transform: General Transform is designed to reduce complexity by addressing single-Gaussian variables and categorical variables with many categories.It encodes columns in the generator’s output range, while avoiding unnecessary mixture modeling for simple distributions.
  • Encoding choices: General Transform is selectively applied to single-mode Gaussian variables, while Mode-Specific Normalization remains preferred for categorical and complex variables.Applying General Transform broadly can lose mode-indicator information and does not suit complex distributions.

F. Treat Long Tails

CTAB-GAN+ treats long-tailed variables with logarithmic preprocessing so Gaussian-mixture encoding can represent tail values, while its private training uses clipped and noisy gradients under a tracked privacy budget.

  • Long-tail preprocessing: Long-tailed variables are preprocessed with a logarithm transformation before variational Gaussian-mixture encoding.The transformation is applied to variables with lower bound l, using different expressions depending on whether l is positive.
  • Long-tail preprocessing: The log-transform compresses the distance between tail and bulk values, making tail values easier for VGM to encode.The paper reports this method as effective for encoding all values, including rare tail points.
  • Private optimization: DP-SGD clips each individual gradient and adds Gaussian noise before updating the network.The clipped gradient uses parameter C, and the noisy gradient replaces the original gradient in SGD.
  • Private optimization: Noise is applied only to gradients computed from the real training dataset; synthetic-data and gradient-penalty gradients remain undisturbed.This distinction defines which training contributions enter the privacy mechanism.
  • Privacy limitation: The Was+GP objective can consume the privacy budget faster because it encourages multiple discriminator updates per generator update.The stronger discriminator requires more frequent private updates.

IV. EXPERIMENTAL ANALYSIS FOR DATA UTILITY

The study evaluates CTAB-GAN+ across seven tabular datasets and compares its machine-learning utility and statistical similarity with multiple GAN-based generators under matched experimental settings.

  • Evaluation design: CTAB-GAN+ is evaluated on seven commonly used tabular datasets using machine-learning utility and statistical similarity.The evaluation also includes ablation analyses of CTAB-GAN+ components.
  • Datasets: The benchmark includes five classification datasets and two regression datasets.Adult, Covertype, Credit, Intrusion, and Loan support classification; Insurance and King support regression.
  • Dataset construction: Covertype, Credit, and Intrusion are randomly sampled to 50K stratified rows because of computing resource limitations.The remaining four datasets are used in their entirety.
  • Baselines: The comparison includes CTAB-GAN, CTGAN, TableGAN, CWGAN, and MedGAN, with implementations using matching generator and discriminator descriptions.Continuous-variable Gaussian-mixture estimation uses the same 10-mode setting as the CTGAN evaluation.

B. Evaluation Metrics

The evaluation measures synthetic-data quality through machine-learning utility and statistical similarity, using task-specific metrics and distributional comparisons.

  • Machine Learning Utility: ML utility compares models trained on equal-sized real and synthetic datasets using five classification and four regression algorithms.Classification uses accuracy, F1-score, and AUC; regression uses MAPE, EVS, and R2.
  • Machine Learning Utility: Classification utility is evaluated with accuracy, F1-score, and AUC, while regression utility uses MAPE, EVS, and R2.F1-score and AUC are more stable than accuracy for imbalanced targets.
  • Statistical Similarity: Statistical similarity uses Jensen-Shannon divergence for categorical distributions, Wasserstein distance for continuous or mixed variables, and pair-wise correlation differences for feature interactions.Wasserstein distance is used because JSD was numerically unstable for continuous variables without distributional overlap.
  • Statistical Similarity: Jensen-Shannon divergence is bounded between 0 and 1 and symmetric, enabling straightforward comparison of categorical-variable distributions.

C. Results Analysis

CTAB-GAN+ outperforms prior tabular GANs in machine-learning utility and statistical similarity across classification and regression evaluations, while ablations identify component-specific effects.

  • ML Utility: CTAB-GAN+ outperforms all compared methods in classification ML-utility metrics.Its AUC difference decreases from 0.094 for the best baseline to 0.041, a 56.4% reduction; accuracy difference decreases from 8.9% to 5.23%, a 41.2% reduction.
  • Statistical Similarity: CTAB-GAN+ improves classification average JSD over CTAB-GAN, CTGAN, and TableGAN by 37.1%, 44.3%, and 51.3%, respectively.
  • Statistical Similarity: For regression datasets, CTAB-GAN+ outperforms CTAB-GAN by 63.4% in average JSD and 74.5% in average WD, while maintaining better correlations.
  • Ablation Analysis: The ablation studies remove classifiers, information loss, normalization, and long-tail treatment to assess how CTAB-GAN components affect F1-score differences.Experiments are repeated three times on the same five machine-learning algorithms.
  • Ablation Analysis: Component effects vary by dataset: removing long-tail treatment most affects Intrusion, while removing MSN can improve Credit when variables are nearly single-mode Gaussian.

2) With CTAB-GAN+:

CTAB-GAN+ addresses several difficult variable distributions with specialized transformations and encodings, reproducing representative distributions more closely than prior methods.

  • Single Gaussian variables: CTAB-GAN+ reproduces the single-mode Gaussian bmi distribution with minor differences.The result is attributed to the General Transform.
  • Mixed data type variables: For the mixed-type Mortgage variable, CTAB-GAN+ generates clear zero values with frequency close to the real data.The variable is encoded as mixed type.
  • Long tail distributions: CTAB-GAN+ perfectly recovers the long-tail Amount distribution in Credit.Log-transform preprocessing helps it learn this structure better than the SOTA methods shown in Figure 1(c).
  • Skewed multi-mode continuous variables: For skewed multimode Hours-per-week data, CTAB-GAN+ is compared against methods that struggle with side peaks or capture modes less effectively.The comparison concerns a dominant peak at 40 and multiple side peaks.
  • Differential privacy: The privacy evaluation compares CTAB-GAN+ with three SOTA differential-private GAN algorithms after adding DP.

A. Experiment Setup

The experiments compare CTAB-GAN+ with DP and non-DP tabular GAN baselines across classification datasets, privacy budgets, machine-learning utility, and statistical similarity. CTAB-GAN+ outperforms the other state-of-the-art algorithms under both evaluated privacy budgets, while competing methods show limitations in capturing complex tabular distributions.

  • Experimental design: The privacy experiments use five classification datasets and compare CTAB-GAN+ with PATE-GAN, DP-WGAN, and GS-WGAN under ϵ = 1 and ϵ = 100.Privacy costs are computed with RDP accounting for DP-SGD approaches, while PATE-GAN uses the moment accountant.
  • ML utility: CTAB-GAN+ outperforms all other state-of-the-art algorithms on ML utility under both privacy budgets.The ML utility differences measure performance between models trained on original and synthetic data, where lower is better.
  • ML utility: Higher ϵ generally improves ML utility because less noise is injected and more training epochs are available before the privacy budget is exhausted.GS-WGAN is the exception for AUC, although its change is minor.
  • Statistical similarity: Among DP models, CTAB-GAN+ and GS-WGAN consistently improve statistical similarity across metrics as the privacy budget increases.Statistical similarity is summarized over five datasets using the metrics in Table VIII.
  • Statistical similarity: GS-WGAN performs significantly worse than CTAB-GAN+ on statistical similarity, while PATE-GAN and DP-WGAN worsen on selected metrics at higher ϵ.PATE-GAN's Avg WD and DP-WGAN's correlation difference increase with the looser privacy budget.
  • Statistical similarity: Imbalanced categorical columns and skewed continuous columns create complex tabular distributions that existing training frameworks struggle to capture.These distributional challenges are identified as a reason for weaker baseline performance.
Loading 2204.00401v1…