Source-linked AI summary
Modeling Tabular data using Conditional GAN
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni
TL;DR
Tabular synthesis must model mixed data types, multimodal or non-Gaussian continuous columns, sparse categorical representations, and imbalanced categories. The paper proposes CTGAN, using mode-specific normalization, conditional generation, and training-by-sampling, and evaluates it in a broad benchmark. CTGAN outperforms Bayesian networks on at least 87.5% of datasets and outperforms TVAE on 3 datasets.
Problem
Tabular data combines discrete and continuous columns with multimodal, non-Gaussian, sparse, and imbalanced distributions that existing statistical and deep generative models do not properly model.
Method
CTGAN uses mode-specific normalization, architectural changes, conditional generation, and training-by-sampling to model tabular distributions and address categorical imbalance.
Results
87.5% of datasets is the reported minimum share on which CTGAN surpasses Bayesian networks, and CTGAN outperforms TVAE on 3 datasets.
Takeaways & Limitations
CTGAN provides a competitive tabular-data generator evaluated against Bayesian networks and other deep-learning methods across simulated and real datasets.
Takeaways & Limitations
Sparse one-hot representations can let a discriminator detect fake data from sparsity rather than overall row realism.
Abstract
from arXiv · showhide
Modeling the probability distribution of rows in tabular data and generating realistic synthetic data is a non-trivial task. Tabular data usually contains a mix of discrete and continuous columns. Continuous columns may have multiple modes whereas discrete columns are sometimes imbalanced making the modeling difficult. Existing statistical and deep neural network models fail to properly model this type of data. We design TGAN, which uses a conditional generative adversarial network to address these challenges. To aid in a fair and thorough comparison, we design a benchmark with 7 simulated and 8 real datasets and several Bayesian network baselines. TGAN outperforms Bayesian methods on most of the real datasets whereas other deep learning methods could not.
1 Introduction
The paper establishes a benchmark for tabular-data synthesizers and proposes CTGAN, which combines conditional generation with techniques for difficult tabular distributions and imbalance. CTGAN performs significantly better than Bayesian baselines and other tested GANs on the benchmark.
- The paper builds a benchmarking system using real datasets, three recent techniques, and two Bayesian-network baselines to evaluate tabular GANs.
- CTGAN combines mode-specific normalization, architectural changes, conditional generation, and training-by-sampling for tabular-data synthesis.
- 87.5% of datasets is the reported minimum share on which CTGAN surpasses Bayesian networks.
- CTGAN outperforms all methods to date and outperforms TVAE on 3 datasets while remaining competitive across many datasets.
- The benchmark contains 5 deep-learning methods, 2 Bayesian-network methods, 15 datasets, and 2 evaluation mechanisms.
2 Related Work
Earlier synthetic-data methods model tabular variables with statistical structures such as decision trees, Bayesian networks, spatial decomposition trees, and copulas. VAEs and GANs offer greater flexibility and have been adapted for heterogeneous tabular and healthcare data.
- Traditional methods model columns as random variables in a joint distribution, using structures including decision trees, Bayesian networks, spatial decomposition trees, and copulas.
- These statistical models are constrained by distribution types and computational issues.
- VAEs, GANs, and their extensions provide flexible generative-modeling approaches that have been applied to tabular data.
- Prior GAN-based systems include methods for continuous-time medical records, discrete tabular data, and heterogeneous continuous or binary data.
3 Challenges with GANs in Tabular Data Generation Task
Tabular GANs must handle mixed data types, non-Gaussian and multimodal continuous columns, sparse categorical representations, and severe categorical imbalance. These properties create distinct modeling and training difficulties beyond those typical of image generation.
- Mixed data types: Real-world tables mix discrete and continuous columns, requiring GAN outputs to use both softmax and tanh.
- Non-Gaussian distributions: Non-Gaussian continuous values make min-max normalization prone to vanishing gradients.
- Multimodal distributions: 57/123 continuous columns in 8 real-world datasets have multiple modes, which vanilla GANs may fail to model fully.
- Sparse one-hot-encoded vectors: Sparse one-hot representations let a discriminator distinguish real from fake samples using sparsity rather than overall row realism.
- Highly imbalanced categorical columns: 636/1048 categorical columns are highly imbalanced, with major categories appearing in more than 90% of rows.
4 CTGAN Model
CTGAN models mixed-type tabular distributions with mode-specific normalization and conditional generation. Its training-by-sampling strategy addresses imbalanced discrete columns while preserving conditional distributions.
- Overview: CTGAN combines mode-specific normalization, architectural changes, and conditional training-by-sampling to model challenging tabular data distributions.The method targets non-Gaussian and multimodal continuous columns alongside imbalanced discrete columns.
- Mode-specific Normalization: Each continuous value is represented by a mode one-hot vector and a scalar normalized within that mode.A variational Gaussian mixture estimates modes and their parameters before mode-conditioned normalization.
- Mode-specific Normalization: A row concatenates normalized continuous scalars, continuous-mode indicators, and one-hot discrete columns.This representation lets the network jointly process mixed continuous and discrete data.
- Conditional Generator: The conditional generator targets a selected discrete value and reconstructs the original distribution from conditional row distributions.The desired relationship is PG(row|Di*=k*) = P(row|Di*=k*).
- Conditional Generator: A cross-entropy penalty encourages the generator to copy the selected condition into the corresponding generated one-hot vector.This makes the generated row preserve the requested category as training advances.
- Training-by-Sampling: Training-by-sampling selects a discrete column uniformly and samples its category using probabilities based on category log-frequency.The resulting condition and real training rows help the model explore discrete values more evenly.
- TVAE Model: TVAE is an adapted variational autoencoder using the same preprocessing and an evidence lower-bound loss for mixed-type tabular generation.It models pθ(rj|zj) and qφ(zj|rj) with two neural networks.
5 Benchmarking Synthetic Data Generation Algorithms
The benchmark compares Bayesian networks and deep generative models across simulated and real tabular datasets using likelihood fitness and machine-learning efficacy. CTGAN performs strongly on continuous simulated data and outperforms Bayesian baselines on real datasets, while model strengths vary by setting.
- 5.1 Baselines and Datasets: The suite compares CLBN, PrivBN, MedGAN, VeeGAN, TableGAN, CTGAN, and TVAE across 7 simulated and 8 real datasets.Simulated datasets include Gaussian-mixture and Bayesian-network oracles; real datasets include UCI, Kaggle, and transformed MNIST data.
- 5.2 Evaluation Metrics and Framework: Simulated data are evaluated with likelihood fitness, whereas real data are evaluated by machine-learning efficacy using task-specific metrics.Classification uses accuracy and F1; regression uses R2.
- 5.2 Evaluation Metrics and Framework: Ltest addresses Lsyn’s overfitting sensitivity and can detect mode collapse, but requires prior knowledge of the simulated oracle structure.Ltest retrains an oracle with the same structure but different parameters before evaluating test-set likelihood.
- 5.3 Benchmarking Results: On Gaussian-mixture simulations, Bayesian networks suffer from discretizing continuous data, while other GANs suffer from mode collapse; CTGAN performs well with mode-specific normalization.The result concerns the 2-dimensional continuous simulated datasets.
- 5.3 Benchmarking Results: On Bayesian-network simulations, CTGAN performs slightly better than MedGAN and TableGAN, although CLBN and PrivBN have a natural advantage.TableGAN also performs well, possibly because most simulated variables have fewer than four categories.
- 5.3 Benchmarking Results: On real datasets, TVAE and CTGAN outperform CLBN and PrivBN, while CLBN- and PrivBN-trained models are 36.1% and 51.8% worse than real-data-trained models.TVAE outperforms CTGAN in several cases, and the authors do not conclude that VAEs should always replace GANs.
- 5.4 Ablation Study: The ablation study evaluates mode-specific normalization, conditional generation, training-by-sampling, and network architecture using absolute changes on real classification datasets excluding MNIST.The architecture comparison includes WGANGP, vanilla GAN loss, and PacGAN variants.
6 Conclusion
The paper targets robust modeling of complicated tabular distributions. It proposes CTGAN components for multimodal continuous columns and imbalanced discrete columns, and reports better empirical distributions than Bayesian networks.
- 6 Conclusion: CTGAN uses mode-specific normalization to represent continuous values with arbitrary ranges and distributions as bounded vectors for neural networks.The method also uses a conditional generator and training-by-sampling to address imbalanced training data.
- 6 Conclusion: The authors report that CTGAN learns better distributions than Bayesian networks and can generate data with specified discrete values for data augmentation.They identify theoretical justification for GANs on mixed discrete-continuous distributions as future work.
7 Dataset Details
The benchmark covers simulated and real datasets, with selected predictive models and aggregate rankings used to organize evaluation. The section also specifies the CTGAN training inputs and conditional-data construction steps.
- Datasets: Table 4 organizes the benchmark into simulated and real datasets and reports their statistical information.The raw data for eight real datasets are available online.
- Predictive models: For each real dataset, the benchmark selects classifiers or regressors that provide reasonable performance.Table 5 lists the selected models and their corresponding performance.
- Benchmark results: Table 6 ranks methods across Gaussian-mixture simulated, Bayesian-network simulated, and real-data experiments using average ranks over metrics.Algorithms are ranked within each experiment set, their average rank scores are ranked again, and lower final ranks are better.
- CTGAN training: CTGAN training takes training data, generator and critic parameters, batch size, and pac size as inputs, then updates both parameter sets.The training step creates masks, derives condition vectors, and groups conditional vectors and generated records into pacs.
- CTGAN training: For real data, records and conditional vectors are concatenated in pac-sized groups during the training procedure.The procedure forms each group by concatenating pac consecutive conditional vectors and generated records.