PaperlayerView exact revision on arXiv

Source-linked AI summary

How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data

Mihaela Cătălina Stoian, Salijona Dyrmishi, Maxime Cordy, Thomas Lukasiewicz, Eleonora Giunchiglia

arXiv:2402.04823v1cs.LG

TL;DR

Standard DGMs can model complex tabular distributions yet violate domain constraints needed for realistic synthetic data. The paper converts linear inequalities into differentiable Constraint Layers for C-DGMs, whose outputs are guaranteed compliant. Across multiple models and tasks, constrained models eliminate non-compliance and improve utility and detection by up to 6.5%, while the layer can also serve as an inference-time guardrail.

  • Problem

    Standard DGMs may approximate tabular distributions well but provide no guarantee that generated samples obey background constraints.

  • Method

    The method automatically parses linear-inequality constraints into a differentiable Constraint Layer integrated with a DGM.

  • Results

    C-DGMs are never non-compliant and improve utility and detection by up to 6.5% over standard counterparts.

  • Takeaways & Limitations

    Constraint integration can enforce domain knowledge while improving synthetic-data quality, including when used as an inference-time guardrail.

  • Takeaways & Limitations

    The implementation may face exponential representation growth in the worst case when constraints share many variables.

Abstract

from arXiv · show

Deep Generative Models (DGMs) have been shown to be powerful tools for generating tabular data, as they have been increasingly able to capture the complex distributions that characterize them. However, to generate realistic synthetic data, it is often not enough to have a good approximation of their distribution, as it also requires compliance with constraints that encode essential background knowledge on the problem at hand. In this paper, we address this limitation and show how DGMs for tabular data can be transformed into Constrained Deep Generative Models (C-DGMs), whose generated samples are guaranteed to be compliant with the given constraints. This is achieved by automatically parsing the constraints and transforming them into a Constraint Layer (CL) seamlessly integrated with the DGM. Our extensive experimental analysis with various DGMs and tasks reveals that standard DGMs often violate constraints, some exceeding $95\%$ non-compliance, while their corresponding C-DGMs are never non-compliant. Then, we quantitatively demonstrate that, at training time, C-DGMs are able to exploit the background knowledge expressed by the constraints to outperform their standard counterparts with up to $6.5\%$ improvement in utility and detection. Further, we show how our CL does not necessarily need to be integrated at training time, as it can be also used as a guardrail at inference time, still producing some improvements in the overall performance of the models. Finally, we show that our CL does not hinder the sample generation time of the models.

1 INTRODUCTION

Realistic tabular synthesis requires both distributional fidelity and compliance with domain constraints. The paper introduces C-DGMs that guarantee constraint satisfaction and reports improved utility and detection over standard DGMs.

  • 1 INTRODUCTION: Standard DGMs capture complex tabular distributions but may generate unrealistic samples that violate background constraints.Examples include clinical measurements requiring maximum values to be at least minimum values.
  • 1 INTRODUCTION: C-DGMs automatically convert linear-inequality constraints into differentiable Constraint Layers integrated with standard DGMs.The resulting sample space is guaranteed to comply with user-defined constraints.
  • 1 INTRODUCTION: WGAN generated 100% non-compliant samples on one dataset, while five standard models exceeded 95% non-compliance on another.The experiments evaluate WGAN, CTGAN, TableGAN, TVAE, and GOGGLE across six tasks with up to 31 constraints.
  • 1 INTRODUCTION: Adding constraints improves detection and utility by up to 6.5% across the evaluated datasets.The paper also evaluates inference-time use of the Constraint Layer as a guardrail and reports no hindrance to sample generation time.

2 PROBLEM STATEMENT

Constrained generative modeling seeks distributions that approximate real data while restricting generated samples to admissible values. Here, admissibility is represented by finite sets of linear inequalities over continuous features.

  • 2 PROBLEM STATEMENT: The problem requires learning a model that both approximates the unknown data distribution and generates only constraint-compliant samples.This extends ordinary distribution learning with explicit background knowledge about admissible data.
  • 2 PROBLEM STATEMENT: The paper represents background knowledge as a finite set of linear inequalities over variables corresponding uniquely to dataset features.Inequality relations may include ≥, >, and equivalent converted forms of other linear relations.
  • 2 PROBLEM STATEMENT: Linear constraints are useful because they support consistency checking, compiled satisfying representations, and convex feasible spaces.These properties motivate their use as the paper’s constraint language.
  • 2 PROBLEM STATEMENT: A sample satisfies the constraint set only when it satisfies every individual constraint; a model is compliant when all generated samples satisfy the set.A constraint set is satisfiable when at least one admissible sample exists.
  • 2 PROBLEM STATEMENT: For an unconstrained model, the target C-DGM should produce compliant outputs while minimally changing each original sample according to user preferences over modified features.This formalizes constraint enforcement as a correction problem rather than merely distribution fitting.

3 CONSTRAINED DEEP GENERATIVE MODELS

The Constraint Layer incrementally corrects DGM outputs using an ordered representation of linear constraints, while preserving differentiability and minimizing changes where possible. The resulting C-DGMs are guaranteed to satisfy feasible constraints, with strict inequalities handled by arbitrarily small positive adjustments.

  • 3 CONSTRAINED DEEP GENERATIVE MODELS: The Constraint Layer is inserted after sample generation, passes gradients during training, and returns corrected samples satisfying the constraints.For GANs, generated samples can be transformed before CL and mapped back before discriminator evaluation.
  • 3 CONSTRAINED DEEP GENERATIVE MODELS: The layer processes features in a user-defined order and incrementally derives constraints for each feature from previously processed variables.Constraint subsets are reduced across variables to obtain bounds for the next feature.
  • 3 CONSTRAINED DEEP GENERATIVE MODELS: For each feature, CL computes lower and upper bounds and minimally adjusts the original value to remain within the feasible interval.Strict inequalities may require replacing a boundary value with the boundary plus a small positive epsilon.
  • 3 CONSTRAINED DEEP GENERATIVE MODELS: Theorem 3.3 guarantees that incorporating CL into any deep generative model yields a model compliant with every satisfiable finite constraint set.The guarantee applies to the generated sample space under the selected variable ordering.
  • 3 CONSTRAINED DEEP GENERATIVE MODELS: If an input sample already satisfies the constraints, CL returns it unchanged and it is optimal under the paper’s definition.The method can select which features to change using preferences tied to how well their distributions are approximated.
  • 3 CONSTRAINED DEEP GENERATIVE MODELS: Without strict inequalities, corrected outputs are optimal; with strict inequalities, they can approach the relaxed problem’s optimal solution as epsilon tends to zero.This preserves compliance while explaining the boundary behavior of the correction mechanism.

4 EXPERIMENTAL ANALYSIS

The experiments evaluate constrained and standard DGMs across diverse datasets, measuring constraint compliance, synthetic-data quality, post-processing, and generation time. C-DGMs guarantee compliance, often improve utility and detection, can serve as inference-time guardrails, and have comparable generation time.

  • 4.4 POST-PROCESSING ABILITY: At inference time, the constraint layer can act as a guardrail while slightly improving overall DGM performance.The experiments compare post-processed P-DGMs with their standard versions.
  • 4.5 IMPACT ON SAMPLES GENERATION TIME: C-DGM sample generation time is comparable to that of standard DGMs, although the constraint representation can have exponential size in the worst case.The experiments report no observed exponential blow-up, which occurs when constraints share many variables.
  • 4.1 EXPERIMENTAL ANALYSIS SETTINGS: C-DGMs are evaluated using five DGM families across six real-world datasets spanning binary classification, multiclass classification, and regression.The datasets range from 2K to 1M rows, 24 to 109 features, and 4 to 31 constraints.
  • 4.1 EXPERIMENTAL ANALYSIS SETTINGS: The evaluation measures utility through TSTR metrics and detection through classifiers distinguishing synthetic from real data, with metrics averaged over five runs.Reported metrics include F1-score, AUROC, weighted F1-score, explained variance, and mean absolute error as appropriate.
  • 4.2 BACKGROUND KNOWLEDGE ALIGNMENT: C-DGMs achieve zero constraint violation rates across all tested models, datasets, and runs, whereas standard DGMs exceed 50% violation in more than half of experiments.Standard models reach 100% CVR for WGAN on LCLD and exceed 95% for four of five DGMs on WiDS.
  • 4.3 SYNTHETIC DATA QUALITY: 6.5% higher utility for GOGGLE with C-GOGGLE demonstrates that constraint integration can improve synthetic-data performance.For News, every C-DGM also improves or preserves utility on at least one metric.

5 RELATED WORK

Prior work develops tabular-data generators for distributional, feature-type, imbalance, privacy, and relational challenges, while neuro-symbolic methods incorporate background knowledge with varying expressiveness and guarantees. This paper combines tabular synthesis with topology-level constraint integration for linear inequalities.

  • Tabular Data Synthesis: Tabular synthesis methods include GANs, TVAE, score-based models, privacy-preserving models, and GOGGLE for learning relational structure.These approaches target mixed features, imbalance, privacy, healthcare, and distribution modeling.
  • Neuro-symbolic AI Methods: Neuro-symbolic methods commonly encode logical constraints in losses, but those penalties do not guarantee that constraints are satisfied.Solver-based and topology-based alternatives address background knowledge differently, with limitations in scalability or expressiveness.
  • Neuro-symbolic AI Methods: The paper differs from prior generative neuro-symbolic work in both application domain and the type of background knowledge incorporated.Its approach targets tabular data and constraints expressed as linear inequalities.

6 DISCUSSION AND CONCLUSIONS

The paper presents C-DGMs that translate linear-inequality constraints into an integrated layer, guaranteeing compliant tabular samples. Training with the layer improves sample quality, while the approach remains limited to linear inequalities.

  • 6 DISCUSSION AND CONCLUSIONS: C-DGMs translate complex linear-inequality constraints into a seamlessly integrated layer that guarantees compliant generated samples.The layer incorporates domain-specific knowledge directly into the DGM architecture.
  • 6 DISCUSSION AND CONCLUSIONS: Including the constraint layer during training enhances generated-sample quality, showing value beyond applying constraints only during generation.
  • 6 DISCUSSION AND CONCLUSIONS: The method is limited to constraints expressible as linear inequalities, while some feature relationships may require more expressive representations.The paper identifies more complex constraint representations as future work.

ETHICS STATEMENT

The paper acknowledges responsible-use risks for high-quality synthetic data and documents reproducibility materials alongside technical proofs and experimental details. The supplied appendix passages also include proofs for the constraint-layer theorems.

  • ETHICS STATEMENT: High-quality synthetic data could be misused for deceptive purposes, although synthetic generation can also support privacy protection.
  • REPRODUCIBILITY STATEMENT: The paper supports reproducibility with appendix proofs, experimental settings, dataset information, evaluation details, hyperparameters, and publicly available code.
  • A.1 PROOF OF THEOREM 3.3: Theorem 3.3 is proved by induction over variables to establish that the constructed constraint layer yields compliant samples.The proof considers the base case and recursively handles variables through lower and upper bounds.
  • A.2 PROOF OF THEOREM 3.5: Theorem 3.5 establishes that applying the constraint layer leaves an already compliant sample unchanged.The argument uses the lowest index at which a transformed sample could differ and rules out violations of the corresponding bounds.

A.3 PROOF OF THEOREM 3.6

The proof establishes that the constraint layer’s sequential construction yields an optimal compliant correction, while the experiments compare five underlying generative models.

  • A.3 PROOF OF THEOREM 3.6: The proof decomposes Theorem 3.6 into lemmas showing CL≥(x̃) is optimal, CL(x̃) is optimal when equal to CL≥(x̃), and CL(x̃) approaches CL≥(x̃) as ϵ tends to 0.The argument proceeds from a one-variable base case and substitutes already computed variables to reduce each step to that case.
  • A.3 PROOF OF THEOREM 3.6: The inductive proof handles constraints involving progressively more variables by preserving earlier computed values and showing no alternative feasible sample is closer at each step.The base cases cover empty, lower-bound, upper-bound, and two-sided constraints.
  • B.1 MODELS: The model study evaluates WGAN, TableGAN, CTGAN, TVAE, and GOGGLE as the base DGMs for the constrained variants.The models include GAN-, VAE-, and graph-based approaches with different tabular-data transformations and architectures.

B.2 DATASETS

The study uses six real-world tabular datasets with known feature constraints and evaluates models through repeated utility and detection protocols with tuned hyperparameters.

  • B.2 DATASETS: Six datasets cover four binary-classification, one multiclass-classification, and one regression task, selected because each has at least three feature-relationship constraints.The datasets span URL phishing, diabetes, lending, credit repayment, steel defects, and news-sharing prediction.
  • B.3 CONSTRAINTS DATASHEET: Constraint structure varies from 4 to 31 constraints per dataset, with 15%–56.88% of variables involved and usually two features per constraint.URL includes a 17-feature constraint, while LCLD includes single-variable constraints.
  • B.4 EVALUATION PROTOCOL: Utility is measured by training classifiers or regressors on synthetic data and testing them on real data across F1-score, weighted F1-score, ROC AUC, or regression metrics.The protocol selects hyperparameters using validation data and averages results over five repetitions.
  • B.4 EVALUATION PROTOCOL: Detection is evaluated by classifying concatenated real and synthetic records, using a new real-versus-synthetic target and a separate validation and test procedure.The best classifier is selected on the constructed validation set before final detection testing.
  • B.5 HYPERPARAMETER SEARCH: Hyperparameter search proceeds in two phases, first narrowing optimizer, learning-rate, and batch-size choices, then varying model-specific parameters while reusing the best initial configuration.The selected settings are shared between each DGM and its constrained counterpart.

C.1 BACKGROUND KNOWLEDGE ALIGNMENT

C-DGMs eliminate constraint violations and better match real boundary behavior, while largely preserving feature-distribution similarity to unconstrained models.

  • C.1 BACKGROUND KNOWLEDGE ALIGNMENT: All unconstrained samples violate at least 50% of constraints on five of six datasets, whereas every C-DGM sample satisfies every constraint.Constraint violation coverage reaches 100% for WiDS, FSP, and News, while C-DGMs guarantee zero violations.
  • C.1 BACKGROUND KNOWLEDGE ALIGNMENT: C-DGMs have zero samplewise constraint-violation coverage, while unconstrained models average as much as 29.8% of constraints violated per sample.The results indicate that even a single violation can make a generated example infeasible in practice.
  • C.1 BACKGROUND KNOWLEDGE ALIGNMENT: Visualizations show constrained models produce samples only within feasible regions for WiDS, Heloc, and FSP constraints.The examples include hemoglobin ordering, insolvent-trade ordering, and steel-plate coordinate ordering.
  • C.1 BACKGROUND KNOWLEDGE ALIGNMENT: C-DGMs reproduce real boundary populations more closely than unconstrained DGMs, with maximum deviations of 14.7% versus 51.5%, respectively.The comparison uses boundary bands defined at p = 1%, 5%, and 10%.
  • C.2 FEATURE DISTRIBUTION ANALYSIS: Feature-distribution differences are generally similar across DGMs, P-DGMs, and C-DGMs, except that C-GOGGLE improves WiDS Wasserstein distance from 0.22 to 0.05.Categorical-feature divergences change little because the datasets contain no constraints over categorical features.

C.3 VARIABLE ORDERING STUDY

The variable ordering controls how the constraint layer computes corrected features, and data-informed orderings generally outperform random ordering, though their best choice depends on the model and setting.

  • C.3 VARIABLE ORDERING STUDY: The constraint layer computes features sequentially, minimally changing each value when it conflicts with previously computed features and the constraints.The user-defined variable ordering determines the computation sequence.
  • C.3 VARIABLE ORDERING STUDY: Correlation-based ordering ranks features by differences between real and generated correlations, while KDE-based ordering ranks them by KL divergence between estimated marginal distributions.Both heuristics use the unconstrained model’s generated samples alongside real data.
  • C.3 VARIABLE ORDERING STUDY: Ordering choice has limitations because the marginals are discretely approximated, and an ordering that improves utility does not necessarily improve detection.The authors identify selecting the best ordering for each model and scenario as future work.
  • C.3 VARIABLE ORDERING STUDY: KDE ordering performs best for detection in C-WGAN and C-TableGAN, while correlation ordering performs best for C-TVAE and C-GOGGLE uses random ordering across all three detection metrics.Utility preferences also vary: KDE leads for C-WGAN, C-TVAE, and C-CTGAN, whereas correlation leads for C-TableGAN and C-GOGGLE.
  • C.3 VARIABLE ORDERING STUDY: On WiDS, causal- and Wasserstein-based orderings beat random ordering in 12 and 11 of 15 detection and utility cases, respectively, but postprocessing gains are small.The causal ordering uses a DAG learned with DAG-GNN, whose outputs required removing self-loops and breaking cycles.

C.4 FULL RESULTS ON DGMS VS. C-DGMS

Across six datasets and multiple metrics, constrained DGMs generally outperform unconstrained models in utility and detection, while inference-only guardrails provide smaller gains. The strongest improvements vary by architecture, with C-TableGAN, C-CTGAN, and C-GOGGLE showing notable results.

  • C.4 FULL RESULTS ON DGMS VS. C-DGMS: C-DGMs generally outperform standard DGMs in utility and detection, particularly when at least one constraint ordering is used.The full evaluation covers multiple datasets, DGMs, three orderings, and five random seeds.
  • C.4 FULL RESULTS ON DGMS VS. C-DGMS: C-WGAN and C-TVAE generally improve utility on most datasets, with no improvement across all three metrics only on LCLD for C-WGAN and Heloc for C-TVAE.For C-TVAE on WiDS, gains are 4.0%, 3.6%, and 1.5% in F1-score, weighted F1-score, and Area Under the ROC Curve, respectively.
  • C.4 FULL RESULTS ON DGMS VS. C-DGMS: C-TableGAN improves utility for all binary and multiclass datasets on at least two of three metrics, with gains of at least 4.5% in F1-score across four datasets.The highest reported F1-score improvement is 7.5% on WiDS.
  • C.4 FULL RESULTS ON DGMS VS. C-DGMS: C-CTGAN yields moderate or near-baseline gains in most cases but improves News utility by 64.9 points in mean absolute error, the largest improvement among C-DGMs.Four of six datasets improve across all three metrics, while the remaining two improve on two metrics.
  • C.4 FULL RESULTS ON DGMS VS. C-DGMS: C-GOGGLE outperforms GOGGLE on all three utility metrics for five of six datasets, with a largest F1-score improvement of 17.1% on URL.FSP is the only dataset without improvement under any ordering.
  • C.4.2 DGMS VS. C-DGMS: DETECTION RESULTS: Detection improves for constrained models across several datasets, while inference-only P-DGMs produce smaller utility and detection gains than models constrained during training.C-WGAN improves or preserves detection across all datasets and metrics except URL Area Under the ROC Curve.

C.6 REAL DATA PERFORMANCE

Real-data utility provides a reference for assessing whether synthetic-data performance approaches the achievable task performance. C-WGAN and C-CTGAN are often similar to real-data scores, while gaps remain for FSP.

  • C.6 REAL DATA PERFORMANCE: C-WGAN and C-CTGAN achieve utility scores similar to real data overall, while C-TableGAN and C-TVAE often bring synthetic performance closer to real-data performance.For LCLD, C-TableGAN matches and slightly exceeds real-data F1-score by 0.3%, whereas none of the models approaches real FSP performance.
Loading 2402.04823v1…