PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Distributional Fidelity: Causal-Penalized Diffusion for Synthetic Tabular Data

Lan Tao, Yongxian He, Shirong Xu, Yidong Ouyang, Guang Cheng

arXiv:2610.11407v1stat.MLcs.LG

TL;DR

Statistical similarity does not guarantee preservation of causal effects in synthetic tabular data. The paper adds a causal-discrepancy penalty to generative training and instantiates it with an on-policy score-function estimator. Across simulations and benchmarks, causal fidelity improves while distributional fidelity is largely preserved.

  • Problem

    Distributional fidelity does not necessarily establish whether synthetic data preserve the inferential information needed for causal effects.

    Source: §1 · p. 1, §3.2 · p. 4

  • Method

    The framework adds a causal discrepancy penalty to the generative objective and instantiates it in TabDDPM with an on-policy score-function estimator.

    Source: §1 · p. 2

  • Results

    Across simulation settings and benchmarks, the proposed method improved causal fidelity while largely preserving distributional fidelity.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The results support treating causal fidelity as a separate objective alongside distributional fidelity in the evaluated settings.

    Source: §1 · p. 2, §6 · p. 10

    ATE identification relies on consistency, unconfoundedness, and overlap.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Synthetic tabular generators are commonly optimized for distributional fidelity, but statistical similarity alone does not guarantee preservation of causal effects. In this paper, we study whether causal fidelity can be improved directly within a fully generative tabular model. Causal Fidelity is defined with respect to a target estimand as the discrepancy between inferential distributions obtained from real and synthetic data, and theoretical results show that high statistical fidelity does not generally imply high causal fidelity. We then propose a causal-fidelity-aware training framework which adds a causal discrepancy penalty to the generative objective. The framework is instantiated with a causal-penalized TabDDPM and optimized using an on-policy score-function estimator. We further establish conditions under which causal regularization improves expected causal fidelity. Experiments across diverse treatment-effect simulations and two benchmark datasets evaluate the ability of our method to improve causal fidelity while preserving competitive statistical fidelity.

1 Introduction

Statistical similarity does not ensure that synthetic data preserve information needed for causal inference. The paper formalizes causal fidelity and introduces a causal penalty for generative training, motivated by distortions in standard generators’ ATE posteriors.

1 Introduction

  • Conventional feature-wise, pairwise, and multivariate distributional criteria measure statistical similarity but do not necessarily determine whether data preserve information for downstream causal inference.Synthetic tabular data are also used for data sharing, privacy protection, augmentation, and downstream machine learning.Source: §1 · p. 1
  • ATE posterior preservation varies across generator families: CTGAN distorts it severely, TVAE reduces but does not remove the discrepancy, and TabDDPM is strongest but remains shifted.The comparison uses CausalPFN to infer ATE posteriors from real and synthetic datasets.Source: §1 · p. 2, Figure 1 · p. 2
  • The paper asks whether causal fidelity can be improved directly within the training objective of a fully generative tabular model, contrasting with evaluation, structured-generation, and hybrid approaches.The cited approaches include causal-aware evaluation and structured generation (Amad et al., 2026) and hybrid generation (Xu, 2026).Source: §1 · p. 2
  • The authors formalize causal fidelity as discrepancy between real- and synthetic-data inferential distributions, and show high distributional fidelity does not generally guarantee ATE preservation.They add a causal discrepancy penalty to the generative objective and instantiate it in an on-policy causal-penalized TabDDPM using a score-function estimator.Source: §1 · p. 2

2 Related Works

The related work spans tabular generators, causal-aware synthetic-data methods, and causal inference operators. The paper distinguishes its approach by adding a causal discrepancy penalty directly to generator training.

2 Related Works

  • The review organizes relevant literature into tabular generative models, synthetic data for causal inference, and causal effect estimation.Source: §2 · p. 2

2.1 Generative Models for Synthetic Tabular Data

  • Tabular synthesis includes GAN-, VAE-, autoregressive language-model-, and diffusion-based generators, with TabDDPM and TabSyn among diffusion approaches.The paper contrasts its direct causal penalty with ReTabSyn’s reinforcement-learning feedback for feature correlations and predictive signals.Source: §2.1 · pp. 2–3

2.2 Synthetic Data for Causal Inference

  • Prior causal synthetic-data work includes STEAM, hybrid synthesis separating covariate from treatment and outcome generation, and CausalWrap’s post-hoc correction; this paper incorporates non-differentiable discrepancies during training.The cited work is Amad et al. (2026), Xu (2026), and Asiaee et al. (2026), respectively.Source: §2.2 · p. 3

2.3 Causal Effect Estimation

  • Causal effect estimation ranges from regression, weighting, and TMLE to Bayesian tree methods and amortized causal models; the experiments use CausalPFN as the inference operator.The passage lists g-computation, IPW, AIPW, BART, Bayesian Causal Forests, Do-PFN, and CausalFM among these methods.Source: §2.3 · p. 3

3 Problem Setup and Modeling

The paper sets causal inference in the potential-outcomes framework, defines causal fidelity through inferential distributions, and adds a causal discrepancy penalty to generative training. Its TabDDPM procedure estimates that penalty on-policy using sampled reverse-diffusion trajectories.

3.1 Preliminaries

  • Under consistency, unconfoundedness, and overlap, the ATE is identified from the observed-data distribution; individual treatment effects are generally unidentifiable because only one potential outcome is observed per unit.The setup adopts the potential-outcomes framework and also introduces the CATE as a population-level quantity.Source: §3.1 · p. 3

3.2 Problem Statement

  • Causal fidelity measures whether synthetic data preserve real-data inferential information about a target estimand, with smaller discrepancy corresponding to higher fidelity.Overall causal fidelity loss is the expected discrepancy across synthetic datasets.Source: §3.2 · p. 4

3.3 Causal-Fidelity-Aware Training

  • Because causal discrepancy is dataset-level and may involve non-differentiable inference after discrete sampling, the paper adds it to the generative objective and uses a score-function gradient.The causal discrepancy weights generator scores, shifting probability mass away from datasets associated with poor causal fidelity; λ controls the tradeoff with conventional generative fidelity.Source: §3.3 · pp. 4–5

3.4 Algorithm

  • The TabDDPM algorithm combines standard diffusion gradients with periodic on-policy updates that sample reverse trajectories, produce datasets, and weight trajectory scores by causal discrepancy.The trajectory-space formulation avoids integrating over intermediate reverse-diffusion states; a moving-average baseline reduces variance.Source: §3.4 · p. 5

4 Theory

Theory shows that joint distributional closeness does not generally control ATE error, while causal regularization can lower expected causal discrepancy under stated conditions.

4.1 Joint Distributional Fidelity and the ATE

  • As total variation distance approaches zero, ATE discrepancy can remain fixed, ruling out a uniform ATE error bound that vanishes with joint distributional discrepancy under pointwise overlap alone.Exact equality of joint distributions still implies equality of the corresponding identified ATEs.Source: §4.1 · p. 6

4.2 Effect of Causal Regularization

  • Exact minimization of the regularized objective yields expected causal discrepancy no greater than an unpenalized global minimizer, with a sufficient condition for strict improvement.Along exact global optima, causal loss is nonincreasing in the penalty weight while generative loss is nondecreasing.Source: §4.2 · p. 7
  • In the Gaussian omitted-intercept example, every positive penalty strictly reduces ATE error, and the generated ATE equals the target once λ ≥ |β0|/(2σ^2).The unpenalized minimizer is θgen = τ + β0, while the generated ATE is θ.Source: §4.2 · p. 7
  • Reducing causal loss tightens the regularized generator’s estimation-error bound but does not imply a monotone reduction in actual estimation error.The bound separates reference estimation error from inferential discrepancy introduced by synthesis.Source: §4.2 · p. 8

5 Experiments

Experiments compare causal-penalized TabDDPM with baseline generators in simulations and two real-world datasets, finding consistent causal-fidelity gains alongside strong distributional fidelity.

5 Experiments

  • The experiments evaluate simulations and two benchmarks, using ATE-posterior discrepancies for causal fidelity and distributional metrics for statistical fidelity.The benchmarks are IHDP, a semi-synthetic dataset with known treatment effects, and IST, data from a large randomized clinical trial.Source: §5 · p. 8, §5.1 · p. 8, §5.2 · p. 9

5.1 Simulation Study

  • Across all three simulation settings, causal-penalized TabDDPM improves causal fidelity; all distributional metrics improve in the linear setting, with modest losses in some metrics otherwise.In the linear setting, Figure 2 shows the synthetic-data posterior shifting toward the real-data posterior and true ATE.Source: Table 1 · p. 8, §5.1 · p. 8, Figure 2 · p. 9

5.2 Real Application

  • On IHDP, causal regularization reduces ATE posterior Wasserstein distance by 27.3% and posterior mean gap by 30.4%; on IST, reductions are 22.7% and 24.5%.Across both datasets it improves causal fidelity over vanilla TabDDPM while maintaining strong distributional fidelity.Source: §5.2 · pp. 9–10, Table 2 · p. 10

6 Conclusions

The paper formalizes causal fidelity, establishes that strong distributional fidelity does not generally guarantee ATE preservation, and introduces causal-penalized TabDDPM. Simulations and benchmarks show improved causal fidelity while largely preserving distributional fidelity.

6 Conclusions

  • The paper defines causal fidelity through inferential-distribution discrepancies and shows strong distributional fidelity does not generally guarantee ATE preservation.Its causal-fidelity-aware objective adds a causal discrepancy penalty and is instantiated in TabDDPM with an on-policy score-function estimator.Source: §6 · p. 10
  • Across simulations and benchmarks, the proposed method improves causal fidelity while largely preserving distributional fidelity.Future work may extend the framework to broader estimands, alternative inference operators, and other tabular generator classes.Source: §6 · p. 10

AI Use Statement

The authors report AI assistance in code development and manuscript work; an appendix comparison motivates focusing subsequent experiments on TabDDPM.

AI Use Statement

  • The authors report using OpenAI Codex for code development and deployment, and ChatGPT for manuscript editing, language polishing, and refinement of parts of the theoretical analysis.They state that AI-assisted outputs were reviewed and verified and that the authors take full responsibility.Source: AI Use Statement · p. 11

A.1 Comparison of Baseline Tabular Generative Models

  • A preliminary comparison of CTGAN, TVAE, and TabDDPM examines causal-information preservation and motivates the later focus on TabDDPM.The comparison uses a relatively simple causal data-generating setting before more challenging mechanisms.Source: §A.1 · p. 13

A.1.1 Experimental Setup

  • The baseline study uses Setting 1 with τ ∈ {−2, 1, 5}, 10 training runs per generator, and 100 synthetic datasets per method.Causal fidelity is evaluated with CausalPFN; final configurations are selected using validation distributional fidelity for CTGAN and TVAE, and training loss for TabDDPM.Source: §A.1.1 · p. 13

A.1.2 Results

Under Setting 1, TabDDPM provides the strongest baseline in both distributional fidelity and preservation of the ATE posterior. The following setup varies simulation regimes and dimensions to test whether its causal-fidelity performance remains consistent.

A.1.2 Results

  • TabDDPM has the lowest discrepancy across all five reported distributional metrics and the smallest ATE posterior-mean discrepancy at all three treatment-effect magnitudes.TVAE is intermediate to TabDDPM and CTGAN on ATE discrepancies, but its categorical-fidelity and correlation-structure performance is worse than CTGAN's.Source: §A.1.2 · pp. 13–14, Table 4 · p. 14
  • For Setting 1 with τ = 1, CTGAN's ATE posterior is substantially shifted and variable across synthetic datasets, TVAE reduces the discrepancy, and TabDDPM aligns more closely with the reference.Distributional fidelity alone does not establish whether treatment-effect information is preserved for downstream causal inference.Source: §A.1.2 · p. 13

A.2.1 Experimental Setup

  • The broader evaluation tests vanilla TabDDPM across linear, nonlinear, and heterogeneous-effect mechanisms while varying covariate dimension over d ∈ {10, 20, 30}.The simulated data use mixed continuous and categorical covariates, randomized treatment, and fixed outcome-noise level σ = 1.Source: §A.2.1 · p. 14, §A.2 · p. 14

A.2.2 Results

Causal-TabDDPM generally improves causal-fidelity metrics over vanilla TabDDPM across the considered simulation settings. Vanilla TabDDPM's causal fidelity varies by regime and dimension, motivating the broader evaluation and penalty-based approach.

A.2.2 Results

  • Vanilla TabDDPM's causal-fidelity gaps vary across dimensions and mechanisms, with larger gaps in several higher-dimensional settings but no uniformly monotonic pattern.Representative posteriors range from close alignment in d = 10 sinusoidal data with τ = 1 to substantial displacement in d = 30 interaction data with τ = 5.Source: §A.2.2 · p. 15, Figure 3 · p. 15, Figure 4 · p. 15

A.3.1 Simulation Data-Generating Mechanisms

  • The simulation mechanisms cover linear outcomes, nonlinear outcomes, and subgroup-specific heterogeneous treatment effects.The supplied setup also describes continuous and categorical covariates, an auxiliary categorical outcome contribution, and Gaussian noise.Source: §A.3.1 · pp. 15–16

A.3.2 Implementation Details

  • The simulation implementation uses Gaussian diffusion for numerical variables, multinomial diffusion for categorical variables, and Wasserstein-1 distance between CausalPFN-inferred ATE posteriors as the causal discrepancy.Causal updates follow 10,000 warmup updates and are applied every 100 iterations, using m = 10 synthetic tables and λ = 1.Source: §A.3.2 · p. 16

A.3.3 Additional Results

  • Across most considered settings, Causal-TabDDPM generally achieves lower MAE, RMSE, and posterior Wasserstein-1 distance than vanilla TabDDPM, with larger gains where vanilla discrepancies are larger.The comparison spans multiple causal outcome models and treatment-effect values.Source: §A.3.3 · p. 16, Table 5 · p. 17

A.4.1 Implementation Details

  • For the real-application experiments, each method is evaluated on 20 synthetic datasets, with a shared data split and CausalPFN inference protocol across methods.Vanilla and causal-penalized TabDDPM share the generator architecture and training budget; the latter adds on-policy causal regularization, with causal-gradient caps differing between IHDP and IST.Source: §A.4.1 · pp. 17–18

A.4.2 Additional Results

On IHDP and IST, causal regularization more consistently improves causal metrics than it changes feature-level distributional discrepancies. The visual comparisons show gains without a uniform deterioration in marginal data fidelity.

A.4.2 Additional Results

  • On IHDP and IST, causal regularization moves synthetic ATE posteriors toward the reference and decreases both posterior Wasserstein distance and posterior mean gap, while feature-level discrepancies change modestly and nonuniformly.On IHDP, feature-level discrepancies are generally slightly larger, particularly for numerical covariates; on IST, some covariates improve and others show small increases in discrepancy.Source: §A.4.2 · p. 18, Figure 6 · p. 19

A.5 Computing Infrastructure

  • The experiments were conducted on a workstation equipped with an NVIDIA GeForce RTX 4090 GPU.Model training, synthetic-data generation, and causal-fidelity evaluation used this GPU environment.Source: §A.5 · p. 19

B.1 Proof of Theorem 1

The construction changes only the treated-group outcome distribution, making the joint-distribution discrepancy depend on treatment prevalence while the ATE shift remains fixed. It illustrates why joint-distribution proximity alone does not uniformly control ATE error.

B.1 Proof of Theorem 1

  • The construction preserves the joint covariate-treatment distribution and control-arm outcome law, but shifts the treated-arm outcome from N(0, σ²) to N(∆, σ²).The real and generated ATEs are τ = 0 and eτθ = ∆, respectively, under the stated identification assumptions.Source: §B.1 · p. 20
  • Because only the treated arm differs, its contribution to joint total variation is weighted by treatment probability ϵ, while the ATE discrepancy remains ∆.The argument also notes that taking ϵ ↓ 0 at fixed sample size does not establish separation of inferred ATE distributions.Source: §B.1 · p. 20

B.2 Proof of Theorem 2

The proof establishes conditions for causal-loss improvement and its tradeoff with generative fit, then relates inferential discrepancy to population ATE error. A Gaussian omitted-intercept example illustrates strict error reduction under penalization.

B.2 Proof of Theorem 2

  • A comparator with lower causal loss yields strict improvement under the sufficient condition, and the proof requires neither convexity nor correct generator specification.Source: §B.2 · p. 21
  • Along exact optima, increasing the penalty weight makes causal loss nonincreasing while generative loss is nondecreasing.Source: §B.2 · p. 21

B.3 From Causal Discrepancy to ATE Error

  • The population ATE-error bound combines synthetic estimation error, causal discrepancy, and reference estimation error; its common bound can decrease even when actual discrepancy is not monotone.W1 bounds the absolute difference between means, which is applied to inferred ATE distributions.Source: §B.3 · p. 21

B.4 Proof of the Gaussian Example

  • In the omitted-intercept Gaussian example, penalization strictly reduces ATE error for every λ > 0, with an explicit increase in generative loss.The closed-form solution is a population illustration, not the original finite-sample optimization problem.Source: §B.4 · p. 22
Loading 2610.11407v1…