PaperlayerView exact revision on arXiv

Source-linked AI summary

MIND: Marginal-Invariant Neural Dependency Diffusion for Mixed-Type Tabular Generation

Pengfei Li, Mohammad Khalil

arXiv:2609.39628v1cs.LG

TL;DR

Mixed-type tabular generation must balance column-level marginals with cross-column dependencies. MIND transports columns into a unified dependency space and uses conditional diffusion to model their relationships. Across diverse datasets, it achieves a balance among marginal fidelity, dependency preservation, and downstream utility.

  • Problem

    Unified mixed-type tabular generation must balance column-level marginal distributions and cross-column joint structures under an end-to-end objective.

    Source: Introduction · p. 1

  • Method

    MIND transports columns into a unified dependency space, then uses conditional diffusion to learn cross-column dependencies, with copula-tangent denoising and rank projection to reduce marginal drift.

    Source: Overview · p. 3, Conclusion · p. 7

  • Results

    Across diverse datasets, MIND balances marginal fidelity, dependency preservation, and downstream utility; on Covertype, its TSTR score is 0.940 versus 0.565 for TabSyn and 0.594 for TabDiff.

    Source: Conclusion · p. 7, Discussion · p. 7

  • Takeaways & Limitations

    The results support explicit marginal-dependency decoupling as a practical design principle for mixed-type tabular generation.

    Source: Conclusion · p. 7

    MIND assumes a designated target column and batch-level generation, limiting direct use in target-free or multi-target settings; rank-based calibration depends on sufficiently large generation batches.

    Source: Discussion · p. 7

Abstract

from arXiv · show

This paper proposes MIND, a marginal-invariant neural dependency diffusion model for mixed-type tabular data. MIND does not directly learn the joint distribution in the original heterogeneous feature space. Instead, it first maps different variable types into a unified latent dependency space via column-wise marginal transport. A conditional diffusion model then learns cross-column relationships. Copula-tangent denoising separates known marginal components from learnable dependency residuals. Rank projection during the sampling phase further mitigates marginal shift in reverse diffusion. Experiments across nine diverse tabular benchmarks show that MIND consistently improves marginal fidelity and dependency preservation over existing unified approaches. By explicitly isolating marginal modelling from dependency learning, MIND achieves a strong and stable balance among marginal fidelity, joint dependency preservation, and downstream prediction utility. This work supports separating marginal and dependency modelling as a principled and highly effective paradigm for complex mixed-type tabular generation.

Introduction

High-quality mixed-type tabular generation must preserve both heterogeneous column marginals and nonlinear cross-column structure; MIND addresses their coupled modelling by separating marginal transformations from dependency learning.

Introduction

  • Tabular generation must preserve both column-level marginals and pairwise, conditional, or full-joint structure; coupling them in one representation and objective can weaken low-frequency subgroup coverage.Failures involving skewed columns, long-tail categories, or missing patterns can introduce bias in marginals or dependencies.Source: Introduction · p. 1
  • MIND maps mixed-type columns into a unified normalized dependency space, then uses neural diffusion to learn cross-column relationships before inverse transformations generate tabular data.Separating heterogeneous marginal processing lets the model focus on dependency modelling.Source: Introduction · p. 2
  • MIND combines copula decomposition with neural generation, treating marginal transformations as part of a marginal-invariant framework rather than mere preprocessing.Sklar’s theorem separates univariate marginals from dependencies, while MIND uses diffusion to model flexible dependency structures in a unified copula-normalized space.Source: Introduction · p. 2
  • MIND’s dependency diffusion combines copula-aware denoising, latent alignment, and progressive marginal projection to capture high-order relationships while suppressing marginal shift.These mechanisms align dependency learning with marginal calibration.Source: Introduction · p. 2
  • The paper proposes comparing MIND with statistical and deep tabular generators on public mixed-type datasets using marginal fidelity, dependency fidelity, and downstream utility.The introduction describes this as a systematic evaluation, but supplies no comparative results.Source: Introduction · p. 2

Related Work

Prior tabular generators span statistical, neural, sequence, and diffusion approaches, while most learn marginals and dependencies together. Copula theory separates them, but classical copula models rely on structural assumptions.

Synthetic Tabular Data Generation

Diffusion and Score-Based Generative Models for Tabular Data

  • TabDDPM, STaSy, TabSyn, and TabDiff advance diffusion or score-based tabular generation, yet retain shared-representation learning of marginal variation and cross-column dependence.These methods are TabDDPM [Kotelnikov et al. 2023], STaSy [Kim, Lee, and Park 2023], TabSyn, and TabDiff [Shi et al. 2025]; MIND instead diffuses in a copula-normalised space to separate marginal calibration from dependency learning.Source: Diffusion and Score-Based Generative… · p. 2

Copula Dependency Modelling

  • Copula theory decomposes joint distributions into univariate margins and a copula, matching column-level semantics, but Gaussian and vine copulas depend on chosen families or structure.The decomposition follows Sklar’s theorem; Gaussian and vine copulas are representative tabular methods [Tagasovska, Ackerer, and Vatter 2019; Sun, Cuesta-Infante, and Veeramachaneni 2019]. MIND inherits this marginal-dependency perspective while using neural generation in a normalised dependence space rather than predefined copula assumptions [Sun, Cuesta-Infante, and Veeramachaneni 2019; Fan et al. 2017; Genest and Nešlehová 2007].Source: Copula Dependency Modelling · pp. 2–3

Methodology

MIND separates marginal calibration from target-conditional dependency learning by transporting heterogeneous columns into a latent space, then modeling dependencies and correcting marginal drift during sampling.

Overview

  • Column-wise marginal transports encode mixed-type records, calibrating feature marginals separately from a conditional diffusion model that learns feature dependencies given the sampled target.The target is fully observed, and its transported empirical marginal serves as the sampling prior.Source: Overview · p. 3

Marginal-Invariant Representation

  • Type-specific quantile transports map continuous, categorical, binary, and ordinal columns to approximately Gaussian-calibrated marginals; missing values receive explicit missingness representations.Numerical missing positions use auxiliary Gaussian values excluded from denoising loss, while categorical missing values use a dedicated token.Source: Marginal-Invariant Representation · p. 3

Copula-Tangent Dependency Diffusion

  • Under ideal standard-normal marginals, the analytically determined marginal noise component is separated from predictable dependency information, which MIND learns through a tangent-constrained residual branch.The low-noise branch instead predicts total noise to preserve local details and discrete interval boundaries.Source: Copula-Tangent Dependency Diffusion · pp. 3–4

Noise-Adaptive Dependency Network

  • A column-wise Transformer interpolates between identity-based attention at high noise and sample-value-based attention at low noise, with target-conditioning mechanisms preserving the signal.These include time-dependent target-attention bias, an asymmetric condition mask, target-anchor reinjection, and a bounded column-tied readout.Source: Noise-Adaptive Dependency Network · p. 4

Objective and Copula-Projected Sampling

  • Normal-score correlation terms anchor second-order dependencies, while target-conditional DDPM sampling intermittently applies rank projection to recalibrate latent marginals while preserving empirical ranks.The diffusion Transformer learns nonlinear and higher-order structure; rank projection corrects accumulated marginal drift.Source: Objective and Copula-Projected Sampling · p. 4

Experiment Setup

MIND is evaluated across classification, regression, and controlled HeavyTail benchmarks using established generative baselines, shared data splits, and complementary measures of generation quality and predictive utility.

Experiment Setup

  • The benchmark suite comprises six classification datasets, two regression datasets, and a HeavyTail stress test; Covertype uses a class-stratified 50,000-row subset.The regression tasks predict hourly PM2.5 concentration and social-media shares; HeavyTail includes heavy-tailed numerical variables, a long-tail categorical attribute, and structured missingness.Source: Experiment Setup · p. 4
  • Baselines range from independent empirical-marginal and Gaussian Copula samplers to CTGAN, TVAE, TabSyn, and TabDiff, with shared splits and synthetic sample sizes.For each seed in {42, 43, 44}, methods use the same stratified 64%/16%/20% train/validation/test split and generate |Dtrain| synthetic records.Source: Experiment Setup · p. 4
  • Marginal fidelity is measured by Kolmogorov–Smirnov statistic, Wasserstein distance, total variation, and Jensen–Shannon divergence.The cited passage also specifies Pearson/Spearman correlation-matrix and mutual-information errors for dependency preservation.Source: Experiment Setup · pp. 4–5
  • Evaluation also covers predictive utility, distributional fidelity and coverage, and real–synthetic distinguishability, with results reported as mean ± standard deviation over three seeds.Measures include TSTR AUC, accuracy, macro-F1, α-Precision, β-Recall, C2ST AUC, and pMSE.Source: Experiment Setup · p. 5

Results

Across nine datasets, MIND achieves strong aggregate marginal fidelity and dependency preservation, the highest average TSTR utility, and accurate recovery of challenging marginals. Ablations identify copula projection and correlation regularisation as important components.

Dependency Preservation

  • MIND reduces locally concentrated pairwise errors in Default, FICO, and Telco, although TabDiff retains a slightly lower overall Pearson error.The independent marginal model has broad high-error regions, while Gaussian Copula leaves concentrated error blocks in FICO, Covertype, and Shoppers; CTGAN and TVAE show dataset-dependent deviations.Source: Figure 2 · p. 5, Dependency Preservation · p. 5

Marginal Distribution Analysis

  • MIND recovers sharp, asymmetric, skewed, and long-tailed marginals while avoiding spurious density modes and tracking true flat or bimodal shapes.Examples include Hours/week in Adult, Install burden in FICO, Elevation in Covertype, HeavyTail, Shoppers PageValues, and Telco Total charges; competing methods show peak shifts or hallucinated modes.Source: Marginal Distribution Analysis · pp. 5–6

Overall Generation Quality

  • Across nine datasets, MIND leads six of ten Table 1 metrics, including KS 0.003, JS 0.005, Column JS 0.009, Pairwise MI error 0.008, and C2ST gap 0.167.Its KS, JS, and Column JS errors are 62.5%, 58.3%, and 40.0% below the next-lowest means, respectively. Both α-Precision and β-Recall are 0.981; TabDiff has lower Pearson error (0.014 versus MIND's 0.016), while the independent model's lowest pMSE accompanies much worse utility and dependency errors.Source: Overall Generation Quality · p. 5, Table 1 · p. 6

Downstream Utility

  • MIND has the highest average TSTR score (0.693), exceeding TVAE, TabDiff, and TabSyn by 0.046, 0.059, and 0.117, respectively.It ranks first on Covertype, Default, and FICO and second on Telco, with 0.940 versus TVAE's 0.882 on Covertype; it also shows better cross-seed stability than Tabdiff and Tabsyn. Beijing favors TabDiff (0.560 versus MIND's 0.525), and all methods have negative mean R2 on News, where CTGAN leads at −0.038 versus MIND's −0.137.Source: Downstream Utility · p. 5, Table 2 · p. 6

Ablation Study

  • Removing copula projection raises Column JS from 0.0135 to 0.0432, while removing correlation regularisation raises Pairwise MI error from 0.0082 to 0.0089.Removing CTD, dependency regularisation, or attention enhancements raises the C2ST gap from 0.140 to about 0.144; the full model performs best in marginal fidelity, pairwise dependency preservation, and distinguishability while maintaining competitive utility and coverage.Source: Ablation Study · p. 6

Discussion

The discussion presents MIND as competitive with diffusion-based state of the art, with strengths in marginal fidelity and selected dataset results. It highlights sampling projection’s role in marginal calibration and notes limitations tied to target conditioning and batch-level generation.

Discussion

  • MIND leads on Covertype, Default, and FICO and has lower average marginal, mutual-information, and C2ST errors than the strongest baselines.It does not dominate every metric or dataset, and the aggregate TSTR mean favoring MIND is substantially influenced by Covertype.Source: Discussion · p. 6
  • Removing sampling projection raises Column JS from 0.0135 to 0.0432, while other ablations shift the C2ST gap only from 0.140 to 0.144–0.146.The C2ST differences are small relative to reported variation and do not establish component interactions; removing correlation regularisation changes Pairwise MI error from 0.0082 to 0.0089 and TSTR from 0.844 to 0.841.Source: Table 3 · p. 7, Discussion · p. 7
  • Unlike Gaussian and vine copula synthesis (Patki, Wedge, and Veeramachaneni 2016; Sun, Cuesta-Infante, and Veeramachaneni 2019), MIND uses target-conditional neural diffusion and corrects marginal drift during sampling.It replaces a fixed copula family with diffusion in a normalized dependence space.Source: Discussion · pp. 6–7
  • On Covertype, MIND’s TSTR score is 0.940, versus 0.565 for TabSyn and 0.594 for TabDiff, while empirical target-marginal sampling avoids reconstructing class proportions.Despite TabDiff’s lower pairwise Pearson error, it has a substantially larger C2ST gap and target-class mass shift; MIND learns pθ(x | y).Source: Discussion · p. 7
  • An independent marginal model achieves low pMSE and small marginal errors but poor TSTR and dependency preservation, so MIND uses normal-score correlation only as an auxiliary regularizer.On Covertype, TabDiff has lower Pearson error than MIND but substantially worse TSTR and C2ST results; the diffusion model learns broader dependency structures.Source: Discussion · p. 7
  • MIND assumes a designated target column and batch-level generation, limiting direct use in target-free or multi-target settings.Rank-based calibration depends on sufficiently large generation batches; future work targets flexible conditioning, batch-independent sampling, and privacy-aware training.Source: Discussion · p. 7

Conclusion

MIND separates marginal modelling from dependency learning for mixed-type tabular generation. Across diverse datasets, it balances marginal fidelity, dependency preservation, and downstream utility.

Conclusion

  • MIND maps heterogeneous variables into a unified latent space through column-wise marginal transport, allowing conditional diffusion to focus on cross-column dependencies.This separates marginal modelling from the learning of dependencies.Source: Conclusion · p. 7
  • Copula-tangent denoising and rank projection further reduce marginal drift during generation.Source: Conclusion · p. 7
  • Across diverse datasets, MIND achieves a strong balance among marginal fidelity, dependency preservation, and downstream utility.These results support explicit marginal-dependency decoupling as a practical design principle for mixed-type tabular generation.Source: Conclusion · p. 7

Related papers

This paper cites

Loading 2609.39628v1…