PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Distribution Matching: Semantics-Consistent Tabular Diffusion with Weak Semantic Priors

Yili Wang, Ruxue Shi, Mengnan Du, Hangting Ye, Yi Chang, Xin Wang

arXiv:2609.16069v1cs.LGcs.AI

TL;DR

Tabular generators can match distributions without explicitly modeling the weak semantic priors needed for valid rows. SCTab-Diff constructs intra- and inter-column priors and uses them to condition diffusion generation; experiments report improvements across six benchmarks and robustness to incomplete priors.

  • Problem

    Existing tabular generators primarily optimize distributional fidelity without explicitly modeling weak semantic priors, even though distributionally plausible rows can violate semantic constraints.

    Source: §1 · pp. 1–2

  • Method

    SCTab-Diff extracts intra-column semantics and inter-column rules from metadata and validates them on training data, then uses these priors to condition column-wise diffusion denoising.

    Source: §1 · p. 2, §3.1 · p. 4, §3.2 · p. 4

  • Results

    Across six real-world benchmarks, SCTab-Diff improves distributional fidelity, semantic consistency, and downstream utility over representative generators, with robustness under incomplete priors.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    Synthetic tabular generation should account for both distributional fidelity and semantic validity, including when semantic knowledge is only partially available.

    Source: §1 · p. 1, §4.3 · p. 8

    Prior quality depends on informative schema and descriptions, and the framework does not explicitly model causal relations or procedural constraints.

    Source: F Limitations · p. 25

Abstract

from arXiv · show

Synthetic tabular data can match real data distributions while still violating the semantic constraints that govern valid tabular rows. This reveals a key limitation of existing tabular generators: they mainly optimize distributional fidelity, but do not explicitly model weak semantic priors encoded in tabular schema and textual descriptions. In this paper, we propose \ours, a semantics-consistent tabular diffusion framework for high-fidelity synthetic data generation under weakly specified semantic priors. \ours\ first constructs two types of priors, namely intra-column semantics and inter-column symbolic rules, with LLM-assisted extraction from metadata and validation on the real training split. These priors are then used as generation conditions rather than post-hoc filters. Specifically, \ours\ maps heterogeneous column values, column identities, and semantic priors into a unified semantic space, and performs column-wise forward corruption and prior-conditioned reverse denoising to preserve both marginal distributions and rule-consistent cross-column dependencies. Extensive experiments on six real-world tabular benchmarks show that \ours\ consistently improves distributional fidelity, semantic consistency, and downstream task utility over representative VAE-, GAN-, LLM-, and diffusion-based baselines. Additional analyses further demonstrate the robustness of \ours\ when semantic priors are partially unavailable.

1 Introduction

Tabular generators can reproduce distributions while producing semantically invalid rows, despite tables’ typed columns and cross-column dependencies. SCTab-Diff makes weak semantic priors part of diffusion generation, and experiments report improvements in fidelity, consistency, and utility.

1 Introduction

  • Synthetic rows may match distributions yet violate plausibility or cross-column rules, a concern because they are reused in downstream pipelines.The cited uses include data augmentation, privacy-preserving sharing, and model training or evaluation.Source: §1 · p. 1
  • Tabular columns have distinct semantic types and marginal behaviors, while valid tuples must respect dependencies that are often only weakly specified.Intra-column meaning is commonly implicit in names, schema, and documentation; inter-column validity is governed by symbolic rules.Source: §1 · pp. 1–2
  • Existing VAE-, GAN-, and diffusion-based methods generally treat rows as holistic vectors and do not explicitly preserve column identity or condition denoising on weak priors.This leaves semantic consistency as an implicit byproduct of distribution learning.Source: §1 · p. 2
  • SCTab-Diff conditions reverse-time score estimation on intra-column semantics and inter-column symbolic rules rather than applying them as post-hoc filters.The framework couples column-wise diffusion and prior-conditioned denoising in a unified semantic space.Source: §1 · p. 2
  • Experiments report improved distributional fidelity, semantic consistency, and downstream utility over representative tabular generators.The introduction also reports robustness when semantic priors are incomplete.Source: §1 · p. 3

2 Preliminaries

The paper formulates generation from a tabular dataset and weak semantic sources as producing samples that are both distributionally faithful and valid under the constructed priors. It uses a variance-exploding score-based diffusion process as its backbone.

2.1 Problem Formulation

  • Rows are represented as attributes x_i corresponding to named columns f_i, with numerical and categorical columns distinguished.The formulation defines n columns, divided into N_n numerical and N_c categorical attributes.Source: §2.1 · p. 3
  • The generator is framed as learning from dataset D and weak semantic sources to produce samples faithful to D and valid under priors P.P consists of intra-column semantics and inter-column symbolic rules.Source: §2.1 · p. 3

2.2 Score-Based Diffusion Preliminaries

  • The score-based diffusion preliminaries adopt the variance exploding formulation as SCTab-Diff’s diffusion backbone.The passage identifies the perturbed-data distribution and reverse-time Wiener process in the denoising formulation.Source: §2.2 · p. 3

3 Proposed method: SCTab-Diff

SCTab-Diff turns weak semantic priors into generation conditions for tabular diffusion. Its prior-conditioned denoising combines statistical recovery with semantic consistency and cross-column context.

3.1 Weak Semantic Priors as Generation Conditions

  • Candidate priors encode plausible values within columns and Boolean dependency rules across columns, and are extracted from schema and descriptions then validated against observed data.The two prior types address attribute-level plausibility and tuple-level dependency validity.Source: §3.1 · p. 4

3.2 From Distribution-Centric Diffusion to Semantics-Consistent Generation

  • Column-wise corruption preserves column-level modeling granularity, while reverse denoising conditions each column on its identity, other columns, and semantic priors.The column-specific noise schedule is learnable, and the target is compatibility with semantic priors and cross-column context, not only statistical plausibility.Source: §3.2 · p. 4
  • Because priors may be incomplete or unavailable, training randomly masks prior tokens, with pu controlling the masking probability.The reverse-time process is written after this masking step.Source: §3.2 · p. 5

3.3 Instantiating SCTab-Diff

  • SCTab-Diff maps heterogeneous column values and identities into a unified semantic space, encoding categorical values with column names and numerical values as value-sensitive representations.Weak semantic priors are encoded in the same latent space so they can condition generation rather than remain an external rule set.Source: §3.3 · p. 5
  • Cross-attention supplies semantic-prior context, and a Transformer-based dependency module integrates it with column identity and cross-column context for noise prediction.The resulting conditioned score guides each reverse denoising step.Source: §3.3 · p. 6

4 Experiments

Across six real-world datasets, SCTab-Diff improves distributional and semantic fidelity and downstream predictive performance. The experiments also examine robustness to missing priors, component contributions, and fine-grained categorical and dependency patterns.

4.1 Experimental Setups

  • The evaluation covers six real-world datasets—four classification and two regression—and compares SCTab-Diff with seven traditional, LLM-based, and diffusion-based baselines.Metrics assess distributional, structural and semantic, and task fidelity; results are averaged over five random-seed runs.Source: §4 · p. 6, §4.1 · p. 7, §4.2.1 · p. 7

4.2.1 Distributional and Semantic Fidelity

  • 98.29% average Shape versus 98.03% for TABDIFF, 98.05% average Trend (0.73 percentage points above TABDIFF), and 82.23% average SA versus TABDIFF’s 81.38%.SCTab-Diff achieves the best Shape, Trend, and SA results on 5 of 6 datasets for each metric.Source: §4.2.1 · pp. 7–8, Table 1 · p. 7

4.2.2 Task Fidelity

  • SCTab-Diff achieves the best downstream performance on 5 of 6 datasets and second-best on Adult; Beijing RMSE falls from 0.582 to 0.498 versus the strongest baseline.It also achieves the highest AUC on Shoppers, Default, and Magic.Source: §4.2.2 · p. 8, Table 2 · p. 8

4.3 Analysis of Weak Semantic Priors

  • With increasing prior missingness, Shape stays around 98% on Beijing and close to 99% on Adult, while Trend is more sensitive and drops sharply when nearly all priors are removed.On Adult, Trend decreases from around 98% to nearly 93%.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 Ablation Study

  • USS raises AUC from 0.741 to 0.752, Shape from 96.84% to 97.18%, and Trend from 93.98% to 95.21%; ICS improves all metrics, while ISR raises Trend from 95.94% to 97.95%.USS unifies values, column identities, and priors; ICS constrains attribute-level plausibility, and ISR constrains tuple-level dependencies.Source: §4.4 · p. 9, Table 3 · p. 9
  • With Cw-FP, SCTab-Diff achieves the best AUC and Shape scores, 0.771 and 98.57%, while maintaining a Trend score of 97.89%.Cw-FP performs column-level corruption in the semantic latent space.Source: §4.4 · p. 9, Table 3 · p. 9

4.5 Visualization of Fine-Grained Tabular Fidelity

  • Across News Weekday and Shoppers Region, SCTab-Diff’s generated category counts follow real-data patterns more closely than the compared baselines.The visualizations compare category-wise counts across real data and synthetic data; on Shoppers, SCTab-Diff better matches minority categories 5, 8, and 9.Source: §4.5 · p. 9, Figure 4 · p. 9
  • Additional visualizations show closer fits for BounceRates in Shoppers and n_tokens_title in News, and more faithful pair-wise dependency patterns than the baselines.These supplementary results are shown in Appendix E.1 and E.2.Source: §4.5 · p. 9

5 Conclusion

SCTab-Diff makes weak semantic priors conditions for reverse denoising, and experiments on six benchmarks report gains in distributional fidelity, semantic consistency, and downstream task utility. Earlier tabular generators largely prioritize distributional or task-level fidelity over explicit semantic validity.

5 Conclusion

  • SCTab-Diff conditions reverse denoising on intra-column semantics and inter-column rules, aligning values, column identities, and priors to preserve distributions and cross-column dependencies.Experiments on six real-world benchmarks report improvements in distributional fidelity, semantic consistency, and downstream task utility over representative baselines; analyses also report robustness to partially unavailable priors.Source: §5 · pp. 9–10

A.1 Distribution-Centric Tabular Generation

  • GAN- and VAE-based generators model tabular data through distribution-oriented approaches, including mode-specific normalization, conditional generation, and latent representations.Examples include CTGAN [13], CTAB-GAN+ [24], TVAE [13], and VAEM [14].Source: §A.1 · p. 13
  • LLM-based synthesis and augmentation methods mainly target distributional or task-level fidelity and rarely model explicit constraints on whether generated rows are logically plausible.Examples include P-TA [25], EPIC [12], and Curated LLM [11].Source: §A.1 · p. 13

A.2 Diffusion Models for Mixed-Type Tabular Data

Tabular diffusion has advanced in handling mixed data types and heterogeneous features, but remains largely distribution-driven and may miss semantic validity constraints. SCTab-Diff conditions denoising on semantic priors, column identities, and cross-column context, then decodes denoised column representations into synthetic rows.

A.2 Diffusion Models for Mixed-Type Tabular Data

  • Tabular diffusion methods handle mixed types with separate or co-evolving processes, while latent-space and feature-wise approaches target heterogeneous feature distributions.Examples include TabDDPM [5], CoDi [28], TABSYN [4], and TABDIFF [3].Source: §A.2 · p. 13

A.3 Semantics-Aware Tabular Generation

  • Existing tabular diffusion models largely learn statistical regularities, so they may generate plausible samples that violate column meanings or cross-column symbolic rules.SCTab-Diff extracts intra-column semantics and inter-column rules from schema and textual descriptions, then injects them into prior-conditioned denoising.Source: §A.2 · p. 13, §A.3 · p. 13

B.1 Algorithms

  • Training encodes priors in a unified semantic space, corrupts each column representation, adds column identity, injects masked priors through cross-attention, and captures cross-column context.The denoiser predicts injected noise, while the decoder uses a reconstruction objective; training minimizes the combined loss.Source: §B.1 · pp. 14–15
  • Generation starts from independent Gaussian latent variables for each column, iteratively denoises using column identity, semantic priors, and cross-column context, then decodes columns into synthetic rows.Repeating this process produces the synthetic dataset.Source: §B.1 · pp. 15–16

B.2 Analysis of Prior-Conditioned Denoising

The analysis interprets semantic conditioning as changing the reverse denoising direction toward rows that satisfy weak priors, without adding a training objective. Random prior masking trains across partial-prior conditions to reduce reliance on a complete fixed set.

Bayesian view of semantic conditioning

  • The semantics-aware score combines the standard distributional score with a term that raises the probability of denoising into a semantically valid row.Thus semantic priors affect denoising before a complete sample is generated, rather than serving as post-hoc filters.Source: §B.2 · p. 16

Effect on invalid regions

  • Noisy states more likely to denoise into valid rows receive greater relative density, while trajectories likely to end in invalid rows are down-weighted.This offers an intuitive explanation for reducing tuple-level violations while maintaining distributional fidelity.Source: §B.2 · p. 16

Robustness induced by prior masking

  • Randomly masking prior tokens trains SCTab-Diff under multiple partial-prior conditions, acting as a regularizer against over-reliance on a complete fixed prior set.A larger pu weakens guidance, while a moderate pu improves robustness to incomplete priors.Source: §B.2 · p. 17

C.1 Datasets

The experiments use six UCI datasets, construct weak semantic priors from metadata and validate them on training data, and compare representative generators using distributional, semantic, and task metrics. Reported results include gains over TABDIFF [3] on joint-distribution measures and strong C2ST performance; guidance-level changes track across several metrics.

C.1 Datasets

  • The study uses six UCI datasets: Adult, Default, Shoppers, and Magic for classification, and Beijing and News for regression.Table 4 reports dataset statistics and counts of intra-column semantics and inter-column symbolic rules.Source: §C.1 · p. 17, Table 4 · p. 17

C.2 LLM-based Prior Construction and Validation

  • GPT-4o mini proposes intra-column semantics and inter-column rules from metadata without seeing raw records; candidates are checked on training data and retained, discarded, or revised.Validated constraints are verbalized without adding new rules and then used as semantic conditions.Source: §C.2 · pp. 17–18

C.3 Baselines

  • Comparisons include GAN-, VAE-, LLM-, and diffusion-based generators; post-hoc filtering and repair are excluded because they use a different mechanism from conditioned denoising.The study uses representative generators under the same evaluation protocol and ablates components of SCTab-Diff.Source: §C.3 · pp. 18–19

C.4 Metrics

  • Evaluation covers column-wise Shape, pairwise Trend, α-Precision and β-Recall for joint fidelity and coverage, plus MLE, Sample Accuracy, and C2ST.MLE uses TSTR with XGBoost; classification is scored by AUC and regression by RMSE.Source: §C.4 · p. 19, §C.4.1 · pp. 19–20, §C.4.2 · p. 20

D.1 Joint Distribution

D.3 Analysis of Condition Guidance Level Parameters

  • Across Adult, Magic, Beijing, and News, MLE, SA, Shape, and Trend show similar patterns as the Condition Guidance Level parameter pu changes.The authors report that the nearly consistent trends suggest these metrics may be correlated and that pu influences them in a related manner.Source: Figure 5 · p. 22, §D.3 · p. 22

D.2 Detection Score (C2ST)

  • SCTab-Diff achieves either the best or second-best Detection Score measured by C2ST, which evaluates how distinguishable synthetic data is from real data.The reported discriminator is logistic regression.Source: Table 8 · p. 22, §D.2 · p. 22

D.4 Analysis of Prior Masking Probability

Prior masking tests probe robustness to partially incomplete schema and textual metadata. The main experiments use pu=0.2, where degradation is generally moderate.

D.4 Analysis of Prior Masking Probability

  • The main experiments set pu=0.2 to reflect partially incomplete schema and textual descriptions while most semantic information remains available.The paper describes extremely large masking probabilities as overly pessimistic for real-world datasets.Source: §D.4 · pp. 22–23, Table 9 · p. 23
  • As pu increases, Shape, Trend, and SA tend to decline, but degradation is generally moderate when pu ≤0.2.Table 9 reports results for different prior masking probabilities.Source: §D.4 · p. 23, Table 9 · p. 23
  • Higher pu masks more prior tokens, weakening semantic guidance, while moderate masking may regularize against over-reliance on complete priors.Source: §D.4 · p. 23

D.5 Visualization of Authenticity Comparison between Synthetic Data and Real Data

The Adult sample-level visualization contrasts real rows with synthetic rows to show local semantic violations that aggregate distribution metrics may miss. SCTab-Diff shows fewer highlighted violations than TABDIFF.

D.5 Visualization of Authenticity Comparison between Synthetic Data and Real Data

  • Highlighted cells mark intra-column semantic or inter-column rule violations, revealing row-level patterns that distribution-level metrics may not fully capture.Source: §D.5 · p. 23
  • Real Adult samples generally satisfy the weak priors, while TABDIFF can produce mismatched education and education_num values and incompatible demographic combinations.Source: §D.5 · pp. 23–24
  • SCTab-Diff produces fewer highlighted violations than TABDIFF and better preserves logical consistency among related columns.The paper attributes this improvement to prior-conditioned denoising, with weak semantic priors encoded in a unified semantic space and used as generation conditions.Source: §D.5 · p. 24

E.1 Detailed Description of Numerical Column Distribution Visualization

The visual analyses show SCTab-Diff better matching numerical column distributions and preserving pair-wise dependencies than the compared methods on Shoppers and News.

E.1 Detailed Description of Numerical Column Distribution Visualization

  • For BounceRates on Shoppers, SCTab-Diff follows the real KDE more closely than TabDDPM, TABSYN, and TABDIFF, preserving the dominant near-zero region and secondary high-value region.Source: §E.1 · p. 24

E.2 Detailed Description of Inter-Column Dependency Visualization

  • Lighter Fig. 8 heatmap regions indicate stronger preservation of real pair-wise dependencies; SCTab-Diff better retains these patterns on Shoppers and News.TabDDPM and TABSYN show larger discrepancies, especially on the more complex News dataset, while TABDIFF retains some mismatches.Source: Figure 8 · p. 25, §E.2 · p. 25

E.1 Detailed Description of Numerical Column Distribution Visualization

  • For n_tokens_title on News, SCTab-Diff aligns better with the real density shape than TabDDPM, TABSYN, and TABDIFF, whose curves show collapse or remaining mismatches.The passage describes News as challenging because it has many features; SCTab-Diff's unified semantic space and column-wise diffusion help preserve value-sensitive numerical semantics.Source: §E.1 · pp. 24–25

F Limitations

SCTab-Diff depends on useful metadata, models only two types of semantic priors, and has been evaluated on six real-world benchmarks whose coverage may not include highly specialized domains.

F Limitations

  • Prior quality depends on the availability and informativeness of schema and textual descriptions, and noisy or incomplete metadata may yield imperfect guidance.Source: F Limitations · p. 25
  • The framework models intra-column semantics and inter-column symbolic rules, but not more complex domain knowledge such as causal relations or procedural constraints.Source: F Limitations · p. 25
  • Experiments cover six real-world tabular benchmarks, which may not represent highly specialized domains with stricter semantic requirements.Source: F Limitations · p. 25
  • Although the LLM for prior extraction does not access raw records, the generative model is still trained on real data.The authors identify stronger privacy auditing and privacy-preserving training as future work.Source: F Limitations · p. 25

Related papers

Loading 2609.16069v1…