PaperlayerView exact revision on arXiv

Source-linked AI summary

Generating Benchmark Health Data Using a Tabular Diffusion Transformer

Hao Yan, Lisa Pilgram, Dan Liu, Linglong Kong, Fida Dankar, Khaled El Emam

arXiv:2608.14496v1cs.LGcs.AI

TL;DR

Existing synthetic tabular data generators are largely limited to single input tables, leaving cross-tabular generation from heterogeneous datasets insufficiently addressed. The paper introduces a two-stage CTDG framework that standardizes tables into statistical representations, models them with a diffusion transformer, and reconstructs synthetic raw tables. Experiments report high fidelity in the learned representations and a favorable fidelity-diversity trade-off in generated data.

  • Problem

    Existing synthetic tabular data generation methods are largely restricted to single-table settings and struggle with multiple heterogeneous tables.

  • Method

    The framework converts each raw table into a standardized statistical table, trains a diffusion transformer on these tables, and reconstructs synthetic raw tables through Gaussian sampling and inverse probability integral transformation.

  • Results

    The proposed approach achieves high fidelity in learned statistical representations and favorable fidelity-diversity trade-offs in generated synthetic data.

  • Takeaways & Limitations

    Learning at the table level supports a unified generative model for producing synthetic heterogeneous datasets that reflect the plurality of real inputs.

  • Takeaways & Limitations

    The statistical-table representation may miss extreme outliers, even when marginal distributions capture heavy tails.

Abstract

from arXiv · show

Cross-Tabular Data Generation (CTDG) seeks to learn a generative model from multiple heterogeneous tables and produce new synthetic tabular datasets. However, existing synthetic tabular data generation methods are largely restricted to single-input-table scenarios and struggle to effectively handle multiple heterogeneous tables with diverse feature sets. To address this limitation, we propose a two-stage framework for cross-tabular data generation. In the first stage, each heterogeneous raw table is transformed into a standardized statistical table with the same set of columns across all tables. Each statistical table captures the marginal distributions of the original columns and the pairwise correlations among them. In the second stage, a diffusion transformer model is trained to capture structural patterns across these homogeneous statistical tables and to generate synthetic statistical tables. Synthetic raw tables are subsequently reconstructed from the generated statistical tables via multivariate Gaussian sampling followed by an inverse probability integral transform. This two-stage CTDG framework enables the learning of a unified generative model from multiple heterogeneous tables and supports the generation of an unlimited number of realistic synthetic heterogeneous tables. Experimental results demonstrate high fidelity in the learned statistical representations and a favorable fidelity-diversity trade-off in the generated synthetic data, validating the effectiveness of the proposed approach.

1. Introduction

Synthetic tabular data can support healthcare benchmarking, collaboration, and data-scarcity mitigation, but existing benchmark repositories often lack standardization, diversity, or grounding in real health data. The study develops CTDG to learn across heterogeneous tables and generate diverse synthetic datasets reflecting shared and source-specific patterns.

  • Synthetic healthcare data can support privacy-preserving collaboration, benchmarking, reproducibility, and analyses of data scarcity and bias.
  • Existing benchmark repositories often use only a few datasets, making dataset selection arbitrary and comparability vulnerable to cherry-picking.
  • Raw-data benchmarks require user-dependent preprocessing, so reproducibility and comparability can vary even for the same dataset.
  • Some synthetic benchmark approaches lack diversity or are not based on real data, limiting their reflection of actual health-data distributions and structure.
  • Pooling tables can capture shared correlations, distributional shapes, and feature interactions that may be unavailable in any individual dataset.
  • CTDG learns from multiple heterogeneous tables to produce diverse, realistic synthetic datasets that reflect shared domain structure and source-specific distributions.

2. Background

Existing tabular generators largely model rows from a single fixed schema, limiting their ability to learn across heterogeneous tables. Cross-table research broadens the setting, but prior generative approaches have mainly been evaluated for single-table generation rather than diverse heterogeneous dataset generation.

  • Single-table methods assume homogeneous feature spaces and cannot directly learn across tables whose columns, data types, and semantics differ.
  • Diffusion models such as TabDDPM and TabSyn have achieved strong tabular fidelity and privacy results, but remain within standard single-table generation settings.
  • Standard synthetic tabular data generation methods generally treat each row as an independent instance within one fixed schema.
  • Prior cross-table generative work pretrains on multiple tables but fine-tunes and evaluates on individual tables rather than diverse heterogeneous dataset generation.
  • Cross-table models including XTab and CM2 primarily target predictive tasks, while CTSyn was evaluated mainly for single-table generation.
  • The proposed method instead treats each table as one instance, learning statistical properties and structural patterns across multiple tables.

3. Methods

The method standardizes heterogeneous raw tables into fixed-format statistical tables containing marginal and correlational information, trains a Tabular Diffusion Transformer over them, and reconstructs synthetic raw tables. Its representations prioritize shared statistical patterns over exact original labels or value ranges while preserving dependencies for efficient sampling.

  • 3.1 Statistical tables: Each heterogeneous raw table is converted into a homogeneous statistical table summarizing column-wise marginal distributions and the correlation matrix.
  • 3.2 Diffusion transformer: A Tabular Diffusion Transformer learns the joint distribution of statistical tables and generates new synthetic statistical tables for reconstruction.
  • 3.1.1 Marginal statistics: Categorical values are frequency-sorted and mapped to indices before fitting a Beta–Binomial representation; numerical values are discretized into fixed intervals.
  • 3.1 Statistical tables: The objective is to reproduce shared statistical patterns rather than exact category labels or original numerical value ranges.
  • 3.1.1 Marginal statistics: Marginal statistics use standardized parameter summaries, including distribution parameters, cardinality, and missing rate, across categorical and numerical columns.
  • 3.3 Reconstruction: Reconstruction transforms mapped columns to normal variables, samples jointly from a multivariate Gaussian, and preserves dependency relationships through Gaussian correlations.
  • 3.1.2 Representation of correlational structure: Correlation matrices are eigen-decomposed into a fixed-dimensional representation, with padding or column dropping used to maintain consistency across tables.
  • 3.1.2 Representation of correlational structure: The resulting column representation combines marginal parameters with correlation vectors, while correlations are recovered through inner products.

3.2 Learning from multiple tables

The framework standardizes heterogeneous tables into homogeneous statistical representations and uses a diffusion transformer to learn and generate combinations of statistical rows across tables.

  • Cross-table generation: The model learns the distribution of statistical-row combinations across multiple tables and generates new combinations with similar statistical properties.Synthetic raw tables can subsequently be reconstructed from the generated combinations.
  • Statistical-table representation: Each statistical table represents a unique combination of homogeneous statistical rows corresponding to columns in an original raw table.Rows share the same number of parameters and semantic meanings across tables.
  • Diffusion transformer: A simplified diffusion transformer uses self-attention and variable-length sequence processing to model correlations among embeddings from multiple homogeneous tables.Each row is treated as an embedding, while rows within a table form a sequence.
  • Statistical-table representation: The representation captures column distributions and correlations while standardizing values to comparable scales for transformer modeling.Global quantile transformations are learned across tables and later inverted during generation.
  • Diffusion transformer: DDPM training adds timestep-dependent Gaussian noise and trains the transformer to reconstruct the noise through iterative denoising.Sampling starts from Gaussian noise and progressively produces synthetic statistical tables.
  • Training strategy: Pretraining on general OpenML tables followed by health-data fine-tuning addresses the limited number of statistical-table training instances.The authors report improved convergence stability and generation quality on domain-specific datasets.

3.3 Model training

Training uses broad OpenML pretraining followed by health-domain fine-tuning, with fixed-dimensional statistical representations and a transformer diffusion configuration.

  • Datasets: The evaluation uses OpenML datasets and 13 healthcare-related datasets, including 131 Health-tagged OpenML datasets for fine-tuning.The datasets are treated as representative health-domain benchmarks.
  • Datasets: Pretraining uses 4,095 verified OpenML datasets selected to satisfy the diffusion transformer’s size requirements.The collection spans health, economics, engineering, finance, games, and statistics.
  • Representation settings: Numerical features are discretized into 100 intervals to capture distributional shape while limiting cardinality and computational complexity.The same configuration supports uniform treatment of numerical and categorical columns.
  • Representation settings: Correlation matrices use a hidden dimension K=256, while tables exceeding the supported dimensionality are truncated and narrower tables are zero-padded.This fixed-dimensional representation enables cross-table modeling across varying column counts.
  • Model configuration: The diffusion transformer uses K+4=260 input dimensions, 768-dimensional embeddings, 12 encoder layers, and 12 attention heads.Training uses T=1000 diffusion timesteps with sinusoidal timestep embeddings.

3.4 Model Evaluation

Evaluation separately measures reconstruction fidelity and synthetic health-table quality, including similarity to real health data and diversity across generated tables.

  • Evaluation objectives: The first evaluation tests whether reconstructed raw tables retain high fidelity to their original raw tables after statistical-table encoding and reconstruction.This assesses the representational quality of the statistical tables.
  • Evaluation objectives: The second evaluation compares generated health tables with real health data and real data from engineering, economics, and gaming domains.It also assesses whether generated tables cover the domain’s diversity.
  • Reconstruction metric: The dimension-Normalized Wasserstein Distance divides mean and covariance terms by dimension, enabling comparisons across tables with different column counts.Each table is treated as a multivariate random variable.
  • Synthetic-data metric: The Wasserstein computation maps distributions to multivariate normals through the cumulative distribution function and uses means and covariance matrices.The formulation includes the matrix trace and symmetric square root of the covariance matrix.
  • Synthetic-data metric: Closest Wasserstein Distance measures each synthetic statistical table against all real tables, records the minimum distance, and averages those minima.The resulting average is reported as the overall fidelity measure for the diffusion transformer.

4. Results

The evaluation finds that reconstructed tables generally preserve the originals’ first- and second-order statistics, while correlation ablations substantially degrade reconstruction fidelity. Synthetic tables show systematic proximity to real Health data across folds, alongside broad coverage indicating diversity.

  • Reconstruction from statistical tables: The reconstruction experiment compares raw tables regenerated from extracted statistical tables with their original Health datasets using dNWD.The evaluation uses Health, including Ours-13 and OpenML Health, as source tables and computes distances between real and reconstructed tables.
  • Reconstruction from statistical tables: Most of the 144 real–synthetic table comparisons have dNWD below 0.2, with an average of 0.095 and standard deviation 0.061.These results support close agreement between reconstructed and real tables in their first- and second-order statistics.
  • Effect of correlation representation: Accurate correlation modeling is essential: both underestimating dependencies and introducing spurious ones substantially degrade reconstruction quality.Correlations must reflect the data’s true dependency structure to preserve the original tables’ statistical properties.
  • Synthetic-to-real evaluation: Across five folds, synthetic tables achieve the smallest CWD to real Health tables, averaging 0.72 versus 0.97 for Economics and 0.98 for Statistics.The domain ranking remains stable across folds, with Engineering, Finance, and Games farther away.
  • Synthetic-to-real evaluation: Health CWD distributions are left-shifted while less-related domains move rightward and develop heavier tails, preserving the same domain ordering across the distribution.The pattern is not driven by a small number of favorable cases.

5. Discussion

The paper presents a two-stage approach for learning from multiple heterogeneous tabular datasets and generating plausible synthetic heterogeneous tables. The authors report high fidelity and meaningful diversity, while identifying privacy, outlier, and dimensionality boundaries.

  • The objective is to generate large-scale realistic health datasets for benchmarking studies.
  • The approach learns from multiple heterogeneous tables and generates new, plausible synthetic heterogeneous tables with shared statistical patterns.
  • The two-stage methodology converts heterogeneous raw tables into homogeneous statistical tables, then trains a diffusion transformer to generate new statistical tables.The representation captures marginal and correlation statistics, after which synthetic raw tables are reconstructed by sampling from generated statistical tables.
  • The experiments report high fidelity in reconstructing raw tables and synthetic statistical tables with strong fidelity and meaningful diversity.
  • The generated collection contains 1000 health datasets, with 10,000 rows per dataset and between 10 and 256 columns.These datasets support benchmarking generative models and simulating classifier performance under different outcome balances.
  • Extreme outliers may be missed, so important outliers would need to be inserted into generated datasets for some benchmarking studies.

Funding statement

The research was funded by Canadian research and health organizations, with additional support from the Deutsche Forschungsgemeinschaft.

  • Funding came from the Canada Research Chairs program through the Canadian Institutes of Health Research.
  • Additional support came from an NSERC Discovery Grant, a CIFAR catalyst grant, and the Canadian Children Inflammatory Bowel Disease Network.
  • LP was funded by the Deutsche Forschungsgemeinschaft through its German Research Foundation program.

Competing Interests Statement

KEE and LP report financial interests in Woodway Assurance, a spin-off from their academic research lab that develops privacy-enhancing technologies.

  • KEE and LP have financial interests in Woodway Assurance, a spin-off company from their academic research lab.
  • The disclosed company originates from the authors’ academic research lab.
  • Woodway Assurance develops privacy-enhancing technologies.

A1. Datasets

The experiments use public OpenML data and 13 healthcare-related datasets covering several types of medical and health records. Dataset details are provided in Table 3.

  • The experiments use public OpenML datasets and 13 healthcare-related datasets.
  • The healthcare datasets include inpatient, discharge, pregnancy, COVID-19, and health-survey records.
  • Table 3 lists the details of the 13 healthcare-related datasets.
Loading 2608.14496v1…