PaperlayerView exact revision on arXiv

Source-linked AI summary

Constraint-Aware Synthetic Tabular Data Generation via Inter-Column Constraint Discovery with LLM Agents

Jianxing Zhao, Mao Guan, Dongyu Liu

arXiv:2608.15109v1cs.AI

TL;DR

Synthetic tabular data can remain structurally invalid despite strong statistical fidelity and downstream utility. This paper discovers executable equations, linear inequalities, and logical dependencies with LLM agents, validates and revises them using counterexamples, and postprocesses generator outputs; the workflow improves held-out violation detection and yields zero measured violations for retained, applicable constraints.

  • Problem

    Existing methods do not jointly automate discovery across heterogeneous inter-column constraint families and generator-agnostic enforcement.

  • Method

    LLM agents propose executable equations, linear inequalities, and logical dependencies, which deterministic tools validate and revise before coordinated post-hoc repair of unchanged generator outputs.

  • Results

    The workflow improves held-out violation detection over one-shot prompting, while postprocessing yields zero measured violations for every retained, applicable validator.

  • Takeaways & Limitations

    Counterexample-grounded discovery with coordinated repair offers a practical path to structurally reliable tabular synthesis.

  • Takeaways & Limitations

    The method covers only inter-column constraints evaluated within each record and excludes inter-row, cell-level, and formatting constraints.

Abstract

from arXiv · show

Generating structurally valid synthetic tabular data remains difficult: outputs with high statistical fidelity and downstream utility can still violate semantically meaningful domain constraints. We study the discovery and enforcement of three complementary inter-column constraint families---equations, linear inequalities, and logical dependencies. Our unified tool-grounded workflow represents all three as machine-executable hypotheses and applies a common interface for full-table validation, deterministic diagnosis, and counterexample-guided revision. A generator-agnostic postprocessor coordinates family-specific repairs on outputs from unchanged tabular generators. Across curated behavioral audits and end-to-end evaluations, the complete workflow improves held-out violation detection over one-shot direct prompting, while postprocessing yields zero measured violations for every retained, applicable constraint, improves downstream utility on most datasets, and largely preserves univariate marginals.

1 Introduction

The paper presents a tool-grounded framework that discovers and enforces machine-executable inter-column constraints on synthetic tabular data. It combines deterministic validation and revision with generator-agnostic postprocessing, improving held-out violation detection and eliminating measured violations for retained applicable constraints.

  • Motivation: Strong distributional similarity and downstream utility can coexist with structurally implausible synthetic records that violate domain constraints.Such violations can undermine simulation and decision support, distort downstream models, and expose synthetic records through simple logical checks.
  • Framework: The framework discovers equations, linear inequalities, and logical dependencies as machine-executable hypotheses from metadata, column profiles, and sampled records.Family-specific deterministic tools validate proposals against complete reference tables and return diagnostics and counterexamples for iterative revision.
  • Framework: A generator-agnostic postprocessor coordinates categorical and numerical repairs across shared columns without backbone access or retraining.This is one of the paper’s stated contributions alongside the unified typed interface for validation, diagnosis, and counterexample-guided revision.
  • Evaluation: Higher mean violation-detection accuracy, precision, and recall than one-shot direct prompting occur in all 18 family–backbone–metric comparisons.The complete executable validation-and-revision workflow was evaluated on three curated discovery benchmarks with two LLM backbones.
  • Evaluation: Zero measured violations result under every retained, applicable constraint after postprocessing.End-to-end evaluation covers seven public datasets across four generator paradigms.

2 Related Work and Problem Setting

This section frames constraint-aware tabular generation as learning synthetic tables that approximate mixed-type real-data distributions while satisfying executable, record-level inter-column constraints. It distinguishes three complementary constraint families and motivates unified discovery, validation, and coordinated repair for unconstrained generator outputs.

  • Problem setting: Rows are modeled as i.i.d. samples from a mixed-type joint distribution, while generators learn pθ from real data to approximate pdata.Metadata M includes dataset descriptions, column names, data types, and column descriptions.
  • Related work: Existing tabular generators include copula-based models, deep generators such as CTGAN and TVAE, and language-model generators.Language-model approaches include serialized-row fine-tuning or pretraining and prompting or LLM-guided curation.
  • Constraint scope: The paper defines constraints as row-separable predicates over at least two columns, excluding uniqueness, cross-table keys, and dataset-level statistical constraints.Each predicate is evaluated independently on a record and never compares different records.
  • Constraint families: Equations capture numerical derivations, linear inequalities numerical feasibility, and logical dependencies categorical admissibility across deterministic and set-valued semantics.The families jointly cover numerical and categorical columns while remaining executable and verifiable at record level.
  • Paper objectives: The objectives are to discover, validate, and revise executable constraints from real data, metadata, and unconstrained synthetic tables, then jointly enforce them while preserving fidelity and utility.Shared columns make independent repairs unsafe because repairing one constraint can invalidate another or distort the generated distribution.

3 Methodology

The methodology discovers inter-column constraints as executable hypotheses, validates them deterministically on the full reference table, and revises failed candidates using counterexamples. It then consolidates accepted constraints and repairs generated tables post hoc without changing the backbone generator.

  • Constraint discovery: Column profiles summarize semantic descriptions, types, and type-specific statistics before constraint discovery.Numerical profiles include ranges, quantiles, means, standard deviations, and mass points; categorical profiles include frequent values and frequencies.
  • Constraint discovery: Executable representations provide a common interface for proposing, evaluating, diagnosing, and revising equation, inequality, and logical-dependency hypotheses.Candidates include identifiers, natural-language descriptions, participating columns, and family-specific machine-executable fields.
  • Validation and revision: Every hypothesis is validated on the complete reference dataset and retained only when its violation rate satisfies the threshold, with logical dependencies additionally requiring sufficient applicability support.Failed candidates return aggregate diagnostics and up to ncex violating rows for revision, with at most Rrev attempts before rejection.
  • Constraint enforcement: Postprocessing enforces discovered constraints on already-produced tables without accessing or changing the backbone generator, avoiding retraining and rejection sampling.This makes the pipeline applicable to black-box generators and motivates direct post-hoc repair when rejection acceptance becomes impractically low.
  • Constraint enforcement: Repairs apply logical dependencies, equations, and linear inequalities sequentially, with equation repair preceding linear projection to prevent reintroduced inequality violations.During projection, equation-participating columns remain fixed so repaired equations are preserved.

4 Evaluation

Evaluation spans curated audits and end-to-end generation across multiple constraint families, model backbones, datasets, and generators. The tool-grounded workflow improves held-out discovery and achieves exact applicable-constraint satisfaction through postprocessing while largely preserving utility and fidelity.

  • Evaluation setup: The audit covers 12 equations in NBA, 17 inequalities in URL, and 13 logical dependencies in categorical ANXIETY.Separate benchmarks provide family-specific constraints and controlled tasks for each typed representation.
  • Discovery evaluation: 18/18 family–backbone–metric comparisons show higher mean held-out violation-detection scores for the complete workflow.Logical-dependency precision and recall rise from .442 to .974 with GPT-5.6 and .963 with Claude-5; Claude-5 linear accuracy rises from .815 to .959.
  • End-to-end evaluation: Raw generators show family-specific reversals: TabDDPM has 3.1% LD CVR but 2.23 LFD, while TVAE has 0.11 LFD and equational R2 = 0.45.CTGAN is weakest on equations with R2 = −0.97, whereas Gaussian Copula has the highest LD CVR at 36.4%.
  • End-to-end evaluation: Postprocessing reaches exact optima for every applicable constraint metric: CVR, sCVC, and LFD become zero, while equational consistency reaches R2 = 1.Raw equational CVR ranges from 98.2% to 100%, LD CVR reaches 65.9%, and linear CVR reaches 77.8%.
  • Sensitivity analyses: On NEWS, removing counterexamples lowers validated-hypothesis yield from 46.33 to 19.33, while one discovery round lowers it to 11.33.Five rounds more than double token use but yield only 38.67, making the default (3, 100, 20) the best observed configuration.
  • Repair analyses: KS-complement scheduling preserves higher marginal fidelity than random feasible scheduling while both achieve zero equational CVR on 180 matched inputs.Projection last also avoids cross-family interference: E→L has zero final CVR and LFD, whereas L→E leaves linear violations in 24 settings (66.7%).

5 Conclusion

The paper presents a unified framework for executable inter-column constraint discovery and post-hoc enforcement on unchanged tabular generators. Across evaluations, validation-and-revision improves held-out violation detection, while postprocessing eliminates measured violations under retained, applicable validators and generally preserves utility and univariate marginal fidelity.

  • Unified framework: The framework represents equations, linear inequalities, and logical dependencies as executable hypotheses and coordinates their post-hoc enforcement on outputs from unchanged generators.It unifies discovery and enforcement across three inter-column constraint families.
  • Evaluation results: Improved held-out violation detection over direct prompting was observed across multiple LLMs, datasets, and generator families.The complete validation-and-revision workflow produced this improvement.
  • Evaluation results: Zero measured violations under retained, applicable validators were achieved while utility and univariate marginal fidelity were generally preserved.Counterexample-grounded discovery coupled with coordinated repair provides a practical path to structurally reliable tabular synthesis.

Limitations

The method covers only inter-column constraints evaluated within individual records and depends on rich metadata for LLM-based discovery. Its zero-violation result is empirical and limited to retained, applicable validators, not complete semantic correctness or discovery.

  • Constraint scope: The method does not model inter-row constraints or cell-level validity and formatting rules, including uniqueness, regular expressions, and domain-specific semantic types.Extending discovery and enforcement to these constraint families is outside this work’s scope.
  • Validation limits: Zero measured violations establish empirical consistency only for retained, applicable validators, excluding uncovered logical-dependency configurations and unavailable repairs.Full-table validation does not establish semantic correctness or complete constraint discovery.
  • Metadata dependence: Discovery agents rely on rich dataset- and column-level metadata, so limited, ambiguous, or noisy annotations may reduce discovery accuracy.Robustness to limited-quality metadata remains future work.

Ethics Statement … A.3 KS-Guided Equational Repair

The paper specifies executable representations and prompt protocols for discovering equations, inequalities, and logical dependencies, then uses dependency-safe KS-guided scheduling to repair equations while addressing data-protection and bias risks. The workflow prioritizes semantically defensible constraints, full-data verification, and schedules that avoid especially damaging repairs.

  • Ethics Statement: Public datasets are used only for methodological evaluation, not individual identification or decision support, including in sensitive domains such as credit risk and mental health.Applying the framework to confidential or personally identifiable data requires authorization and data-protection measures before records are sent to third-party model providers.
  • Ethics Statement: Discovered constraints may encode historical biases, so structural validity does not guarantee privacy or fairness; generated Python must also be securely sandboxed.
  • A.1 Constraint Hypothesis Representations: Each constraint family has a distinct machine-executable representation, including vectorized equation checks, coefficient-based linear inequalities, and value-table logical dependencies.The representations support full-table validation through executable schemas.
  • A.2 Discovery LLM Prompts: Discovery prompts require semantically grounded, row-wise constraints supported by metadata, samples, and full-data evidence while excluding correlations, fitted formulas, arbitrary groupings, and accidental patterns.Logical dependencies use determinant columns, one dependent column, and admissible-value tables; inequalities emphasize whole–part and minimum–average–maximum relationships.
  • A.2 Discovery LLM Prompts: Refinement uses failed validation samples and candidate history to submit one revision with the same constraint ID or reject it with a concrete reason.
  • A.3 KS-Guided Equational Repair: For each equation, repair-target and repair-order choices are evaluated by changes in KS-complement marginal similarity, with repaired columns frozen so later repairs cannot invalidate satisfied constraints.A positive ∆c,j indicates expected improvement and a negative value expected loss.
  • A.3 KS-Guided Equational Repair: The selected repair schedule maximizes its worst individual repair score, then total score, preventing one highly damaging repair from being hidden by favorable others.Dynamic programming searches dependency-safe schedules by descending score thresholds and returns the first threshold admitting a complete schedule.

B Datasets · C Constraint Discovery Details · C.1 Expert Annotation and Ground-Truth Constraints

The evaluation uses eight public tabular datasets, with Anxiety reserved for constraint discovery and no utility task. Ground truth comprises expert-reviewed logical dependencies, equations, and linear inequalities for Anxiety, NBA, and URL, respectively.

  • B Datasets: Eight public tabular datasets support downstream utility evaluations, with Table 3 reporting train-test sizes and categorical and numerical column counts.Column counts include the utility target; Anxiety is excluded from the end-to-end evaluation task.
  • B Datasets: Anxiety is used only for logical-dependency discovery after discretizing twelve numerical columns into two to four ordinal bands and deriving Occupation Group.This produces a view of 11,000 rows and 20 categorical columns, with no utility task.
  • C.1 Expert Annotation and Ground-Truth Constraints: Ground truth is constructed through a two-annotator protocol in which candidates are proposed from documentation and independently reviewed for soundness, support, and semantic meaning.The second annotator checks near-universal satisfaction among applicable records and nontrivial support, including determinant configurations for logical dependencies.
  • C.1 Expert Annotation and Ground-Truth Constraints: 13 annotator-verified logical dependencies form the ground truth for the ANXIETY detection task.The rules encode conditional admissibility and mappings among categorical attributes, including high-anxiety subgroup rules and caffeine-related one-to-many implications.
  • C.1 Expert Annotation and Ground-Truth Constraints: 12 annotator-verified equations form the ground truth for the NBA detection task.They capture exact accounting identities among per-100-possession basketball statistics, including point decompositions and reconstructions of makes, attempt shares, and efficiency.
  • C.1 Expert Annotation and Ground-Truth Constraints: 17 annotator-verified linear inequalities form the ground truth for the URL detection task.URL features support containment, ordering, and bounded-total constraints; hyperlink fractions sum to at most one, while corresponding media features sum to at most 100.

C.2 Detection Evaluation Protocol … D.3 Computational resources

The paper evaluates constraint detection on hidden audit splits using executable discovered constraints and support-weighted metrics, then specifies generator configurations, constraint-quality metrics, utility and marginal-fidelity measures, and computational resources for end-to-end experiments.

  • C.2 Detection Evaluation Protocol: Five 70/30 discovery–audit splits hide audit rows and annotations during discovery, while agents inspect two disjoint 100-row samples and may revise proposals for three rounds.The system verifies proposals against the complete discovery partition.
  • C.2 Detection Evaluation Protocol: 34,000 URL and 24,000 NBA shuffled rows per split are formed from matched one-cell mutations that violate only the targeted ground-truth constraint.ANXIETY uses 1,950 matched pairs per split with allocation proportional to each rule’s applicability support.
  • C.2 Detection Evaluation Protocol: Accuracy, precision, and recall are computed from an ensemble of executable discovered constraints, with support-weighted means and standard deviations reported across five splits.The protocol evaluates behavioral detection rather than exact formula recovery, and malformed constraints produce no violation predictions.
  • D.1 Generator Training Details: The experiments use adapted official implementations of CTGAN, TVAE, and TabDDPM, alongside SDV 1.32.1’s GaussianCopulaSynthesizer.Tables 7–10 report the resolved configurations, including a FLIGHTS-specific TabDDPM override.
  • D.2 Definition of Metrics: CVR measures the fraction of rows violating any constraint, whereas sCVC measures average violation density across row–constraint checks; both range from 0 to 1.Zero indicates no measured violations, and sCVC is more informative when many constraints make CVR approach one.
  • D.2 Definition of Metrics: Equation consistency is scored by the best verified target-direction reconstruction R2, while linear feasibility distance is normalized by real-data standard deviations and equals zero when all rows are feasible.Higher equation scores are better; lower feasibility distances are better.
  • D.2 Definition of Metrics: TSTR reports ROC-AUC for classification and R2 for regression, while Column Shapes measures mean univariate similarity using KS and total-variation complements.Models are selected by the highest mean TRTR score, and higher marginal-fidelity scores indicate greater similarity.
  • D.3 Computational resources: The end-to-end design covers 84 dataset–split–synthesizer settings and 252 synthetic-sample runs on 128 CPU cores and four 48-GB GPUs.Recorded training and generation required approximately 18 hours of cumulative computation.

D.4 Full End-to-End Results · E Ablation Studies

The end-to-end evaluation reports complete utility, marginal-fidelity, and raw constraint results across dataset–generator combinations. Constraint postprocessing attains perfect measured constraint scores throughout, while its effects on utility and univariate marginals are reported relative to raw outputs.

  • D.4 Full End-to-End Results: Table 11 reports Column Shapes for every dataset, generator, and output variant, where higher values indicate greater univariate marginal fidelity to real data.Entries aggregate nine runs across three independent data splits, and the final column gives ∆= Postprocessed −Raw for displayed means.
  • D.4 Full End-to-End Results: Table 11 summarizes before-and-after constraint-postprocessing Column Shapes with means, standard deviations, and gain or loss annotations.Green denotes gains and orange denotes losses; higher Column Shapes values are better.
  • D.4 Full End-to-End Results: Table 12 reports complete downstream utility results, including selected dataset-level TRTR scores and raw versus postprocessed TSTR scores for every dataset–generator pair.TRTR aggregates three splits, while TSTR aggregates nine runs across three splits; regression uses R2 and classification uses ROC-AUC.
  • D.4 Full End-to-End Results: Postprocessing attains zero CVR, sCVC, and LFD, while equational R2 is one throughout the applicable end-to-end results.Only raw scores are tabulated because postprocessing effectively addresses the applicable constraints.
  • D.4 Full End-to-End Results: Table 12 reports postprocessing deltas as Postprocessed − Raw for displayed TSTR means, with gains and losses identified separately.Higher is better for both R2 and ROC-AUC.
  • D.4 Full End-to-End Results: Table 13 reports raw constraint results for every dataset–generator combination across nine runs spanning three independent data splits.CVR, sCVC, and LFD are lower-is-better; equational R2 is higher-is-better, and unavailable constraint families are shown as dashes.

E.1 Constraint-Discovery Hyperparameter Sensitivity

A one-factor sensitivity study on NEWS supports (Rdisc, nctx, ncex) = (3, 100, 20) as an efficient operating point. More rounds, context, or counterexamples increased cost without improving retained-constraint yield, while removing counterexample feedback sharply reduced yield.

  • Discovery rounds: 46.33 mean yield at three rounds exceeded 11.33 at one and 38.67 at five, making three rounds the best tested cost–yield balance.Five rounds more than doubled token usage, while one round cut token use.
  • Context rows: 100 context rows were most efficient: 50 and 200 rows used more tokens but yielded 48.00 and 44.33 versus 46.33.The comparison is relative to the 100-row default.
  • Counterexample feedback: Removing counterexample feedback reduced mean yield from 46.33 to 19.33 without lowering token usage, while increasing ncex to 50 yielded 47.33 at greater cost.The default uses ncex = 20.
  • Interpretation and limitation: The results support (3, 100, 20) as efficient in this setting, but the one-dataset, one-backbone study establishes neither universal optimality nor statistical significance.The study evaluated seven configurations across three NEWS splits with GPT-5.6 Luna.

E.2 Equational Repair Order and Target Selection

This ablation isolates equational-repair scheduling and target selection while holding the synthetic data, constraints, generators, splits, and repair functions fixed. KS-complement guidance improves marginal fidelity over random feasible scheduling while both schedules achieve exact equation satisfaction.

  • Experimental setup: The ablation reruns only equational repair, disabling logical-dependency and linear-inequality enforcement while holding inputs, generators, splits, constraints, and repair functions fixed.The intervention is limited to the scheduling policy: KS-complement guidance versus a random feasible scheduler.
  • Scheduling policy: KS-guided scheduling prioritizes schedules by their highest minimum predicted KS-complement change, then selects the schedule with the largest total predicted change.After an equation is repaired, all participating columns are frozen as future targets; random scheduling chooses targets and order randomly, with backtracking when needed.
  • Evaluation: 180 matched inputs across FLIGHTS, NBA, NEWS, TAXI, and STEEL evaluate univariate marginal fidelity with Column Shapes and constraint satisfaction with equational CVR.The design uses four generators, three data splits, and three synthetic samples per split.
  • Results: 0 equational CVR is achieved by both schedules on all 180 matched inputs, so the scheduling difference concerns fidelity rather than equation satisfaction.Equational CVR is omitted from Table 14 because it is zero under both schedules on every input.
  • Results: 0.020–0.053 mean paired Column Shapes improvement favors KS guidance over random feasible scheduling on four datasets, with the largest gain on FLIGHTS and an effective tie on NEWS.The equal-weight mean improvement across datasets is 0.025, without claiming global optimality.

E.3 Equational–Linear Repair Order

On NBA, numerical repair order determines whether overlapping equational and linear constraints remain jointly satisfied. Equational-then-linear repair is safer because the final projection protects repaired equation columns while enforcing linear feasibility.

  • E.3 Equational–Linear Repair Order: Under E→L, equational repair establishes equations before projection, which protects equation-participating columns and adjusts only remaining mutable columns.Under L→E, projection lacks equation-derived protected columns and may modify any participating numerical columns.
  • E.3 Equational–Linear Repair Order: Only E→L achieves joint satisfaction in all 36 matched settings, with final linear CVR and LFD both zero while equations remain exactly satisfied.The ablation covers three splits, four generators, and three synthetic draws per split, yielding 36 matched settings.
  • E.3 Equational–Linear Repair Order: 66.7% of L→E settings retain linear violations, raising aggregate CVR to 4.574% and LFD to 0.863, despite zero final equational CVR.Linear violations occur in 24 of 36 settings; Gaussian Copula and CTGAN have mean CVRs of 9.919% and 7.331%, respectively.
  • E.3 Equational–Linear Repair Order: When numerical families overlap, projection last is safer because equational reconstruction after projection can move rows outside the linear feasible region.Logical-dependency repair is applied first because its categorical columns are disjoint from numerical constraint columns and cannot interfere with either numerical family.
Loading 2608.15109v1…