PaperlayerView exact revision on arXiv

Source-linked AI summary

Generate in the Chart, Not on the Boundary: Function-Symbol Grounding for Hard Constraints in LTN-GANs

Nijesh Upreti, Vaishak Belle

arXiv:2608.21605v1cs.AI

TL;DR

Constrained generators must produce realistic samples that also obey hard structural rules, but validity alone can erase the real distribution of constraint margins. This paper uses function-symbol grounding and finds that it preserves margins with exact validity, outperforming clamping on high-resolution datasets while a hybrid matches or exceeds it elsewhere.

  • Problem

    Constrained generation requires realistic samples that are provably valid under orderings, positivity requirements, and definitional identities.

  • Method

    Function-symbol grounding computes constrained variables inside a feasible-region chart, while a resolution ratio selects charting or clamping per constraint.

  • Results

    Function-symbol grounding cuts margin KS by up to 25× at equal validity on four high-resolution datasets, while the hybrid matches or exceeds clamping on the benchmark.

  • Takeaways & Limitations

    Validity metrics alone can miss unrealistic margin distributions, so the resolution ratio helps determine which grounding can learn each constraint.

  • Takeaways & Limitations

    The proposed charts cover linear constraints, leaving non-convex feasible sets from nonlinear or disjunctive constraints for future work.

Abstract

from arXiv · show

Logic Tensor Network-Enhanced Generative Adversarial Networks (LTN-GANs) inject background knowledge by grounding each logical axiom as a predicate and training the generator to raise its satisfaction, a fuzzy truth value in $[0,1]$. Previous LTN-GAN work grounded every constraint this way, at the predicate level, and improved constraint satisfaction. A predicate, however, only scores a sample, so it cannot embed hard structural constraints, rules such as orderings, positivity, and definitional identities that must hold in every generated sample. In this work, we investigate grounding each axiom as a function symbol inside the LTN framework. We compare against the state-of-the-art alternative, a constraint layer that clamps each violating sample onto the feasible boundary and so produces outputs that are always valid. Our investigation shows that a valid sample is not always a realistic one. An inequality is not merely satisfied or violated. It holds by a margin, and a faithful generator should also reproduce the margin's real distribution. We find that the resolution ratio $R$, the data's scale over the margin's spread, is a diagnostic, computable before training, of which constraints a chosen grounding can learn. When $R$ is large, the predicate receives no learning signal, the clamp pushes every sample onto the boundary, and the margin distribution is lost while every standard metric still looks fine. A function symbol avoids both failures, computing the constrained variable rather than scoring it. Together the function symbols form a chart, a coordinate system inside the feasible region, where every sample is valid by construction and the margin is learned like any other quantity.

1. Introduction

Constrained generation must produce data that are both realistic and provably valid, because standard validity metrics can miss distorted distributions of constraint margins. Function-symbol grounding generates valid samples inside the feasible region while learning margins as coordinates, with the resolution ratio R guiding when charting or clamping is appropriate.

  • Motivation: Constrained generation must preserve known orderings, positivity requirements, and definitional identities while approximating realistic data distributions.Examples include drop-off after pick-up, departure equal to schedule plus delay, and U0 < U < H.
  • Motivation: Validity alone can hide unrealistic constraint margins, such as trip durations or energy steps whose distributions differ from real data.A sample may satisfy every inequality while producing implausible values for drop-off minus pick-up or U − U0.
  • Function-symbol grounding: Function-symbol grounding computes constrained variables directly, generating inside the feasible region with exact validity and treating each margin as a learned coordinate.For b > a, the generator emits a free value for a and constructs b by adding a positive, smoothly parameterised increment.
  • Resolution ratio: The resolution ratio R predicts before training which constraints a grounding can learn and acts as a condition number for constrained density estimation.High-R continuous constraints favor charting, whereas low-R and discrete margins are better clamped.
  • Hybrid model: A per-constraint hybrid combines function-symbol grounding and clamping because high-R continuous margins benefit from charting while low-R and discrete margins do not.The hybrid uses a short pre-run to choose the treatment for each constrained variable.

2. Background: Logic Tensor Networks and LTN-GANs

LTNs ground predicates as truth-valued scores and function symbols as real-valued computations. In LTN-GANs, generator training combines adversarial loss with knowledge-base satisfaction, while function-symbol grounding can enforce constraints by construction rather than merely encouraging them.

  • Real Logic and grounding: An LTN grounding assigns tensors to first-order symbols, with predicates scoring arguments in [0, 1] and function symbols computing real-valued terms.A k-ary function symbol is grounded as a real map from R^Dk to R^D.
  • The LTN-GAN objective: LTN-GANs train a generator to fool a discriminator while increasing knowledge-base satisfaction through LG = Ladv + λ(1 − Sat(KB)).The generator maps Gaussian latent noise to samples, and the discriminator scores real and generated samples.
  • Predicate grounding of a constraint: G-LTN-GANs ground ordering constraints such as b > a as predicates, producing a soft satisfaction signal that encourages but does not guarantee validity.The predicate-based formulation pushes samples toward satisfying the ordering through the knowledge-base objective.
  • Predicate grounding of a constraint: Predicate gradients are informative only where the logistic predicate is unsaturated, and the usable region covers Θ(1/R) of samples as R grows.Function-symbol grounding instead makes the constrained variable a computed term, while satisfaction remains 1 by construction.

3. Problem Statement

The problem concerns learning generators whose samples satisfy linear inequality axioms while preserving the data’s conditional margin distributions. It compares validity-by-clamping with a resolution-ratio diagnostic that identifies when constraints become difficult to learn.

  • Problem formulation: The generator maps noise to samples x ∈ R^D and is trained so its output distribution pθ approximates the data distribution pX.Background knowledge is a finite set Π of linear inequality axioms over the features.
  • Problem formulation: A generator is compliant when every generated sample satisfies every axiom in Π, including orderings, positivity, and definitional identities encoded as inequalities.The studied axioms have the form Σk wkxk + b ⊵ 0, with ⊵ ∈ {≥, >}.
  • Margin distribution: For each inequality, the margin mϕ measures how far a sample lies from its boundary, and the target is its conditional distribution pX(mϕ | mϕ ≥ 0).Generated and real margins are compared using Kolmogorov–Smirnov distance on a fixed reference sample from the real data.
  • Resolution ratio: The resolution ratio Rϕ = σs/σm is computed before training from raw data, quantifying when a margin is a tiny difference between large, nearly equal feature quantities.σm is the dataset margin spread, while σs is the largest standard deviation among features related by the constraint.
  • The constraint layer (clamping): C-DGM’s constraint layer clamps each generated variable into a Fourier–Motzkin-derived admissible interval, guaranteeing validity but moving violations to near-zero-margin boundaries.The resulting margin distribution is lost at a rate governed by R.

4. Function-Symbol Grounding as a Change of Coordinates

Function-symbol grounding reparameterizes the feasible region by mapping unconstrained coordinates to constrained samples, making linear inequalities and identities valid by construction. Unlike clamping, this chart preserves the distribution of constraint margins by learning them as coordinates inside the feasible region.

  • Change of coordinates and chart: The decoder φ maps one unconstrained coordinate per free variable into the relatively open convex polytope of feasible samples.The feasible set is diffeomorphic to R^d and covered by a single global chart.
  • Grounding construction: Function-symbol grounding assembles variables in Fourier-Motzkin order, using one-sided or box transformations and deriving identity-dependent variables.Softplus and sigmoid parameterizations keep bounded variables within their admissible intervals.
  • Validity by construction: Every generator output produces a sample satisfying any satisfiable finite set of linear inequalities, so these axioms have Sat(KB) = 1.Validity follows from non-empty admissible intervals and transformations mapping R into each interval’s interior.
  • Margin representation: The chart differs from clamping only in where samples land within admissible intervals, preserving the constraint-margin distribution rather than forcing violations to the boundary.For b > a, clamping maps violators to b = a, whereas function-symbol grounding emits mφ = softplus(z).
  • Chart properties: The chart is a smooth bijection onto the relative interior of the feasible set and can realize every margin distribution on (0, ∞) exactly.It has a closed-form inverse, places no probability on the boundary, and is smooth except where the active Fourier-Motzkin bound switches.

5. Experimental Analysis

Across four high-resolution datasets and multiple generator architectures, FSG-LTN-GAN preserves validity while recovering constraint-margin distributions that clamping distorts. Its selective hybrid extends this advantage to high-R benchmark cases, while predicate grounding has usable gradients that shrink as R grows.

  • RQ1: Validity and realism: FSG-LTN-GAN and the constraint layer are both 100% valid, but margin KS is 0.04–0.10 versus 0.57–1.00 for clamping.The difference is significant at p < 0.01 across all ten seeds, while per-property moment error remains comparable.
  • RQ1: Validity and realism: The constraint layer’s clamp collapses every high-R margin to the feasible boundary, whereas FSG-LTN-GAN reproduces the real margin distribution.The resulting distributional distortion is invisible to standard per-property moment metrics.
  • RQ2: Architecture generality: Across CTGAN, TVAE, and the MLP generator, all methods reach 100% validity, but margin KS is 0.35–0.67 for CTGAN and TVAE clamping versus 0.04–0.10 for FSG-LTN-GAN.The in-loop MLP constraint layer performs worst at 0.57–1.00, showing that the failure transfers across architectures.
  • RQ3: Benchmark generalization: On the six-dataset benchmark, the hybrid wins three tasks, ties three, and never loses, improving on the high-R faults dataset with p=0.002.On low-R datasets it clamps and matches the constraint layer; FSG-all is worse everywhere except faults, making per-constraint selection necessary.
  • RQ4: Predicate grounding: Predicate grounding’s usable-gradient fraction is bounded at Θ(1/R), motivating tests of generator-loss, discriminator-reweighting, and discriminator-feature placements.On Alchemy, their satisfaction fractions are 0.58±0.02, 0.51 ± 0.03, and 0.14 ± 0.08, compared with 0.49 ± 0.03 without a mechanism.

6. Related Work

Related work includes predicate-style neuro-symbolic logic constraints, hard-constraint output layers, and constrained tabular generation. LTNs maximize grounded satisfaction, while related approaches implement logic as differentiable losses.

  • Neuro-symbolic generation and LTNs: LTNs train by maximizing grounded satisfaction, alongside related methods that realize logic as a differentiable loss.The cited approaches include Badreddine et al. (2022), Xu et al. (2018), and Fischer et al. (2019).
  • Neuro-symbolic generation and LTNs: Neuro-symbolic generation methods described here use predicate-style soft constraints.The passage identifies all such methods as predicate-style soft constraints under RQ4.
  • Constrained tabular generation: Hard-constraint output layers guarantee specific constraint fragments, while Stoian et al. (2024) introduces a constraint layer for constrained tabular generation.The hard-constraint output-layer examples are Ahmed et al. (2022), Hoernle et al. (2022), and Giunchiglia and Lukasiewicz (2020).

7. Discussion and Conclusions … Appendix C. Proof of Proposition 1

Function-symbol grounding provides exact validity while preserving the real distribution of constraint margins, whereas predicate grounding and clamping fail for high-resolution constraints. The method is established for linear constraints, with nonlinear and disjunctive feasible sets left for future work.

  • 7. Discussion and Conclusions: Function-symbol grounding recovers margin distributions with exact validity, outperforming the constraint layer on four high-resolution datasets.Predicate grounding cannot learn high-resolution constraints, while clamping collapses margins despite standard metrics appearing satisfactory.
  • 7. Discussion and Conclusions: The charts currently cover only linear constraints, using Fourier-Motzkin elimination to construct admissible intervals.Non-convex feasible sets from nonlinear or disjunctive constraints remain future work; discrete margins are boundary point masses that the hybrid correctly clamps.
  • Appendix A. Extended Related Work: Prior work injects logic through differentiable losses that penalize violating assignments or otherwise encourage constraint satisfaction.Examples include semantic loss, pseudo-semantic extensions, DL2, straight-through estimators, distribution-aware objectives, and bilevel formulations.
  • Appendix A. Extended Related Work: Other approaches guarantee satisfaction by construction for specific logical, differential, or linear constraint classes, including the constraint layer of Stoian et al.The cited families include Semantic Probabilistic Layers, MultiplexNet, coherent hierarchical classifiers, and physics-informed networks.
  • Appendix A. Extended Related Work: Rejection sampling can recover the conditional distribution in principle, but its validity-based acceptance rate collapses at high R and is zero for definitional identities.Validity is 0.05 on Alchemy, Table 5, and Corollary 3 states that an absolutely continuous generator satisfies an identity with probability 0.
  • Appendix B. The Hybrid Procedure: The hybrid procedure reduces linear constraints to per-variable bounds, selects chart or clamp grounding after a brief pre-run, and trains with charted variables valid by construction.Fourier-Motzkin elimination supplies bounds; the pre-run measures interval satisfaction and margin discreteness before grounding choices are made.
  • Appendix C. Proof of Proposition 1: In the proof, Fourier-Motzkin bounds depend only on finalized earlier variables, and satisfiability guarantees non-empty intervals, with interior placement requiring ℓi < ui.If inequalities imply ℓi = ui, the interval is pinched and interior placement is unavailable.

Appendix D. The Resolution Ratio as Condition Number

The resolution ratio R—the data scale relative to margin spread—determines whether predicates and clamps can learn constraint margins. Function-symbol grounding changes coordinates so sub-resolution margins become learnable while validity holds by construction.

  • Resolution-ratio regimes: R compares data scale with margin spread, and high R makes the margin invisible to discriminators even when the ordering remains structurally important.At high R, the feasible margin is tiny relative to the data scale; at low R, it remains a visible fraction of that scale.
  • Resolution-ratio regimes: R ≈9 × 10^4 makes Alchemy’s ordering margin occupy only relative width ∼10^-5 after standardization, beyond a Lipschitz discriminator’s resolution.A free generator matching feature marginals still violates the ordering on about half of samples on average.
  • Resolution-ratio regimes: The predicate’s active gradient band contains Θ(1/R) probability mass, so its learning signal becomes negligible as R grows.The gradient is Θ(1/s) only when the margin is within a width-Θ(σm) band; outside it, the logistic derivative is exponentially suppressed.
  • Clamp failure: Clamping preserves satisfying margins but maps violators to the boundary, producing a boundary atom and a large KS discrepancy from the real margin distribution.The bound is KS ≥ 1/2 − Θ(1/R), and KS approaches 1 as the violating fraction v approaches 1; empirically, the boundary atom appears on every dataset.
  • Function-symbol chart: Function-symbol links form a smooth chart that maps admissible interiors bijectively to unconstrained coordinates, placing margins at learnable scale and recovering valid samples without boundary mass.Dependent coordinates are computed from identities, while the inverse is defined on the relative interior; the resulting coordinate change gives Θ(1) conditioning and preserves realizable margin distributions.

Appendix E. Constraint-Layer Code Corrections and Evaluation Protocol

Appendix E documents corrections to the released constraint-layer code and the evaluation settings needed for genuine 100% validity and fair margin comparisons.

  • Constraint-layer corrections: A latent Fourier–Motzkin bug was corrected because stale terms remained after variable coefficients canceled, as triggered by equalities encoded as two inequalities.The corrected layer was run in double precision with strict-inequality ε set above one unit in the last place; the default ε = 10−12 underflowed and falsely reported 0% validity.

Appendix F. Datasets, Constraints, and Resolution Ratios · Appendix G. Architecture and Training Protocol

The study uses four standardized real high-resolution datasets, computes resolution ratios from raw data before training, and evaluates explicit structural constraints including positivity, an algebraic identity, and thermochemical orderings. All methods share the same generator and discriminator architectures, isolating the constraint mechanism as the experimental difference.

  • Appendix F. Datasets, Constraints, and Resolution Ratios: Appendix F. Datasets, Constraints, and Resolution Ratios: Four real high-resolution datasets support RQ1, RQ2, and RQ4.Table 4 summarizes the datasets and reports N rows, D properties, and the resolution ratio for each evaluated ordering margin.
  • Appendix F. Datasets, Constraints, and Resolution Ratios: Appendix F. Datasets, Constraints, and Resolution Ratios: Each property is standardized to zero mean and unit variance before training.
  • Appendix F. Datasets, Constraints, and Resolution Ratios: Appendix F. Datasets, Constraints, and Resolution Ratios: The resolution ratio R = σ_s/σ_m is computed on raw data before training.Here, R denotes the data’s scale over the margin’s spread.
  • Appendix F. Datasets, Constraints, and Resolution Ratios: Appendix F. Datasets, Constraints, and Resolution Ratios: Alchemy contains 12 molecular properties constrained by positivity of mu, gap, and zpve, plus the identity lumo = homo + gap.The molecular properties follow QM9 naming, and the identity concerns frontier-orbital energies.
  • Appendix F. Datasets, Constraints, and Resolution Ratios: Appendix F. Datasets, Constraints, and Resolution Ratios: Alchemy also imposes the thermochemical orderings U0 < U < H and G < H.U0 and U are internal energies at 0 K and 298 K, while H is enthalpy and G is free energy.
  • Appendix G. Architecture and Training Protocol: Appendix G. Architecture and Training Protocol: All methods use one shared generator architecture, so the constraint mechanism is the only experimental difference.The generator is an MLP with latent dimension 64, three hidden layers of width 256, LeakyReLU(0.2), dropout 0.1, batch normalization on two inner layers, and a linear output map.
  • Appendix G. Architecture and Training Protocol: Appendix G. Architecture and Training Protocol: The discriminator is an MLP with architecture D → 256 → 128 → 1, LeakyReLU(0.2), dropout 0.1, and sigmoid output.Generator weights use Kaiming-uniform initialization.

Appendix H. Full Per-Method Results (RQ1 and RQ2)

Across full RQ1 and RQ2 results, C-DGM and FSG-LTN-GAN achieve exact validity, but only FSG-LTN-GAN recovers the real constraint-margin distribution. Raw architectures often violate constraints, while clamping attains validity by concentrating violating mass at the boundary and can leave margin-KS unchanged.

  • RQ1: The paired Wilcoxon test comparing FSG-LTN-GAN with C-DGM returns p = 0.00195 on every dataset.At n = 10, this is the minimum attainable p-value.
  • RQ1: Only FSG-LTN-GAN recovers the constraint-margin distribution, whereas C-DGM is exactly valid without matching that distribution.Table 5 reports validity as the fraction satisfying all constraints and margin KS against a fixed real reference sample.
  • RQ2: Raw architectures reach below 3% validity on chemistry datasets and 30 to 46% on Transition1x and Taxi.The +CL rows, C-DGM, and FSG-LTN-GAN all reach 100% validity.
  • RQ2: Only FSG-LTN-GAN recovers the constraint-margin distribution across the tested architectures.On tmQM and Taxi, raw and +CL margin-KS entries coincide exactly because clamping maps violating mass to the boundary point ε while the reference margin lies above ε.

Appendix I. The Constraint-Layer Benchmark, Full (RQ3)

The full RQ3 benchmark shows that C-DGM, FSG-all, and the hybrid achieve 100% validity, while the hybrid charts selected constraints and clamps the rest. Its charting behavior varies by dataset, reducing to the constraint layer on heloc and lcld but charting one or two variables on url, news, and wids.

  • Full benchmark: C-DGM, FSG-all, and the hybrid all reach 100% validity, whereas unconstrained validity ranges from 0.005 (wids) to 0.736 (url).Table 7 reports mean ± standard deviation across n = 10 seeds, with margin KS averaged over each dataset’s constraints.
  • Full benchmark: The hybrid charts constraints selected before training and clamps the remainder.It selects constraints when free-generator satisfaction is below 0.9 and the margin is not discrete.
  • Dataset behavior: On heloc and lcld, the hybrid charts nothing and therefore reduces to the constraint layer.The hybrid’s charted count is zero on both datasets.
  • Dataset behavior: On url, news, and wids, the hybrid charts one or two variables as function symbols.The charted count records how many variables the hybrid represents as function symbols.

Appendix J. Additional Analyses

Additional analyses show that function-symbol grounding preserves realistic margins across orderings and generator backbones, while separating validity-by-construction from chart-based discrimination. Manual margin preprocessing and conditional inverse-design experiments further support the chart’s role in recovering realistic constrained quantities.

  • Variable-ordering robustness: Across every dependency-valid ordering, the constraint layer remains near 1.0 margin KS, whereas function-symbol grounding reaches 0.065 with 100% validity.The ordering failure is therefore intrinsic to clamping rather than an elimination-order artifact.
  • Other backbones: CTGAN and TVAE trained in chart coordinates become exactly valid and achieve margin KS 3.6 to 8.6× below their constraint-layer counterparts.The recipe uses unmodified backbones, φ−1-encoded training data, package-default hyperparameters, and decoding through φ; paired Wilcoxon p=0.002 holds for every dataset–backbone pair.
  • Sources of margin-KS gains: Replacing chart-coordinate discrimination with decoded ambient discrimination worsens margin KS on every dataset, although it remains below the clamp’s.The degradation has paired Wilcoxon p=0.002 on each dataset and tracks R, showing that both chart generation and chart discrimination contribute.
  • Margin-form preprocessing: On Taxi, margin-form preprocessing reduces R from 1.1 × 103 to R=1 and lowers margin KS from 0.574 to 0.134 ± 0.020 at exact validity.Duration KS simultaneously falls from 1.000 to 0.118 ± 0.033, demonstrating that manually charting the margin largely preserves its distribution.
  • Conditional inverse design: In conditional inverse design, only the chart is simultaneously valid, on target, and realistic in its margins.The experiment conditions on a target designable property and compares constraint mechanisms while holding the conditional architecture fixed.
  • Threshold sensitivity: Sweeping the hybrid thresholds changes the charted set on 5 to 19 of 25 cells, yet retraining every alternative set changes no conclusion.All configurations retain exact validity, while alternatives alter margin KS by at most the amount reported in the appendix passage.
Loading 2608.21605v1…