PaperlayerView exact revision on arXiv

Source-linked AI summary

Probabilistic Counterfactual Inference for Discrete Outcomes in Gaussian-Process Causal Models

Juliette Sinnott, Amir-Hossein Karimi, Mohammad Kohandel

arXiv:2610.08689v1cs.LG

TL;DR

Existing GP-SCM counterfactual methods focus on continuous outcomes, leaving discrete child variables with continuous parents outside their scope. This paper pairs GP predictors with type-specific noise mechanisms for discrete outcomes. On synthetic SCMs, the coupling choice can change counterfactual accuracy even when observational and interventional quantities agree.

  • Problem

    Existing GP-SCM counterfactual methods focus on continuous outcomes, leaving discrete child variables with continuous parents outside their scope.

    Source: §1 · p. 2, §1.1 · p. 2

  • Method

    The framework pairs GP predictors with explicit exogenous noise mechanisms, using uniform thresholds for binary, Gumbel-max for nominal, and latent Gaussian cutpoints for ordinal outcomes.

    Source: §1 · p. 2, §2.2 · p. 4

  • Results

    The coupling choice can change counterfactual accuracy despite agreement on observational and interventional quantities, with an ordinal-versus-categorical mismatch producing roughly 0.25 counterfactual distance that does not shrink with more data.

    Source: §5 · p. 9, §4.2 · p. 9, §6 · p. 10

  • Takeaways & Limitations

    The coupling is a structural modeling choice that cannot be relied on to be recovered from model fit and should reflect the variable’s type.

    Source: §5 · pp. 9–10

    No mechanism that abducts and reuses noise can be optimal simultaneously for every pair of interventional distributions, so neither proposition is uniformly best.

    Source: §5 · p. 10

Abstract

from arXiv · show

Counterfactual inference in Gaussian-process structural causal models (GP-SCMs) has been developed primarily for continuous endogenous variables, limiting applicability to causal graphs that contain discrete child nodes with continuous parents. We introduce a unified probabilistic framework for counterfactual inference with heterogeneous variable types by pairing GP predictors with explicit exogenous noise mechanisms. For discrete outcomes, we derive exact conditional noise-abduction procedures using a uniform threshold for binary variables, a Gumbel-max race for nominal categories, and a latent Gaussian cut-point model for ordinal ones. In each case, we propagate abducted noise through interventions while accounting for posterior uncertainty in the GP latent functions, and prove that the resulting mechanisms reproduce the fitted model's observational and interventional distributions. On synthetic SCMs with known ground-truth counterfactuals, we evaluate estimation accuracy, consistency, and robustness to coupling misspecification. A key finding is that applying a categorical coupling to ordinal data inflates counterfactual error roughly threefold even when observational fit remains comparable, and that this error does not diminish with more data. As the training set grows, the fitted structural equation converges to the truth while the counterfactual error flattens onto a floor. In the reverse direction, forcing a false order onto nominal data instead degrades the fitted equation itself. The choice of coupling must therefore be justified on structural grounds rather than read off the fit.

1 Introduction

The paper extends GP-SCM counterfactual inference beyond real-valued child variables to binary, categorical, and ordinal outcomes. It assumes the causal graph is given and focuses on fitting structural equations and selecting noise mechanisms for counterfactuals.

1 Introduction

  • Existing GP-SCM counterfactual methods address real-valued relationships, leaving discrete child variables with continuous parents outside their scope.The paper assumes graph discovery is complete and focuses on fitting parent-child structural equations.Source: §1 · pp. 1–2, §1.1 · p. 2

1.1 Counterfactuals

  • Counterfactuals use a factual instance to infer what would have changed under an intervention, through abduction, action, and prediction.Abduction isolates exogenous noise; action modifies the model; prediction reuses the noise under the intervention.Source: §1.1 · p. 2

1.2 Existing Approaches

  • The paper takes the causal graph as given and addresses equation fitting with GP predictors paired with exact noise-abduction identities.Prior approaches also address graph discovery or coupling a fixed interventional law, rather than this combination.Source: §1.2 · pp. 2–3

2 Preliminaries

Discrete counterfactuals require a coupling because factual and counterfactual class probabilities do not determine their joint law. The paper realizes couplings by abducting exogenous noise from the factual observation and reusing it after intervention.

2.1 Introducing Gumbel Noise

  • The Gumbel distribution provides an explicit exogenous noise for categorical outcomes, enabling exact rather than relaxed noise abduction.The paper uses the non-relaxed Gumbel form, unlike the differentiable Gumbel-softmax/Concrete construction [Jang et al., 2017, Maddison et al., 2017].Source: §2.1 · p. 3

2.2 Coupling and Abduction

  • Factual and counterfactual class probabilities alone do not determine their joint law; reusing abducted exogenous noise specifies a coupling.The noise is inferred from the factual outcome and held fixed while the counterfactual parents change.Source: §2.2 · p. 3
  • For binary outcomes, a uniform scalar threshold gives the unique coupling consistent with Bernoulli marginals under the stated monotonicity requirement.At C = 2, the categorical and ordinal couplings coincide exactly.Source: §2.2 · pp. 3–4
  • For unordered categories with C ≥3, the Gumbel-max coupling abducts one independent Gumbel perturbation per class and reuses them in the counterfactual race.This treats every pair of classes symmetrically and is characterized by counterfactual stability [Oberst and Sontag, 2019].Source: §2.2 · p. 4
  • For ordered categories, a shared Gaussian noise passed through monotone thresholds gives the comonotone coupling and respects class order.The categorical and ordinal constructions therefore use different exogenous noise objects.Source: §2.2 · p. 4

3 Results

The results section develops categorical and ordinal GP counterfactual mechanisms within a shared compatibility, abduction, action, and prediction framework. The ordinal construction uses a Gaussian latent with cutpoints, while the categorical construction uses Gumbel noise.

3.1 Categorical

  • The categorical proposition pairs class-probability scores with independent per-class Gumbel noise and marginalizes counterfactual predictions over joint GP latent draws.Its class probabilities are formed by renormalizing independent binary Laplace GP classifiers; the construction also states compatibility with the fitted class probabilities.Source: §3.1 · pp. 4–5
  • Under the categorical proposition, intervening at the factual parent value recovers the observed factual outcome with certainty.For C = 2, the categorical construction has a closed form with uniform noise derived from the difference of two Gumbels.Source: §3.1 · p. 5

3.2 Ordinal

  • The ordinal proposition models a latent GP score plus Gaussian noise, thresholded by ordered cutpoints that may be learned jointly with the latent function.When cutpoints are known, the construction reduces to a single-latent GP-ordinal model with fixed cutpoints.Source: §3.2 · pp. 5–6
  • For fixed cutpoints, the ordinal construction uses the same Gaussian noise through monotone thresholds to form the comonotone coupling.At C = 2, the two propositions select the same coupling, though their link functions differ; the agreement does not extend to C ≥3.Source: §3.2 · pp. 6–7
  • After marginalizing over the joint posterior of the latent functions and cutpoints, the estimator is a mixture of comonotone couplings that need not itself be comonotone.The comonotonicity identification applies at a fixed cutpoint setting.Source: §3.2 · p. 7

4 Experiments

Experiments compare categorical and ordinal counterfactual mechanisms on four synthetic SCMs, testing fit, counterfactual accuracy, and coupling misspecification. Correcting ordinal mismatches improves counterfactual accuracy, but more data does not resolve a wrong coupling.

4.1 Setup

  • The experiments fit GP classifiers to synthetic data from known SCMs and evaluate eight factual/intervention scenarios across four structures.Each experiment uses n = 500 training samples; reported TVint and TVcf means and standard deviations aggregate 10 seeds.Source: §4.1 · pp. 7–8
  • For the Interaction and Chain SCMs, ordinal ground truth uses cutpoints b1 = −0.5 and b2 = 0.5; fitted estimators learn the score and either infer or receive those cutpoints.Ground-truth counterfactuals are computed in closed form from the known score and cutpoints.Source: §4.1 · p. 7

4.2 Findings

  • Ordinal refitting recovers most of the roughly 0.25 TVcf error by factors of 3.4 and 6.1, while TVint moves far less.Categorical fits on ordinal data retain TVint at the same order as correctly specified fits.Source: §4.2 · pp. 8–9, Table 1 · p. 8
  • Forcing an ordinal fit on nominal data raises Single-parent TVint sixfold, from 0.030 to 0.184.The fitted equation degrades because the imposed monotone score is harder to learn for classes with no true order.Source: §4.2 · p. 9
  • On the Binary SCM, switching to the matching probit link barely changes TVcf, within one standard deviation across seeds.The two couplings coincide in this binary case.Source: §4.2 · p. 9
  • Learned cutpoints are indistinguishable within seed noise from using the true cutpoints on the ordinal models.The estimators learn the score with a GP classifier and either infer or receive the cutpoints.Source: §4.1 · p. 7, §4.2 · p. 9
  • As n grows from 100 to 1000, TVcf under a wrong coupling flattens near 0.21–0.24 even as TVint improves.Where the coupling is correct, TVint and TVcf fall together.Source: §4.2 · p. 9
  • Abduction drives accuracy: skipping it costs an order of magnitude on Single-parent, while an oracle recovers only about a tenth of Interaction’s misspecification error.Posterior averaging contributes almost nothing to the point estimate.Source: §4.2 · p. 9

5 Discussion

The coupling is a modeling choice above the fitted model: observational and interventional agreement cannot establish counterfactual correctness. The authors argue variable type should constrain that choice, while noting limits on validation and scope.

5 Discussion

  • Propositions 1 and 2 agree on observational and interventional quantities but can disagree on counterfactuals because they use different couplings.The coupling is a modeling choice made above the fitted model, not a parameter the fit can recover.Source: §5 · p. 9
  • Variable type should constrain admissible couplings, since a good observational and interventional fit does not imply a correct counterfactual.The authors argue this structural constraint is more consequential than tuning within one coupling family.Source: §5 · p. 9
  • Validation uses synthetic SCMs with known mechanisms, while real-data mechanisms are not directly observable.The coupling choice requires knowing whether categories carry an order, not the structural equation itself.Source: §5 · pp. 9–10
  • No single noise-reuse mechanism can be optimal for every pair of interventional distributions, and shared latent functions across correlated ordinal nodes remain future work.The authors note that Proposition 2 jointly infers cutpoints with the latent function and recovers the fixed-cutpoint case exactly.Source: §5 · p. 10

6 Conclusion

The paper presents exact noise-abduction procedures for heterogeneous outcomes in GP-SCMs and shows that coupling choice can dominate counterfactual accuracy. Its experiments use synthetic SCMs with known ground truth and a specified evaluation protocol.

6 Conclusion

  • Couplings that agree on observational and interventional quantities can differ in counterfactual total variation distance by roughly 0.25, with error that does not shrink as data grows.Mistaking ordinal data for categorical can hide the gap from goodness-of-fit, whereas forcing order on categorical data degrades the fitted equation.Source: §6 · p. 10
  • The framework derives exact noise-abduction procedures for binary, categorical, and ordinal outcomes, each reproducing the fitted model’s observational and interventional distributions.The authors conclude that variable type, rather than a downstream fitting criterion, should guide coupling choice.Source: §6 · p. 10

Reproducibility statement

  • Theoretical claims are proved in Appendix A.6, and experiments use synthetic SCMs with explicit generating equations and closed-form ground-truth counterfactuals.The appendix specifies sample sizes, seeds, and Monte Carlo budgets and includes a convergence check.Source: Reproducibility statement · p. 10

A.1 Definitions

The appendix defines Gaussian-process priors through their finite-dimensional Gaussian distributions and mean and covariance functions. It then introduces the posterior predictive distribution by conditioning on training observations.

A.1 Definitions

  • A Gaussian process is a collection of random variables whose every finite subset has a joint Gaussian distribution.The prior is specified by its mean function and covariance function; the mean is often set to zero for convenience.Source: §A.1 · p. 12
  • The GP posterior is obtained by conditioning the joint prior over training targets and test-function values on the observations.The setup defines training observations as noisy function values and gives the joint prior before conditioning.Source: §A.1 · p. 12
  • For one test point, the posterior predictive expressions reduce to a single-point form using its covariance with the training inputs.The passage identifies k_* as the covariance between the training inputs and the test point.Source: §A.1 · p. 12

A.2 Worked Numerical Example

The worked example abducts categorical noise from a factual class, carries it through an intervention on the parent, and predicts a counterfactual class. A separate fitted-GP example compares estimates with known ground truth.

A.2 Worked Numerical Example

  • The categorical example holds the fitted latent scores fixed at factual and counterfactual parent values, illustrating the compatibility case of Proposition 1.The setup intervenes on the parent, and both score vectors come from the already-fitted classifier.Source: §A.2 · p. 12
  • From factual class 1, the worked example abducts a noise vector and holds it fixed while replacing the parent’s factual score with its post-intervention score.The factual probabilities are (0.7, 0.2, 0.1); after moving the parent from 0.5 to 2.5, the probabilities are (0.2, 0.3, 0.5).Source: §A.2 · p. 13
  • The abducted noise makes class 3 the counterfactual outcome even though class 1 was factual and class 3 was least likely factually.The example also reports consistency when intervening at the factual parent value.Source: §A.2 · p. 13
  • The fitted GP estimate and exact ground truth agree that the counterfactual class is 3, with total variation distance 0.022.The example uses Monte Carlo over 4,000 draws of the joint latent and abducted noise.Source: §A.2 · pp. 13–14
  • Under the trivial intervention, the fitted estimator places probability 1.0 on the factual class, confirming the stated consistency check empirically.The synthetic example uses one continuous parent and a three-class categorical child.Source: §A.2 · pp. 13–14

A.3 Experimental Protocol

The protocol measures variability across the full fitting and counterfactual pipeline and checks that Monte Carlo estimates are sufficiently converged.

A.3 Experimental Protocol

  • At n = 500, each Table 1 value is a mean ± standard deviation over 10 seeds that redraw data, refit every GP, and rerun all eight scenarios.The reported spread therefore reflects variability across data, kernel fitting, and Monte Carlo draws together.Source: §A.3 · p. 14
  • Each counterfactual probability averages 4,000 joint-latent and abducted-noise draws.The convergence check varies the draw budget from 250 to 16,000 on the Interaction SCM.Source: §A.3 · p. 14
  • Beyond 2,000 draws, TVcf changes by at most 0.0015, far below the seed-to-seed standard deviation of 0.100.The first budget increase, from 250 to 500, moves TVcf by +0.012.Source: §A.3 · p. 14

A.4 Additional Ablations

The ablations show that counterfactual error can persist under coupling misspecification despite improving fit, while noise abduction is central to the estimator's accuracy.

A.4 Additional Ablations

  • With misspecified coupling, TVint falls 61% (0.144 → 0.056) on Interaction and 74% (0.091 → 0.024) on Chain, while TVcf plateaus at 0.207 and 0.243.Table 2 reports TVcf against training-set size and marks rows whose coupling does not match the true mechanism with †.Source: §A.4 · p. 14, Table 2 · p. 14
  • TVcf/TVint rises from 1.8 to 3.7 on Interaction and 3.1 to 10.2 on the Chain mediator under misspecification, but stays near 1 when specified correctly.The coupling choice must be made on structural grounds; observational fit does not reveal it at any sample size.Source: §A.4 · p. 14
  • On the Single-parent SCM, full-estimator TVcf is 0.031, versus 0.295 when resampling independently and 0.300 when conditioning the latent without abducting noise.Table 3 compares five estimators on the same data; abduction, coupling, and posterior averaging are varied separately.Source: §A.4 · pp. 14–15, Table 3 · p. 15
  • Replacing posterior draws with the posterior-mean latent leaves TVcf unchanged to three decimals on both SCMs, but discards posterior spread needed for credible intervals.The comparison indicates marginalization contributes calibration rather than point-estimate accuracy at these sample sizes.Source: §A.4 · p. 15
  • With true class probabilities, oracle TVcf on Interaction falls only from 0.231 to 0.205, leaving roughly nine tenths of misspecification error attributable to the coupling.On the correctly coupled Single-parent SCM, oracle error is 0.005±0.001, the Monte Carlo floor at 4,000 draws.Source: §A.4 · p. 15, Table 3 · p. 15

A.5 Algorithms

The algorithms share a fit-and-sample structure, then use type-specific noise abduction and propagation through interventions, including for multi-node SCMs.

A.5 Algorithms

  • Both algorithms fit the model, compute joint posterior moments, then repeat abduction, action, and prediction in a Monte Carlo loop over latent uncertainty.They differ in the exogenous noise and how it is abducted.Source: §A.5 · p. 15
  • Fixing the cutpoints to a point mass recovers the fixed-cutpoint algorithm and reduces training to a single Laplace fit.This is stated as the fixed-cutpoint special case.Source: §A.5 · p. 15
  • Categorical abduction uses logaddexp, which is equivalent in exact arithmetic to the alternative expression but avoids cancellation and overflow.The categorical procedure is identified as Algorithm 1 for a categorical node.Source: §A.5 · p. 16
  • For categorical nodes, Algorithm 1 fits one-versus-rest binary Laplace GP classifiers and uses a single joint posterior draw across factual and counterfactual inputs within each class.The classes are independent; drawing the two input values marginally would discard covariance and break the stated consistency result.Source: §A.5 · p. 16
  • For ordinal nodes, Algorithm 2 samples cutpoints from their approximated posterior, fits the latent GP, and abducts the factual class's Gaussian noise within its interval.The noise is integrated out in closed form during action and prediction.Source: §A.5 · p. 16
  • For multi-node SCMs, nodes are processed in topological order, with each sampled counterfactual parent becoming the next node's intervened input.The naive form loses precision for classes that were probable factually but lost the arg max.Source: §A.5 · p. 17

A.6 Proofs

The proofs establish exact counterfactual consistency for the discrete mechanisms, covering multiclass, binary, and ordinal models. They also clarify the role of approximation methods and account for uncertainty where specified.

A.6.1 Laplace machinery for the multiclass GPC

  • The multiclass Laplace lemma establishes a strictly concave posterior objective with a unique MAP and derives its Gaussian latent-posterior approximation and predictive moments.The GP prior removes softmax shift degeneracy and makes the MAP well defined.Source: §A.6.1 · pp. 17–18
  • One-versus-rest Laplace fits change probability estimation, not counterfactual construction, but require coupling on log p; at C = 2, the joint and binary approximations coincide.Only the accuracy of p is affected, as measured by the paper’s TVint column.Source: §A.6.1 · p. 18, §A.6.2 · p. 19

A.6.2 Proof of Proposition 1, Corollary 1 and Corollary 2

  • The Gumbel-max mechanism coupled to log probabilities reproduces the fitted model’s observational and interventional distributions for any probability vector.The categorical counterfactual uses the same joint posterior draw of the latent function at factual and intervention inputs; independent draws break the shared-latent construction.Source: §A.6.2 · pp. 19–20
  • For binary outcomes, the uniform-threshold mechanism yields exact abduction by restricting the uniform noise to the interval consistent with the factual outcome, and counterfactual probabilities are monotone in the intervention probability.When factual and counterfactual probabilities are equal, the mechanism returns the factual outcome exactly.Source: §A.6.2 · p. 21

A.6.3 The probit approximation

  • The probit convolution gives a closed form for the interventional class probability, but not for the binary counterfactual probability in equation 11.That counterfactual expression is nonlinear in the factual and counterfactual probabilities, so separate probit approximations do not yield its posterior expectation.Source: §A.6.3 · pp. 21–22
  • The probit approximation is generally biased by Jensen’s inequality and breaks exact consistency, so the proof retains Monte Carlo; reparameterized draws still permit pathwise gradients.The approximation no longer forces pCF = pF pathwise.Source: §A.6.3 · p. 22

A.6.4 Proof of the ordinal-GP counterfactual proposition

  • For the experiments’ low-dimensional cutpoints (C ≤3), the exact cutpoint-posterior Hessian is computed numerically, with central finite differences incurring O(ϵ2) truncation error.This avoids symbolically deriving and implementing the dependence of the fitted latent mode on cutpoints.Source: §A.6.4 · p. 23
  • The ordinal probit mechanism reproduces the fitted model’s class probabilities exactly, both observationally and under intervention, including after averaging over the joint posterior of latent functions and cutpoints.Compatibility follows from the standard-normal noise and ordinal cutpoint intervals; posterior averaging preserves the fitted law by the tower rule.Source: §A.6.4 · p. 23
  • Given cutpoints, factual ordinal abduction yields a truncated standard-normal noise posterior; uncertainty in cutpoints makes the overall noise posterior a mixture of truncated normals.Counterfactual prediction reuses that abducted noise under intervention and averages interval-overlap probabilities over the joint latent-function and cutpoint posterior.Source: §A.6.4 · pp. 23–24
  • The nested ordinal counterfactual Monte Carlo estimator is unbiased and converges almost surely by successive averaging over latent-function and cutpoint posterior draws.Each inner contribution is bounded, and the outer average is likewise over bounded quantities.Source: §A.6.4 · p. 24
  • Under the factual intervention, the ordinal counterfactual distribution is exactly a point mass on the observed class for every cutpoint and posterior draw.The proof uses the shared factual and counterfactual noise and the fact that the class intervals partition the real line.Source: §A.6.4 · p. 24
Loading 2610.08689v1…