Source-linked AI summary
Causal Effect Inference with Deep Latent-Variable Models
Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, Max Welling
TL;DR
The paper studies individual-level causal-effect inference from observational data when confounders are hidden and observed only through noisy proxies. It uses a VAE-based latent-variable model to estimate the confounder space and causal effects jointly, showing greater robustness to noisy proxies and competitive performance on established benchmarks.
Problem
The problem is estimating individual-level causal effects from observational data when important confounders are unmeasured but noisy proxy variables may be available.
Method
CEVAE uses a neural-network latent-variable model based on VAEs to infer hidden confounders and approximately recover their relationships with proxies, treatment, and outcome.
Results
The method is more robust to hidden confounding with noisy proxies and competitive with state-of-the-art methods on benchmark datasets.
Takeaways & Limitations
Latent-variable modeling can support causal-effect estimation in surrogate-rich settings where the exact structure of hidden confounders is unknown.
Takeaways & Limitations
The approach assumes the latent joint distribution can be approximately recovered from observed variables, and VAEs lack guarantees that optimization reaches the true model.
Abstract
from arXiv · showhide
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers. The most important aspect of inferring causal effects from observational data is the handling of confounders, factors that affect both an intervention and its outcome. A carefully designed observational study attempts to measure all important confounders. However, even if one does not have direct access to all confounders, there may exist noisy and uncertain measurement of proxies for confounders. We build on recent advances in latent variable modeling to simultaneously estimate the unknown latent space summarizing the confounders and the causal effect. Our method is based on Variational Autoencoders (VAE) which follow the causal structure of inference with proxies. We show our method is significantly more robust than existing methods, and matches the state-of-the-art on previous benchmarks focused on individual treatment effects.
1 Introduction
The paper addresses individual-level causal-effect inference from observational data when important confounders are hidden but noisy proxies are available. It proposes learning the latent confounders and their causal effects jointly with a VAE, reporting greater robustness to noisy proxies and competitive benchmark performance.
- Motivation: Hidden confounders can make causal effects impossible to estimate without additional assumptions, even when observational data are abundant.Confounders affect both treatment and outcome; socioeconomic status is given as an example involving medication access and health.
- Motivation: Proxy variables such as zip code and job type can provide noisy information about otherwise unmeasured confounders.The paper describes big data as a source of many potential proxies for unmeasured confounders.
- Challenges: Treating proxies as ordinary confounders can induce bias, so their appropriate use depends on relationships among hidden confounders, proxies, treatment, and outcome.Existing proxy-based methods provide guarantees under specific structural conditions.
- Approach: The proposed approach estimates a latent-variable model to discover hidden confounders and infer how they affect treatment and outcome when the confounder structure is uncertain.The method targets surrogate-rich settings with many proxies and uses approximate maximum-likelihood methods.
- Approach: The paper uses VAEs because they offer computationally efficient, flexible latent modeling with weaker data-generating assumptions, despite limited theory for identifying the true model.The authors also report that the method is more robust to noisy proxies and competitive on existing causal-inference benchmarks.
2 Identification of causal effect
The identification strategy assumes the latent confounder model can be approximately recovered from observed proxies, treatment, and outcome. Under that assumption and the paper’s causal model, recovering the joint distribution identifies individual and average treatment effects through back-door adjustment.
- Assumptions: The method assumes that p(Z, X, t, y) can be approximately recovered from observations of (X, t, y).Recovery is possible only when the hidden confounder is related to observed variables, with examples including multiple independent views and noisy-or measurements.
- Assumptions: The identification result depends on the assumed causal model and recoverability of the latent joint distribution, not merely on observing proxies.The paper explicitly notes that recovery is impossible when the hidden confounder has no relation to observed variables.
- Estimands: The paper targets both individual treatment effects and average treatment effects, defining ITE as the difference between potential outcomes under treatment and control.ATE is defined as the expectation of ITE(x).
- Identification: Recovering p(Z, X, t, y) identifies the ITE under the stated causal model, and ATE is identified whenever ITE is identified.The proof invokes Pearl’s back-door adjustment formula and identifies the required quantities from the recovered joint distribution.
- Identification: The resulting estimator integrates p(y|X, t = 1, Z) with p(Z|X) over the latent confounder.The derivation uses do-calculus and the model’s conditional independence of y and X given Z.
3 Causal effect variational autoencoder
CEVAE uses variational autoencoders to model latent confounders and infer causal effects from observed covariates, treatments, and outcomes. Its generative and inference networks combine latent-variable modeling with treatment-specific outcome pathways and auxiliary predictions for new subjects.
- CEVAE uses variational autoencoders to infer nonlinear relationships among observed covariates, latent confounders, treatments, and outcomes.The approach approximately recovers the joint latent-variable model p(Z, X, t, y).
- The generative model conditions covariate, treatment, and outcome distributions on the latent confounder z.Outcomes use treatment-specific neural-network functions, with Gaussian modeling for continuous outcomes and Bernoulli modeling for discrete outcomes.
- Because nonlinear neural networks make exact inference intractable, CEVAE marginalizes over z using variational inference and inference networks.The inference networks output parameters of a fixed-form posterior approximation over the latent variables.
- The posterior approximation conditions on observed covariates, treatment, and outcome, using treatment-specific branches after a shared representation.The inference network follows a TARnet-inspired architecture with neural networks parameterized by variational parameters.
- Auxiliary distributions predict treatment assignments and outcomes for out-of-sample subjects when treatment and outcome are unavailable during latent-confounder inference.The outcome distribution is Gaussian for continuous outcomes and Bernoulli for discrete outcomes, and both auxiliary terms are added to the variational lower bound.
4 Experiments
Experiments evaluate CEVAE on benchmark, synthetic, and Twins datasets using causal-effect and counterfactual tasks. CEVAE is competitive on established benchmarks and more robust to hidden confounding and noisy proxies, especially when proxy information is weak.
- Benchmark datasets: CEVAE performs comparably to the Balancing Neural Network on IHDP, using absolute error for individual and average treatment effects.The IHDP evaluation reports individual-effect error and absolute ATE error.
- Benchmark datasets: CEVAE is competitive with the state of the art on Jobs and achieves the best out-of-sample estimate of ATT overall.Jobs results average performance over 10 train/validation/test splits and report policy risk alongside ATT error.
- Synthetic experiment on toy data: In the synthetic hidden-confounding experiment, LR1, LR2, and TARnet do not reduce ATE error as sample size increases, whereas CEVAE achieves substantially lower error.CEVAE uses either a binary latent variable or a 5-dimensional continuous latent variable across sample sizes from 1,000 to 30,000.
- Synthetic experiment on toy data: CEVAE with the correctly specified binary latent model performs well at small sample sizes, while the misspecified continuous model needs more samples to approximate the binary latent variable.The comparison tests robustness to latent-model misspecification.
- Binary treatment outcome on Twins: The Twins experiment hides one twin to simulate observational treatment assignment and constructs noisy proxies by independently flipping replicated gestation-category indicators.Flip probabilities range from 0.05 to 0.5, and three independent views are used to support latent-feature recovery.
- Binary treatment outcome on Twins: On Twins, CEVAE is more robust than LR1, LR2, and TARnet as proxy noise increases, including when direct proxies are completely useless at noise level 0.5.The study evaluates counterfactual mortality and average treatment-effect inference while varying hidden-layer counts and proxy noise.
5 Conclusion
The paper connects causal inference with proxy variables to latent-variable modeling through CEVAE, a neural model for estimating individual and population causal effects. Experiments report state-of-the-art competitiveness on benchmarks and greater robustness to hidden confounding in synthetic and modified real data.
- Conclusion: CEVAE connects causal inference with proxy variables to latent-variable models and estimates both individual and population causal effects.The paper describes CEVAE as a first attempt to tie these ideas together.
- Conclusion: CEVAE is competitive with the state of the art on benchmark datasets and more robust to hidden confounding on toy and modified real datasets, including Twins.The conclusion identifies robustness as the central empirical outcome.
A. Simple example where one should not adjust for proxy variables
A binary example shows that adjusting for a noisy proxy X can give the wrong interventional outcome, even when the resulting treatment-effect contrast happens to be correct under symmetric conditions.
- Setup: The example models a binary latent confounder Z, proxy X, treatment t, and deterministic outcome y = t ⊕ Z.The latent confounder is balanced, while X and treatment are conditionally related to Z through parameters ρx and ρt.
- Incorrect adjustment: The proposed calculation estimates p(y = 1|do(t = 1)) by treating X as the only confounder and applying covariate adjustment.The calculation averages treatment-conditioned outcome probabilities across X rather than adjusting for the latent variable Z.
- Special cases: The adjustment is valid exactly when treatment is randomized or X is a perfect proxy for Z or 1 − Z.These conditions correspond to ρt = 0.5 or ρx ∈ {0, 1}, respectively.
- Why adjustment fails: The crucial error is that conditioning on X does not equate observational and interventional outcome distributions, unlike conditioning on the true confounder Z.Specifically, p(y = 1|do(t = 1), x) ≠ p(y = 1|t = 1, x), whereas equality holds when conditioning on Z.
- Effect contrast: Under symmetric proxy distributions, the incorrect and correct treatment-effect contrasts are equal despite incorrect interventional outcome estimates.Breaking that symmetry makes both interventional estimates and their treatment-effect contrast incorrect.