PaperlayerView exact revision on arXiv

Source-linked AI summary

The Blessings of Multiple Causes

Yixin Wang, David M. Blei

arXiv:1805.06826v3stat.MLcs.LGstat.ME

TL;DR

Observational causal inference commonly assumes that all confounders are observed, although ignorability is untestable. The paper proposes the deconfounder, which models dependencies among multiple causes to infer substitute confounders and then performs causal inference. Its theory and studies support causal estimation under weaker assumptions, while identifying limitations when the assignment model fits poorly and when single-cause confounders remain.

  • Problem

    Observational causal inference relies on ignorability, the untestable assumption that no relevant unobserved confounders remain.

  • Method

    The deconfounder fits a latent-variable model of the assignment mechanism, infers a latent variable for each individual, and uses it as a substitute confounder.

  • Results

    The deconfounder provides causal estimates closer to truth than ignoring confounding in the studied smoking, GWAS, and movie settings, with predictive checks assessing assignment-model fit.

  • Takeaways & Limitations

    Multiple causes can make observational causal inference possible under weaker assumptions than classical causal inference requires.

  • Takeaways & Limitations

    The approach remains challenged by spatially generated SNP assignments, and assumes there are no unobserved single-cause confounders.

Abstract

from arXiv · show

Causal inference from observational data often assumes "ignorability," that all confounders are observed. This assumption is standard yet untestable. However, many scientific studies involve multiple causes, different variables whose effects are simultaneously of interest. We propose the deconfounder, an algorithm that combines unsupervised machine learning and predictive model checking to perform causal inference in multiple-cause settings. The deconfounder infers a latent variable as a substitute for unobserved confounders and then uses that substitute to perform causal inference. We develop theory for the deconfounder, and show that it requires weaker assumptions than classical causal inference. We analyze its performance in three types of studies: semi-simulated data around smoking and lung cancer, semi-simulated data around genome-wide association studies, and a real dataset about actors and movie revenue. The deconfounder provides a checkable approach to estimating closer-to-truth causal effects.

1 Introduction

The paper frames multiple causal inference as a setting where many causes are studied simultaneously, and argues that this multiplicity can support causal inference under weaker assumptions than classical observational approaches. It introduces the deconfounder, which models dependencies among observed causes to construct a substitute for unobserved confounders.

  • Motivation: Multiple causal inference studies several possible causes whose effects contribute to a shared outcome, unlike classical inference focused on one cause.Examples include actors and movie revenue, genes and traits, neurons and behavior, social programs and outcomes, and treatments and disease progression.
  • Motivation: Unobserved confounders can bias causal estimates because they affect both assigned causes and outcomes, as movie genre affects cast selection and revenue.The resulting dependence makes E[Yi(a)| Ai = a] differ from E[Yi(a)].
  • Classical causal inference: Classical adjustment requires identifying and controlling for confounders, but assuming that no other confounders exist is uncheckable.The paper characterizes this assumption as a leap of faith in many applied observational studies.
  • The deconfounder: The deconfounder fits a factor model of assigned causes, infers per-unit latent variables, and uses them as substitutes for unobserved confounders.The factor model is intended to render assigned causes conditionally independent, with predictive checks assessing whether it captures the observed data.
  • The deconfounder: The method uses patterns of causes appearing together as indirect evidence about confounding, replacing an uncheckable confounder search with a checkable modeling goal.The paper develops this approach within the potential outcomes framework rather than Pearl’s causal-graph framework.

2 Multiple causal inference with the deconfounder

The deconfounder uses a latent factor model of multiple causes and predictive checks to replace an uncheckable search for multi-cause confounders with a testable modeling task. Under SUTVA, overlap, and single ignorability, it provides unbiased causal estimates, while trading reduced confounding bias for higher variance.

  • Ignorability is untestable because observational data reveal only one value of each potential outcome function.
  • The deconfounder fits a latent assignment model, infers a per-individual variable, and conditions causal inference on it as a substitute confounder.The assignment model must capture the population distribution of the assigned causes.
  • A well-fitted factor model captures multiple-cause confounders because those confounders induce dependence among observed causes.This shifts the goal from imagining possible confounders to modeling dependence in the assigned causes.
  • Single ignorability requires marginal independence between each individual cause and the potential outcome, weaker than classical joint ignorability.It allows unobserved multi-cause confounders to be captured while retaining the assumption that unobserved single-cause confounders are absent.
  • Under SUTVA, single ignorability, and overlap, the deconfounder provides an unbiased estimate of the average causal effect.
  • Predictive checks assess whether the assignment model reproduces observed assignments; scores above 0.1 are considered satisfactory, with an ideal score around 0.5.The reported assignment-model predictive score is larger than 0.1.
  • In a GWAS simulation, the deconfounder produces closer-to-truth causal estimates, measured by RMSE of the estimated causal coefficients.
  • The deconfounder trades confounding bias for estimation variance because it allocates information to estimating a substitute confounder before estimating causal effects.When full ignorability holds, both approaches are unbiased but the deconfounder has higher variance; under only single ignorability, classical estimates are biased.

3 Empirical studies

Across smoking, GWAS, and movie data, predictive checks helped identify assignment models whose substitute confounders supported closer-to-truth causal estimates. Performance improved with better factor models, but depended on observing many causes and could trade lower bias for higher variance.

  • Two causes: Smoking and medical expenses: Conditioning on the reconstructed causes produced better outcome-model predictions than conditioning on the substitute confounder alone.The predictive scores were 0.05 for conditioning on z-hat and 0.18 for conditioning on reconstructed causes a(z-hat).
  • Two causes: Smoking and medical expenses: A quadratic assignment model passed predictive checking and produced less biased causal estimates, whereas the mismatched linear model yielded biased estimates.The one-dimensional quadratic substitute remained biased, but the outcome check revealed this issue.
  • Two causes: Smoking and medical expenses: The smallest quadratic factor model passing the check achieved the best mean squared error, while higher-capacity models generally increased variance.Models passing the check produced similar bias, but variance grew with model capacity.
  • Two causes: Smoking and medical expenses: Adding observed covariates preserved unbiasedness or bias reduction but inflated variance, supporting their use mainly when single-cause confounding is plausible.The reported covariates included gender, race, seat belt usage, education, and age of starting to smoke.
  • Many causes: Genome-wide association studies: Across GWAS outcomes, PPCA, PF, LFA, LMMs, and DEFs produced lower RMSEs than regressions that ignored confounding, while DEFs almost always performed best.Predictive checks also revealed when factor-model misspecification would harm downstream causal estimates, as with GMMs.
  • Many causes: Genome-wide association studies: The deconfounder remained challenged by spatially generated SNPs and partially observed causes, although it still improved over ignoring confounding in the spatial setting.With fewer observed causes, single strong ignorability became more likely to fail and RMSE ratios moved toward one.
  • Summary: The empirical studies support checking both assignment and outcome models and using factor models such as DEF to obtain closer-to-truth causal estimates.The paper’s summary emphasizes that the deconfounder performs especially well when many assigned causes are observed.
  • Case study: Actors and movie earnings: In the movie case study, GMM failed predictive checking, whereas PPCA, PF, and DEF passed and the deconfounder predicted better than no control and observed-covariate conditioning.The reported predictive scores were below 0.01 for GMM and 0.12, 0.14, and 0.15 for PPCA, PF, and DEF.

4 Theory

The theory connects factor models to ignorability and establishes when the deconfounder identifies causal effects in multiple-cause settings. It also characterizes the substitute confounder and specifies assumptions and uncertainty handling when identification conditions fail.

  • Factor models and the substitute confounder: Factor models connect the distribution of assigned causes to ignorability through Kallenberg constructions, justifying inference with a factor-model-derived substitute confounder.The connection requires the factor model to capture the population distribution of assigned causes.
  • Factor models and the substitute confounder: The substitute confounder captures all multi-cause confounders and does not capture mediators, supporting its use as a pre-treatment covariate.Single-cause confounders, if present, must still be accounted for separately.
  • Causal identification of the deconfounder: The deconfounder identifies average causal effects for all causes, subsets of causes, and conditional potential outcomes under SUTVA, single ignorability, and consistency assumptions.Subset effects additionally require overlap, while conditional potential outcomes require queried assignments to yield the same substitute-confounder estimate.
  • Factor models and the substitute confounder: Many factor models yield consistent substitute-confounder estimates when the number of causes is large, under (n + m)·log(nm)/(nm) → 0.The stated examples include probabilistic PCA and Poisson factorization.
  • Causal identification of the deconfounder: Identification of the average effect of all causes additionally requires a piece-wise constant substitute confounder and separability of the outcome in the substitute confounder and causes.These requirements are stated as conditions for Theorem 6.
  • Causal identification of the deconfounder: When identification assumptions fail, the posterior distribution of the deconfounder estimate reflects uncertainty and can expose non-identifiability.With non-informative priors, a quantity non-identifiable over R yields a posterior uniform over R.

5 Discussion

The discussion frames multiple causes as enabling causal inference under weaker assumptions than classical ignorability and identifies open directions for extending the approach.

  • Multiple causal inference can provide observational causal inference under weaker assumptions than the classical approach requires.
  • The deconfounder fits a factor model, infers a substitute confounder, and then performs causal inference.The paper states that a good factor model must capture all multi-cause confounders and that model adequacy is checkable.
  • Future work includes trading off confounding bias and estimation variance when estimating potential outcomes under all cause configurations.
  • The paper identifies rigorous model checking for causal inference as an open problem because model checking remains imprecise.
  • Other open directions include testing significant causes while controlling family-wise error or false discovery rates and handling multiple outcomes.

A Detailed Results of the GWAS Study

The GWAS results section presents tables for high- and low-SNR settings, including a high-SNR Balding-Nichols simulation where the deconfounder outperforms (G)LMM.

  • The section presents GWAS study results from Section 3.2.
  • Tables 6 to 10 report results under the high-SNR setting.
  • In the high-SNR Balding-Nichols simulation, the deconfounder outperforms (G)LMM, while DEF performs best among five factor models.Predictive checking offers a good indication of when the deconfounder fails.
  • Tables 11 to 15 report results under the low-SNR setting.

B Detailed Results of the Movie Study

The movie-study results section presents tables reporting results from the movie study in Section 3.3.

  • The section presents tables of results from the movie study in Section 3.3.
  • The supplied passage identifies the movie-study tables as a dedicated detailed-results section.
  • No specific movie-study outcome or quantitative comparison is stated in the supplied passage.

C Proof of Lemma 1

The proof constructs cause variables from a shared latent variable and independent randomization variables, then uses conditional-independence arguments to establish the lemma.

  • The proof first represents assigned causes as measurable functions of a latent variable Z and randomization variables U_j.
  • The constructed randomization variables are shown to be independent of the latent variable and potential outcomes, yielding conditional independence for the assigned causes.
  • The construction uses measurable spaces and Borel cause spaces as part of its formal setup.
  • The argument invokes Kallenberg results and concludes through necessity and sufficiency steps.
  • The proof repeatedly introduces uniform random variables and measurable functions to construct the cause variables.

D Proof of Lemma 2

The proof represents each cause through a factor model with a per-unit latent variable and independent residuals. This structure yields the conditional independence needed for the substitute confounder.

  • Point-mass latent parameters are independent of potential outcomes and other latent variables, supporting the proof’s ignorability argument.
  • The proof assumes continuous causes, latent variables, and no single-cause confounders, while noting extensions to discrete variables and observed single-cause confounders.
  • The factor-model representation expresses each cause as a function of the substitute confounder and an independent residual.
  • The residual variables are jointly independent because conditional independence of the causes would otherwise fail.
  • Conditioning on the inferred latent variable makes the cause-specific residuals independent of the potential outcomes.
  • Observed pre-treatment single-cause confounders can be incorporated by expanding the substitute confounder with those covariates.

E Proof of Lemma 3

The proof argues that a factor-model substitute confounder captures confounders affecting multiple causes, while observed covariates must account for single-cause confounders. Together they cover all confounders under single ignorability.

  • A multi-cause confounder affects at least two causes and the potential outcome through a pre-treatment covariate structure.
  • If an omitted multi-cause confounder were not measurable from the substitute confounder, the factor model could not satisfy its required conditional independence.
  • Under single ignorability, every confounder is measurable with respect to the substitute confounder and observed covariates.
  • The substitute confounder handles multi-cause confounders but requires the absence of unobserved single-cause confounders.
  • The paper’s identification results also hold under a weaker technical version of single ignorability.

F Proof of Lemma 4

The two-stage deconfounder infers the substitute confounder using causes alone before estimating potential outcomes. This separation prevents the substitute confounder from encoding a mediator.

  • The deconfounder separates substitute-confounder inference from causal-effect estimation, making the inferred variable pre-treatment.
  • A mediator would remain dependent on the potential outcome conditional on the causes, so it cannot be measurable from the substitute confounder.
  • Because the substitute confounder is inferred only from assigned causes, it is conditionally independent of potential outcomes given those causes.
  • Under single ignorability, the substitute confounder and observed covariates together capture all confounders.

G Proof of Proposition 5

The proposition constructs factor-model representations using point masses and copulas, clarifying how the deconfounder captures dependence among multiple causes. These constructions also expose limits of overly flexible factor models.

  • Point-mass parameters allow the distribution of assigned causes to be written in factor-model form.
  • Setting the substitute confounder equal to the assigned causes does not help causal inference because of overlap considerations.
  • A factor model that captures the observed cause distribution is not necessarily the true assignment model.
  • The proposition’s construction is intended to demonstrate the large capacity of factor models rather than establish that every such model is substantively correct.
  • The copula-based construction uses Sklar’s theorem to represent the joint cause distribution.
  • The construction targets multi-cause confounders through the dependence structure among causes, represented by their copula.

H Proof of Theorem 6

Theorem 6 identifies the average causal effect of all causes using substitute confounders under a step-function condition and a separable outcome model. These assumptions avoid requiring overlap while supporting ignorability given observed covariates and substitute confounders.

  • Proof strategy: The proof relies on single ignorability and pre-treatment causes to show that observed covariates and substitute confounders capture confounders but not mediators.Consistent factor-model estimates require P(Zi | Ai) = δf (Ai).
  • Identification conditions: Theorem 6 identifies the average causal effect of all causes without assuming overlap.This requires ∇a f (a1,...,am) = 0 almost everywhere and a separable outcome model.
  • Identification conditions: The substitute confounder must be constant within countably many regions partitioning the possible values of all causes.It may take different constant values across regions.
  • Identification conditions: Separability requires that the outcome model has no interaction between the substitute confounder and the causes.Under this condition, the average causal effect is identified for all values of a and a′.
  • Proof strategy: The gradient argument uses SUTVA, ignorability given Xi,Zi, separability, the chain rule, and differentiability conditions.The cited equalities also use the fundamental theorem of calculus and algebraic rearrangement.

I Proof of Theorem 7

Theorem 7 establishes identification for partially intervened causes by combining substitute-confounder ignorability with overlap for the intervened causes. The resulting quantity is estimable when the substitute confounder is consistently estimated.

  • Identification: The partially intervened causes must satisfy overlap with respect to the substitute confounder and observed covariates.This overlap condition supports estimation from observed data.
  • Identification: Consistent estimation of the substitute confounder is also required for estimating the identified quantity from observed data.The substitute confounder is denoted Z in the estimation condition.

J Proof of Theorem 8

Theorem 8 identifies an individual’s mean potential outcome under a new assignment when the current and new assignments yield the same substitute confounder estimate. This identification does not require overlap.

  • Identification: Theorem 8 identifies the mean potential outcome conditional on an individual’s current multiple-cause assignment.The current assignment is Ai = (a1,...,am).
  • Identification: The new assignment must satisfy f (a1,...,am) = f (a′1,...,a′m), so both assignments produce the same substitute confounder estimate.This equality makes the target conditional mean estimable from the observed data.
  • Proof strategy: The identification argument combines the consistency requirement P(Zi | Ai) = δf (Ai) with ignorability given Zi and Xi.The proof also uses the tower property and the estimability implied by equal substitute-confounder values.
  • Scope: The nonparametric identification result does not require overlap.This scope follows explicitly from the theorem’s concluding statement.

K Details of Section 3.2

The GWAS simulations generate genotype matrices from structured allele frequencies, latent population confounding, and controlled signal-to-noise settings. Across several scenarios, the deconfounder often outperforms linear mixed models, while predictive checking indicates failure cases.

  • Simulation design: Genotypes are simulated as Ai j ∼Binomial(2,Fi j), with allele frequencies factorized as F = ΓS.S encodes genetic population structure, while Γ maps that structure to SNP allele frequencies.
  • Simulation design: The simulations include 100 independent studies with one continuous and one binary trait.Each trait combines causal signals, confounding, and random effects.
  • Simulation design: The first 1% of SNPs are set as true causal SNPs, while the remaining SNP coefficients are zero.The causal coefficients for the remaining SNPs satisfy βj = 0.
  • Simulation design: Latent population structure affects both SNPs and confounding, with confounders formed from three K-means clusters.The clustering uses Euclidean distance on the columns of S.
  • Simulation design: Low-SNR settings allocate variance as νgene = 0.1, νconf = 0.2, and νnoise = 0.7; high-SNR settings use 0.4, 0.4, and 0.2.These settings control the relative contributions of causal signals, confounding, and noise.
  • Simulation results: The deconfounder outperforms (G)LMM in the PSD high-SNR simulation and often does so in the spatial model.DEF often performs best among the five factor models, while predictive checking indicates when the deconfounder fails.
  • Simulation results: In four low-SNR scenarios, the deconfounder outperforms LMM and DEF performs best among the five factor models.These scenarios are Balding-Nichols, TGP, HGDP, and PSD.
  • Simulation results: In the low-SNR spatial model, spatially induced SNPs challenge latent-variable models, and the deconfounder does not outperform using unobserved confounder information.Predictive checking offers an indication of when the deconfounder fails.
Loading 1805.06826v3…