PaperlayerView exact revision on arXiv

Source-linked AI summary

A Class of Algorithms for General Instrumental Variable Models

Niki Kilbertus, Matt J. Kusner, Ricardo Silva

arXiv:2006.06366v3cs.LGstat.ML

TL;DR

Observational causal estimation with unobserved confounding often depends on restrictive assumptions, and unrestricted continuous-treatment instrumental-variable models can have vacuous bounds. The paper optimizes over flexible response-function models compatible with observed data to produce causal-effect bounds. In experiments, the bounds capture effects when additive methods fail, with their width reflecting response flexibility and confounding or instrument strength.

  • Problem

    Many continuous-treatment instrumental-variable methods identify effects under a special outcome form such as additive error, while unrestricted structural equations can leave even average-treatment-effect bounds vacuous.

    Source: §2 · pp. 2–3

  • Method

    The method flexibly parameterizes causal response functions and optimizes bounds over models compatible with observed data using a smooth constrained objective, gradient methods, and Monte Carlo approximations.

    Source: §2 · p. 3, §3.1 · p. 4

  • Results

    In the nonlinear non-additive setting, 2SLS fails, while KIV fails under strong confounding and captures the true effect in data-rich regions when the instrument is strong and confounding weak; bounds depend on response flexibility and can narrow with stronger instruments and weaker confounding.

    Source: §5 · pp. 8–9

  • Takeaways & Limitations

    Bounds can represent causal effects compatible with observations without relying on unwarranted structural assumptions, and become looser as the allowed response functions become more flexible.

    Source: §1 · p. 2, §5 · p. 8

    Without restrictions on the structural equations, continuous-treatment instrumental-variable effects cannot be identified and even average-treatment-effect bounds are vacuous.

    Source: §2 · p. 3

Abstract

from arXiv · show

Causal treatment effect estimation is a key problem that arises in a variety of real-world settings, from personalized medicine to governmental policy making. There has been a flurry of recent work in machine learning on estimating causal effects when one has access to an instrument. However, to achieve identifiability, they in general require one-size-fits-all assumptions such as an additive error model for the outcome. An alternative is partial identification, which provides bounds on the causal effect. Little exists in terms of bounding methods that can deal with the most general case, where the treatment itself can be continuous. Moreover, bounding methods generally do not allow for a continuum of assumptions on the shape of the causal effect that can smoothly trade off stronger background knowledge for more informative bounds. In this work, we provide a method for causal effect bounding in continuous distributions, leveraging recent advances in gradient-based methods for the optimization of computationally intractable objective functions. We demonstrate on a set of synthetic and real-world data that our bounds capture the causal effect when additive methods fail, providing a useful range of answers compatible with observation as opposed to relying on unwarranted structural assumptions.

1 Introduction

Instrumental variables can permit causal effect analysis despite unobserved confounding, but existing identification methods often rely on restrictive assumptions. This work develops flexible bounds for continuous settings, allowing practitioners to trade background assumptions against informativeness.

1 Introduction

  • Unobserved confounding obstructs trustworthy causal estimation from observational data, while identification despite confounding is available only in specific settings such as front-door adjustment or linear/additive instrumental-variable regression.Outside such settings, the paper frames meaningful bounds as the alternative.Source: §1 · p. 1
  • The method bounds effects over compatible instrumental-variable models in general continuous settings, with a continuum of model spaces to trade stronger background knowledge for tighter bounds.The approach avoids requiring a fixed structural assumption such as additivity or monotonicity.Source: §1 · pp. 1–2
  • Existing symbolic methods characterize observable constraints by eliminating hidden variables, but enumerating all inequality constraints can be super-exponential even for discrete models.Balke & Pearl (1994) instead match latent-model marginals to the observed distribution, connecting the problem to stochastic integral evaluation.Source: §1 · p. 2
  • The proposed approach parameterizes causal responses and compatible instrumental-variable structure, then uses Monte Carlo integral approximation and gradient-based optimization to find lower and upper bounds.It optimizes across plausible models consistent with observed data, including when treatment is continuous.Source: §1 · p. 2

2 Current Approaches and Their Limitations

Prior continuous-treatment instrumental-variable methods often obtain identification by imposing an additive outcome equation, while unrestricted models can yield vacuous bounds. The paper proposes flexible response-function bounds optimized as a constrained continuous program.

2 Current Approaches and Their Limitations

  • For continuous treatments, much recent machine-learning instrumental-variable work identifies effects by assuming a special outcome equation, commonly an additive error model.The error need not be independent of treatment, so the setup permits unobserved confounding.Source: §2 · pp. 2–3
  • Without restrictions on structural equations, continuous-treatment instrumental-variable effects cannot be identified and even average-treatment-effect bounds are vacuous; finite-discontinuity conditions can support meaningful bounds.Gunsilius (2020) gives a theoretical framework but the authors report frequent, hard-to-diagnose stability issues in its proposed wavelet method.Source: §2 · p. 3
  • The proposed alternative flexibly parameterizes causal response functions and encodes instrumental-variable compatibility, using an augmented Lagrangian method for non-convex optimization with inequality constraints.It formulates the bounds and compatibility requirements as a smooth constrained objective optimized with gradient methods and Monte Carlo approximations.Source: §2 · p. 3
  • The authors report matching prior estimation results in additive settings and meaningful causal-effect bounds in general non-additive models.The method replaces symbolic-combinatorial constraints with a continuous program.Source: §2 · p. 3

3 Problem Setting

The paper represents continuous-treatment instrumental-variable models through response functions and bounds intervention outcomes by optimizing over response distributions compatible with observed marginals. It constructs compatibility constraints for treatment and outcome distributions while using flexible response classes to encode assumptions.

3 Problem Setting

  • The instrumental-variable setting assumes Z is independent of the exogenous process U, associated with X, and conditionally independent of Y given X and U.The target is lower and upper bounds on E[Y | do(x⋆)] for a chosen intervention level.Source: §3 · p. 3, §3.1 · p. 3

3.1 Goal

  • The goal is to optimize minimum and maximum intervention outcomes over latent structural models that reproduce the observed treatment and outcome marginals conditional on Z.The paper notes that matching p(x | z) and p(y | z) suffices for this formulation.Source: §3.1 · pp. 3–4
  • Response-function indices represent confounder states by their implied functions of treatment, allowing optimization over distributions of outcome functions rather than an unclear-dimensional distribution over U.The model focuses on RY; it does not require counterfactuals for X, and matching p(x | z) and p(y | z) avoids requiring simultaneous measurements of X and Y.Source: §3.1 · p. 4
  • The response class is parameterized by θ, with each response fθ(x) formed as a linear combination of basis functions; the paper focuses on this class although the method works for differentiable responses.Without restrictions on the function space, non-trivial inference is impossible.Source: §3.1 · p. 4

3.2 Matching p(x | z) and Enforcing Z ⊥⊥U

  • The model factors pη(x, θ | z) into the observed p(x | z) and a parameterized conditional response distribution, using a copula to enforce independence between θ and Z.The described implementation uses Gaussian marginal densities for θ components and a Gaussian copula.Source: §3.2 · p. 5

3.3 Matching p(y | z)

  • Matching p(y | z) creates infinitely many constraints involving non-continuous indicators, complicating smooth gradient optimization.The method approximates the constraints with a finite grid of instrument values and expectations over a finite dictionary of basis functions.Source: §3.3 · p. 5
  • Constraining moments such as the outcome mean and variance relaxes the full constraint set, so it can make bounds looser but does not invalidate them absent local-optimum, Monte Carlo, or estimation errors.The paper relates the alternative of an infinite dictionary sampled at infinitely many instrument values to mean embeddings.Source: §3.3 · p. 5

4 Optimization Strategy

The method frames causal-effect bounds as a smooth constrained optimization over response models compatible with observed data. An augmented Lagrangian strategy and empirical estimation choices make the optimization practical while allowing response-function complexity to be varied.

4 Optimization Strategy

  • The final problem optimizes lower or upper causal-effect bounds subject to constraints matching model-implied quantities to observed-data targets.The bounds are obtained by minimizing or maximizing over η.Source: §4 · p. 5

4.2 Empirical Estimation and Implementation Choices

  • Empirical estimation fixes conditional outcome targets from data and repeatedly estimates model-implied quantities using samples during optimization.Monte Carlo estimates and automatic differentiation provide gradients; Gaussian-copula sampling with reparameterized parameters makes the draws differentiable with respect to η.Source: §4.1 · p. 6, §4.2 · p. 6

4.1 Augmented Lagrangian Strategy

  • The augmented Lagrangian strategy alternates between minimizing the constrained objective and updating multipliers λ and temperature τ.Positive absolute and relative tolerances relax equality constraints; as tolerances approach zero, the exact constraints are recovered, and the relaxation can only loosen bounds.Source: §4.1 · p. 6

4.2 Empirical Estimation and Implementation Choices

  • Polynomial and learned MLP basis functions let practitioners vary response-function complexity and examine how that choice changes bound tightness.More plausible complexity can be selected at the cost of looser bounds; the formulation also permits shape and smoothness assumptions.Source: §4.2 · p. 7

5 Experimental Results

Across synthetic and expenditure data, the bounds capture the true or plausible causal effects while reflecting uncertainty under different response-function families and instrument conditions. Additive point-estimation methods can fail in non-additive settings, while the bounds can become uninformative when the response family is too restrictive.

Synthetic results

  • The true effect stays within the bounds across synthetic settings, which also contain most compatible alternative models; bounds widen with response flexibility and narrow as instrument strength rises and confounding weakens.Flexible MLP bounds remain loose under strong confounding, consistent with the uncertainty warranted by that setting.Source: Figure 2 · p. 8, §5 · p. 8

Linear Gaussian case

  • In the linear-Gaussian weak-instrument setting, 2SLS recovers the true effect, while KIV reverts to its prior mean of 0; with a stronger instrument, KIV matches the true effect in data-rich regions.Figure 2 compares weak- and strong-instrument settings across response-function families.Source: §5 · pp. 7–8, Figure 2 · p. 8

Non-additive, non-linear case

  • In the nonlinear, non-additive setting, 2SLS fails, and KIV fails under strong confounding but captures the effect in data-rich regions when the instrument is strong and confounding weak.With a strong instrument and weak confounding, bounds using an adequate response family can become narrow; linear responses cannot explain the data in that setting.Source: Figure 2 · p. 8, §5 · pp. 8–9

Expenditure data

  • On UK expenditure data, bounds provide information about the sign and magnitude of effects and widen as response functions become more flexible.They include most effects from fitted latent-variable models; the few outside the bounds may reflect suboptimal local-likelihood fits.Source: Figure 3 · p. 9, §5 · p. 9

6 Conclusion

The paper proposes optimization-based algorithms for bounding causal effects and identifies several directions for extending and assessing them. These include quantifying statistical uncertainty, testing IV-model validity, and accommodating softer causal constraints.

6 Conclusion

  • Bounds do not capture statistical uncertainty; confidence or credible intervals for the bounds remain future work.The authors point to Silva & Evans (2016) as providing directions for credible intervals.Source: §6 · p. 9
  • The authors assumed the IV model was correct, while model falsification remains possible when optimization fails to find a solution.They identify methods focused on testing models, rather than deriving bounds, as future work.Source: §6 · p. 9
  • The authors envision extending causal modeling from conditional-independence statements to softer constraints, such as allowing a weak Z → Y edge.They note that computationally feasible implementation remains a challenge.Source: §6 · p. 9

Broader Impact

The paper frames causal effect estimation from observational data as relevant to decisions where randomized experimentation is unavailable and assumptions are unavoidable. Its bounds can widen as support for functional-form assumptions weakens, but the method itself also depends on assumptions and non-convex optimization.

Broader Impact

  • Observational-data decisions often lack randomized experiments, and causal inference depends on assumptions that may be untestable.The authors note that even slight assumption violations can lead to substantially different conclusions.Source: Broader Impact · p. 10
  • Bounds widen as less prior information supports assumptions about the functional form of effects, allowing stricter assumptions to be assessed before judgment.The authors present this as a way of deferring judgment while assumptions are assessed and verified.Source: Broader Impact · p. 10
  • The method requires domain-expert assessment of assumptions, and non-convex local optimization has no theoretical guarantee of finding correct bounds.Prematurely accepted wrong assumptions or failure to find global optima may wrongly inform decisions; biased training data may replicate group biases.Source: Broader Impact · p. 10

A Gunsilius’s Algorithm

This section describes Gunsilius’s approach to bounding causal effects and contrasts its sampling and optimization choices with the paper’s method. The comparison covers convergence behavior and computational cost; Figure 4 reports Gunsilius-algorithm results at three outcome quantiles.

A Gunsilius’s Algorithm

  • Gunsilius’s basic algorithm reweights sampled response functions to match the observed bivariate CDF on a grid, using approximate L2 constraints.The sampled functions come from a Gaussian process in the experiments described.Source: A Gunsilius’s Algorithm · p. 13
  • The non-adaptive Monte Carlo version may require many samples, while its adaptive variant can have severe, hard-to-diagnose convergence problems.The adaptive variant replaces response-function samples with low weights after each solution for the weights is found.Source: A Gunsilius’s Algorithm · p. 13
  • The paper instead continuously optimizes η with gradient estimates and constrains a univariate density, avoiding discrete mixture reweighting that the authors say introduces instability.The approach can use p(y | z), or p(y | x, z) in an appendix variation.Source: A Gunsilius’s Algorithm · p. 13
  • About 30 minutes for one Gunsilius setting versus about 20 minutes for our algorithm’s bounds at 15 x⋆ values on the same quad-core Intel Core i7.The comparison follows the authors’ acceleration and parallelization efforts for Gunsilius’s algorithm.Source: A Gunsilius’s Algorithm · p. 13
  • Figure 4 shows Gunsilius-algorithm results for y⋆ values 0.25, 0.5, and 0.75.The caption specifies the three settings but supplies no outcome values.Source: Figure 4 · p. 14

B The Shape of pη(θ | x, z) and Conditional Effects

The appendix discusses the response-function distribution, extensions to discrete outcomes and covariates, and optimization and sampling details. It describes both flexibility choices and practical constraints, including parametric modeling and computational cost.

B The Shape of pη(θ | x, z) and Conditional Effects

  • The Gaussian parameterization is illustrative rather than universal: the authors describe copulas usable with the reparameterization trick as alternatives.They identify reliance on a parametric pη(θk), and its related copula, as the main apparent limitation; more flexible mixtures are possible at greater computational cost.Source: B The Shape of pη(θ | x, z) and… · pp. 14–15
  • Covariates W can define separate optimization problems for each configuration, while a multilevel pη(w) can encode smooth variation across W.The authors leave more flexible modeling of pη(·), including covariates, for future work.Source: B The Shape of pη(θ | x, z) and… · p. 15

C Discrete Outcomes and Discrete Features

  • For discrete outcomes, discontinuous response functions complicate gradient optimization; the authors report less robust results for the alternative approach and therefore report the main-text dictionary-function approach.The alternative uses a smoothly differentiable approximation to the indicator and gradually increases ρ to approximate the constraints.Source: C Discrete Outcomes and Discrete… · pp. 15–16

D.1 Additional Details of the Optimization

  • The optimization details include smoothing empirical constraints across instrument bins with spline regression to reduce jumps between neighboring values.The implementation uses a cubic univariate spline with smoothing factor 0.2.Source: §D.1 · p. 16

D.2 Augmented Lagrangian Optimization Strategy

  • The inequality-constrained problem is reformulated with Lagrange multipliers and a penalty term that discourages large multiplier changes between iterations.The penalty parameter is increased during optimization; the authors report no issues with up to 54 parameters and 40 constraints in their experiments.Source: §D.2 · pp. 17–18

D.3 Sampling from the Copula

  • Copula sampling incorporates assumptions about p(X | Z) and Z ⊥⊥ U directly into samples used for Monte Carlo estimates.The described sampling procedure is differentiable with respect to η.Source: §D.3 · p. 18

D.4 Parameter Initialization

  • Initialization uses a broad initial distribution over response functions, with larger σk favored to improve coverage.The lower triangle of L is sampled from N(0, 0.05), while µk and ln(σ2k) initialization depends on the response-function family.Source: §D.4 · p. 18

E Response Functions

The section presents flexible response-function choices and alternative modeling and optimization strategies for bounding continuous-treatment effects. On the sigmoidal design, bounds recover the true effect across several response families when they can represent its shape.

E Response Functions

  • Response functions can use polynomial, neural-network, or Gaussian-process bases, trading stronger shape assumptions for potentially more informative bounds.Neural and GP bases assume the causal effect can be approximated by combinations of learned activations or sampled functions.Source: E Response Functions · pp. 18–19

F Why Discretization is not a Good Idea

  • Discretizing treatment can violate the exclusion restriction, make interventions ambiguous, incur super-exponential costs, and yield vacuous bounds in the limit.The section argues that fixed-knot regression splines can represent response constraints without discretizing treatment.Source: F Why Discretization is not a Good Idea · pp. 19–20

G Modeling p(y | x, z) and Monte Carlo Alternatives

  • Modeling p(y | x, z) uses more of the observed information and avoids estimating p(x | z), but requires functionals of the joint conditional distribution.The approach trades additional constraints for removing integration over X from each constraint; Monte Carlo is useful for differentiable features whose expectations lack analytic solutions.Source: G Modeling p(y | x, z) and Monte Carlo… · p. 20

H Fitting Latent Variable Models

  • Latent-variable models use encoders and decoders to fit response functions consistent with the chosen response-family form.The outcome decoder is specified as an MLP, linear function, or quadratic function according to the response family being evaluated.Source: H Fitting Latent Variable Models · pp. 20–21

I.2 Sigmoidal Design

  • On the sigmoidal design, the true logistic effect is well recovered across cubic-polynomial, GP-basis, and MLP-basis response families.The bounds are relatively narrow when the instrument is strong and the response family is flexible enough to capture a sigmoidal shape.Source: Figure 5 · p. 21, §I.2 · p. 21

I.1 Hyperparameter Settings

  • The reported hyperparameter settings worked across datasets and response families without much manual tuning.The experiments use 5000 synthetic observations and gradually tighten the relative tolerance from 0.3 to 0.05.Source: §I.1 · p. 21

I.3 Expenditure Data

The supplied passages describe expenditure-data sample restrictions and identify two figures, but do not report expenditure-data results.

I.3 Expenditure Data

  • The expenditure-data sample is restricted to married couples living together, with household heads aged 20–55, at most two children, and no unemployment.After these restrictions, the dataset contains 1650 observations.Source: §I.3 · pp. 21–22
  • Figure 6 presents method performance with 500 observations, contrasting strong confounding with a weak instrument against weak confounding with a strong instrument.The left column uses α = 0.5, β = 3; the right uses α = 3, β = 0.5.Source: Figure 6 · p. 22
  • Figure 7 covers the Acemoglu et al. (2001) small-dataset results using linear response functions and M = 5 z-bins.Source: Figure 7 · p. 22

I.4 Small Data Regime

The method provides useful bounds in the reported small-data settings, including a synthetic setting with 500 observations and an economic-instrument dataset with 70 observations.

I.4 Small Data Regime

  • With 500 synthetic observations and M = 6 z-bins, the bounds are looser but still provide useful information.Source: §I.4 · p. 22
  • With 70 observations, linear-response bounds using M = 5 z-bins include the 2SLS estimate but do not fully include KIV results.The data come from the Acemoglu et al. (2001) setting on settler mortality as an instrument for institutional effects on economic performance.Source: §I.4 · p. 23

J KIV Heuristic for Tuning Hyperparameters

KIV's off-the-shelf heuristic fails in the strongly confounded linear Gaussian setting, while manual tuning can improve its fit there. In the non-linear, non-additive setting, even manually selected hyperparameters do not recover the true effect.

J KIV Heuristic for Tuning Hyperparameters

  • Manual hyperparameter tuning improves KIV's approximation in the strongly confounded linear Gaussian setting, but selection requires knowing the true causal effect.The authors also note uncertainty about choosing such values without that knowledge and whether the result is coincidental.Source: J KIV Heuristic for Tuning… · p. 23
  • KIV's internal hyperparameter heuristic fails in the strongly confounded linear Gaussian setting despite the setting satisfying its assumptions.A grid search scored settings using out-of-sample mean-squared error for the known true causal effect.Source: J KIV Heuristic for Tuning… · p. 23
  • In the non-linear, non-additive setting, KIV's manually selected hyperparameters do not make its regression capture the true causal effect.That setting violates KIV's additive-confounding assumption.Source: J KIV Heuristic for Tuning… · pp. 23–24

Related papers

Loading 2006.06366v3…