PaperlayerView exact revision on arXiv

Source-linked AI summary

On a Class of Bias-Amplifying Variables that Endanger Effect Estimates

Judea Pearl

arXiv:1203.3503v1stat.MEcs.AI

TL;DR

The paper asks why some covariates, especially instrumental variables, amplify confounding bias when conditioned on. It derives and explains the phenomenon, then studies nonlinear systems and selection-induced bias, finding linear amplification, possible nonlinear new bias, and no effect on pure selection bias.

  • Problem

    Covariate adjustment can worsen causal-effect bias, but the conditions and scope of this bias-amplifying behavior require analysis beyond standard confounder-selection practice.

  • Method

    The paper derives bias amplification in a structural model, gives an intuitive explanation, and extends the analysis to nonlinear systems and selection-induced bias.

  • Results

    In linear systems, conditioning on an IV amplifies existing confounding bias; in nonlinear systems it may amplify or attenuate bias and introduce new bias, while it has no effect on pure selection-induced bias.

  • Takeaways & Limitations

    Covariates should be selected for their likely bias-reducing effect and outcome importance rather than treatment-assignment prediction alone.

  • Takeaways & Limitations

    In nonlinear models, IV-sensitivity can rank estimates, but its usefulness is limited because low sensitivity may also be assessed through dependence between Y and Z given X.

Abstract

from arXiv · show

This note deals with a class of variables that, if conditioned on, tends to amplify confounding bias in the analysis of causal effects. This class, independently discovered by Bhattacharya and Vogt (2007) and Wooldridge (2009), includes instrumental variables and variables that have greater influence on treatment selection than on the outcome. We offer a simple derivation and an intuitive explanation of this phenomenon and then extend the analysis to non linear models. We show that: 1. the bias-amplifying potential of instrumental variables extends over to non-linear models, though not as sweepingly as in linear models; 2. in non-linear models, conditioning on instrumental variables may introduce new bias where none existed before; 3. in both linear and non-linear models, instrumental variables have no effect on selection-induced bias.

1 INTRODUCTION

The paper identifies instrumental variables as a class of covariates that can amplify existing confounding bias, challenging the practice of selecting covariates mainly for treatment-assignment prediction. It derives this phenomenon and extends the analysis to nonlinear models and selection bias.

  • 1 INTRODUCTION: Adjustment is unbiased only when the selected variables satisfy admissibility conditions such as the back-door criterion.Nonadmissible covariates, including colliders and intermediates, can increase bias rather than remove it.
  • 1 INTRODUCTION: Instrumental variables can amplify existing confounding bias even though they exhibit statistical properties commonly associated with confounders.In linear systems, conditioning on an IV invariably increases confounding bias when such bias exists.
  • 1 INTRODUCTION: Instrumental variables are associated with treatment but not with other outcome-affecting factors when treatment is fixed by intervention.This property helps explain why IVs can appear to be ordinary confounders requiring control.
  • 1 INTRODUCTION: The finding challenges covariate-selection practices that favor powerful predictors of treatment assignment without considering their relationship to outcome.The paper argues that ignoring outcome relevance can produce unwanted consequences when residual bias remains.
  • 1 INTRODUCTION: The paper derives and explains bias amplification in structural models, then extends the analysis to nonlinear systems and selection-induced bias.The extensions address when confounding variables become amplifiers, nonlinear IV behavior, and IV effects on selection bias.

2 ANALYSIS

The linear structural analysis compares the causal effect with unadjusted and IV-adjusted regression effects. It shows that conditioning on the instrumental variable increases the magnitude of confounding bias.

  • 2 ANALYSIS: The linear model represents treatment X, outcome Y, unobserved confounder U, and instrumental variable Z, assuming zero means and unit variances.The analysis evaluates whether conditioning on Z changes the bias in estimating the causal coefficient c0.
  • 2 ANALYSIS: The linear model therefore shows that conditioning on an instrumental variable is harmful when unobserved confounding produces nonzero bias.The paper frames the result as an increase in the bias of the estimate of c0.
  • 2 ANALYSIS: The analysis compares the causal effect, the unadjusted regression coefficient, and the coefficient of X after conditioning on Z.The unadjusted coefficient is the crude or naive estimate of the causal effect.
  • 2 ANALYSIS: The derivation obtains the conditional regression of U on X and Z and uses the instrumental assumption cov(Z, U) = 0.These relationships determine the coefficients needed to evaluate the conditional bias.
  • 2 ANALYSIS: Conditioning on Z increases bias magnitude: |Bz| ≥ |B0|, with strict inequality whenever |B0| > 0 and |c3| > 0.The result holds regardless of the signs of c1 and c2 and also extends to a vector of confounding variables.

3 INTUITION

The intuition is that conditioning on Z removes variation in Z that would otherwise absorb part of the treatment difference. This makes the confounder difference larger and transmits more of it into the outcome contrast.

  • 3 INTUITION: With X = U + cZ, conditioning on Z compares units whose confounder values differ by the full treatment contrast.Within a fixed Z stratum, the mean difference in U between X = 0 and X = 1 is unity.
  • 3 INTUITION: The confounder difference is transmitted to Y and becomes bias in the conditional treatment contrast.The conditional outcome contrast differs from the causal coefficient c0 by c2.
  • 3 INTUITION: Without conditioning on Z, variation in Z absorbs part of the change in X, reducing the portion transmitted through U to Y.In the uniform unit-square example, the resulting mean difference in U is half the corresponding conditional difference.
  • 3 INTUITION: For any joint distribution of U and Z, E(U|x) = x − cE(Z|x), making the role of Z explicit in the unconditioned confounder mean.The unconditioned expectation averages conditional confounder means over Z given X.
  • 3 INTUITION: The second term remains positive regardless of whether X and Z are positively or negatively correlated, so refraining from conditioning on Z diminishes the confounder difference.The direction of the correlation does not change the qualitative conclusion.

4 THE LINE BETWEEN INSTRUMENTS AND CONFOUNDERS

When an imperfect instrumental variable also affects the outcome, its bias effect depends on the relative strength of those pathways. Strong treatment prediction can make such variables especially dangerous to condition on, while instrumental-variable adjustment can also produce Simpson’s reversal.

  • Imperfect instruments: Allowing Z to affect Y directly turns the perfect-instrument analysis into a comparison of Z’s effects on treatment and outcome.The paper asks when this imperfect instrument becomes a bias reducer rather than a bias amplifier.
  • Simpson’s reversal: An instrumental variable may produce Simpson’s reversal, so associations within Z strata can have the opposite sign from the crude association.With unobserved U and continuous X, the IV model is statistically indistinguishable from a model in which Z causes both X and Y.
  • Imperfect instruments: Z becomes a bias reducer only when its effect on Y exceeds its effect on X by the specified threshold.The threshold is expressed as c2c1/1 −c2.
  • Imperfect instruments: When c3 is close to unity, the threshold is difficult to meet, making the best predictors of X the most dangerous bias amplifiers.Thus, strong treatment-selection prediction compromises a covariate’s bias-reducing potential.
  • Practical implication: The practical implication is to select pre-treatment covariates for expected bias reduction rather than for predictive power over treatment assignment.The paper recommends ranking covariates by their importance for the outcome.

5 A GLIMPSE AT NON-LINEAR SYSTEMS

The bias-amplification result extends to nonlinear systems but no longer holds universally. Nonlinearity can make conditioning on an instrumental variable reduce existing bias or create bias where none existed.

  • Nonlinear extension: In nonlinear models, instrumental-variable conditioning can still amplify bias, but there are cases where the unconditioned bias exceeds the conditioned bias.Thus, the universal linear-model result does not carry over unchanged.
  • Bias reduction: Conditioning on Z may reduce confounding bias when B0 ≥0 and c1c3g′(x)z > 0.This can occur even when Z is a perfect instrument and Y and X are linear in U.
  • Bias reduction: Because Y is nonlinear, conditional bias depends on the value of Z; at Z = 0, the linear-case bias amplification is recovered.The nonlinear specification changes how bias varies across values of the instrument.
  • New bias: Conditioning on Z can introduce bias when B0 = 0, including the case c1 > 0 and g(x) = A/x, which yields Bz > 0.This possibility is suppressed in linear systems but becomes possible in nonlinear systems.
  • New bias: New bias from conditioning on Z requires Z and Y to remain dependent given X and does not occur when zero bias is structural.Structural zero bias means one of the structural equations is trivial in its U argument.

6 THE RESILIENCE OF SELECTION BIAS

Selection bias caused by preferential inclusion can arise through collider mechanisms and is fundamentally different from confounding bias. Unlike confounding bias, pure selection-induced bias is unaffected by conditioning on an instrumental variable, although mixed selection and confounding bias can respond to such conditioning.

  • Selection mechanisms: Preferential selection into the data pool conditions on S and creates spurious X–Y association through collider-related mechanisms.S is affected by both X and Y; conditioning on it induces an association between its parents, while a second mechanism involves a descendant of a virtual collider.
  • IV resilience: Conditioning on an instrumental variable Z has no effect on pure selection-induced bias.The result extends beyond the parametric model because the diagram implies that Y and Z are independent given X and S.
  • Selection versus confounding: Selection bias differs from confounding bias because the former is insensitive to conditioning on an IV, whereas confounding bias can change under such conditioning.This distinction motivates using IV-sensitivity to diagnose confounding in observed effect estimates.
  • Diagnostic scope: IV-sensitivity can rank effect estimates in nonlinear models, but its usefulness is limited because causal effects may be nonidentifiable.Low sensitivity and low bias may instead be assessed indirectly through dependence between Y and Z given X.
  • Mixed bias sources: Mixed selection and confounding bias can arise when exclusion mechanisms involve ancestors of treatment.In the illustrated model, S1 induces both components, S2 induces pure selection bias, and S3 induces pure confounding bias.
  • Bias removal: Conditioning on U2 eliminates the bias induced by S2, whereas the selection component induced by S1 cannot be eliminated by any method.The confounding component of S1 can be removed by conditioning on U1, while pure confounding from S3 can be removed by conditioning on either U1 or U2.

7 CONCLUSIONS

The paper examines how instrumental variables affect confounding and selection bias across linear and nonlinear structural models. It concludes that covariate choice should emphasize outcome relevance, while structural causal assumptions remain important for avoiding unprincipled selection.

  • Conclusions: In linear systems, conditioning on an IV always amplifies existing confounding bias; in nonlinear systems, it may amplify or attenuate bias and can introduce new bias.The nonlinear result is less pervasive than the linear result.
  • Conclusions: For selection-induced bias without a confounding component, conditioning on an IV has no effect.The paper also proposes IV-sensitivity as a diagnostic for distinguishing confounding from selection bias.
  • Practical implications: Covariates should be chosen according to their importance for the outcome rather than solely their predictive power for treatment.Variables with weak outcome effects and strong treatment effects should be discarded, while strong outcome predictors should be retained in the propensity score.
  • Practical implications: Outcome-focused propensity scores are safer when residual unobserved confounders are present.In the absence of unobserved confounders, propensity scores based on direct causes of treatment or outcome give the same asymptotic effect estimate.
  • Methodological significance: Avoiding structural causal considerations can foster unprincipled covariate selection.The paper connects this problem to methodological differences between structural and experimentalist approaches.
Loading 1203.3503v1…