PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Causal Inference

Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, Aidong Zhang

arXiv:2002.02770v1stat.MEcs.AIcs.LGstat.ML

TL;DR

Causal inference from observational data offers a lower-cost alternative to randomized trials but requires methods for recovering counterfactual outcomes under limited treatment control. This survey synthesizes potential-outcome methods, organizes them by their assumptions, and reviews statistical, machine-learning, application, and resource landscapes. It concludes that the field contains methods both relying on and relaxing the framework’s assumptions, while highlighting important assumption and estimator-stability constraints.

  • Problem

    Observational data are attractive because randomized controlled trials are costly, time-consuming, ethically constrained, and limited in representativeness, yet causal inference must recover counterfactual outcomes without treatment control.

  • Method

    The survey reviews causal inference methods under the potential outcome framework, categorizing them by whether they require all three assumptions and covering statistical, machine-learning, application, dataset, and code resources.

  • Results

    The survey provides comprehensive discussions, comparisons, and summaries of methods in both assumption-dependent and assumption-relaxing categories, including representative applications and resources.

  • Takeaways & Limitations

    The survey serves as a consolidated guide to potential-outcome causal inference methods and their use across advertising, recommendation, medicine, and reinforcement learning.

  • Takeaways & Limitations

    IPW can become unstable when estimated propensity scores are small, and trimming is sensitive to its chosen threshold; positivity is also harder to satisfy in high-dimensional data.

Abstract

from arXiv · show

Causal inference is a critical research topic across many domains, such as statistics, computer science, education, public policy and economics, for decades. Nowadays, estimating causal effect from observational data has become an appealing research direction owing to the large amount of available data and low budget requirement, compared with randomized controlled trials. Embraced with the rapidly developed machine learning area, various causal effect estimation methods for observational data have sprung up. In this survey, we provide a comprehensive review of causal inference methods under the potential outcome framework, one of the well known causal inference framework. The methods are divided into two categories depending on whether they require all three assumptions of the potential outcome framework or not. For each category, both the traditional statistical methods and the recent machine learning enhanced methods are discussed and compared. The plausible applications of these methods are also presented, including the applications in advertising, recommendation, medicine and so on. Moreover, the commonly used benchmark datasets as well as the open-source codes are also summarized, which facilitate researchers and practitioners to explore, evaluate and apply the causal inference methods.

1 INTRODUCTION

Causal inference distinguishes causal relationships from correlation and seeks counterfactual outcomes, especially when randomized trials are costly, limited, or impractical. This survey organizes potential-outcome methods by their assumptions, reviews statistical and machine-learning approaches, and catalogs applications and resources.

  • Correlation describes trends between variables, whereas causality concerns effects in which a cause is partly responsible for and an effect depends on it.
  • Randomized controlled trials support causal inference through random treatment assignment, but they can be expensive, time-consuming, ethically constrained, and limited in representativeness.
  • Observational-data causal inference centers on estimating counterfactual outcomes without researcher control over treatment assignment.
  • The survey focuses on the potential outcome framework, its three identification assumptions, methods that require them, and methods that relax some assumptions.
  • The survey compares representative statistical and machine-learning methods and summarizes applications, benchmark datasets, and open-source codes.
  • Its taxonomy separates methods into two major categories based on whether they require all three assumptions, with the assumption-requiring category divided into seven confounder-handling subcategories.

2 BASIC OF CAUSAL INFERENCE

Causal inference estimates how outcomes would change under alternative treatments, using potential outcomes to define treatment effects at population, group, subgroup, and individual levels. Observational estimation is valuable but must address confounding, which can make treatment comparisons reflect spurious effects.

  • Task description: Causal inference estimates outcome changes that would result if another treatment had been applied.Randomized experiments approximate the ideal of applying different treatments to the same cohort, while observational data records treatments and outcomes without directly revealing treatment mechanisms.
  • Definitions: Under the potential outcome framework, a unit is the atomic research object, treatment is the action applied to it, and each unit-treatment pair has a potential outcome.The observed or factual outcome is the potential outcome corresponding to the treatment actually applied.
  • Definitions: A counterfactual outcome is the potential outcome for a treatment the unit did not actually receive, so only one potential outcome is observed for each unit.In the binary-treatment case, the counterfactual outcome corresponds to the alternative treatment, YCF = Y(W = 1 −w).
  • Treatment effects: Treatment effects can be measured as ATE for the population, ATT for treated units, CATE for subgroups, and ITE for individual units.CATE captures heterogeneous treatment effects when effects vary across subgroups.
  • Confounders and general solutions: Confounders are pre-treatment variables affecting both treatment assignment and outcomes, so direct treated-versus-control averages can include spurious effects.In the medicine example, age influences recovery and treatment choice; the calculated 5% ATE cannot establish that Treatment A is more effective because 270 of 350 Treatment A patients are young.

3 CAUSAL INFERENCE METHODS RELYING ON THREE ASSUMPTIONS

The survey organizes causal inference methods that rely on the three potential-outcome assumptions according to how they control confounders. It covers seven method categories.

  • Method categories: Methods relying on the three potential-outcome assumptions are categorized by how they control confounders.The categories are re-weighting, stratification, matching, tree-based, representation-based, multi-task, and meta-learning methods.

3.1 Re-weighting Methods

Re-weighting methods address selection bias by assigning weights that balance treated and control groups, commonly through balancing or propensity scores. The section reviews IPW, robustness improvements, and methods that distinguish confounders from other variables.

  • Core idea: Sample re-weighting assigns appropriate weights to observational units to create a pseudo-population with more comparable covariate distributions.This targets selection bias caused by treatment assignment being correlated with covariates.
  • Balancing and propensity scores: A balancing score satisfies W ⊥⊥ x | b(x), and the propensity score is a commonly used special case based on observed covariates.The propensity score is the conditional probability of treatment given background variables.
  • Propensity-score weighting: Inverse propensity weighting assigns each sample a weight based on its estimated probability of treatment assignment.The section also introduces normalized IPW estimators, especially when propensity scores are estimated rather than known.
  • Propensity-score adjustment: Adjustment for the scalar propensity score can remove bias due to observed covariates and supports matching, stratification, regression adjustment, or combinations of these methods.This theoretical result motivates propensity-score-based adjustment across several causal-effect estimators.
  • Robust estimation: The doubly robust estimator combines propensity-score weighting with outcome regression and remains consistent when either model is correct.It is designed to reduce sensitivity to misspecification of one nuisance model, but not both.
  • Limitations and extensions: IPW can become unstable when estimated propensity scores are small, while trimming is sensitive to its threshold; bounded-weight alternatives and corrected procedures address this issue.The section also discusses CBPS and CBGPS, which target covariate balance directly to alleviate propensity-score model misspecification.
  • Variable-aware re-weighting: D2VD separates confounders, adjustment variables, and irrelevant variables, while DCB selects and differentiates confounders for distribution balancing.These methods target high-dimensional, noisy settings with limited prior knowledge about interactions among observed variables.

3.2 Stratification Methods

Stratification adjusts confounding by splitting units into homogeneous blocks and combining within-block treatment effects. The section covers pre-treatment blocking, bias reduction, block construction, and principal stratification for post-treatment variables.

  • Stratification: Stratification splits the population into homogeneous blocks where treated and control units are similar on measured covariates.Within-block comparisons are intended to reduce bias from differences between treatment groups.
  • Estimation: The stratification estimator combines treated and control outcome averages across blocks using each block’s share of the full sample.The block-specific averages and weights define the aggregate treatment-effect estimate.
  • Bias reduction: Compared with the difference estimator, stratification reduces bias per covariate by a factor determined by within-block treated-control covariate differences.Under a linear outcome model, the section contrasts overall covariate imbalance with weighted within-block imbalance.
  • Block construction: Equal-frequency blocking partitions units by the appearance probability of a score, such as the propensity score, and weights block-specific CATEs by block size.The central design choices are how to create blocks and how to combine their estimates.
  • Principal stratification: Principal stratification forms subgroups using potential values of post-treatment variables rather than their observed values.This avoids directly comparing treated and control groups whose observed intermediate outcomes may differ because treatment affects those outcomes.

3.3 Matching Methods

Matching methods estimate counterfactual outcomes by pairing units across treatment groups using similarity measures, thereby reducing confounding bias in observational data. The survey organizes matching around distance design, neighbor selection, and feature choice, while highlighting bias–variance trade-offs.

  • Distance Metric: Matching estimates a unit’s counterfactual outcome from matched neighbors in the opposite treatment group.For treated units, the estimated control outcome uses matched control neighbors; for control units, the estimated treated outcome is defined analogously.
  • Distance Metric: Distance metrics compare unit closeness through transformations of covariates, with methods differing mainly in how the transformation function is designed.A general form is D(x_i, x_j) = ||f(x_i) − f(x_j)||2.
  • Distance Metric: Propensity-score matching measures closeness by score differences, with logit-based distance recommended because it can reduce bias.Propensity scores can also be combined with other distances for finer comparison using key covariates.
  • Distance Metric: Outcome-informed transformations learn representations that preserve relationships between transformed covariates and observed outcomes, complementing propensity-score balancing.BNR integrates balancing and nonlinear representation advantages, while RNNM uses random projections and median effects across nearest-neighbor matches.
  • Choosing a Matching Algorithm: Nearest-neighbor, caliper, stratification, kernel, local linear, and coarsened exact matching offer different ways to construct counterfactuals.Kernel and local linear matching use weighted control-group averages to lower variance, whereas CEM coarsens selected covariates before exact matching and predicts outcomes in extrapolation regions.
  • Choosing a Matching Algorithm: Matching methods should be selected with attention to sample size and the bias–variance trade-off, while feature selection should exclude post-treatment variables.Asymptotically, matching methods converge toward exact comparisons, but method choice matters more in small samples.

3.4 Tree-based Methods

Tree-based methods use adaptive partitions and ensembles to estimate treatment effects and their heterogeneity. The survey covers CART, random forests, BART, and extensions to flexible treatment settings.

  • Tree-based Methods: Decision trees predict outcomes through simple decision rules inferred from data, supporting both classification and regression.Classification trees use discrete targets, while regression trees model continuous targets using squared prediction error.
  • Tree-based Methods: CART-based approaches partition data into subpopulations with different treatment-effect magnitudes and can construct valid confidence intervals without sparsity assumptions.The cited approach targets conditional average treatment effects rather than conventional prediction alone.
  • Tree-based Methods: BART models outcomes as a regularized sum of regression trees estimated with Bayesian backfitting.Its formulation combines tree-specific functions with an error term, while regularization constrains individual weak learners.
  • Tree-based Methods: BART handles many predictors, continuous treatments, missing data, and uncertainty intervals without requiring parametric relationships among inputs.It uses the outcome, treatment assignment, and confounding covariates as inputs.
  • Tree-based Methods: In examined nonlinear simulations, BART produced more accurate average treatment-effect estimates than propensity-score matching, propensity-score weighting, and regression adjustment.BART also identifies heterogeneous and individual-level causal effects.
  • Tree-based Methods: Tree-based causal methods define neighborhoods through shared leaves, allowing narrower partitions where signals change quickly and wider partitions elsewhere.This adaptive geometry can increase power when feature dimensionality is moderately large.
  • Tree-based Methods: Tree-based frameworks extend to multidimensional treatments with discrete or continuous components and require minimal manual tuning.The cited framework is described as robust to model misspecification.

3.5 Representation Learning Methods

Representation-learning methods address causal estimation by learning covariate embeddings that balance treatment groups while preserving information relevant to factual and counterfactual outcomes. The survey also discusses local similarity and textual covariates as unresolved or specialized challenges.

  • Balanced representation learning: Domain adaptation motivates representing treatment groups similarly so counterfactual inference can use shared structure across distributions.The representation objective combines factual prediction, counterfactual prediction informed by factual outcomes, and treatment-distribution distance.
  • Balanced representation learning: Balanced representation objectives combine weighted factual loss, model-complexity regularization, and an integral probability metric between treatment-group representations.The weights compensate for treatment-group size differences, and the IPM compares probability distributions.
  • Balanced representation learning: Separate treatment and control heads help prevent high-dimensional representations from obscuring treatment information in outcome prediction.Each head estimates the outcome under its corresponding treatment condition.
  • Balanced representation learning: SITE jointly preserves local similarity and balances treatment-group distributions through representation learning, metric losses, and an outcome-prediction network.Its components include triplet-pair selection, position-dependent deep metric learning, middle-point distance minimization, and mini-batch training.
  • Balanced representation learning: Textual covariates remain an open treatment-effect-estimation problem because nearly instrumental variables can be more predictive of treatment than outcome and amplify bias when conditioned on.This challenge motivates conditional treatment-adversarial learning approaches.
  • Balanced representation learning: Matching remains more interpretable than regression after representation learning because each counterfactual is directly assigned from an opposite-treatment neighbor’s factual outcome.Nearest-neighbor matching uses the nearest control or treated unit as the counterfactual reference.

3.6 Multitask Learning Methods

Multitask causal models share information across treatment groups while retaining treatment-specific components. The survey covers Bayesian uncertainty modeling, selection-bias mitigation, and extensions to multiple treatments with continuous dosage.

  • Multitask Learning Methods: Causal inference can be formulated as multitask learning with shared layers for both treatment groups and separate layers for each group.A propensity-dropout regularization scheme is used to alleviate selection bias.
  • Multitask Learning Methods: A multitask Gaussian process places a nonparametric Bayesian prior over vector-valued functions and provides individualized pointwise credible intervals.These intervals are described as important for precision medicine.
  • Multitask Learning Methods: Risk-based empirical Bayes adaptation jointly minimizes factual-outcome error and uncertainty in counterfactual outcomes.The method adapts the multitask Gaussian-process prior to address selection bias.
  • Multitask Learning Methods: The multitask architecture extends to multiple treatments with continuous dosage through shared base layers, treatment-specific layers, and dosage-associated heads.Shared layers train on all samples, while treatment layers train on samples from their respective categories.

3.7 Meta-Learning Methods

Meta-learning methods separate confounder control from CATE estimation into distinct stages, with T-, S-, X-, and R-learners offering different ways to construct treatment-effect estimates.

  • Overview: Meta-learning algorithms separate confounder control and CATE estimation into two steps.Unlike methods that address both requirements simultaneously, meta-learning treats them as separate tasks.
  • T- and S-learners: T-learner trains separate base models for treated and control outcomes, then subtracts their predictions to estimate CATE.Its two models estimate µ1(x) and µ0(x), with τT(x) = µ1(x) − µ0(x).
  • T- and S-learners: S-learner incorporates treatment assignment as a feature in one combined outcome model and contrasts predictions under treatment values.The model estimates µ(x,w), producing the CATE by comparing its treated and control predictions.
  • X-learner: X-learner uses cross-group information to improve treatment-effect estimation when treated and control groups are highly unbalanced.It imputes treatment effects, trains group-specific effect models, and combines them with a weighting function.
  • R-learner: R-learner constructs its CATE loss from the Robinson transformation rather than the regular X-learner loss.The transformation uses pre-trained outcome and propensity-score estimators under ignorability, with regularization on the CATE function.

4 METHODS RELAXING THREE ASSUMPTIONS

The survey reviews causal-inference methods for settings where SUTVA, ignorability, or positivity may fail, including dependent data, treatment versions, hidden confounding, and limited overlap.

  • SUTVA: SUTVA may fail for networked, time-series, or other non-i.i.d. data because units can be dependent or influence one another.Interference occurs when one subject’s treatment affects another subject’s outcome, complicating causal-parameter identification.
  • SUTVA: Graph convolutional networks can represent network features and confounders before inferring potential outcomes in networked data.The approach maps subject features and network structure into a representation space.
  • SUTVA: Continuous dosage parameters and dynamic treatment regimes violate the single-treatment-version aspect of SUTVA.Dynamic regimes use sequential decision rules and support individualized treatment choices such as precision medication.
  • SUTVA: Q-learning and A-learning estimate optimal dynamic treatment regimes from observational data using different regression-modeling strategies.Q-learning models outcome quality at each decision point, whereas A-learning models only part of the relevant regression.
  • Ignorability: Ignorability is difficult to satisfy because collecting all background variables, especially hidden confounders, is generally impossible.The assumption requires treatment assignment to be independent of potential outcomes conditional on observed covariates.
  • Ignorability: Approaches to hidden confounding include variational autoencoders, predictive representations, combined experimental-observational data, substitute confounders, and instrumental variables.These methods aim to infer, control, correct, or bypass confounding under weaker or alternative assumptions.

5 GUIDELINE ABOUT EXPERIMENT

The survey recommends using benchmark datasets and open-source implementations while recognizing that counterfactual outcomes are unobservable and many benchmarks are semi-synthetic.

  • Resources: Experimental resources include commonly used causal-inference datasets and open-source code for methods reviewed in the survey.Toolboxes list supported methods and programming languages, while separate tables list code for individual methods.
  • Benchmark design: Because counterfactual outcomes are unobservable, observational benchmarks rarely provide perfectly realistic ground-truth treatment effects.Many studies therefore use semi-synthetic datasets generated from randomized or real-world covariates.
  • Benchmark datasets: IHDP is a benchmark derived from a randomized infant-health experiment, with 25 pretreatment covariates describing children and mothers.Observed outcomes are generated and biased subsets are removed to mimic observational settings.
  • Benchmark datasets: Jobs combines Lalonde experiment data with a PSID comparison group to study job-training effects on employment.Its pretreatment variables include age, education, ethnicity, and prior earnings.
  • ACIC challenges: ACIC challenges vary observational-study difficulty through nonlinearities, sparsity, targeted selection, censoring, scaling, and dimensionality.The challenges include tasks for CATE and ATE estimation across datasets with differing sample and variable sizes.
  • Benchmark datasets: Networked, dosage, time-series, and randomized benchmarks cover BlogCatalog, Flickr, News, MVICU, TCGA, yeast, and STAR settings.These datasets represent social networks, multiple treatments or doses, longitudinal gene expression, cancer treatment, and class-size experiments.

6 APPLICATIONS

Causal inference supports decision evaluation, counterfactual estimation, and selection-bias correction across advertising, recommendation, medicine, and policy-oriented applications.

  • Application categories: Applications of treatment-effect estimation fall into decision evaluation, counterfactual estimation, and dealing with selection bias.These directions connect causal estimates to evaluating interventions, comparing alternative decisions, and improving population-level generalization.
  • Advertising: Advertising studies estimate whether campaigns change clicks or sales, often using observational data because randomized experiments are costly and time-consuming.Complex advertisements may involve discrete or continuous, unidimensional or multidimensional treatments.
  • Advertising: Small randomized datasets can be combined with large control datasets to address advertising settings with limited experimental data.The cited approach connects the datasets using a minimal set of modeling assumptions.
  • Advertising: Selection bias from advertisement display policies can make click prediction inaccurate and cause revenue loss if ignored.Propensity-score-based and propensity-score-free methods are discussed for correcting this bias.
  • Recommendation: Recommendation exposure can be treated as a treatment, while self-selection makes recommendation data unrepresentative of the target population.Propensity-score weighting provides an unbiased quality estimate, with extensions addressing variance, sparsity, and doubly robust estimation.
  • Recommendation: CausalEmbed uses a small randomized recommendation dataset to correct selection bias in large logged-feedback datasets.The method applies a matrix-factorization approach to combine control-policy and randomized data.
  • Medicine: Medical applications seek personalized treatment rules while accounting for observed and unobserved confounders.These rules can be derived from observational data or combined experimental and observational data, although unobserved confounding remains difficult.
  • Reinforcement learning: Individual treatment-effect estimation can be viewed as a contextual multi-armed bandit problem, linking treatments to actions and outcomes to rewards.Offline policy evaluation commonly uses importance sampling, but high variance motivates doubly robust alternatives.

7 CONCLUSIONS

The survey organizes potential-outcome causal inference methods by whether they rely on all three framework assumptions, while comparing methods and resources across applications.

  • Methods are divided into categories according to whether they rely on all three potential outcome framework assumptions.
  • The survey provides thorough discussions, comparisons, and summaries of methods in both categories.
  • Benchmark datasets and open-source codes are listed to support exploration and evaluation of causal inference methods.
  • Representative applications include advertising, recommendation, medicine, and reinforcement learning.
Loading 2002.02770v1…