Source-linked AI summary
DoWhy: An End-to-End Library for Causal Inference
Amit Sharma, Emre Kiciman
TL;DR
Causal inference requires explicit assumptions and robustness checks, but many libraries focus mainly on statistical estimation. DoWhy addresses this gap with a causal-graph-centered API covering modeling, identification, estimation, and refutation, while supporting external estimators and multiple robustness tests.
Problem
Many causal-inference libraries focus on statistical estimators while leaving assumption modeling, identification, and robustness testing to analysts.
Method
DoWhy uses causal graphs and a four-step Model, Identify, Estimate, and Refute API, integrating graphical-model and potential-outcomes approaches.
Results
DoWhy provides an end-to-end, extensible causal-inference library that models assumptions, identifies effects, estimates them, and tests robustness through methods including placebo, bootstrap, and unobserved-confounding checks.
Takeaways & Limitations
DoWhy gives analysts a unified interface for explicit causal assumptions, multiple estimation methods, and robustness assessment across a causal analysis.
Abstract
from arXiv · showhide
In addition to efficient statistical estimators of a treatment's effect, successful application of causal inference requires specifying assumptions about the mechanisms underlying observed data and testing whether they are valid, and to what extent. However, most libraries for causal inference focus only on the task of providing powerful statistical estimators. We describe DoWhy, an open-source Python library that is built with causal assumptions as its first-class citizens, based on the formal framework of causal graphs to specify and test causal assumptions. DoWhy presents an API for the four steps common to any causal analysis---1) modeling the data using a causal graph and structural assumptions, 2) identifying whether the desired effect is estimable under the causal model, 3) estimating the effect using statistical estimators, and finally 4) refuting the obtained estimate through robustness checks and sensitivity analyses. In particular, DoWhy implements a number of robustness checks including placebo tests, bootstrap tests, and tests for unoberved confounding. DoWhy is an extensible library that supports interoperability with other implementations, such as EconML and CausalML for the the estimation step. The library is available at https://github.com/microsoft/dowhy
1 INTRODUCTION
DoWhy addresses the difficulty of making causal assumptions explicit and testing them, rather than focusing only on effect estimation. Its four-step API models, identifies, estimates, and refutes causal effects while integrating graphical and potential-outcomes frameworks.
- Motivation: DoWhy targets the challenges of translating domain knowledge into causal graphs and validating assumptions when causal tasks lack ground-truth answers.The paper emphasizes sensitivity testing and assumption checks as critical for confidence in causal results.
- DoWhy: Its API organizes causal analysis into Model, Identify, Estimate, and Refute steps.Model encodes assumptions in a causal graph; Identify finds the causal effect; Estimate applies statistical methods; Refute tests robustness to model assumptions.
- Novelty: DoWhy differs from estimation-focused causal libraries by covering the path from data to a causal estimate and a measure of its robustness.Existing libraries often assume analysts already specified a causal model, identified the estimand, and conducted sensitivity checks.
- Framework: DoWhy combines graphical models for assumptions and identification with potential-outcomes methods for estimation.It also supports interoperability with EconML and CausalML estimators.
- Contributions: The library provides principled causal-graph modeling with explicit assumptions and identification of desired causal effects.This makes the assumptions underlying the analysis explicit.
- Contributions: DoWhy offers a unified interface for popular estimation methods across graphical-model and potential-outcomes frameworks.The interface is designed to combine the two major causal-inference frameworks.
- Contributions: DoWhy automatically tests causal-assumption validity when possible and assesses estimate robustness to violations.Robustness assessment is part of the library’s stated contribution.
2 DOWHY AND THE FOUR STEPS OF CAUSAL INFERENCE
DoWhy implements a four-step causal-analysis pipeline that separates causal modeling and identification from statistical estimation, then applies refutation methods to test robustness. The pipeline supports multiple identification and estimation approaches, including integrations with external estimator libraries.
- I. Model the causal question: DoWhy first creates a causal graphical model so assumptions are explicit, while allowing analysts to provide only a partial graph.The library treats remaining variables as potential confounders.
- II. Identify the causal estimand: It identifies causal estimands from the graph using graph-based criteria and do-calculus.Supported criteria include back-door, front-door, instrumental variables, and mediation identification.
- III. Estimate the causal effect: The estimation step supports propensity, outcome-model, instrumental-variable, front-door, and mediation methods, plus confidence intervals and permutation tests.The listed methods include propensity score matching, inverse propensity weighting, linear models, two-stage least squares, regression discontinuity, and two-stage linear regression.
- III. Estimate the causal effect: DoWhy can directly call EconML and CausalML estimators, including machine-learning estimators for conditional average treatment effects.The paper motivates such integrations for data with high-dimensional confounders.
- IV. Refute the obtained estimate: Refutation validates effect estimates through multiple methods, including placebo treatment, dummy outcomes, simulated outcomes, and added common causes.These tests target either the full causal analysis or specific steps such as estimation; data-subset and bootstrap validation test estimator stability.
3 AN EXAMPLE CAUSAL ANALYSIS
DoWhy structures causal analysis as four independent steps, from building a causal model through identification, estimation, and refutation. Its interface separates causal identification from statistical estimation while supporting external estimators and robustness checks.
- 3 AN EXAMPLE CAUSAL ANALYSIS: DoWhy represents a causal analysis through four steps: Model, Identify, Estimate, and Refute.The example begins with a causal model, identifies target estimands, estimates them statistically, and refutes the obtained estimate with robustness checks.
- 3 AN EXAMPLE CAUSAL ANALYSIS: Identification is treated as a causal problem, while estimation is treated as a statistical problem.This separation allows multiple estimation methods for one identified estimand and supports using available statistical estimators for a target estimand.
- 3 AN EXAMPLE CAUSAL ANALYSIS: DoWhy supports machine learning-based estimators such as double machine learning for data with high-dimensional confounders.The example uses an EconML double machine learning estimator with gradient-boosting nuisance models and a LassoCV final model.
- 3 AN EXAMPLE CAUSAL ANALYSIS: The library makes sensitivity and robustness checks part of the workflow for evaluating obtained estimates against unverified assumptions.These checks are presented as a critical part of causal analysis that is often skipped.
- 3 AN EXAMPLE CAUSAL ANALYSIS: The four-step design is extensible, allowing implementations from other libraries to coexist within the workflow.DoWhy provides example notebooks for applying the API to different causal problems.
4 CONCLUSION
DoWhy is an extensible, end-to-end causal-inference library that helps analysts devise causal models, test their assumptions, and estimate causal effects. The authors identify extending refutation analyses and adding estimation methods as future directions.
- 4 CONCLUSION: DoWhy combines causal-model design, assumption testing, and causal-effect estimation in an extensible end-to-end library.The conclusion contrasts this focus with libraries centered mainly on estimation.