PaperlayerView exact revision on arXiv

Source-linked AI summary

Matching Methods for Causal Inference: A Review and a Look Forward

Elizabeth A. Stuart

arXiv:1010.5586v1stat.ME

TL;DR

Matching-method research and guidance are scattered across disciplines, leaving no single overview for researchers. This paper synthesizes the literature and practical advice, emphasizing balance diagnostics and combining matching with regression adjustment.

  • Problem

    Matching-method research and applied guidance are scattered across disciplines, with no single overview for researchers selecting and implementing these methods.

  • Method

    The paper coalesces research across disciplines and structures guidance around matching choices, balance diagnostics, and subsequent outcome analysis.

  • Results

    The review summarizes available matching methods and advises assessing covariate balance before outcome analysis, generally combining matching with regression adjustment.

  • Takeaways & Limitations

    Applied researchers can use matching to diagnose insufficient overlap and design balanced samples before estimating effects with outcome models.

  • Takeaways & Limitations

    Selecting among matching methods remains difficult because defining the best balance requires trading off balance across multiple covariates.

Abstract

from arXiv · show

When estimating causal effects using observational data, it is desirable to replicate a randomized experiment as closely as possible by obtaining treated and control groups with similar covariate distributions. This goal can often be achieved by choosing well-matched samples of the original treated and control groups, thereby reducing bias due to the covariates. Since the 1970s, work on matching methods has examined how to best choose treated and control subjects for comparison. Matching methods are gaining popularity in fields such as economics, epidemiology, medicine and political science. However, until now the literature and related advice has been scattered across disciplines. Researchers who are interested in using matching methods---or developing methods related to matching---do not have a single place to turn to learn about past and current research. This paper provides a structure for thinking about matching methods and guidance on their use, coalescing the existing research (both old and new) and providing a summary of where the literature on matching methods is now and where it should be headed.

1. INTRODUCTION

Matching methods seek to make treated and control groups comparable on observed covariates in observational studies, approximating a key benefit of randomized experiments. This review consolidates scattered methodological research, explains the implementation process, and offers guidance for applied researchers and future work.

  • Review contribution: The expanding literature lacked a single overview of available methods, techniques, and practical advice because relevant research was scattered across disciplines.The review addresses this gap by bringing together research from statistics, epidemiology, sociology, economics, and political science.
  • Definition and context: Matching methods aim to balance covariate distributions between treated and control groups, using approaches such as 1:1 matching, weighting, or subclassification.The paper situates matching within the broader design of nonexperimental studies.
  • Motivation: Matching complements regression adjustment, highlights regions with insufficient covariate overlap, and can clarify how much causal information the data provide.These advantages distinguish matching from treating it as an alternative that conflicts with regression-based adjustment.
  • Scope: The paper focuses on treatments assigned at a defined time, pre-treatment covariates, and post-treatment outcomes, excluding more complex longitudinal treatment settings.Marginal structural models and balanced risk set matching are identified as useful methods for settings outside this scope.
  • Causal framework: Strongly ignorable treatment assignment requires treatment to be independent of potential outcomes given covariates, together with a positivity condition.This assumption provides the framework for causal comparisons in nonexperimental studies.
  • Methodological development: Exact multivariate matching can be difficult: among 671 treated and 523 control subjects, only 23 pairs matched exactly on six categorical covariates.The propensity score, introduced in 1983, addressed computational and data challenges in matching on multiple covariates.

2. DEFINING CLOSENESS

Defining closeness requires selecting covariates and combining them into a distance measure. Covariate selection should preserve ignorability, while distance choices must balance similarity against retaining enough matched individuals.

  • Covariate selection: Include variables related to both treatment assignment and outcome to support the ignorability assumption.Ignorability assumes no unobserved treatment-control differences conditional on observed covariates.
  • Covariate selection: Exclude variables affected by treatment; if adjustment is critical, omit them from matching and include them in the outcome model.This concern is especially important when covariates, treatment, and outcomes are measured simultaneously.
  • Covariate selection: Variables nearly predictive of treatment assignment should be excluded only with strong justification, because they may prevent separating the treatment effect.Exclusion requires believing the variable is unrelated to outcomes and that ignorability still holds.
  • Distance measures: Propensity scores reduce matching to one scalar and justify matching on that score when treatment assignment is ignorable given covariates.The propensity score is the probability of treatment given observed covariates; exact and Mahalanobis matching can perform poorly in high dimensions.
  • Distance measures: Caliper matching combines propensity-score proximity with especially close matching on key covariates.The key covariates often include pretreatment outcome measures, such as baseline test scores.
  • Propensity-score estimation: Propensity-score models should be assessed by covariate balance rather than conventional predictive fit or parameter diagnostics.Logistic regression is common, while boosted CART and generalized boosted models can also perform well.

3. MATCHING METHODS

Matching methods use a selected distance measure to form matched samples, and they primarily differ in how many individuals remain and how individuals are weighted.

  • Method overview: Matching begins by applying a selected distance measure to choose matched individuals.The paper introduces this step as the basis for comparing available matching methods.
  • Method overview: The available methods span a spectrum defined primarily by the number of individuals retained after matching.Different methods leave different numbers of individuals in the matched sample.
  • Method overview: Matching methods also differ in the relative weights assigned to different individuals.The section identifies individual weighting as a second primary dimension along which methods vary.

3.1 Nearest Neighbor Matching

Nearest neighbor matching is a common, accessible approach that usually estimates the ATT by matching controls to treated individuals and discarding unused controls. Its variants trade off match quality, bias, variance, and inference complexity depending on competition for controls and reuse of matches.

  • Nearest Neighbor Matching: Nearest neighbor matching is common and easy to implement, generally estimates the ATT, and is especially effective when selecting individuals for follow-up.It matches control individuals to the treated group and discards controls not selected as matches.
  • Nearest Neighbor Matching: In 1:1 matching, each treated individual is paired with the control having the smallest distance, although many observations may be discarded.Despite this concern, the reduction in power is often minimal.
  • Nearest Neighbor Matching: Optimal matching minimizes a global distance measure and avoids order dependence, but greedy matching performs poorly under intense control competition and well when competition is limited.Optimal matching generally produces similarly balanced groups while reducing within-pair distances more effectively.
  • Nearest Neighbor Matching: Ratio matching uses multiple controls per treated individual when many good controls exist, increasing bias while potentially reducing variance.The additional controls are farther from treated individuals than the closest match, creating a bias:variance trade-off.
  • Nearest Neighbor Matching: Matching with replacement can reduce bias and eliminate order dependence when comparable controls are scarce, but repeated controls complicate inference because matched observations are not independent.Frequency weights are one example of an adjustment needed in outcome analysis.

3.2 Subclassification, Full Matching and Weighting

Subclassification, full matching, and weighting use all individuals rather than discarding some controls, offering approaches to estimate the ATE or ATT. Subclassification can substantially reduce covariate-induced bias, while weighting requires attention to extreme-weight variance.

  • Weighting: Weighting, full matching, and subclassification use all individuals, unlike k :1 nearest-neighbor matching, which can discard controls.Weighting can assign individuals implicit or explicit weights to construct comparison groups.
  • Subclassification: Subclassification groups similar individuals, often using propensity-score quintiles, and can estimate either the ATE or ATT.Its historical use includes subclassification on age in studying lung cancer and smoking.
  • Full matching: Full matching automatically selects subclasses and forms matched sets containing at least one treated and one control individual, estimating either the ATE or ATT.Each matched set may contain many individuals from either group.
  • Weighting: IPTW uses propensity scores as inverse weights for ATE estimation, whereas odds weighting estimates the ATT by weighting both groups to represent the treatment group.IPTW weights treated and control groups up to the full sample; odds weighting gives treated individuals a weight of 1 and reweights controls.
  • Weighting: Extreme propensity-score weights can produce very large variance, motivating weight trimming when scores approach 0 or 1.The passage notes that large variance may be appropriate under a correctly specified model but may also reflect estimation rather than true probabilities.

3.3 Assessing Common Support

Assessing common support is essential because nonoverlapping propensity-score or covariate distributions can make some subjects inappropriate comparisons and affect which causal estimands can be reliably estimated.

  • 3.3 Assessing Common Support: Common support requires overlap between treated and control covariate distributions; subjects outside the other group’s range may be inappropriate comparisons.Limited overlap can arise when many controls differ substantially from all treated individuals, creating concerns for estimating causal effects.
  • 3.3 Assessing Common Support: Researchers commonly assess support by trimming propensity-score values outside the other group’s range or examining the covariates’ convex hull.The convex-hull approach identifies multidimensional regions permitting interpolation rather than extrapolation.
  • 3.3 Assessing Common Support: Common-support restrictions can change the estimand: limited overlap may prevent reliable ATE estimation, whereas discarding out-of-range controls may benefit ATT estimation.For the ATE, controls outside the treated range lack credible untreated-to-treated outcome comparisons without extensive extrapolation.

4. DIAGNOSING MATCHES

Matching must be followed by diagnosing covariate balance, defined as similarity between the empirical covariate distributions of matched treated and control groups. Poorly balanced matches should be rejected, while diagnostics should assess distributional overlap using numerical and graphical summaries aligned with the planned outcome analysis.

  • Balance assessment: Assess balance after every match by comparing the empirical distributions of the full covariate set in matched treated and control groups.The target is treatment unrelated to covariates, represented by equal empirical distributions.
  • Balance assessment: Reject highly imbalanced matches and try alternatives; diagnostics may reveal that reliable estimation requires implausibly strong modeling assumptions when groups are too dissimilar.
  • Numerical diagnostics: Compute balance measures in the same way as the planned outcome analysis, including subclass-specific aggregation and analysis weights when applicable.This applies to subclassification, IPTW, variable-ratio matching, and full matching.
  • Numerical diagnostics: For regression adjustment, absolute standardized mean differences should be below 0.25 and propensity-score variance ratios should lie between 0.5 and 2.These guidelines are presented as thresholds for trustworthy regression adjustment.
  • Numerical diagnostics: Do not use sample-size-dependent hypothesis tests or p-values as balance measures or stopping rules for selecting matched samples of varying size.Reduced power can make randomly discarding controls appear to improve balance.
  • Graphical diagnostics: Graphical diagnostics complement numeric checks by examining propensity-score overlap, QQ plots for continuous covariates, weighted boxplots, and standardized-difference plots.QQ plots compare corresponding empirical quantiles, while standardized-difference plots show whether balance improved across covariates.

5. ANALYSIS OF THE OUTCOME

After matching creates adequately balanced treated and control groups, researchers proceed to outcome analysis, generally using regression adjustments tailored to the matching structure. Weighting and subclassification require corresponding weighted analyses, while variance estimation remains debated despite evidence that estimated propensity scores can improve precision when matching nearly eliminates bias.

  • General outcome analysis: Matching methods design balanced treated and control groups but are not themselves causal-effect estimators; outcome analysis generally uses regression adjustments on the matched samples.The modeling details depend on the matching structure, and matching is not intended to compete with outcome modeling.
  • General outcome analysis: Weights enter outcome models directly for weighting methods, including weighted least squares, while permutation-based tests such as Fisher’s exact test are also appropriate.The paper focuses on parametric models because they are most commonly used.
  • Weighted matching: Matching with replacement assigns controls frequency weights based on reuse, whereas variable-ratio matching weights controls proportionally to the number matched to their treated individual.These weights are needed in the subsequent analysis for both matching designs.
  • Subclassification: Subclassifications estimate effects within subclasses and aggregate them; weighting by treated counts targets the ATT, while weighting by all individuals targets the ATE.Residual imbalance within subclasses makes regression adjustment important, and full matching can model subclass-specific treatment effects while assuming a common covariate-outcome relationship across subclasses.
  • Variance estimation: Variance estimation remains debated, but when matching on true or estimated propensity scores achieves nearly 100% bias reduction, estimated scores can produce more precise average treatment-effect estimates.The paper reports approximately half the variance of corresponding random-sample matched-mean differences with large control pools when matching on estimated linear propensity scores.

6. DISCUSSION

The discussion addresses missing covariates, unobserved confounding, method selection, and multiple treatment levels as unresolved issues in matching. It concludes with practical guidance for choosing covariates, distance measures, estimands, matching methods, and balance diagnostics, while identifying open research questions.

  • Missing data: Missing covariates can be handled with generalized boosted models or single imputation plus missingness indicators, but balance diagnostics for missing and nonignorable data mechanisms need further development.Qu and Lipkovich (2009) report good results for an adaptation that also includes indicators of missing data patterns in the propensity score model.
  • Unobserved confounding: Sensitivity analyses assess how plausible unobserved confounding could alter treatment-effect conclusions because ignorability cannot be directly tested.Approaches examine effects on variables unaffected by treatment or the correlations an unobserved covariate would need with treatment assignment and outcome to eliminate the observed effect.
  • Choosing among methods: Applied researchers have many matching methods but limited selection guidance, with balance-based choice complicated by trade-offs across multiple covariates.The discussion identifies selecting the method yielding the best balance as the primary advice while noting that defining best balance is complex.
  • Multiple treatment levels: For multiple treatment levels, generalized propensity functions can balance covariates through low-dimensional adjustment, although diagnostics become more complicated.Imai and van Dyk generalize propensity scores to ordinal, categorical, and multidimensional treatment regimes and advocate subclassification rather than matching.
  • Practical guidance: Researchers should carefully select covariates, estimate a distance measure, assess common support and the estimand, implement matching, and examine resulting covariate balance.The recommended workflow uses balance to determine whether to proceed with regression adjustment, refine matching, change the method, or reconsider the estimand or data.
  • Practical guidance: Matching diagnostics remain valuable even without matching-based effect estimates because standard regression diagnostics do not reveal insufficient overlap reliably.Estimating propensity scores and assessing balance before and after matching can help determine whether the data answer the question of interest.

7. SOFTWARE APPENDIX

Recent advances have made advanced matching methods more accessible, and the appendix catalogs major software procedures across R, Stata, and SAS. The listed tools support varied matching designs, diagnostics, effect estimation, weighting, and sensitivity analysis.

  • Software availability: Recent advances have made advanced matching methods increasingly accessible, with a continuously updated online software list.Earlier software limitations had made advanced methods difficult to implement.
  • Matching software for R: R software includes coarsened exact, propensity-score, nearest-neighbor, Mahalanobis, caliper, exact, full, optimal, and subclassification matching.Packages also provide balance diagnostics, effect and variance estimation, weighting, propensity-score estimation, and sensitivity analysis.
  • Matching software for Stata: Stata software supports k :1, radius, subclassification, full Mahalanobis, propensity-score, and kernel-weighted matching, with ATT or ATE estimation.The listed procedures also include common-support graphing, covariate-imbalance testing, robust variance estimation, diagnostics, and sensitivity analyses.
  • Matching software for SAS: SAS resources provide greedy, optimal, Mahalanobis-within-propensity-score-caliper, and variable-ratio matching procedures.The listed macros and procedures match one or more controls to cases and include variable optimal matching.
Loading 1010.5586v1…