PaperlayerView exact revision on arXiv

Source-linked AI summary

The Augmented Synthetic Control Method

Eli Ben-Michael, Avi Feller, Jesse Rothstein

arXiv:1811.04170v3stat.MEecon.EM

TL;DR

The paper addresses settings where SCM cannot achieve excellent pre-treatment fit. It introduces Augmented SCM, especially Ridge ASCM, to model and remove mismatch bias while penalizing extrapolation; simulations report large gains over alternative estimators, and the Kansas application finds a substantial negative effect of the 2012 tax cuts on economic output.

  • Problem

    Traditional SCM is advised against when a synthetic control cannot closely match pre-treatment outcomes, creating a need for an approach for infeasible-fit settings.

  • Method

    Augmented SCM uses an outcome model to estimate bias from imperfect pre-treatment fit and de-bias the SCM estimate, with Ridge ASCM regularizing extrapolation.

  • Results

    ASCM shows large gains over alternative estimators, especially under model misspecification, and estimates a substantial negative effect of the 2012 Kansas tax cuts on economic output.

  • Takeaways & Limitations

    Augmented SCM provides a framework for using synthetic-control-style estimation when excellent pre-treatment fit is not feasible.

  • Takeaways & Limitations

    Under a factor model with greater noise, the gains from augmentation relative to SCM are more limited.

Abstract

from arXiv · show

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in panel data settings. The "synthetic control" is a weighted average of control units that balances the treated unit's pre-treatment outcomes as closely as possible. A critical feature of the original proposal is to use SCM only when the fit on pre-treatment outcomes is excellent. We propose Augmented SCM as an extension of SCM to settings where such pre-treatment fit is infeasible. Analogous to bias correction for inexact matching, Augmented SCM uses an outcome model to estimate the bias due to imperfect pre-treatment fit and then de-biases the original SCM estimate. Our main proposal, which uses ridge regression as the outcome model, directly controls pre-treatment fit while minimizing extrapolation from the convex hull. This estimator can also be expressed as a solution to a modified synthetic controls problem that allows negative weights on some donor units. We bound the estimation error of this approach under different data generating processes, including a linear factor model, and show how regularization helps to avoid over-fitting to noise. We demonstrate gains from Augmented SCM with extensive simulation studies and apply this framework to estimate the impact of the 2012 Kansas tax cuts on economic growth. We implement the proposed method in the new augsynth R package.

1 Introduction

The paper proposes Augmented SCM for cases where traditional SCM cannot achieve excellent pre-treatment fit, using outcome-model bias correction while controlling extrapolation. Ridge ASCM improves fit through regularized, potentially negative weights, with theoretical error bounds, simulations, and a Kansas tax-cut application.

  • Motivation: Traditional SCM is recommended only when its synthetic control closely matches the treated unit’s pre-treatment outcomes.When good fit is infeasible, the original proposals advise against using SCM.
  • Augmented SCM: Augmented SCM estimates bias from imperfect pre-treatment fit with an outcome model, then de-biases the original SCM estimate.The method is intended as a middle ground between SCM and approaches that extrapolate more freely.
  • Ridge ASCM: Ridge ASCM uses ridge regression to control pre-treatment fit while penalizing distance from SCM weights.Unlike SCM, its modified synthetic-controls formulation permits negative weights and therefore extrapolation.
  • Theory: Under linear and latent-factor models, better pre-treatment fit reduces bias, while regularization negotiates bias-variance trade-offs and limits over-fitting risk.The penalty directly parameterizes the trade-off, making hyperparameter choice important in practice.
  • Extensions: The framework extends to auxiliary covariates by including them in SCM and outcome models or residualizing outcomes before fitting Ridge ASCM.The residualized approach controls estimation error under a linear factor model with auxiliary covariates.
  • Evidence and application: Simulations report large gains for ASCM over alternative estimators, especially under model misspecification, and the Kansas application finds a substantial negative effect of the 2012 tax cut on economic output.The methodology is implemented in the augsynth R package.

2 Overview of the Synthetic Control Method

SCM estimates a treated unit’s missing post-treatment counterfactual using non-negative donor weights chosen to balance pre-treatment outcomes. Its usefulness depends on achieving excellent pre-treatment fit, which is often infeasible in high-dimensional settings.

  • SCM imputes the treated unit’s missing post-treatment control outcome using a weighted average of control-unit outcomes.
  • The SCM weights are constrained to the simplex, enforcing non-negativity and supporting interpretability.
  • When the treated unit’s lagged outcomes lie inside the controls’ convex hull, SCM can achieve perfect pre-treatment fit.
  • Due to the curse of dimensionality, perfect or near-perfect pre-treatment fit is not always feasible under simplex-constrained weights.
  • The authors propose using a modified SCM approach in many cases where SCM alone is infeasible.

3 Augmented SCM

Augmented SCM corrects imperfect synthetic-control fit by modeling outcome differences associated with pre-treatment imbalance. Ridge regression provides a central implementation whose penalty yields a penalized synthetic-control formulation.

  • ASCM estimates bias from imperfect pre-treatment balance with an outcome model and adjusts the original SCM estimate.
  • The augmented estimator can use models linear in pre-treatment outcomes or comparison-unit outcomes, including ridge regression and other approaches.
  • More predictive pre-treatment periods receive larger adjustment when their imbalance contributes more strongly to the modeled post-treatment outcome.
  • Ridge-regularized linear augmentation produces an estimator that is itself a penalized synthetic-control estimator.
  • The resulting overall weights can be negative even though the original SCM weights remain simplex-constrained.

4 Ridge ASCM improves pre-treatment fit while penalizing extrapolation

Ridge ASCM improves pre-treatment fit by permitting controlled extrapolation beyond simplex-constrained SCM weights. Its ridge penalty regulates that extrapolation and balances fit against variance and over-fitting.

  • Ridge ASCM improves pre-treatment fit relative to SCM by allowing negative weights and extrapolation beyond the controls’ convex hull.
  • Ridge ASCM directly penalizes distance from sparse, non-negative SCM weights, with λridge controlling the amount of extrapolation.
  • When SCM achieves exact pre-treatment balance, Ridge ASCM and SCM weights are identical.
  • Ridge ASCM weights remain closer to the simplex than ridge-regression weights when both achieve comparable balance.
  • Ridge ASCM’s pre-treatment imbalance is at least a factor of λ d2m+λ smaller than SCM’s, except in specified corner cases.

5 Error under restrictions on the data generating processes

The paper analyzes Ridge ASCM estimation error under linear and latent-factor outcome models, showing how pre-treatment imbalance, extrapolation, noise, and regularization interact. Better fit can reduce bias, but latent factors and noisy pre-treatment outcomes create approximation and over-fitting risks.

  • Linear model: Under a linear model, improving pre-treatment fit directly reduces bias, while the Ridge ASCM penalty negotiates a bias-variance trade-off.The penalty controls the balance between fit and variance from larger weights.
  • Linear model: Exactly balancing lagged outcomes eliminates systematic error, and sparse outcome dynamics may require balancing only the lags with non-zero coefficients.For an AR(K) process, balancing the first K lags is sufficient.
  • Linear model: Proposition 1 bounds finite-sample error using imbalance in lagged outcomes and the L2 norm of the weights, with high-probability control over post-treatment noise.The result applies to Ridge ASCM and includes SCM as the λridge = ∞ special case.
  • Latent factor model: When SCM lacks perfect pre-treatment fit, finite-λ Ridge ASCM extrapolates beyond the convex hull to improve fit and reduce bias, unlike SCM’s perfect-fit equivalence case.If SCM achieves perfect pre-treatment fit, ASCM and SCM weights are equivalent and error comes from weight variance and post-treatment noise.
  • Latent factor model: Under a latent factor model, balancing observed pre-treatment outcomes need not balance latent factor loadings, creating additional approximation error.This approximation error, and thus the associated bias, converges to zero in probability as T0 →∞.
  • Latent factor model: Latent-factor estimation also faces over-fitting to noisy pre-treatment outcomes, so a long pre-period alone may not control imbalance error without excellent pre-treatment fit.A small amount of extrapolation can lower the error bound, but excessive extrapolation worsens it; high noise can make ASCM worse than SCM.
  • Latent factor model: The error bound improves with modest extrapolation, but further extrapolation eventually worsens it, especially when noise is large.Sufficient regularization can still yield a lower error bound than SCM, while lower noise permits larger augmentation gains.
  • Hyper-parameter selection: The paper proposes selecting the extrapolation penalty λ through cross-validation inspired by in-time placebo checks.The simulations examine the resulting trade-offs across multiple data-generating processes.

6 Auxiliary covariates

The paper extends ASCM to incorporate auxiliary covariates alongside lagged outcomes. A residualization-based two-step procedure exactly balances auxiliary covariates while regularizing extrapolation in residualized outcomes.

  • Incorporating auxiliary covariates: Auxiliary covariates can enter both the SCM balance objective and the outcome model used for ASCM augmentation.The combined setup assigns user-specified importance and regularization weights to covariate groups.
  • Incorporating auxiliary covariates: When auxiliary covariates are few relative to donors, the outcome model can regularize lagged-outcome coefficients while leaving auxiliary-covariate coefficients unregularized.This yields weights that perfectly balance the auxiliary covariates.
  • Residualization: A two-step procedure residualizes pre- and post-treatment outcomes on auxiliary covariates, then estimates Ridge ASCM on the residualized outcomes.The resulting optimization targets balance in residualized lagged outcomes rather than the original outcomes.
  • Residualization: After residualization, the method penalizes extrapolation in the residualized outcome space while allowing potentially large extrapolation in auxiliary covariates.This separates exact covariate balance from regularized control of the remaining lagged-outcome component.
  • Error properties: With a small number of auxiliary covariates, exact covariate balance and targeting residual imbalance can decrease pre-treatment-fit error with only a small increase in approximation and post-treatment-noise error.The appendix reports this result for a linear factor model when λridge →∞ and residualized weights do not extrapolate beyond the convex hull.
  • Error properties: With larger numbers of auxiliary covariates, incorporating them in parallel with lagged outcomes becomes less favorable.The supplied discussion identifies covariate dimension relative to the donor count as an important boundary.

7 Simulations and empirical illustrations

Simulations show that augmenting SCM, especially with ridge regression, reduces bias and RMSE when pre-treatment fit is imperfect. The Kansas application finds consistently negative estimated effects, while augmentation improves fit and covariate balance with limited extrapolation.

  • Simulation design: Extensive simulations compare SCM, ridge regression, Ridge ASCM, fixed effects, and de-meaned SCM across factor, noisy factor, fixed-effects, and AR(3) models.The study also evaluates alternative outcome models and augmented versions using the same data-generating processes.
  • Simulation results: Ridge augmentation reduces bias relative to SCM in all four simulations without conditioning on excellent pre-treatment fit.Under the baseline factor model and fixed-effects model, bias falls by more than 75% and over 90%, respectively; gains are more limited in the AR(3) and noisier factor models.
  • Simulation results: Ridge ASCM has slightly lower bias than ridge regression alone across all simulation settings and lower RMSE than SCM in every simulation.The RMSE improvement reflects the large bias decrease despite a slight increase in variance.
  • Regularization: Augmentation substantially reduces bias across SCM fit quartiles, but the best-fit quartile has non-monotonic bias in λ and an intermediate λ minimizes RMSE.Too-small λ can increase RMSE when pre-treatment fit is already good, indicating a regularization trade-off.
  • Alternative outcome models: Across alternative outcome models, augmenting SCM almost always reduces bias relative to SCM and nearly always outperforms outcome modeling alone.The gains from flexible outcome models are more limited when SCM pre-treatment fit is good, except with the oracle gsynth estimator.
  • Kansas application: Ridge ASCM improves pre-treatment fit and retains minimal extrapolation, with donor weights remaining similar to SCM and only Louisiana receiving a meaningful negative weight.Adding auxiliary covariates further improves pre-treatment fit and covariate balance, while residualization perfectly balances those covariates by design.

A Inference

This section studies conformal and jackknife+ inference for synthetic control estimators, characterizing when prediction intervals attain nominal coverage and examining their finite-sample performance. It also records computational and theoretical limitations of these procedures.

  • Conformal inference: Conformal inference constructs prediction sets for the treated unit’s post-treatment counterfactual outcome using pre-treatment residual quantiles.The procedure inverts a hypothesis test and treats the counterfactual outcome as random.
  • Validity: Approximate coverage is close to nominal when pre-treatment fit is good and approximation error is small.
  • Validity: Exact finite-sample validity requires exchangeable residuals or time periods, while factor-value differences across periods can violate the needed proxy relationship.
  • Jackknife+: Jackknife+ uses leave-one-out residuals and requires refitting for each pre-treatment period, unlike the computationally intensive full conformal approach.It incorporates estimator variability through leave-one-out post-treatment estimates.
  • Limitations: Extending approximate-validity results to non-exchangeable residuals is left for future work.
  • Simulation study: Simulations evaluate 95% full conformal and jackknife+ intervals for SCM and ridge ASCM, with SCM sometimes undercovering because of finite-sample bias.
  • Appendix results: For SCM, the appendix specializes Ridge ASCM results by taking λ →∞ and states corollaries under linear and linear-factor models with sub-Gaussian noise.

B.2 Error under a partially linear model with Lipshitz deviations from linearity

This section bounds estimation error under a partially linear model with Lipschitz deviations from linearity. The bounds emphasize balancing pre-treatment outcomes while keeping weighted donor units near the treated unit.

  • Model: Under Lipschitz deviations from linearity, moving away from the nearest-neighbor match creates approximation error.
  • Ridge ASCM: The Ridge ASCM error bound separates imbalance in covariates, excess approximation error, and SCM approximation error.
  • SCM: The SCM-only bound is obtained from the Ridge ASCM result by taking λ to infinity.
  • Weight choice: Controlling pre-treatment fit and restricting weights to nearby control units is sufficient to control estimation error.
  • Weight choice: The penalized SCM hyperparameter λ plays the role of L/∥β∥2, governing the relative importance of imbalance and proximity.

B.3 Error under a linear factor model with covariates

This section analyzes error under a linear factor model with auxiliary covariates and explains why fitting residualized lagged outcomes can control part of the estimation error.

  • Model: The covariate model allows time-varying functions of auxiliary covariates and decomposes deviations into linear approximation residuals.
  • Bounds: The covariate-weight bound separates imbalance in residualized outcomes, excess approximation error, and SCM approximation error.
  • Special case: When the covariate function is exactly linear, all approximation residuals are zero.
  • Error control: Under the covariate factor model, controlling fit in residualized lagged outcomes partially controls estimation error.
  • Error control: Targeting residualized rather than raw lagged outcomes is justified by the additive, separable structure of the auxiliary covariates.
  • Approximation: The excess approximation error is expected to be small when the number of covariates is small relative to the number of control units.

C Simulation data generating process

The simulations generate synthetic panel data from calibrated factor, fixed-effects, and autoregressive models, then estimate treatment effects under a sharp null. Selection mechanisms are tied to factors, fixed effects, or recent outcomes.

  • Linear factor model: The main simulation calibrates a linear factor model to log GSP per capita for 50 units over 89 pre-treatment and 105 post-treatment periods, using three factors.
  • Calibration: The simulations use estimated fixed effects, factor loadings, and residual variation to generate outcomes, including a setting with quadrupled noise standard deviation.
  • Treatment effect: All data-generating processes impose a sharp null of zero treatment effect and estimate the ATT at the final time point.
  • Selection: Treatment selection is modeled through marginal propensity scores based on factors and fixed effects, with one treated unit per simulation.
  • Autoregressive model: An alternative data-generating process uses an AR(3) outcome model and selection proportional to the last three outcomes.

D.1 Proofs for Section 4

The appendix derives finite-sample error bounds for SCM and ridge ASCM under linear, random, factor-model, and Lipschitz approximation structures. Ridge ASCM controls imbalance while adding an excess approximation-error term governed by its regularized weights.

  • General finite-sample bounds: Proposition A.1 decomposes estimation error into pre-treatment imbalance, approximation error, and post-treatment noise under independent sub-Gaussian errors.The bound applies to weights independent of post-treatment residuals.
  • Linear model: Under the linear model, ξ_i = 0, so the approximation-error component vanishes from the resulting bound.This specialization yields Corollary A.1.
  • Random approximation error: For mean-zero sub-Gaussian ξ_i, the appendix separately bounds approximation error and then combines it with noise and imbalance bounds.The ridge ASCM result uses λridge = λN0 and includes an excess approximation-error term.
  • Linear factor model: The linear factor model is represented as a special case of the general model, allowing the general error bounds to apply under factor-structured outcomes.The mapping controls the relevant regression coefficient norm using the factor structure.
  • Lipschitz approximation: For Lipschitz unit-specific terms, approximation error is bounded by pre-treatment outcome distance, with ridge ASCM adding a regularization-dependent term.The ridge-ASCM bound assumes simplex-related quantities and λridge = N0λ.

E Connection to balancing weights and IPW

The appendix connects SCM to balancing weights and inverse propensity-score weighting through a general primal-dual formulation. The balance criterion and dispersion penalty determine the corresponding regularization and propensity-score link, while SCM need not justify a propensity interpretation.

  • Balancing weights: SCM weights are approximate balancing weights that directly minimize L2 covariate imbalance, unlike propensity-score weights that minimize imbalance indirectly through a fitted score model.This places SCM within the broader balancing-weights literature.
  • Generalized formulation: A general balancing-weights problem removes non-negativity when permitted by the dispersion penalty and extends squared L2 balance to criteria such as L∞ constraints.Examples include ridge, entropy, and elastic-net penalties.
  • Primal-dual connection: The dual formulation recovers donor weights from the derivative of the convex conjugate of the dispersion penalty evaluated at an intercept-plus-linear predictor.Strong duality makes the primal balancing problem equivalent to the dual optimization.
  • IPW interpretation: For squared L2 balance, the dual propensity-score coefficients receive ridge regularization; entropy dispersion instead produces logistic-link inverse-propensity weights.The dual representation unifies several balancing-weight constructions.
  • IPW interpretation: The authors caution that an IPW-form dual does not always make propensity-score interpretation appropriate in SCM settings.The algebraic connection is therefore not automatically a substantive propensity model.

F Additional figures

The additional figures document tuning, estimated effects, donor weights, placebo estimates, simulation performance, and latent factors across SCM and augmented variants. They provide visual comparisons of fit, uncertainty, weighting, and estimator error without reporting outcomes in the supplied captions.

  • Tuning: Figure F.1 plots cross-validation MSE with one-standard-error bands and highlights both the minimum and the largest λ within one standard error.The figure supports the tuning choices used for augmented estimators.
  • Kansas application: Figures F.2 and F.3 plot point estimates and point-wise 95% conformal intervals for counterfactual GSP and tax-cut effects.They compare SCM, ridge ASCM, and ridge ASCM with covariates against observed outcomes where specified.
  • Kansas application: Figure F.4 compares ATT estimates with two-standard-error intervals across de-meaned SCM, ridge regression, ridge ASCM, SCM with covariates, and two-way fixed effects.The outcome is log GSP per capita.
  • Placebo analysis: Figures F.6 and F.7 show placebo point estimates with two-standard-error intervals for placebo treatment times in Q2 2009, 2010, and 2011.The scale begins in 2005 to highlight placebo estimates.
  • Donor weights: Figures F.8 and F.9 compare donor-unit weights for SCM, ridge regression, and ridge ASCM, including versions residualized on auxiliary covariates.Both figures focus on weights balancing lagged outcomes.
  • Simulation studies: Figures F.10 and F.12 plot RMSE across augmented and non-augmented estimators and outcome models, while Figure F.11 plots bias conditional on high SCM fit.Figure F.12 labels the conditioning group as the top quintile of SCM fit.
  • Simulation studies: Figure F.13 displays the latent factors used for calibrated simulation studies.The simulations use estimated factors and time fixed effects as inputs.
Loading 1811.04170v3…