PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Synthetic Control

Muhammad Jehangir Amjad, Devavrat Shah, Dennis Shen

arXiv:1711.06940v1econ.EMstat.APstat.ML

TL;DR

The paper addresses synthetic control when observations are noisy or missing and covariate or expert information is limited. It de-noises the data with singular value thresholding, learns a regularized donor relationship, and extends the method Bayesianly for uncertainty quantification. Theoretical results establish finite-sample guarantees and consistency, while experiments report improvement over classical synthetic control.

  • Problem

    Classical synthetic control lacks theoretical coverage for noisy or missing observations and may rely on covariate or expert information that is unavailable.

  • Method

    The method singular-value-thresholds the data to impute and de-noise observations, then uses regularized linear regression to estimate the synthetic control under a latent variable model.

  • Results

    The method establishes finite-sample MSE bounds and, with data aggregation, asymptotic consistency; experiments on synthetic and real-world data report improvement over classical synthetic control.

  • Takeaways & Limitations

    Robust synthetic control can estimate treatment counterfactuals under randomly missing or noisy data while remaining useful without covariates or expert donor selection.

  • Takeaways & Limitations

    The authors do not recommend abandoning supplementary covariates or domain knowledge, which can narrow the donor pool and guide missing-entry imputation.

Abstract

from arXiv · show

We present a robust generalization of the synthetic control method for comparative case studies. Like the classical method, we present an algorithm to estimate the unobservable counterfactual of a treatment unit. A distinguishing feature of our algorithm is that of de-noising the data matrix via singular value thresholding, which renders our approach robust in multiple facets: it automatically identifies a good subset of donors, overcomes the challenges of missing data, and continues to work well in settings where covariate information may not be provided. To begin, we establish the condition under which the fundamental assumption in synthetic control-like approaches holds, i.e. when the linear relationship between the treatment unit and the donor pool prevails in both the pre- and post-intervention periods. We provide the first finite sample analysis for a broader class of models, the Latent Variable Model, in contrast to Factor Models previously considered in the literature. Further, we show that our de-noising procedure accurately imputes missing entries, producing a consistent estimator of the underlying signal matrix provided $p = Ω( T^{-1 + ζ})$ for some $ζ> 0$; here, $p$ is the fraction of observed data and $T$ is the time interval of interest. Under the same setting, we prove that the mean-squared-error (MSE) in our prediction estimation scales as $O(σ^2/p + 1/\sqrt{T})$, where $σ^2$ is the noise variance. Using a data aggregation method, we show that the MSE can be made as small as $O(T^{-1/2+γ})$ for any $γ\in (0, 1/2)$, leading to a consistent estimator. We also introduce a Bayesian framework to quantify the model uncertainty through posterior probabilities. Our experiments, using both real-world and synthetic datasets, demonstrate that our robust generalization yields an improvement over the classical synthetic control method.

1 Introduction

The paper develops a robust synthetic control method for comparative case studies, targeting noisy or incomplete data and limited covariate information. It combines de-noising, theoretical guarantees, and empirical evaluation to improve counterfactual estimation.

  • Motivation: Prop. 63 motivates comparing California with a similar control state to estimate post-intervention violent-crime outcomes.Because only one California exists, randomized experimentation is unavailable and a statistically valid comparison requires a demonstrably similar control.
  • Main contributions: The algorithm first de-noises the data and then learns a linear relationship between the treated unit and donor pool.This two-step procedure is designed to identify a useful donor subset even without supplementary covariate information.
  • Robust algorithm: The method de-noises observations through singular value thresholding, which can impute missing values and reduce overfitting.Thresholding also provides a bias-variance tuning mechanism and may reduce dependence on auxiliary covariates, while still benefiting from useful domain information.
  • Theoretical performance: The paper provides finite-sample analysis, missing-data guarantees, graceful degradation with random missingness, and asymptotic consistency after data aggregation.These results address theoretical gaps beyond prior bias bounds that relied on perfect pre-treatment matches.
  • Theoretical performance: The paper proves conditions supporting continuation of the treatment-donor linear relationship after intervention and uses a latent variable model encompassing many factor models.The common structure is that these models induce approximately low-rank matrices suitable for the proposed method.
  • Experimental results: Experiments on real-world and synthetic data report similar performance without extensive covariates or expert donor selection and detect subtle intervention effects missed by classical synthetic control.The evaluation covers both settings where the underlying truth is known and existing case studies.

2 Background

The paper formulates synthetic control using time-series outcomes from a treatment unit and donor pool under a latent-variable model. Its key assumptions are a pre-intervention span relationship and randomly missing donor observations.

  • Data and notation: The data consist of N units observed over T periods, with T0 pre-intervention periods and T−T0 post-intervention periods.The first unit is designated as the treatment unit, while the remaining units form the donor pool.
  • Latent variable model: Each observed outcome combines a deterministic mean with independent zero-mean noise, while latent unit and time features determine the mean structure.The latent function formulation includes standard econometric factor models as a special case.
  • Counterfactual: The counterfactual post-intervention means are defined by extending the unaffected units’ model relationship to the treatment unit.This treats the untreated treatment unit as following the same latent relationship after the intervention.
  • Synthetic-control assumption: The synthetic control assumes the treatment unit’s pre-intervention mean vector lies in the span of donor mean vectors.The corresponding weights define the synthetic control and represent the treatment unit as a donor combination.
  • Missing data: The model allows donor-pool observations to be missing independently with probability 1−p, including settings where observation probabilities vary by row or column.The constant-p observation model is standard, but the stated results also accommodate dependent observation probabilities.

3 Algorithm

The robust algorithm de-noises a partially observed outcome matrix with singular value thresholding, then learns a regularized donor relationship for counterfactual prediction. A Bayesian extension quantifies uncertainty through posterior distributions.

  • Robust algorithm: The first step approximates the observation matrix with informative singular values while naturally imputing missing observations.The second step uses the de-noised pre-intervention data to learn the treatment-donor relationship before forecasting the counterfactual.
  • Weight learning: The method permits positive and negative donor weights by learning the synthetic control through linear regression rather than requiring a convex combination.This accommodates possible negative correlations between the treatment unit and donor units.
  • Hyperparameters: The thresholding parameter µ trades bias against variance, while the regularization parameter η controls model complexity.The algorithm uses both hyperparameters to tune prediction and limit overfitting.
  • Thresholding procedure: The algorithm computes an SVD, retains singular values at least µ, and constructs a de-noised estimator of the mean matrix.The observed-data matrix is formed with missing entries represented as zero before thresholding.
  • Bayesian extension: A Bayesian version estimates noise variance, computes posterior parameters, defines counterfactual means, and quantifies model uncertainty over time.The model uses a zero-mean isotropic Gaussian noise assumption and square loss.
  • Regularization: Regularization options include ridge regression and LASSO, with LASSO recommended when an interpretable sparse donor combination is desired.Regularization can simplify the fitted model and improve generalization despite potentially increasing training error.

4 Theoretical Results

Theoretical results establish guarantees for de-noising, prediction, consistency, extrapolation, and uncertainty quantification under noisy or incomplete observations. The analysis also characterizes regularization and threshold selection as central controls of estimation error.

  • Assumptions and targets: The analysis models the observation matrix as X = M + E with independent, zero-mean noise and bounded second moments, while evaluating estimation through ℓ2 error.The theoretical setup also assumes bounded matrix entries and examines whether pre-intervention linear relationships persist after intervention.
  • De-noising and imputation: Singular value thresholding de-noises observations, imputes missing entries, and adapts to the structure of M through a universal, data-driven threshold.The procedure is analyzed for low-rank or Lipschitz-generated signal matrices and is presented as an important filtering and imputation method.
  • Prediction error: The prediction bound exposes a bias-variance trade-off: larger thresholds can omit signal, whereas smaller thresholds can retain spurious noise and overfit.The threshold controls the balance between uncaptured signal and noise mistakenly interpreted as signal.
  • Prediction and consistency: Theoretical results support prediction under broad signal conditions, including settings that do not require M to be low rank or the generating function to be Lipschitz.The analysis also establishes asymptotic consistency through a computationally beneficial preprocessing aggregation step.
  • Regularization: Regularization adds a complexity penalty that trades higher pre-intervention bias for lower post-intervention variance and can reduce coefficient-estimation error.The paper discusses ridge regression and LASSO as mechanisms for controlling model complexity and avoiding overfitting.
  • Bayesian uncertainty: A Bayesian extension quantifies uncertainty through posterior probabilities and separates predictive inference from later decisions under different loss functions.The predictive distribution contains the information needed to construct Bayesian estimators, allowing decision rules to change without relearning that distribution.

5 Experiments

Experiments on Basque Country, California, and synthetic data show that robust synthetic control reproduces classical findings while remaining effective with missing data and without supplementary covariates. Thresholding improves estimation, but retaining too many singular values increases predictive uncertainty through overfitting.

  • 5.1 Basque Country: The robust method reproduced similar qualitative synthetic controls for Basque Country without the additional predictor variables used previously.Pre-treatment GDP matched the observed trajectory, while post-1975 divergence suggested a negative economic effect from terrorism.
  • 5.1 Basque Country: Removing de-noising caused predictions to deteriorate drastically, reinforcing thresholding’s role in controlling model complexity and overfitting.The experiments interpret the threshold as a model-complexity choice.
  • 5.1 Basque Country: Placebo comparisons showed the Basque Country’s post-intervention divergence was more extreme than those of the control regions.The Basque trajectory remained the most extreme divergence compared with other regions until about 1990.
  • 5.3 Synthetic simulations: Predictive uncertainty increased when more than two singular values were retained, illustrating the variance side of the bias–variance trade-off.Synthetic simulations likewise found high bias with too few singular values and high variance when noise was treated as signal.
  • 5.2 California Anti-tobacco Legislation: For California, the robust control matched pre-1988 cigarette consumption, diverged afterward, and remained robust to randomly missing data.The post-1988 divergence suggested that Proposition 99 reduced cigarette consumption.
  • 5.2 California Anti-tobacco Legislation: California was the most significant and consistent placebo outlier until about 1995, supporting a significant Proposition 99 effect.The conclusion was unchanged when the twelve states excluded in earlier work were removed.
  • 5.3 Synthetic simulations: Post-intervention MSE closely matched pre-intervention MSE across noise levels, and thresholding outperformed no thresholding across missing-data proportions.The simulations included settings with no missing data and 40% randomly missing data.

6 Conclusion

The paper presents robust synthetic control for missing data, noise, and absent covariate information. Its approach combines singular value thresholding with regularized regression and supports uncertainty quantification through a Bayesian counterpart.

  • The method targets randomly missing data and large noise while remaining effective without covariate or expert information.
  • Singular value thresholding imputes missing data and de-noises observations before regularized linear regression determines the synthetic control.
  • The latent variable model generalizes factor models and supports finite-sample MSE bounds for the estimated synthetic control.
  • A simple data aggregation method can yield an asymptotically consistent estimator when data are plentiful.
  • Experiments on synthetic and real-world case studies demonstrate improvement over the classical method.

A Useful Theorems

This section collects foundational inequalities and matrix facts used in the paper's analysis, together with assumptions governing observed donor data and forecasting.

  • Singular-value perturbation bounds control how matrix changes affect ordered singular values.
  • Poincaré separation relates eigenvalues of a projected symmetric matrix to those of the original matrix, including Cauchy interlacing as a special case.
  • Hoeffding and Bernstein inequalities provide concentration tools for bounded independent random variables.
  • The matrix concentration result requires δ2 ≥ n^-1+ζ, with a logarithmic relaxation also permitted.
  • Projection matrices formed from a matrix and its pseudoinverse are symmetric and idempotent, with eigenvalues only 0 or 1.
  • The analysis assumes bounded entries and models donor observations as independently observed with probability p.

D Imputation Analysis

The imputation analysis shows that thresholding can consistently recover the underlying mean matrix while imputing missing entries and filtering corrupted observations. The guarantees rely on structural assumptions and sufficient observation probability.

  • The de-noising procedure consistently estimates the underlying mean matrix, imputing missing entries and filtering corrupted observations.
  • A Lipschitz latent-variable model on a compact d-dimensional feature space yields a covering-number bound controlled by C(K, d, L)δ^-d.
  • The constructed approximation has at most |P(δ)| distinct rows and columns, implying rank(B) ≤ |P(δ)|.
  • The imputation theorem assumes p ≥ T^-1+ζ/(σ^2+1) for some ζ > 0 and uses a threshold defined earlier in the analysis.
  • The resulting error analysis combines concentration events, boundedness, and a total-probability argument to control estimation error.

D.1 Proof of Theorem 4.1

The proof of Theorem 4.1 applies the imputation result to a rank-k mean matrix under a lower bound on the observation probability.

  • Theorem 4.1 assumes that the underlying matrix M has rank k.
  • The theorem requires p ≥ T^-1+ζ/(σ^2+1) for some ζ > 0 and uses the threshold defined in equation (20).
  • The proof follows directly by applying Lemma D.2.

D.2 Proof of Theorem 4.2

The proof establishes the Lipschitz-function case of Theorem 4.2 and bounds the pre-intervention error for both unregularized and regularized estimators. It uses singular-value-based donor construction, independence of treatment noise from the donor pool, and projection-matrix identities.

  • Theorem 4.2 assumptions: Theorem 4.2 assumes an L-Lipschitz function and an observation fraction satisfying p ≥ T^-1+ζ for some ζ > 0.These are the conditions used for the Lipschitz-function imputation result.
  • Error analysis: The pre-intervention ℓ2 error is bounded to measure the estimator’s prediction power.The proof analyzes this error for both η = 0 and η > 0.
  • Error analysis: The donor-pool matrix excludes the treatment row, so its construction depends only on donor noise and is independent of the treatment noise.This independence makes the relevant cross terms vanish or factorize when taking expectations.
  • Error analysis: Projection-matrix and trace identities reduce quadratic error terms to rank-controlled expressions involving the estimated donor matrix.The projection has rank equal to rank(ˆQ), which is at most the donor subset size.
  • Regularized estimator: For η > 0, the proof derives the closed-form regularized estimator through the singular value decomposition of ˆQ.The regularized analysis parallels the η = 0 argument while incorporating the quadratic penalty.

E.3.1 Proof of Theorem 4.3

Theorem 4.3 combines the unregularized and regularized error analyses to give a common pre-intervention error bound for all η ≥ 0 and μ ≥ 0.

  • Theorem 4.3: Theorem 4.3 bounds the algorithm’s pre-intervention error for any η ≥ 0 and μ ≥ 0.The result follows by amalgamating the bounds established in Lemmas E.2 and E.5.

E.3.2 Proof of Corollary 4.1

Corollary 4.1 specializes the error analysis under the sampling and horizon conditions needed for the theorem, using the singular-value threshold and concentration events to obtain the stated bound.

  • Corollary 4.1: The corollary assumes p ≥ T^-1+ζ and T ≤ αT0 for a constant α > 1.Under these conditions, the pre-intervention error is bounded for every η ≥ 0.
  • Corollary 4.1: The final bound combines concentration events with Lemmas E.1 and E.5.The argument uses the event E = E1 ∩ E2 ∩ E3 and applies total-probability reasoning before invoking the earlier lemmas.

E.3.3 Proof of Theorem 4.4

Theorem 4.4 extends the analysis through block aggregation, controlling the rescaled noise and reapplying the earlier proof strategy to the block-partitioned matrices.

  • Theorem 4.4: Theorem 4.4 fixes γ ∈ (0, 1/2) and ω ∈ (0.1, 1), then defines the aggregation scale using Δ = T^(1/2+γ).Its proof follows the earlier corollary with block-partitioned matrices.
  • Block construction: Block aggregation preserves zero-mean noise while rescaling its variance for the treatment and donor rows.The variance bounds are derived separately for i = 1 and i > 1.
  • Block construction: Each block is assumed to contain at least one observed entry, allowing the block observations to equal the corresponding block-partitioned matrix entries.The subsequent analysis proceeds in the same manner using the newly defined variables and parameters.
  • Error analysis: The proof reuses concentration events and Lemmas E.1 and E.5 after replacing the original noise parameter with its block-aggregated counterpart.The resulting intermediate quantities include φ := p(N − 1) + σ^2 + q̄, with q̄ = q/τ.
  • Error analysis: The block-aggregated estimator’s post-intervention ℓ2 error is then bounded using the same analytical framework.The section concludes by initiating the post-intervention error analysis after establishing the block construction.

F.1 Proof of Theorem 4.5

The proof establishes the theorem by extending the rank-preservation argument as columns are added, then bounding post-intervention error under the stated sample-size condition. It also develops related error bounds and Gaussian posterior-parameter derivations.

  • Rank-preservation argument: rank(M −) = rank(M) allows the argument to extend from the initial matrix to matrices formed by adding columns.The proof proceeds by induction over the number of added columns.
  • Post-intervention bound: α′T0 ≤ T ≤ αT0 with α′, α > 1 is used to upper-bound the post-intervention error.The bound is stated for any η ≥ 0 and uses µ as defined in (20).
  • Post-intervention analysis: The post-intervention analysis applies triangle inequality, expectations, induced norms, and boundedness of ˆβ(η).Its first term parallels earlier pre-intervention bounds but uses ˆM + and M + instead of ˆM − and M −.
  • Post-intervention analysis: The quantities q = σ2p + p(1 −p) and φ = p(N −1) + σ2 + q are reused in the subsequent Jensen-inequality argument.The proof states that these definitions are unchanged from the earlier analysis.
  • Posterior-parameter derivation: The appendix derives Gaussian posterior parameters by forming the joint distribution and collecting quadratic and linear terms.The derivation treats Λ and Σ as precision matrices and obtains the posterior covariance through matrix inversion.
  • Posterior-parameter derivation: The Gaussian derivation expands the quadratic form, identifies the precision matrix, and then obtains the posterior mean from the collected linear terms.The displayed expressions include the terms involving Λ + A^TΣA and the covariance of z.
Loading 1711.06940v1…