PaperlayerView exact revision on arXiv

Source-linked AI summary

Combining multiple observational data sources to estimate causal effects

Shu Yang, Peng Ding

arXiv:1801.00802v4stat.ME

TL;DR

Observational causal-effect estimation is difficult when the large main dataset lacks confounders, while a smaller validation dataset observes them. The paper combines validation-based estimators with information from the main data; simulations and an application report improved efficiency while preserving the validation estimators’ consistency.

  • Problem

    Large observational datasets may lack important confounders, while sensitivity analyses do not provide point estimates and valid instrumental variables can be difficult to find.

    Source: §1 · p. 2

  • Method

    The framework uses the correlation between a consistent validation-data estimator and an error-prone main-data estimator to improve efficiency while retaining consistency.

    Source: §1 · p. 3

  • Results

    Simulations found the proposed estimators unbiased and more efficient than initial estimators, while the COPD application reported a 1.55% increase in HZ development on average.

    Source: §5 · p. 16, §6 · p. 18

  • Takeaways & Limitations

    Main-data information can improve the efficiency of validation-based causal-effect estimators without sacrificing their consistency.

    Source: §1 · p. 3

    The relaxed validation-sampling approach assumes independent selection with known inclusion probabilities conditional on (A, X, Y); otherwise, that selection model must be fitted.

    Source: §7 · p. 19

Abstract

from arXiv · show

The era of big data has witnessed an increasing availability of multiple data sources for statistical analyses. We consider estimation of causal effects combining big main data with unmeasured confounders and smaller validation data with supplementary information on these confounders. Under the unconfoundedness assumption with completely observed confounders, the smaller validation data allow for constructing consistent estimators for causal effects, but the big main data can only give error-prone estimators in general. However, by leveraging the information in the big main data in a principled way, we can improve the estimation efficiencies yet preserve the consistencies of the initial estimators based solely on the validation data. Our framework applies to asymptotically normal estimators, including the commonly-used regression imputation, weighting, and matching estimators, and does not require a correct specification of the model relating the unmeasured confounders to the observed variables. We also propose appropriate bootstrap procedures, which makes our method straightforward to implement using software routines for existing estimators.

1 Introduction

The section develops a way to combine large data missing some confounders with smaller validation data that observe them, aiming to improve efficiency while preserving consistency without requiring a correct model linking the unmeasured confounders to observed variables. It also develops variance estimation and bootstrap procedures for regular asymptotically linear and matching estimators, including a doubly robust correction for the AIPW variance estimator.

2.1 Notation: causal effect and two data sources

  • Validation data observe all confounders, while the larger main data omit U; the framework covers both two-phase sampling and linking a rich, smaller dataset to a larger dataset with fewer covariates.The formulation initially treats validation observations as a simple random sample from the main study population.Source: §1 · p. 2, §2.1 · p. 4

2.2 Identification and model assumptions

  • Under ignorability given (X, U) and adequate overlap, the average causal effect is identifiable and can be estimated using regression imputation, inverse probability weighting, augmented weighting, or matching.In the main data, treatment assignment is only latent ignorable because U is unobserved.Source: §2.2 · p. 5

3.1 Review of commonly-used estimators based on validation data

  • Validation-only estimators include regression imputation, weighting, and matching; the AIPW estimator is consistent if either its outcome or propensity score model is correct and locally efficient if both are correct.The validation data identify the effect under ignorability and overlap, but using them alone may be inefficient.Source: §1 · p. 3, §3.1 · pp. 5–6

3.2 A general strategy

  • The framework combines a consistent validation-data estimate with the difference between matching error-prone estimates from main and validation data, improving efficiency without modeling the relationship between U and (A, X, Y).It accommodates non-RAL estimators such as matching and uses standard software routines for existing methods.Source: §1 · p. 3, §3.2 · pp. 7, 9
  • The proposed estimator remains consistent and has smaller asymptotic variance than the validation-only estimator when the shared error-prone estimates converge to the same finite parameters and are associated with the initial estimate.Its asymptotic variance is v2 − Γ^T V^-1 Γ, compared with v2 for the validation-only estimator.Source: §3.2 · p. 8

3.3 Regular asymptotically linear (RAL) estimators

  • The paper first illustrates its general strategy with RAL estimators, whose validation-data estimates admit an asymptotic representation using an influence function.The passage defines the influence function as a mean-zero random vector with finite second moment.Source: §3.3 · p. 9
  • Under regularity conditions, Theorem 1 establishes (4) for RAL estimators (8) and (9), with variance components defined from their influence functions.The components are v2 = var{ψ(A, X, U, Y)}, Γ = (1−ρ)cov{ψ(A, X, U, Y), φ(A, X, Y)}, and V = (1−ρ)var{φ(A, X, Y)}.Source: §3.3 · p. 10
  • The proposed AIPW correction terms yield variance consistency under either Assumption 4 or 5; Lunceford and Davidian (2004)'s uncorrected form works only under Assumption 5.Without the correction terms, the variance estimator is not consistent in general when Assumption 5 does not hold, as shown by simulation in Funk et al. (2011).Source: §3.3 · p. 11

3.4 Matching estimators

  • Matching estimators are non-RAL because fixed match counts make their forms nonsmooth; Theorem 2 establishes (4) for matching estimators (12) and (14).Conditional outcome means and variances can be estimated by matching units with the same treatment level.Source: §3.4 · pp. 11–12

3.5 Bootstrap variance estimation

  • Theorem 3 establishes consistency of the bootstrap variance components and estimators; RAL estimators can resample original observations, but matching requires resampling asymptotic linear terms.Ordinary resampling for matching does not preserve the distribution of how often units are used as matches.Source: §3.5 · pp. 12–13
  • When n2/n1 is small, an alternative bootstrap scheme guarantees n2 validation observations; very large main samples can use subsampling or the Bag of Little Bootstrap.As n2 →∞ and ρ = 0, only the validation data need to be bootstrapped, treating the error-prone main-data estimator as constant.Source: §3.5 · p. 13

4 Extensions

The framework extends to subset and nonlinear causal estimands, derives cost-constrained sample allocation guidance, and extends efficiency gains to multiple data sources.

4.1 Other causal estimands

  • The strategy extends to causal effects in population subsets, including the treated, and nonlinear estimands such as the log causal risk ratio.For the log risk ratio, the paper estimates each potential-outcome mean from validation data and applies the log ratio; linearization supports an improved estimator.Source: §4.1 · p. 14

4.2 Design issue: optimal sample size allocation

  • The optimal validation and main-data sample sizes are inversely proportional to the square roots of their costs, with larger validation samples favored when estimator association is weak.The association is measured by the squared multiple correlation between the initial and error-prone estimators.Source: §4.2 · p. 15

4.3 Multiple data sources

  • With multiple partial-covariate datasets and one full-information validation dataset, the extended estimator is consistent and has lower asymptotic variance when Γ is nonzero.Using more data sources improves asymptotic estimation efficiency.Source: §4.3 · p. 15

5 Simulation

In simulations, the proposed estimators were unbiased and improved the validation-only estimators, while multiple imputation was biased under imputation-model misspecification. Both asymptotic and bootstrap confidence intervals had coverage close to nominal.

Simulation results

  • Across all scenarios, proposed estimators were unbiased and improved validation-only estimators; same-type error-prone estimators gave substantial efficiency gains.Adding another error-prone estimator gave no important additional gain, and the matching estimator had the smallest gain in these settings.Source: §5 · p. 16
  • The multiple imputation estimator was biased when its imputation model was misspecified.Source: §5 · p. 16
  • Empirical coverage for confidence intervals using either asymptotic variance estimates or bootstrap was close to the nominal rate.The study used Wald-type 95% confidence intervals.Source: §5 · p. 16

Simulation setup

  • The simulations used main-data samples of 1,000 units and validation samples of either 200 or 500 units across 2,000 Monte Carlo samples.Source: §5 · p. 16

6 Application

The COPD–HZ application combines a large claims dataset lacking smoking and alcohol measures with a smaller validation dataset that includes them. The proposed estimators yielded similar point estimates to validation-only estimators with substantially smaller standard errors.

6 Application

  • The main data included 42,430 subjects but lacked smoking and alcohol measures; the validation data included 1,148 comparable subjects with those confounders measured.The validation data also included the covariates available in the main data.Source: §6 · p. 18
  • The analysis fit logistic regression models for propensity scores and outcome means using the main and validation data.The models used observed covariates for error-prone estimates and the additional confounders for validation-data estimates.Source: §6 · p. 18
  • 0.57 (95% confidence interval: 0.41, 0.72) was the proposed error-prone IPW log causal risk-ratio estimate, versus 1.10 (95% confidence interval: 0.02, 2.18) for validation-only IPW.The authors describe the proposed estimate as much more accurate than the initial estimate.Source: §6 · p. 18
  • COPD increased HZ development by 1.55% on average; proposed estimators had smaller standard errors, and all but the proposed matching estimator’s 95% confidence interval excluded zero.The proposed estimators’ point estimates were not greatly different from the corresponding validation-only estimates.Source: §6 · p. 18, Table 1 · p. 19

7 Relaxing Assumption 1

The section extends the framework to validation samples selected with known, outcome-dependent inclusion probabilities, then develops asymptotic results and variance estimation for regression-adjustment and matching estimators. It also relates the approach to missing-data estimators without requiring a model for the missing confounders.

7 Relaxing Assumption 1

  • Validation inclusion may depend on (A, X, Y), with independent Bernoulli selection and known positive inclusion probabilities; otherwise, the selection model must be fit.This relaxes the earlier random-sample assumption.Source: §7 · p. 19

7.1 RAL estimators

  • Under regularity conditions, Theorem 4 establishes the asymptotic variance result for the Hajek-type estimators, while the validation-only AIPW estimator is consistent for τ.The variance result uses the sampling probability and covariance terms specified in the theorem.Source: §7.1 · p. 20

7.2 Matching estimators

  • For matching, inclusion-weighted match-use counts enter the estimators; the resulting main-data estimator is consistent for τ, and the paired error-prone difference is consistent for zero.Theorem 5 gives the asymptotic result for the matching estimators.Source: §7.2 · pp. 20–21

7.3 A bootstrap variance estimation procedure

  • Bootstrap estimates of Γ, V, and the variances are consistent; resampling the asymptotic linear forms is particularly useful for matching.The procedure constructs replicates from the linear forms and estimates the variance components from those replicates.Source: §7.3 · p. 22

7.4 Connection with missing data

  • The framework belongs to a class of missing-data estimators but avoids specifying f(U | A, X, Y), instead using error-prone estimators and an estimator consistent for zero.The paper contrasts this with an optimal adjustment that requires a correctly specified missing-data model.Source: §7.4 · p. 23

8 Discussion

The discussion emphasizes that combining the data can improve efficiency without imposing parametric models linking unmeasured and observed covariates. It also identifies transportability as a way to construct an estimator of zero and proposes sensitivity analysis when the samples cannot support a shared limit.

8 Discussion

  • The framework can combine a small validation sample with a larger main sample, including settings with many covariates, while retaining standard-estimator software and avoiding parametric models linking U to observed variables.The validation-data estimator may use flexible machine-learning methods, with the main data then used to improve efficiency.Source: §8 · p. 23
  • A transportable feature supplies an estimator of zero: when the distribution of (A, X, Y) is transportable, subtracting its estimates across the two datasets yields the needed difference.Under sample heterogeneity, the authors also discuss transportability of the conditional outcome distribution or comparable subgroups.Source: §8 · p. 23
  • If intrinsic sample heterogeneity prevents the two error-prone estimators from sharing a limit, sensitivity parameter δ represents their systematic difference and allows assessment of its impact on causal inferences.The paper gives an adjusted estimator indexed by δ.Source: §8 · pp. 23–24

Supplementary materials for “Combining multiple

The supplementary proofs establish the variance decomposition, an optimality result within a class of smooth consistent estimators, and a property of adding error-prone estimator dimensions. The supplied supplementary passage for the materials heading states no substantive claim.

S1 Proof for Theorem 1

  • The proof of Theorem 1 derives the asymptotic variance formula from the linear forms, including a component V equal to (1 − ρ) times the variance of φ(A, X, Y).The derivation uses the covariance between the validation and main-sample contributions.Source: S1 Proof for Theorem 1 · p. 30

S2 Proof of Remark 2

  • The proposed estimator is optimal among consistent estimators that are smooth functions of the initial estimate and the two error-prone estimates.The proof chooses the variance-minimizing coefficient under their joint normality.Source: S2 Proof of Remark 2 · p. 31

S3 Proof of Remark 3

  • Adding dimensions to the error-prone estimator does not decrease the efficiency gain, under the proof’s invertibility condition on V.The proof reduces this claim to an inequality comparing the full and lower-dimensional variance reductions.Source: S3 Proof of Remark 3 · p. 31

S4 Proof of Lemma 1

Under Assumption 5, the proof obtains the AIPW estimator’s influence function by replacing an expected derivative with an expected outer product. The supplementary lemmas establish error-prone regression imputation, IPW, and AIPW estimators when neither working model is correctly specified.

S4 Proof of Lemma 1

  • Under Assumption 5, the expected derivative can be replaced by the expected outer product to derive the AIPW estimator’s influence function.The proof first expresses the estimator as depending on its parameter estimates and then uses a Taylor expansion.Source: S4 Proof of Lemma 1 · p. 32

S5 Lemmas for error-prone estimators

  • The error-prone estimators are treated as initial estimators with U null, without requiring correct specification of the propensity-score or outcome model.The lemmas cover regression imputation, IPW, and AIPW.Source: S5 Lemmas for error-prone estimators · pp. 32–34
  • Under suitable regularity conditions, the estimating-equation solutions converge in probability to parameters satisfying the population estimating equations.The setup specifies working outcome models and a working propensity-score model.Source: S5 Lemmas for error-prone estimators · p. 33

S6 Assumptions for the matching estimator

The matching-estimator assumptions impose boundedness and smoothness conditions on the covariate distributions and outcome functions, with discrete covariate components allowed. Under these conditions, the matching estimator’s asymptotic variance is the sum of two variance components.

S6 Assumptions for the matching estimator

  • Assumption S1 requires bounded densities on compact convex support and bounded conditional fourth moments; discrete covariate components are also allowed.The text describes the smoothness requirements as mild and states that Assumption S1(ii) implies S1(iii).Source: S6 Assumptions for the matching… · pp. 34–35
  • Assumption S2 requires the estimated mean functions to meet asymptotic conditions, with tuning parameters selected appropriately for nonparametric estimators.Correctly specified parametric models satisfy S2; the condition supports fast convergence of the bias-correction terms.Source: S6 Assumptions for the matching… · p. 35

S7 Proof of Theorem 2

  • Under Assumption S1, the asymptotic variance of n1^1/2(τ̂mat,2 − τ) is vτ^2 + ve^2, with ve^2 = O(1).The proof bounds the relevant matching term and its second moment uniformly over n2.Source: S7 Proof of Theorem 2 · p. 36

S8 Proof of Theorem 3

The proof establishes consistency of the bootstrap variance estimator for both RAL and matching estimators. For matching estimators, the resulting procedure is described as simpler than that of Otsu and Rai (2016).

S8 Proof of Theorem 3

  • For matching estimators, the bootstrap variance estimator is simpler than the procedure of Otsu and Rai (2016).That work showed that the conditional bootstrap distribution approximates the sampling distribution of the estimator.Source: S8 Proof of Theorem 3 · p. 36
  • For the bootstrap replicate of (τ̂2 − τ), the error is written as T*1 + T*2; by (S7) and root-n consistency of ψ̂(Aj, Xj, Uj, Yj), T*2 = oP(1).Source: S8 Proof of Theorem 3 · pp. 36–37
  • The bootstrap variance estimator is consistent for the variance of τ̂, covering both RAL and matching estimators.The proofs for Γ and V are described as similar and omitted.Source: S8 Proof of Theorem 3 · pp. 36–37

S9 Derivation of the optimal sample allocation

The supplement derives the optimal sample allocation by minimizing an objective under a constraint. It also gives an asymptotic covariance calculation and introduces error-prone and bias-corrected matching estimators.

S9 Derivation of the optimal sample allocation

  • Solving the Lagrange-multiplier first-order conditions for the constrained minimization yields the stated optimal allocation in (21).The derivation begins by minimizing (20) subject to (19).Source: S9 Derivation of the optimal sample… · p. 38

S10 Proof of Theorem 4

  • The proof of Theorem 4 uses a multivariate central limit theorem to obtain asymptotic normality and calculates the corresponding covariance matrix.The calculation includes a covariance term involving ψ and φ weighted by π.Source: S10 Proof of Theorem 4 · p. 38

S11 Error-prone matching estimators in Section 7.2

S12 Proof of Theorem 5

The proof shows that two components of the estimator have zero covariance and derives its asymptotic variance. It also uses Taylor expansions to establish an estimator representation under Assumption 4.

S12 Proof of Theorem 5

S13 Proof of (23)

  • The proof establishes consistency of ˆT1 for τ and ˆT2 for 1, then uses Taylor expansion to derive the estimator representation.Source: S13 Proof of (23) · pp. 41–42
  • Under Assumption 4, the Taylor-expansion argument replaces E{∂S(A, X, U; α∗)/∂αT} with Σαα because HAIPW = 0.Source: S13 Proof of (23) · p. 42

S14 Proof of Proposition 2

The proof uses earlier asymptotic representations for RAL and matching estimators, then establishes consistency of the bootstrap variance estimate.

S14 Proof of Proposition 2

S15 Proof of Theorem 6

Loading 1801.00802v4…