PaperlayerView exact revision on arXiv

Source-linked AI summary

Semiparametric doubly robust targeted double machine learning: a review

Edward H. Kennedy

arXiv:2203.06469v2stat.ME

TL;DR

Efficient estimation of causal and other functionals in flexible nonparametric models requires both sharp performance benchmarks and estimators that can attain them. This review develops minimax-style efficiency bounds through influence functions and practical derivation tools, then examines estimator errors and conditions for efficiency. It highlights root-n behavior for functionals, asymptotic normality with valid 95% confidence intervals under stated conditions, and limitations when nuisance functions are too complex.

  • Problem

    The review addresses how to estimate causal and other functionals efficiently in nonparametric models and determine the best possible performance and whether particular estimators attain it.

  • Method

    The review synthesizes minimax-style efficiency bounds, influence-function derivations, worked examples, and analyses of influence-function-based estimators under weak assumptions.

  • Results

    Root-n convergence can be achieved for functionals in nonparametric models, and the reviewed estimator is asymptotically normal with asymptotically valid 95% confidence intervals under stated conditions.

  • Takeaways & Limitations

    Efficient influence functions provide both local minimax lower bounds and guidance for constructing efficient estimators and identifying their efficiency conditions.

  • Takeaways & Limitations

    When nuisance functions are insufficiently smooth or sparse relative to dimension, root-n consistency may be impossible; for example, s = 5 and dimension 50 yields minimax rate n^-1/12.

Abstract

from arXiv · show

In this review we cover the basics of efficient nonparametric parameter estimation (also called functional estimation), with a focus on parameters that arise in causal inference problems. We review both efficiency bounds (i.e., what is the best possible performance for estimating a given parameter?) and the analysis of particular estimators (i.e., what is this estimator's error, and does it attain the efficiency bound?) under weak assumptions. We emphasize minimax-style efficiency bounds, worked examples, and practical shortcuts for easing derivations. We gloss over most technical details, in the interest of highlighting important concepts and providing intuition for main ideas.

1 Introduction

The review frames functional estimation as estimating structured functionals of an unknown distribution rather than the full distribution or individual components. It emphasizes efficiency bounds, worked examples, practical derivations, and intuition while largely omitting technical details.

  • Functional estimation targets a structured combination of distributional components, not the entire distribution or a single regression or density function.
  • The setup assumes independent, identically distributed observations from an unknown distribution in a specified model.
  • Root-n convergence can be achieved for functionals in nonparametric models, unlike ordinary nonparametric regression or density estimation.
  • The review emphasizes minimax-style efficiency bounds, worked examples, and practical shortcuts for easing derivations.

2 Setup: Target Parameters & Model Assumptions

The review defines statistical and causal functionals across regression, treatment, missing-data, mediation, instrumental-variable, and other settings. It focuses on post-identification statistical estimation in nonparametric models while only briefly discussing causal identifying assumptions.

  • Many causal-inference and missing-data functionals are regression functions averaged over covariate distributions.
  • The average treatment effect is identified as an expected contrast of treatment-specific regression functions under positivity, consistency, and no unmeasured confounding.
  • The review focuses on statistical estimation and inference after identification, so subsequent methods apply to statistical functionals even when causal assumptions fail.
  • The review covers functionals for stochastic interventions, instrumental-variable effects, time-varying treatments, mediation, treatment-effect bounds, and expected densities, entropy, and f-divergences.These examples are presented under their respective identification or statistical assumptions.
  • Natural mediation effects require weaker positivity assumptions than controlled mediation effects.

3 Benchmarks: Nonparametric Efficiency Bounds

The review develops nonparametric efficiency bounds by extending parametric lower-bound ideas through submodels and von Mises expansions. Efficient influence functions provide both the bounds and a route to estimator construction, while practical derivation strategies and attainability conditions complete the framework.

  • 3 Benchmarks: Nonparametric Efficiency Bounds: Efficiency lower bounds benchmark the difficulty of estimating a target and whether an estimator makes optimal use of the data.
  • 3.1 Cramér–Rao Bounds and Parametric Submodels: The parametric-submodel device transfers Cramér–Rao-style lower bounds from smaller parametric models to larger nonparametric models.A submodel is contained in the larger model and passes through the truth at its reference point.
  • 3.2 Pathwise Differentiability: The von Mises expansion is a distributional Taylor expansion whose derivative term acts as an influence curve and whose remainder captures higher-order changes.
  • 3.4 Deriving Influence Functions: The efficient influence function is the key component of local minimax lower bounds and also guides efficient estimator construction and efficiency conditions.
  • 3.4 Deriving Influence Functions: Derivative rules using simple influence functions and clever submodels provide practical shortcuts for deriving influence functions.The review presents a discrete regression-function example and notes that the second strategy was not previously seen in the literature by the authors.

4 Methods: Influence Function-Based Estimators

Influence-function expansions provide both efficiency benchmarks and a blueprint for constructing estimators that remove first-order plug-in bias. One-step estimators, TMLE, and cross-fitting can achieve root-n inference under suitable nuisance-error and complexity conditions, though these conditions and remainder calculations depend on the functional.

  • Influence-function estimators: Efficient influence functions define minimax benchmarks and guide estimator construction, including the conditions needed for efficiency.They arise as derivative terms in the von Mises expansion and underpin local minimax lower bounds.
  • Plug-in bias correction: Generic plug-in estimators can inherit first-order nuisance bias, whereas one-step corrections typically reduce the remainder to second-order products of nuisance errors.For the average treatment effect, the plug-in bias is the integrated regression-estimation bias and can exceed the root-n scale.
  • One-step estimators: One-step estimators estimate the influence-function bias correction by a sample average and are also interpretable as generalized Newton updates.For the average treatment effect, the nuisance estimates include the outcome regression and treatment propensity.
  • TMLE: TMLE solves the efficient influence-curve estimating equation through a fluctuated distribution and may improve finite-sample boundedness relative to one-step estimators.The asymptotic equivalence holds despite the potential finite-sample advantage when the parameter and plug-in estimate are bounded.
  • Empirical-process control: Cross-fitting makes the empirical-process term asymptotically negligible under L2 consistency without Donsker or entropy restrictions, accommodating methods such as lasso and random forests.Donsker conditions can be restrictive in high-dimensional settings and may require overly strong sparsity assumptions.
  • Remainder control: If both nuisance estimates converge faster than n^-1/4 in L2(P), the second-order remainder is negligible, yielding root-n consistency, asymptotic normality, and local minimax efficiency when the influence function is efficient.The expected-density example lacks double robustness but retains these properties when density estimation is faster than n^-1/4.

5 Some Extensions & Open Problems

The review identifies extensions and open problems involving new functionals, data structures, identifying assumptions, and regimes where root-n inference is impossible or pathwise differentiability fails.

  • New causal effects, network data structures, and identifying assumptions require studying their functional-specific remainder and bias terms.
  • 5.2 High Complexity Regimes: When nuisance functions are insufficiently smooth or sparse relative to dimension, root-n consistency may be impossible.For a 5-smooth regression with 50-dimensional covariates, the minimax rate is n^-1/12, slower than n^-1/4 rates.
  • 5.2 High Complexity Regimes: Open questions include whether alternative estimators can attain root-n rates and what minimax rates apply when root-n estimation is impossible.
  • Many important parameters satisfy the von Mises expansion, but non-pathwise differentiable parameters do not.
  • Non-pathwise differentiability arises for nonsmooth finite-dimensional parameters and infinite-dimensional curves or functions such as regression and density functions.
  • For conditional treatment-effect functions, exploiting smoothness or sparsity requires adapting the review's ideas to an infinite-dimensional setting.The conditional treatment effect is τ(x) = E(Y | X = x, A = 1) − E(Y | X = x, A = 0).

A.1.1 Integral Equation Approach

The integral-equation approach derives an influence function by differentiating the parameter along smooth parametric submodels, expressing the derivative as an inner product, and solving the resulting equation.

  • The derivation first calculates the parameter derivative along a smooth parametric submodel through the true distribution.
  • For the average treatment effect, the two tasks are handled separately: derivative calculation followed by solving the integral equation for the influence function.
  • It then matches the derivative to an inner product involving the influence function and the submodel score.

Derivative of parameter

For the average treatment effect, the review differentiates the parameter along a generic smooth submodel by decomposing the submodel score into outcome, treatment, and covariate components.

  • The submodel score decomposes into conditional outcome, treatment-given-covariates, and covariate-distribution score terms.
  • The derivative of the average treatment effect is obtained by applying the parameter definition and differentiating the submodel density.
  • After computing the derivative, the remaining task is to express it in inner-product form.

Expressing as inner product

The inner-product approach decomposes the influence function into outcome, treatment, and covariate components, reducing one difficult integral equation to three simpler equations and yielding the ATE influence function.

  • The derivative is written in inner-product form to identify the influence function.
  • Decomposing any mean-zero influence function into outcome, treatment, and covariate components simplifies the inner product using conditional mean-zero restrictions.
  • The decomposition transforms one large integral equation into three smaller equations for the outcome, treatment, and covariate components.
  • For the ATE, the covariate component is ϕx(x) = µ(x) − E{µ(X)} = µ(x) − ψ, while the treatment component is ϕa(a, x) = 0.
  • The remaining outcome component is found by solving an integral equation involving the conditional outcome score.
  • The resulting function satisfies the pathwise differentiability condition for every sufficiently smooth parametric submodel and is therefore the influence function.

A.1.2 Gateaux Derivative Approach

The Gateaux derivative approach constructs a point-mass parametric submodel so the pathwise derivative directly yields the influence function, avoiding the integral equation. Although the derivation requires care, it produces the same influence function and extends beyond discrete settings when π and µ are well-defined.

  • A.1.2 Gateaux Derivative Approach: The Gateaux derivative approach is a special case using a parametric submodel whose point-mass score makes the pathwise derivative equal the influence function.This avoids solving an integral equation.
  • A.1.2 Gateaux Derivative Approach: The submodel perturbs the original distribution toward a Dirac measure at the observed point, represented with a mass function in the discrete setting.The construction uses ǫ(z) = (1 −ǫ)P(z) + ǫδZ and its discrete analogue.
  • A.1.2 Gateaux Derivative Approach: The derivation evaluates the parameter along the submodel, differentiates at ǫ = 0, and simplifies the result using the chain rule and rearrangement.Intermediate expressions involve the joint and conditional mass functions.
  • A.1.2 Gateaux Derivative Approach: The Gateaux derivative approach gives the same influence function as the more involved integral equation approach, despite requiring more than a page of calculations.The resulting influence function remains well-defined outside the discrete setup when regression functions π and µ are well-defined.
Loading 2203.06469v2…