PaperlayerView exact revision on arXiv

Source-linked AI summary

VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatments

Lizhen Nie, Mao Ye, Qiang Liu, Dan Nicolae

arXiv:2103.07861v1cs.LGstat.ML

TL;DR

Estimating ADRFs from observational data with continuous treatments remains challenging, and block-based neural-network heads can yield discontinuous estimates. The paper proposes VCNet and functional targeted regularization; experiments report improved performance, with their combination consistently achieving state-of-the-art performance.

  • Problem

    Estimating ADRFs from observational data with continuous treatments is important, while block-based neural-network approaches can produce discontinuous estimates in practice.

    Source: §1 · pp. 1–2

  • Method

    VCNet varies prediction-head weights continuously with treatment, while functional targeted regularization uses smoothing to estimate the whole ADRF curve with a doubly robust estimator.

    Source: §1 · p. 2, §4.3 · p. 6

  • Results

    Experiments on synthetic and semi-synthetic datasets find that VCNet and targeted regularization boost performance independently and jointly consistently achieve state-of-the-art performance.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper provides theoretical results for consistency and double robustness and reports performance gains from VCNet and targeted regularization, separately and together.

    Source: §7 · p. 9

    Density estimation is not the paper's main focus; its conditional-density estimator uses linear interpolation, with kernel smoothing noted as an alternative.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Motivated by the rising abundance of observational data with continuous treatments, we investigate the problem of estimating the average dose-response curve (ADRF). Available parametric methods are limited in their model space, and previous attempts in leveraging neural network to enhance model expressiveness relied on partitioning continuous treatment into blocks and using separate heads for each block; this however produces in practice discontinuous ADRFs. Therefore, the question of how to adapt the structure and training of neural network to estimate ADRFs remains open. This paper makes two important contributions. First, we propose a novel varying coefficient neural network (VCNet) that improves model expressiveness while preserving continuity of the estimated ADRF. Second, to improve finite sample performance, we generalize targeted regularization to obtain a doubly robust estimator of the whole ADRF curve.

1 INTRODUCTION

Estimating continuous-treatment ADRFs from observational data is important, but neural-network approaches using treatment blocks can produce discontinuous estimates. The paper introduces VCNet to preserve continuity and generalizes targeted regularization to estimate the whole ADRF curve.

1 INTRODUCTION

  • Block-based DRNet can produce discontinuous ADRF estimates in practice, despite being designed for continuous treatments.It partitions treatment levels into blocks and uses a separate prediction head for each block.Source: §1 · pp. 1–2
  • VCNet uses prediction-head weights that vary continuously with treatment, producing continuous ADRF estimates when the activation function is continuous.The varying-coefficient structure also prevents treatment information from being lost in the high-dimensional latent representation.Source: §1 · p. 2
  • Functional targeted regularization generalizes targeted regularization to the whole ADRF curve and yields a doubly robust estimator under mild assumptions.It uses smoothing to control model complexity across infinitely many treatment levels and lets regularization strength depend on sample size.Source: §1 · p. 2
  • Experiments on synthetic and semi-synthetic datasets find that VCNet and targeted regularization independently boost performance and jointly consistently achieve state-of-the-art performance.The reported result concerns their joint use as well as each component independently.Source: §1 · p. 2

2 PROBLEM STATEMENT AND BACKGROUND

The paper considers estimating the average dose-response function from i.i.d. observations of outcomes, covariates, and continuous treatments. Its stated assumptions require treatment-density overlap and covariates that block backdoor paths.

2 PROBLEM STATEMENT AND BACKGROUND

  • The target is the ADRF, the potential expected outcome that would be observed under treatment level t.The setup assumes an i.i.d. sample of outcomes, covariates, and continuous treatments, with T=[0,1].Source: §2 · p. 3
  • The target outcome at each treatment level is the expected potential outcome under that level.This is the paper's operational description of the ADRF.Source: §2 · p. 3
  • The treatment density must be bounded below for all covariates and treatment levels, and measured covariates must block all backdoor paths.The paper states that these conditions support treatment-level availability and identification from observational data.Source: §2 · p. 3

3 VCNET: VARYING COEFFICIENT NETWORK STRUCTURE

VCNet uses treatment-conditioned features and a treatment-varying prediction head to estimate a continuous ADRF. The section also describes conditional-density estimation and joint training, including a doubly robust alternative to the naive ADRF estimator.

3 VCNET: VARYING COEFFICIENT NETWORK STRUCTURE

  • The conditional-density head supplies features z for downstream outcome estimation, and learning π(t|X) helps distill useful information from X for estimating the ADRF.A naive ADRF estimator instead averages the fitted conditional outcome function over observed covariates.Source: §3 · p. 3

3.1 THE VARYING COEFFICIENT PREDICTION HEAD

  • VCNet makes the prediction head’s neural-network parameters depend on treatment level, strengthening treatment’s influence while preserving continuity of the ADRF estimate.With typical spline bases such as B-splines, continuous activations yield continuous ADRF estimators; DRNet’s blockwise heads are a special case under a particular basis choice.Source: §3.1 · p. 4

3.2 CONDITIONAL DENSITY ESTIMATOR

  • The conditional-density network estimates π(t|x) on equally spaced treatment grids and uses linear interpolation for treatment values between grid points.The authors use this simple estimator without additional techniques such as kernel smoothing or smoothness regularization, noting that density estimation is not the paper’s main focus.Source: §3.2 · pp. 4–5

3.3 TRAINING

  • Training jointly optimizes outcome prediction and conditional-density estimation; plugging both estimates into the estimating equation yields a doubly robust ADRF estimator if either function is in the model space.The loss combines outcome prediction loss with conditional-density negative log likelihood, weighted by α.Source: §3.3 · p. 5

4 FUNCTIONAL TARGETED REGULARIZATION

Section develops doubly robust estimation for treatment-specific outcomes and extends targeted regularization from scalar targets to the whole ADRF. Functional targeted regularization approximates its treatment-indexed perturbation with splines and has a doubly robust convergence result under stated conditions.

4.1 DOUBLY ROBUST ESTIMATOR

  • For any treatment level, the efficient influence function yields a doubly robust estimator when either the propensity or outcome estimator is correct.The result assumes the estimated treatment density is bounded below.Source: §4.1 · p. 5

4.2 TARGETED REGULARIZATION FOR INFERRING A FINITE DIMENSIONAL QUANTITY

  • Targeted regularization jointly optimizes the outcome and density models with a scalar perturbation to drive the influence-function term toward zero, avoiding its unstable direct estimation.The instability can be pronounced when the positivity assumption is nearly violated; the resulting estimator has no conditional-density estimate in its denominator.Source: §4.2 · p. 6

4.3 FUNCTIONAL TARGETED REGULARIZATION FOR INFERRING THE WHOLE ADRF

  • For the whole curve, optimizing an unrestricted perturbation function is infeasible and risks overfitting, so the method approximates it with spline basis functions using smoothness of the outcome and density functions.The number of spline basis functions may change with sample size.Source: §4.3 · p. 6
  • The method lets the perturbation coefficient β depend on sample size, with β_n = o(1), to retain double robustness when the outcome model is misspecified.The paper notes that using fixed β might remove double robustness in this case.Source: §4.3 · p. 7
  • Under controlled model complexity and mild assumptions, functional targeted regularization is doubly robust; when both nuisance estimators are consistent, its estimator converges faster than either nuisance estimator.The theorem concerns the estimator of the whole ADRF curve.Source: §4.3 · p. 7

5 RELATED WORK

The related work distinguishes VCNet’s treatment-varying neural-network parameters from prior network approaches and situates its whole-curve targeting extension alongside methods for scalar causal quantities.

The Varying Coefficient Structure

  • VCNet applies a varying-coefficient structure to a complex neural network to enhance treatment-effect expressiveness, unlike hypernetworks, which generate a larger network’s weights to reduce search space.VCNet directly trains the main network with weights that are linear combinations of treatment-dependent spline functions.Source: §5 · p. 8

Neural Network Structure for Treatment Effect Estimation

  • Other neural approaches learn counterfactuals or deep variable models, but the cited approaches are described as heuristic and lack theoretical guarantees for asymptotic correctness.The passage names Yoon et al. (2018), Bica et al. (2020), and Louizos et al. (2017).Source: §5 · p. 8

Doubly Robustness, TMLE and Targeted Regularization

  • Earlier TMLE and targeted-regularization work focused on single quantities, whereas this paper generalizes both approaches to inference on the whole ADRF curve.Examples include binary-treatment contrasts and an averaged treatment effect for continuous treatment.Source: §5 · p. 8

6 EXPERIMENTS

The experiments compare VCNet and targeted-regularization variants with neural and traditional baselines on synthetic and semi-synthetic continuous-treatment data. VCNet plus targeted regularization performs especially well on simulation and IHDP, and is close to the best method on News.

Dataset

  • The evaluation uses one synthetic and two semi-synthetic datasets: IHDP and News.The datasets are used because true ADRFs are rarely available for real-world data.Source: §6 · p. 8

Baselines and Settings

  • The neural baselines are Dragonnet and DRNet, alongside causal forest, BART, and GPS as non-neural baselines.The neural implementations add treatment-block heads for Dragonnet and a conditional-density head for DRNet.Source: §6 · p. 9
  • VCNet uses a degree-2 truncated polynomial basis with knots at 1/3 and 2/3, matched to five blocks in Dragonnet and DRNet for equal model complexity.Hyperparameters are tuned on 20 separate tuning sets for each method and dataset.Source: §6 · p. 9

Estimator and Metrics

  • Neural methods are evaluated in naive, doubly robust, TMLE, and targeted-regularization versions, using test-set AMSE as the metric.Non-neural models use their usual estimators.Source: §6 · p. 9

Results

  • VCNet + targeted regularization outperforms traditional baselines by a large margin on simulation and IHDP, and is close to the best method on News.Table 2 reports AMSE averaged over 100 experiments for simulation and IHDP and 20 for News; ± denotes estimated standard deviation.Source: §6 · p. 9, Table 2 · p. 9
  • Across neural-network methods, the varying-coefficient structure boosts performance, while targeted regularization often performs best and naive versions consistently perform worst.Doubly robust and TMLE performance varies across datasets.Source: §6 · p. 9

7 CONCLUSION

The paper proposes VCNet and extends targeted regularization to continuous ADRF curves. It reports consistency and double-robustness theorems and strong experimental performance when the methods are used separately or together.

Conclusion

  • The paper proposes a varying-coefficient network and generalizes targeted regularization to continuous curves.Source: §7 · p. 9
  • The paper provides theorems establishing consistency and double robustness.Source: §7 · p. 9
  • Experiments report that VCNet and targeted regularization boost performance independently, and together improve over existing methods by a large margin.Source: §7 · p. 9

A.1 ON THE CONSISTENCY OF ¯µ AND ¯π

The consistency argument states that adding functional targeted regularization does not change the large-sample limits of the outcome and treatment-density estimators. Its proof uses a risk comparison and convergence of the estimated functions to the unregularized minimizer.

Consistency

  • Under the stated result, the limits of the outcome and treatment-density estimators are the same with loss (4) as with loss (1).Source: §A.1 · p. 13
  • The proof compares the regularized empirical risk with the unregularized minimizer’s population risk, with an O_p(n^-1/2) remainder.Source: §A.1 · p. 13
  • The argument invokes uniform concentration under boundedness and a bounded Lipschitz constant for log on [1/c, c].Source: §A.1 · p. 13
  • The proof uses convergence in L2 of the estimated outcome and treatment-density functions to the unregularized minimizer.Source: §A.1 · p. 13

A.2 TECHNICAL PROOFS

This appendix section presents proofs of the paper’s two main theorems and additional results mentioned in the main text.

Technical Proofs

  • The section proves Theorems 1 and 2 and gives additional results mentioned briefly in the main text.Source: §A.2 · p. 14

A.2.1 NOTATIONS AND DEFINITIONS

This passage set introduces notation for the targeted-regularization estimator, including its fitted nuisance functions and residual vectors.

A.2.1 NOTATIONS AND DEFINITIONS

  • The fitted treatment density, outcome model, and perturbation function are denoted by π̂_n, μ̂_n, and ε̂_n.The perturbation estimate is represented as a spline regression estimator.Source: §A.2.1 · p. 14
  • Π_n stores fitted treatment densities at each observed treatment-covariate pair, while Z_n records outcome residuals.The residual is defined as observed outcome minus fitted outcome.Source: §A.2.1 · p. 14
  • The passage defines Z̃_n as the n-dimensional vector (z̃_1, z̃_2, …, z̃_n)^T ∈ R^n, with each coordinate given only as z̃_i = P in the supplied excerpt.Source: §A.2.1 · p. 14

A.2.2 USEFUL LEMMAS

The appendix develops supporting lemmas and proofs for the influence-function and convergence results, extends the influence-function result to vector parameters, and documents implementation and dataset settings.

A.2.2 USEFUL LEMMAS

  • The efficient influence function for Γ includes the point-dose parameter ψ(t0), the randomized-trial average outcome, and the binary-treatment average treatment effect as special cases.The proof concludes that the proposed ζ is an efficient influence function for Γ.Source: §A.2.2 · pp. 14–15, §A.2.5 · p. 18
  • The uniform spline-estimation bound is O_p(n^-1/3√log n) with K_n ≍ n^1/6, accounting for estimated nuisance functions that depend on the observations.For a fixed objective, the standard rate is O_p(n^-2/5) with K_n ≍ n^1/5; the uniform bound is larger because the estimated propensity and outcome functions are not fixed.Source: §A.2.2 · p. 15, §A.2.6 · p. 22
  • Under Theorem 2's assumptions, the scaled Gram matrix has eigenvalues bounded between positive constants with probability tending to one, and is therefore invertible.The proof attributes the bounds to the spline-basis result and upper and lower bounds on the estimated propensity.Source: §A.2.2 · p. 15, §A.2.7 · p. 22

A.2.3 PROOF OF THEOREM 1

  • The proof establishes double robustness of the efficient influence function for ψ(t0), using iterated expectation and the treatment-density factorization.The passage states that the desired conclusions follow from the last line of Equation (8).Source: §A.2.3 · p. 15

A.3 ADDITIONAL RESULTS

  • For a d-dimensional parameter Γ, the efficient influence function is the vector of the efficient influence functions for its individual coordinates.The proof notes that each coordinate influence function does not depend on the submodel parameter ε.Source: §A.3 · p. 23

A.4.1 NETWORK STRUCTURE

  • The VCNet and comparison methods use equal parameter counts, while the reported network structure was selected after testing alternatives with similar relative performance.VCNet uses a two-layer prediction head and a degree-two B-spline with knots at {1/3, 2/3}; all networks are trained for 800 epochs.Source: §A.4.1 · p. 23

A.4.2 PARAMETER SETTING

  • Hyperparameters are tuned over 20 runs per dataset using test-set AMSE, and performance was not sensitive to α in the tested settings.The tuning grids include learning rates and spline knot counts; networks are trained for 800 epochs during tuning.Source: §A.4.2 · p. 23

A.4.4 DATASET

  • The evaluation includes tuned statistical baselines and generated continuous-treatment versions of News and IHDP, with the synthetic setting designed to assess performance when noise covariates are present.The passages identify the baseline software packages and state that News and IHDP results are summarized in Table 1, but do not provide the table values.Source: §A.4.3 · p. 24, §A.4.4 · p. 24

Related papers

Loading 2103.07861v1…