PaperlayerView exact revision on arXiv

Source-linked AI summary

Inference in Partially Linear Models under Dependent Data with Deep Neural Networks

Chad Brown

arXiv:2410.22574v1econ.EMstat.ML

TL;DR

The paper addresses inference for a partially linear regression model with stationary β-mixing data after DNN estimation of nuisance components. It constructs the finite-dimensional estimator using those components and shows √n-consistency and asymptotic normality without sample splitting. The paper thereby obtains valid inference in a dependent-data setting while identifying extensions for future work.

  • Problem

    The paper studies how to perform inference in a partially linear regression model under stationary β-mixing data when nuisance components are estimated using DNNs.

    Source: §1 · p. 2

  • Method

    The estimator uses DNN-estimated nuisance components and avoids sample splitting in the dependent-data setting.

    Source: §1 · p. 2, §3 · p. 9

  • Results

    The finite-dimensional parameter estimator achieves √n-consistency and asymptotic normality.

    Source: §1 · p. 2, §3 · p. 9

  • Takeaways & Limitations

    The paper shows that valid inference can be obtained after first-stage DNN estimation in a partially linear regression model with dependent data.

    Source: §3 · p. 9

    The current analysis uses a scalar treatment, while extensions to vector-valued treatments and broader semiparametric or instrumental-variable settings are planned.

    Source: §1 · p. 2, §3 · p. 9

Abstract

from arXiv · show

I consider inference in a partially linear regression model under stationary $\beta$-mixing data after first stage deep neural network (DNN) estimation. Using the DNN results of Brown (2024), I show that the estimator for the finite dimensional parameter, constructed using DNN-estimated nuisance components, achieves $\sqrt{n}$-consistency and asymptotic normality. By avoiding sample splitting, I address one of the key challenges in applying machine learning techniques to econometric models with dependent data. In a future version of this work, I plan to extend these results to obtain general conditions for semiparametric inference after DNN estimation of nuisance components, which will allow for considerations such as more efficient estimation procedures, and instrumental variable settings.

1 Introduction

The paper studies inference for a partially linear model with stationary β-mixing data, using DNNs to estimate nuisance components without sample splitting. It establishes √n-consistency and asymptotic normality for the finite-dimensional parameter under mild regularity conditions.

1 Introduction

  • The model is Y_t = V_tζ_0 + η_0(X_t) + u_t, with E[u_t|V_t, X_t] = 0, where X_t affects outcomes through an unknown nuisance function.The treatment V_t is scalar in the paper, while the framework could generalize to vector-valued treatments.Source: §1 · p. 2
  • The analysis targets ζ_0 when the observed process is strictly stationary and β-mixing, covering dependent-data settings including nonlinear autoregressive applications.Nonlinear autoregressive settings can be represented by including lagged outcomes among the covariates.Source: §1 · p. 2
  • Using DNN-estimated nuisance components and Robinson’s procedure, the estimator achieves √n-consistency and asymptotic normality under mild regularity conditions.The approach applies DNN results from Brown (2024) to estimation of the infinite-dimensional nuisance components.Source: §1 · pp. 2–3
  • The paper avoids sample splitting, which is important for dependent data because independent validation sets can be difficult to construct.This addresses a challenge in applying machine learning methods to econometric models with dependent observations.Source: §1 · p. 3
  • The contribution extends inference-after-machine-learning results beyond predominantly i.i.d. settings to valid large-sample inference under dependent data.It also replaces traditional nonparametric estimators used in related partially linear-model work with DNNs for infinite-dimensional parameters.Source: §1 · p. 3
  • The techniques are presented as potentially applicable beyond the simple partially linear model considered here.The paper connects these results to broader developments in inference after machine learning and semiparametric econometrics.Source: §1 · p. 3

2 Inference in Partially Linear Models

The paper constructs partially linear-model inference by estimating nuisance conditional expectations with DNN sieves under stationary β-mixing data, then establishes asymptotic properties for the finite-dimensional estimator.

2 Inference in Partially Linear Models

  • The two-stage estimator first estimates s0 and f0 with DNN estimators, then uses them to construct a method-of-moments estimator for ζ0.Source: §2 · p. 4

2.1.1 DNN Architectures

  • The first-stage networks are fully connected feedforward DNNs using ReLU activation, with architecture determined by hidden-layer counts and units per layer.Source: §2.1.1 · pp. 4–5

2.1.2 DNN Estimation

  • The nuisance functions f0 and s0 are estimated through DNN sieve spaces following Brown (2024), with approximate estimators allowing plug-in error θn=oP(1).Source: §2.1.2 · p. 6
  • Under stationary β-mixing observations, bounded-support and moment conditions, and Hölder smoothness, the DNN estimators achieve convergence rates governed by ϵn.Source: §2.1.2 · pp. 6–7

2.2 Asymptotic Properties of ˆζ

  • Theorem 1 uses these nuisance rates to establish √n-convergence and asymptotic normality of ˆζ when θn=oP(n^-1/2).Source: §2.2 · p. 8

3 Conclusion

The paper establishes valid inference for a partially linear regression model with stationary β-mixing data after DNN estimation of infinite-dimensional nuisance parameters. The resulting finite-dimensional estimator is √n-consistent and asymptotically normal without sample splitting, with extensions to more complex models planned.

3 Conclusion

  • The results obtain valid inference in a partially linear regression model under dependent data after first-stage DNN estimation of infinite-dimensional parameters.Source: §3 · p. 9
  • The estimator for the finite-dimensional parameter achieves √n-consistency and asymptotic normality after DNN estimation of nuisance components under dependent data.Source: §3 · p. 9
  • Avoiding sample splitting addresses a key challenge in applying machine learning techniques to econometric models with dependent data.Source: §3 · p. 9
  • The results demonstrate practical implications of Brown (2024) and provide techniques that could extend to more complex econometric models.Source: §3 · p. 9
  • Planned extensions include more efficient estimation procedures and instrumental variable models, as well as general conditions for semiparametric inference after DNN nuisance estimation.Source: §3 · p. 9

A Proof of Theorem 1

The proof of Theorem 1 is preceded by three ancillary lemmas modeled on conditions from Chen et al. (2022, Theorem 1).

A.1 Empirical Moment Convergence

The proof concludes using Assumption 2(i) together with L2(PX) convergence of the estimated function to the target function.

  • The proof is completed using Assumption 2(i) and ∥ˆfn −f0∥L2(PX) →0.Source: §A.1 · p. 10

A.2 Stochastic Equicontinuity

The stochastic equicontinuity argument uses independent blocking, a technique commonly applied to stationary β-mixing processes.

  • The proof of Lemma 2 is provided in Appendix A.2.2 before the result is established.Source: §A.2 · p. 11
  • Independent blocking is introduced for handling stationary β-mixing processes.Source: §A.2 · p. 11

A.2.1 Independent blocks

The proof partitions the dependent sequence into blocks and uses β-mixing arguments to construct an identically distributed independent block sequence, while retaining a remainder block.

A.2.1 Independent blocks

  • The sequence is partitioned into alternating blocks of length a plus a remainder block, using index sets T1,j, T2,j, and TR.The remainder has length n − 2ba.Source: §A.2.1 · p. 11
  • Berbee’s Lemma redefines the blocks so that distinct blocks are independent and each redefined block preserves the distribution of its original block.The construction relies on the β-mixing structure and stationarity.Source: §A.2.1 · p. 11
  • By stationarity, all length-a blocks are identically distributed, so the constructed block sequence is i.i.d.The independent blocks are {Zt}t∈T1,1, {Zt}t∈T2,1, …, {Zt}t∈T2,b.Source: §A.2.1 · p. 11
  • The original observations remain dependent within blocks, even though the constructed collection of blocks is independent across blocks.This distinction is essential because the original sequence is not independent.Source: §A.2.1 · p. 11
  • The argument invokes an equivalent representation of the β-mixing coefficient and Eberlein’s lemma to justify the coupling construction.The proof applies Eberlein (1984) after verifying its conditions.Source: §A.2.1 · p. 12

A.2.2 Proof of Lemma 2

The proof of Lemma 2 controls the relevant empirical process under dependence by combining Rademacher-complexity bounds with independent-block coupling and growth conditions.

A.2.2 Proof of Lemma 2

Definition 1. (Rademacher Complexity) For n ∈N, let {W t}n

A.3 Neyman Orthogonality

The score is shown to satisfy Neyman orthogonality by differentiating its expectation with respect to nuisance perturbations and controlling the Taylor remainder.

A.3 Neyman Orthogonality

  • The proof applies Taylor expansion and measure-theoretic differentiation to the expected score functional.Bounded nuisance functions and Yt ∈ L2 justify repeated differentiation and integration steps.Source: §A.3 · pp. 19–20
  • The expected score satisfies Neyman orthogonality because iterated expectations and the definitions of f0 and s0 eliminate the first-order nuisance derivative.The argument cites Corbae et al. (2009, Theorem 7.5.19, p.405) for the differentiation step.Source: §A.3 · p. 20
  • Leibniz’s rule and Fubini’s theorem are used again to control the Taylor remainder after the first-order term vanishes.The resulting expansion separates the orthogonal contribution from higher-order terms.Source: §A.3 · p. 20
  • The proof is similar to Chen et al. (2022, Theorem 1), but is tailored to the dependent-data setting.The section identifies dependence as the relevant adaptation.Source: Proof of Theorem 1 · p. 20

Proof of Theorem 1.

Theorem 1 decomposes the estimator into an asymptotically normal component and remainder components that vanish at suitable rates under dependent data.

Proof of Theorem 1.

  • The proof is tailored to dependent data and uses notation, lemmas, and mixing properties inherited from the underlying sequence.The relevant process inherits the mixing properties of {Zt}n.Source: Proof of Theorem 1 · pp. 21–23
  • The estimator expansion separates an asymptotically normal term from components that converge to zero in probability at suitable rates.This decomposition is the central step in proving the theorem.Source: Proof of Theorem 1 · pp. 21–22
  • The remainder terms are bounded using Lemma 2 together with Bosq (1998, Theorems 1.5 and 1.6).These results establish the required convergence and moment controls for the dependent process.Source: Proof of Theorem 1 · pp. 22–23
  • Result (i) follows from the convergence argument, while result (ii) follows by applying a central limit theorem to the first term.The central limit theorem applies when σ > 0 under Assumption 2(i)(ii).Source: Proof of Theorem 1 · p. 23
Loading 2410.22574v1…