On rate optimality for ill-posed inverse problems in econometrics

Xiaohong Chen; Markus Reiss

Source-linked AI summary

On rate optimality for ill-posed inverse problems in econometrics

Xiaohong Chen, Markus Reiss

arXiv:0709.2003v1Sep 13, 2007math.STcs.NAmath.NAstat.APstat.TH

TL;DR

The paper reports that published work had not discussed the relations among the relevant regularity conditions. It establishes minimax lower bounds and shows rate-optimality for estimators in both NPIR and NPIV models.

Problem
The authors report no published work discussing the relations among the relevant regularity conditions.
Source: §1 · p. 2
Method
The paper establishes mean integrated squared error minimax lower bounds and studies a projection estimator for NPIR and a sieve minimum distance estimator for NPIV.
Source: §1 · p. 3
Results
The NPIR projection estimator and NPIV sieve minimum distance estimator are rate-optimal in the minimax sense.
Source: §1 · p. 3
Takeaways & Limitations
The paper establishes minimax rate-optimality for estimators in both NPIR and NPIV.
Source: §1 · p. 3
The source-condition class has finite radius R, and the paper discusses what may happen when the relative-smoothness link condition changes.
Source: §1 · p. 3, §2.2 · p. 9

Abstract

from arXiv · show

In this paper, we clarify the relations between the existing sets of regularity conditions for convergence rates of nonparametric indirect regression (NPIR) and nonparametric instrumental variables (NPIV) regression models. We establish minimax risk lower bounds in mean integrated squared error loss for the NPIR and the NPIV models under two basic regularity conditions that allow for both mildly ill-posed and severely ill-posed cases. We show that both a simple projection estimator for the NPIR model, and a sieve minimum distance estimator for the NPIV model, can achieve the minimax risk lower bounds, and are rate-optimal uniformly over a large class of structure functions, allowing for mildly ill-posed and severely ill-posed cases.

1 Introduction

The introduction identifies unresolved relationships among NPIV regularity conditions and minimax results, especially for severely ill-posed problems. It presents two basic conditions, minimax lower bounds for NPIR and NPIV, and estimators that attain those bounds across broad settings.

1 Introduction

The paper addresses unclear relationships among existing sufficient conditions and whether prior NPIV minimax bounds and estimators remain valid under other conditions.It also notes the absence of published minimax-rate results for severely ill-posed NPIV.Source: §1 · p. 2
Approximation and link conditions are presented as generalizations of the general source condition and are implied by conditions used in prior NPIV work.Source: §1 · p. 2
Under two basic regularity conditions, the paper derives mean integrated squared error minimax lower bounds for NPIR and NPIV in mildly and severely ill-posed cases.Source: §1 · p. 3
The results recover known minimax-optimal rates in special cases and cover new cases whose specifications satisfy the approximation and link conditions.Source: §1 · p. 3
The introduction also discusses what could happen when the link condition on the conditional expectation operator is not satisfied.Source: §1 · p. 3
The NPIR projection estimator and NPIV sieve minimum distance estimator are rate-optimal uniformly over a large class of structure functions.The result allows for arbitrarily decaying singular values of the conditional expectation operator.Source: §1 · p. 3

2 NPIV models and basic regularity conditions

The section formulates NPIV through a conditional moment equation and develops regularity conditions for studying minimax risk and rate-attaining estimators. It defines approximation and link conditions, relates them to source conditions and existing smoothness assumptions, and notes that one cited convergence-rate result does not hold uniformly over its stated function class.

2 NPIV models and basic regularity conditions

NPIV relates the structural function h0 to observables through E[Y | W] = Kh0(W), with uniqueness imposed by restricting the solution to a chosen class H.Here K is the conditional expectation operator, and H encodes prior information such as smoothness or shape properties.Source: §2 · p. 4

2.1 Basic regularity conditions

The paper seeks a minimax mean integrated squared error lower bound for NPIV that applies to general joint laws without explicit distributional restrictions, and estimators attaining its rate.The stated lower-bound target is over all possible estimators of h in H.Source: §2.1 · p. 5
The approximation condition controls basis-approximation error for structure functions, while the link condition bounds K's smoothing action relative to a Hilbert scale generated by B.The approximation condition also implies that H(r, R) is compact in L2_X.Source: §2.1 · p. 6
The approximation condition controls function-class complexity through finite-dimensional approximation, while the link condition connects its approximation basis to the eigenbasis of K∗K.The paper calls Assumptions 2.1 and 2.2 the approximation and link conditions, respectively.Source: §2.1 · p. 8

2.2 Relation to source conditions

A general source condition uses a continuous ψ with ψ(0)=0 and finite R, includes ψ(λ)=λ^1/2 as the original source condition, and implies Assumptions 2.1–2.2 when u_k=e_k.Here {e_k} is the eigenfunction basis of K∗K.Source: §2.2 · p. 8
Darolles, Florens and Renault (2002) use h0 ∈ HDF_R to establish a mean-squared-error convergence rate, but it does not hold uniformly over that class.The series in (2.6) is not uniformly bounded; the finite R in Hsource supplies that bound.Source: §2.2 · p. 9
Hall and Horowitz (2005) use a hyperrectangle in the K∗K eigenfunction basis; for β > 1/2, their condition implies Assumptions 2.1–2.2 under the stated basis choices.The passage identifies β > 1/2 as playing the role of r + 1/2.Source: §2.2 · p. 9

3 The lower bound

The section derives minimax risk lower bounds for NPIR and NPIV, using a reduction from NPIV to NPIR to establish the latter. The bounds cover mildly and severely ill-posed cases under the paper’s regularity conditions.

3.1 Reduction from NPIV model to NPIR model

The reduction defines a corresponding NPIR class by prescribing the conditional law of X given W through K while allowing the conditional residual law to vary.Source: §3.1 · p. 10
The NPIR model is more informative than NPIV: each NPIV estimator has a corresponding NPIR estimator with the same risk.Source: §3.1 · p. 10

3 The lower bound

The lower-bound argument uses asymptotically least-favorable Bayes priors and assumes Gaussian residuals, though the proof can extend to other error densities.For other densities, bounding distances between alternatives may require additional work.Source: §3 · p. 9, §3.2 · pp. 10–11

3.2 The lower bound

The NPIR and NPIV models share minimax risk lower bounds, including δ_n ≍ (log n)^−2r/a in the severely ill-posed case.The NPIV lower bound follows from the NPIR bound under the stated assumptions.Source: §3.2 · pp. 10–11

4 An upper bound for the NPIR model

A simple projection estimator achieves the NPIR minimax rates under the stated conditions. The section also compares the rates with ordinary nonparametric regression and relaxes the exact link condition.

4 An upper bound for the NPIR model

The NPIR projection procedure uses eigenfunctions of B and is presented as a simple generalization of spectral cut-off using eigenfunctions of K*K.Source: §4 · p. 12

4.1 Assumption.

Under the stated assumptions, the projection estimator is minimax-rate optimal in both mildly and severely ill-posed cases.Source: §4.1 · p. 13
Unlike ordinary nonparametric regression, NPIR has a variance term that grows with the ill-posedness weights; its severe-case rate is independent of ε.Source: §4.1 · p. 13

4.1 Relaxation of the exact link condition

Assumption 4.2 may fail more generally, but the proposed relaxation preserves the minimax rate when estimating the coefficient ⟨h, uℓ⟩.Source: §4.1 · pp. 14–15

5 An upper bound for the NPIV model

The NPIV upper-bound result uses a sieve minimum distance estimator and permits a Riesz basis in place of an ideal orthonormal basis. Under the stated conditions, it attains minimax rates in both ill-posedness regimes.

5 An upper bound for the NPIV model

The NPIV approach uses sieve least squares and a sieve minimum distance estimator to estimate the structure function.Source: §5 · pp. 15–16
Assumption 5.1 permits a Riesz basis instead of the ideal orthonormal basis while preserving the order of approximation error.Construction may require information about the tail behavior of X’s density.Source: §5 · p. 16

5.2 Assumption.

The theorem’s conditions include sieve-basis and design-density requirements, including a continuous density of W bounded away from zero on its support.Source: §5.2 · pp. 16–17
The sieve minimum distance estimator is minimax-rate optimal for NPIV in both mildly and severely ill-posed cases.The theorem gives δ_n ≍ (log n)^−2r/a in the severe case.Source: §5.2 · p. 17

6 More on regularity conditions

The examples show how the approximation space and the operator’s eigenfunctions affect the regularity class and rates. They contrast smooth eigenfunctions with rough ones, emphasizing that rate optimality is tied to the function class.

6.1 Examples of K having infinitely times differentiable eigenfunctions

For infinitely differentiable eigenfunctions, the approximation and link conditions yield minimax-optimal rates for the generated Hilbert scale.Source: §6.1 · p. 18
When a < 1/2, the approximation-space scale need not match the operator’s scale, allowing smooth non-periodic functions to have high regularity.The text recommends the approximation-space approach for smooth, non-periodic functions on [0, 1].Source: §6.1 · p. 18
In the convolution example, the approximation-space rate is roughly n^−1/(2a+1), compared with roughly n^−1/(2a+2) for source-condition methods.Source: §6.1 · p. 19

6.2 Examples of K having non-differentiable eigenfunctions

With non-differentiable eigenfunctions, the Hilbert scale can contain jump functions, and rate optimality depends on whether the structure function is regular in the method’s designed setting.Source: §6.2 · pp. 19–20
The paper’s conclusion is that the associated function class is an essential part of specifying rate optimality.Source: §6.2 · p. 20

7 Perspectives

The paper clarifies regularity conditions for NPIV convergence rates, establishes minimax risk bounds, and shows rate optimality in mildly and severely ill-posed cases. It also identifies adaptive procedure selection as an open problem.

7 Perspectives

The paper clarifies relations among existing regularity conditions for NPIV convergence rates.Source: §7 · p. 20
The simple SMD estimator attains the minimax risk lower bound and is rate-optimal in both mildly and severely ill-posed NPIV cases.The bounds hold under two basic regularity conditions.Source: §7 · p. 20
Many of the paper’s ideas can be adapted to other ill-posed econometric inverse problems; for mildly ill-posed IV quantile regression, a kernel-based Tikhonov estimator reaches the minimax rate under similar conditions.Source: §7 · p. 20
The penalized SMD estimator for nonlinear, possibly nonsmooth conditional moment models also achieves the paper’s NPIV minimax risk lower bound.Source: §7 · p. 20
Choosing regularization parameters without knowing true regularity, or selecting among procedures generated by different operators B, remains an open problem.Source: §7 · p. 20

Appendix: Proofs

The proofs construct minimax lower bounds and match estimator risks to those bounds across mild and severe ill-posedness. They also establish a uniform NPIV risk result over the stated structure-function class.

Appendix: Proofs

The lower-bound proof controls Kullback–Leibler divergence between sign-indexed alternatives before deriving a risk lower bound.Source: Appendix: Proofs · pp. 23–24
For mild ill-posedness, choosing m ≍ n^(1/(2ǫr+2ǫa+1)) yields the asymptotic lower bound.Source: Appendix: Proofs · p. 24
The estimator attains the minimax rate when its variance matches the lower-bound order and its bias is no larger.Source: Appendix: Proofs · p. 25
For mild and severe ill-posedness, the upper-bound arguments choose m using the corresponding case-specific rates.The mild case uses m ≍ n^(1/(2ǫr+2ǫa+1)); the severe case uses m = c log(n)^(1/(aǫ)) with sufficiently small c.Source: Appendix: Proofs · pp. 25–26
Under the proof’s assumptions, the NPIV risk bound holds uniformly over the structure-function class except on an event whose probability tends to zero.Source: Appendix: Proofs · pp. 26–27