PaperlayerView exact revision on arXiv

Source-linked AI summary

PDE-constrained inverse problems at the $\sqrt{n}$ rate via debiased physics-informed neural networks

Yves Atchade, Debarghya Mukherjee

arXiv:2609.12301v1math.STstat.MEstat.MLstat.TH

TL;DR

Standard PINN-based estimators can inherit the slow nonparametric rate of solution estimation, producing biased and inefficient inference for finite-dimensional PDE parameters. The paper introduces influence-function debiasing and a Bayesian analogue, yielding √n-rate inference without neural-network undersmoothing and near-minimax rates for Sobolev regression and derivatives.

  • Problem

    Standard PINN-based parameter estimators can inherit the slower nonparametric rate of solution estimation, leading to biased and statistically inefficient inference.

    Source: §1 · p. 2, §2 · p. 5

  • Method

    A two-step procedure combines neural-network estimation of the PDE solution with an influence-function debiased score; a Bayesian analogue modifies the likelihood to incorporate that score.

    Source: §1 · pp. 2–3

  • Results

    The frequentist estimator is √n-consistent and asymptotically normal without undersmoothing; the Bayesian posterior contracts at √n with matching asymptotic covariance, and neural networks achieve near-minimax rates for Sobolev regression and derivatives.

    Source: §1 · pp. 3–4, §5 · pp. 24–25

  • Takeaways & Limitations

    The framework provides asymptotically agreeing Bayesian and frequentist inference while removing first-order sensitivity to errors in the estimated PDE solution.

    Source: §1 · p. 4, §5 · p. 24

    The limiting variance is not established to attain the semiparametric efficiency bound, and the theory focuses on finite-dimensional parameters.

    Source: §3.2 · p. 17, §5 · p. 25

Abstract

from arXiv · show

We study the problem of estimating unknown parameters in PDE-constrained inverse problems from noisy observations, where the PDE solution is approximated using Physics-Informed Neural Networks (PINNs). While PINNs have demonstrated remarkable empirical success, existing estimators often inherit the slow nonparametric convergence rate of the neural-network solution, leading to biased and statistically inefficient inference for the finite-dimensional parameters of interest. To address this, we propose a two-step debiased estimation procedure that combines neural-network-based nonparametric estimation with an influence-function-based bias correction. By eliminating the first-order sensitivity of the estimator to errors in the nuisance function, our procedure yields a $\sqrt{n}$-consistent and asymptotically normal estimator without requiring undersmoothing of the neural network component. We further extend this framework to Bayesian inference by replacing the original likelihood with a debiased quasi-likelihood and establish a Bernstein-von Mises theorem showing that the resulting posterior contracts at the $\sqrt{n}$-rate with an asymptotic covariance matching that of the frequentist estimator. As a by-product of our analysis, we establish near-minimax optimal convergence rates for estimating a nonparametric regression function and its derivatives in Sobolev spaces using neural networks. Extensive numerical experiments corroborate our theoretical findings and demonstrate the necessity of the proposed debiasing procedure for valid statistical inference in PDE-constrained inverse problems.

1 Introduction

The introduction frames PDE parameter inference as difficult because neural-network estimates of the unknown solution can make parameter estimates biased and slower than the parametric rate. It presents debiasing for frequentist and Bayesian inference, alongside neural-network guarantees for estimating Sobolev regression functions and their derivatives.

1 Introduction

  • Standard PINN parameter estimates can inherit the solution network’s slower nonparametric rate, producing bias and inefficient inference for finite-dimensional PDE parameters.The authors identify whether parametric √n estimation remains possible with PINNs as the motivating question.Source: §1 · p. 2
  • An influence-function-based correction removes the debiased score’s first-order sensitivity to solution-estimation error, yielding √n-consistent, asymptotically normal parameter estimates without undersmoothing.The proposed procedure first estimates the solution with a neural network, then applies the bias correction.Source: §1 · p. 2
  • The analysis establishes near-minimax-optimal simultaneous rates, up to a log factor, for neural-network estimation of Sobolev regression functions and their derivatives.The introduction contrasts this with prior neural-network regression theory that often assumes more restrictive finite Hölder smoothness.Source: §1 · p. 3
  • The Bayesian extension replaces the likelihood with a debiased quasi-likelihood and establishes √n posterior contraction with asymptotic covariance matching the frequentist estimator.The stated posterior result assumes a mild log-concavity condition on the prior.Source: §1 · pp. 3–4
  • Numerical experiments corroborate the theory and demonstrate the necessity of debiasing for valid inference on the PDE parameter.The introduction also previews frequentist and Bayesian theoretical guarantees and the numerical studies that validate them.Source: §1 · p. 4

2 Methodology

Section 2 formulates parameter estimation from noisy PDE-solution observations and explains why standard PINNs can yield slow, biased parameter inference. It develops a two-step frequentist debiasing procedure and a Bayesian correction based on the debiased score.

2 Methodology

  • Noisy solution observations use a known design distribution, while standard PINNs jointly estimate the parameter and solution and may inherit slow nonparametric error in parameter inference.The framework leaves extension to an unknown design distribution as future work.Source: §2 · p. 5

2.1 Debiased estimation of θ⋆

  • The debiased score removes first-order sensitivity to solution-estimation error, leaving nuisance error to affect parameter inference through higher-order terms.The score is constructed from the PDE risk and a representer for the derivative with respect to the nuisance function.Source: §2.1 · pp. 6, 8
  • The first step estimates the solution and its derivatives with neural networks and obtains a consistent preliminary parameter estimate; the second optimizes the debiased score.The preliminary estimate need only be consistent, and its convergence rate may be slow.Source: §2.1 · pp. 6–7
  • The debiased estimator is √n-consistent and asymptotically normal, while the neural-network estimator achieves minimax-optimal rates, up to logarithmic factors, for the function and its derivatives.The rate result for the regression function and its derivatives holds over the specified Sobolev class.Source: §2.1 · pp. 6, 8

2.2 Debiased Bayesian inference

  • Under the original Bayesian PINN formulation, the parameter’s marginal posterior inherits the slow plug-in rate from estimating the solution.This motivates correcting the bias in Bayesian parameter inference.Source: §2.2 · p. 9
  • The Bayesian procedure uses a conditional posterior built from the debiased score, with either a fitted solution or a suitable marginal posterior for the nuisance function.The full joint posterior can be more computationally challenging than the plug-in posterior.Source: §2.2 · pp. 9–10
  • Matching Bayesian inference to frequentist inference requires a careful choice of the positive-definite matrix Vn; extension to the full posterior requires suitable nuisance-posterior contraction.The stated extension applies when the nuisance posterior contracts at rates similar to those of the solution estimator.Source: §2.2 · p. 10

3 Theoretical results

The theoretical results establish optimal-rate neural-network estimation of the PDE solution and its derivatives, characterize the debiased score and plug-in estimator, and develop conditions for √n frequentist and Bayesian inference, including their variance structure and matching limits.

3 Theoretical results

  • The theory assumes a Sobolev-smooth PDE solution and sub-Gaussian errors, with β − 2τ > d for debiasing; under Hölder smoothness, the stated threshold becomes β − 2τ > d/2.The authors note that optimality of these smoothness conditions is unclear.Source: §3 · p. 12
  • The parameter results require a locally well-posed and identifiable inverse problem with positive local curvature, alongside smoothness and growth conditions on Ψθ.The global population-risk minimizer condition rules out distant parameters that fit the true solution equally well; local positive curvature stabilizes the parameter minimizer.Source: §3 · pp. 13–14

3.1 A representation theorem for the Gateaux derivative of ∇θR

  • For β > 2τ + d/2, the Gateaux derivative of ∇θR with respect to the solution function has an integral representation with a bounded, sufficiently smooth representer Gθ,u.The representation supplies the representer needed to construct the debiased score and its smoothness supports the main results.Source: §3.1 · p. 15

3.2 Frequentist approach

  • The neural-network estimator achieves minimax-optimal rates for estimating the Sobolev regression function and its derivatives, and the PINN estimator attains the same bounds.The analysis uses RePU networks with fixed depth and growing width; the PINN result additionally requires the stated penalty and width conditions.Source: §3.2 · pp. 16–17
  • The initial plug-in estimator θ̂PI is consistent and attains at least a nonparametric convergence rate under Assumptions 1–5 and a compact convex parameter space.The supplied theorem statement introduces this result but does not include its rate expression.Source: §3.2 · p. 17
  • The plug-in estimator’s nonparametric rate is driven by estimation of the τ-th derivative of u⋆, but suffices to form a shrinking neighborhood for debiasing.Its rate essentially coincides with posterior contraction rates established in [89, Theorem 3.1].Source: §3.2 · p. 17
  • With β−2τ>d ensuring consistent representer estimation, the debiased estimator is √n-consistent and asymptotically normal under Assumptions 1–5 and compact convex Θ.Source: §3.2 · p. 17
  • The debiased estimator’s limiting variance has a sandwich form, and its comparability with the likelihood-based estimator’s variance remains unresolved.The likelihood-based alternative is computationally intractable because it repeatedly solves a nonlinear PDE, and its efficiency requires potentially stronger forward-map regularity conditions.Source: §3.2 · pp. 17–18
  • A consistent limiting-variance estimate uses the Hessian at the debiased estimate, a sample covariance analog, and residual-based noise-variance estimation.Source: §3.2 · p. 18

3.3 Bayesian theoretical results

  • The debiased posterior contracts at the √n-rate and satisfies a Bernstein–von Mises theorem under Assumptions 1–2 and 4–6.Source: §3.3 · pp. 18–19
  • The shrinking-variance Gaussian prior focuses posterior mass near θ⋆, so strong convexity of R(·, u⋆) is needed only on those neighborhoods.The prior and first-stage estimator have asymptotically vanishing impact on the posterior; posterior intervals are asymptotically equivalent to confidence intervals but may lack good frequentist coverage, while a particular choice yields asymptotic Bayesian–frequentist matching.Source: §3.3 · pp. 19–20

4 Numerical Experiments

The simulations evaluate debiased and standard estimators in a one-dimensional elliptic diffusion problem, using two sample sizes and repeated Monte Carlo runs. They examine finite-sample inference and sensitivity to the Sobolev penalty weight.

4 Numerical Experiments

  • The simulation investigates finite-sample estimator performance and the necessity of debiasing for valid inference in a one-dimensional elliptic diffusion equation.The example has boundary conditions u(0) = u(1) = 0 and a parametric coefficient; its known source function is specified in the paper.Source: §4 · p. 20

4.1 Data-generating process

  • The diffusion example uses true parameter θ⋆ = (0.5, 0.3, 2.0) and Gaussian noise with σ = 0.02, at sample sizes n = 500 and 5000.The forward solution is approximated by a finite element method with 4000 elements.Source: §4.1 · p. 20

4.2 Implementation of the first-stage estimators

  • The first-stage solution estimate uses a fully connected network with three width-64 hidden layers, a W^4,2 penalty, and a Dirichlet boundary penalty.The activation is C^4; the Sobolev penalty weight is cross-validated, and the PINN adds a profiled PDE penalty.Source: §4.2 · p. 21

4.3 Construction of the debiased estimator

  • The debiased estimator uses a representer for the derivative of the PDE risk score, while the Bayesian version targets a debiased posterior with a specified scale matrix and Gaussian prior.The Bayesian posterior is sampled using adaptive random-walk Metropolis; the frequentist search is initialized around the plug-in estimate.Source: §4.3 · p. 21

4.4 Monte Carlo design and results

  • Both plug-in and PINN estimators show substantial bias across coordinates, which the debiased estimator markedly reduces; its 95% intervals have approximately nominal coverage.The debiased estimator's asymptotic normality is supported by Q–Q plots, and the posterior is approximately Gaussian and close to the Bernstein–von Mises limit.Source: §4.4 · p. 22, Figure 1 · p. 22, Figure 2 · p. 24

4.5 Robustness to tuning parameter

  • Across six Sobolev-penalty values spanning five orders of magnitude, the debiased procedures are markedly more robust than the plug-in estimator.Each penalty value was evaluated over 100 simulation runs at n = 500.Source: §4.5 · pp. 22, 24

5 Conclusion and future work

The paper presents frequentist and Bayesian debiasing for PDE-constrained inference, with √n-consistent frequentist estimation without neural-network undersmoothing and a Bayesian posterior asymptotically agreeing with frequentist inference. It also establishes near-minimax-optimal neural-network rates for Sobolev regression functions and derivatives, while leaving efficiency and broader parameter targets open.

Conclusion

  • The influence-function-based debiased score removes first-order sensitivity to nuisance-function errors, yielding √n-consistent, asymptotically normal estimation without undersmoothing.The Bayesian analogue uses a debiased quasi-likelihood and has an asymptotic covariance matching the frequentist estimator.Source: §5 · p. 24
  • Neural-network estimators achieve near-minimax-optimal convergence rates for Sobolev-smooth regression functions and their derivatives simultaneously.The paper frames this as a by-product that may be of independent interest.Source: §5 · p. 25

Future work

  • Whether the debiased estimator’s limiting variance is semiparametrically efficient remains unclear, and extending the framework beyond finite-dimensional targets is open.The discussion also identifies high-dimensional inference with variable selection as a future direction.Source: §5 · p. 25

6 Proof of Theorem 3.8

The proof establishes asymptotic normality by showing the debiased estimator’s expansion has negligible remainder terms and a stable limiting Hessian. Empirical-process control and smoothness conditions bound the terms involving the estimated PDE solution.

Proof of Theorem 3.8

  • Convergence of the debiased estimator and continuity of the Hessian give an invertible limiting Hessian with probability approaching one.The proof then uses the expansion and the central limit theorem with Slutsky’s theorem to obtain asymptotic normality.Source: §6 · pp. 26–27
  • For the T141 remainder, the proof uses Donsker-class stochastic equicontinuity under the condition βα*>d/2.The argument bounds covering numbers for the Sobolev function class and the parameter space.Source: §6 · p. 28
  • The T142 remainder is negligible under β>2τ+d, while the T3 term is controlled through entropy bounds and a shrinking neighborhood around θ*.Together these controls make the scaled remainder terms negligible.Source: §6 · pp. 29–31

S.1.1 (1 + 1) transport PDE

In the transport-PDE experiment, the plug-in and PINN estimators retain substantial √n-scale bias, whereas debiasing reduces bias and brings the scaled-error distribution toward the theoretical limit as sample size grows. The study evaluates three estimators across sample sizes using repeated Monte Carlo runs.

Experimental setup

  • The experiment uses a transport equation with θ*=2 and generates noisy observations at n=500, 3000, and 5000.The observations use Gaussian noise with σ=0.5 and uniformly sampled inputs.Source: §S.1.1 · p. 39

Implementation

  • The first-stage solution is represented by a three-hidden-layer, width-64 RePU network, with automatic differentiation used for first- and second-order derivatives.Training uses fixed tuning parameters, Monte Carlo collocation, and early stopping.Source: §S.1.1 · pp. 39–40

Debiased estimator

  • The debiased score is affine in θ, so its second-stage root is explicit; the same observations fit the first stage and evaluate the score.The root is projected onto a neighborhood to guard against rare finite-sample instability.Source: §S.1.1 · p. 40

Results

  • The plug-in and PINN estimators have substantial bias that does not vanish on the √n scale, while the debiased estimator’s distribution approaches the limiting law as n grows.The figure compares the three estimators’ scaled errors over 100 runs at n=500, 3000, and 5000.Source: §S.1.1 · p. 40, Figure 4 · p. 41

S.1.2 A linear advection-reaction equation

In the advection–reaction experiment, plug-in and PINN estimates are biased downward in both coordinates, while debiased estimates are approximately centered and approach the limiting Gaussian law. Confidence-interval coverage is close to nominal at larger sample sizes but lower at n=500.

Experimental setup

  • The experiment sets θ*=(0.9, 1.2) and generates observations at n=500, 3000, and 5000.The inputs are uniformly sampled, and the observation noise has σ=0.35.Source: §S.1.2 · pp. 41, 43

Implementation

  • The first-stage solution is estimated with a three-hidden-layer, width-64 RePU network, and the PINN adds a PDE penalty to the squared-loss and Sobolev-penalty fit.The networks use automatic differentiation and are trained with early stopping.Source: §S.1.2 · p. 43

Debiased estimator

  • Because the debiased score is affine in θ, its root is available in closed form and is projected onto a ball around the plug-in estimate for stability.The projection constraint is active in only a few n=500 replications and inactive at n=3000 and 5000.Source: §S.1.2 · pp. 43–44

Results

  • At n=3000 and 5000, debiased estimates are approximately centered at θ*, with 95% confidence intervals attaining 95%–98% coverage; coverage is around 90% at n=500.The plug-in and PINN estimates are biased downward in both coordinates, while the debiased scaled-error density concentrates around the limiting law as n grows.Source: Figure 5 · p. 42, §S.1.2 · p. 44

S.2 Some results on Sobolev spaces

This section sets notation for Sobolev and Hölder function classes and states embedding and interpolation results used in the theoretical analysis. It also explains how the Gagliardo–Nirenberg inequality is applied on the unit cube.

S.2 Some results on Sobolev spaces

  • The section uses W^{β,p}(Ω), C^{k,γ}(Ω), and C^K(Ω) for Sobolev, Hölder, and uniformly bounded K-times differentiable function classes, respectively.C^K(Ω) denotes C^{K,0}(Ω).Source: §S.2 · p. 44
  • The Sobolev embedding theorem treats separately the cases where β − d/p is an integer and is not an integer.The stated results concern Ω = (0, 1)^d and functions in W^{β,p}(Ω).Source: §S.2 · p. 44
  • Interpolation bounds derivatives of order j for functions in W^{β,p1}(Ω) ∩ Lp0(Ω), with an endpoint restriction in specified critical cases.For 1 < p < ∞ and nonnegative integer β − j − d/p, the endpoint θ = 1 is excluded.Source: §S.2 · pp. 44–45
  • The Gagliardo–Nirenberg inequality on the cube is obtained by extending functions to R^d and applying the full inequality there.The cube is a bounded Lipschitz domain, and the proof notes the usual endpoint restriction in the critical Sobolev case.Source: §S.2 · p. 45

S.3 Proof of Proposition 3.4

The proof constructs a representer for the Gateaux derivative of the risk with respect to the function argument. Under the stated regularity conditions, it establishes regularity and boundedness properties for this representer and controls its changes across arguments.

S.3 Proof of Proposition 3.4

  • Sobolev embedding and the assumed regularity conditions provide uniform bounds on derivatives of u, supporting the derivative and representer calculations.For u in the specified Sobolev ball, the proof obtains a uniform L∞ bound for ∂^αu when |α| ≤ 2τ.Source: §S.3 · p. 45
  • The Gateaux derivative in the function direction h is represented by G_{θ,u}, which belongs to W^{β−2τ,2}(Ω) and is uniformly bounded in x.Its L∞ bound is given by c0(θ), a function bounded on compact sets.Source: §S.3 · p. 47
  • The proof controls changes in the representer as u and θ vary, completing the stated representer bound.The argument combines the preceding bounds on differences in u and θ.Source: §S.3 · pp. 49, 52

S.4.1 Proof of Theorem 3.5

The proof of Theorem 3.5 balances neural-network approximation and stochastic error to obtain high-probability estimation bounds. It then shows that the derivative bounds also hold for the underlying untruncated network.

S.4.1 Proof of Theorem 3.5

  • The argument balances approximation error against stochastic error using an approximating network and a pseudo-dimension bound.The approximation step selects a network from FNN, and the resulting class complexity controls the stochastic term.Source: §S.4.1 · pp. 53–54
  • With N = (n/log n)^{d/(2β+d)}, the proof derives a high-probability bound for the neural-network estimator.The proof bounds the network class pseudo-dimension by O(W log W) for fixed depth and uses W ≍ N.Source: §S.4.1 · pp. 54–55
  • With high probability, the truncated estimator satisfies bounds for derivatives indexed by multi-indices with 0 ≤ |k| < β.The excerpt does not state the bound values, but identifies the range of derivative orders covered.Source: §S.4.1 · p. 56
  • For sufficiently large n on the same high-probability event, truncation is inactive, so the derivative bounds transfer to the underlying untruncated network.The proof uses m > 2∥u⋆∥∞ to show the truncation map acts as the identity in this regime.Source: §S.4.1 · p. 57

S.4.2 Proof of Corollary 3.6

The proof extends the estimation bounds to the PINN estimator by showing that the additional nonnegative penalty contributes a term of the same order as existing terms. The subsequent Sobolev-norm and derivative-rate arguments therefore carry over.

S.4.2 Proof of Corollary 3.6

  • The additional penalty term is of the same order as terms in the Theorem 3.5 proof, so its subsequent steps yield the claimed Sobolev-norm and derivative-estimation bounds for the PINN estimator.The argument uses R(θ⋆, u⋆) = 0 and bounds the added term using the stated Lipschitz property and Sobolev embedding.Source: §S.4.2 · pp. 57–58

S.4.3 Proof of Lemma 3.7

The proof uses an argmin continuous mapping theorem for the M-estimation problem. It establishes consistency of the PINN parameter estimator and gives an error-rate bound.

S.4.3 Proof of Lemma 3.7

  • The argument follows the argmin continuous mapping theorem for M-estimation.The proof cites Theorem 3.2.2 of the Second edition of.Source: §S.4.3 · p. 58
  • Assumption 1 yields bounded K0u and K1u over Ω, while continuity and compactness bound Ψθ(x,z).The stated smoothness condition is β − 2τ > d/2.Source: §S.4.3 · p. 58
  • The proof concludes that ˆθPI is consistent for θ∗.It then bounds ∥ˆθPI − θ∗∥2 by o_p(r_n^-1/2).Source: §S.4.3 · p. 59

S.5 Supporting technical results for the Proof of Theorem 3.8

This section develops technical ingredients for the proofs of Theorems 3.8 and 3.9, including conditions for the debiased estimator, posterior bounds, and probability, contraction, and local-approximation arguments. It also establishes RePU network approximation results for Sobolev functions and auxiliary empirical-process, truncation, and Donsker properties.

S.5 Supporting technical results for the Proof of Theorem 3.8

  • With probability approaching one, the minimizer of the squared debiased score norm over Θn lies in its interior.The neighborhood Θn is centered at the PINN estimator and shrinks while containing θ* with probability tending to one.Source: §S.5 · pp. 60, 62
  • With probability approaching one, the debiased estimator solves the debiased score equation, supported by a positive lower bound on the minimum singular value of its Jacobian.The proof establishes the Jacobian bound by controlling its difference from the population curvature matrix.Source: §S.5 · pp. 62, 64

S.6 Proof of Theorem 3.9

  • The proof controls exceptional events using the assumed neural-estimator rate and concentration bounds, with several event failures bounded by polynomially small probabilities.The text gives bounds of order n^-C for selected events and 2/n^A for another event.Source: §S.6 · pp. 65–66

S.6.1 Proof of the contraction rate of Theorem 3.9

  • The contraction proof combines a lower bound with curvature and prior-gradient control to bound posterior mass outside a shrinking neighborhood, accounting for exceptional events.The proof uses the curvature parameter and concludes the bound after controlling the probability of the complement of the good event.Source: §S.6.1 · pp. 66, 68, 69

S.6.2 Proof of the Bernstein-von Mises approximation of Theorem 3.9

  • The Bernstein–von Mises proof restricts the posterior and Gaussian approximation to a local ball, then uses a local expansion and bounds on likelihood variation to compare them.The proof first uses contraction to control mass outside the ball and develops the expansion of the debiased score around θ*.Source: §S.6.2 · pp. 69, 70, 73
  • The proof of Theorem 3.9 uses a high-probability bound on Dn leaving the specified event to complete the Bernstein–von Mises approximation.The displayed probability is bounded by n^-C for some C>0, after which the proof invokes the preceding display and (S.6.8).Source: §S.6.2 · p. 73

S.6.3 Technical supporting lemmas for Theorem 3.9

  • The technical lemmas also obtain a first-order Taylor expansion with integral remainder for the debiased score’s change under perturbations of the nuisance function.This expansion is combined with the preceding score identity to establish the stated decomposition.Source: §S.6.3 · pp. 73–74
  • The proof’s supporting lemmas lower-bound posterior normalization on good data events and establish posterior concentration outside fixed neighborhoods under the stated assumptions.The concentration result also uses the sample-size conditions and prior assumptions specified for Lemma S.6.3.Source: §S.6.3 · pp. 74, 77

S.7 Empirical processes results

  • The empirical-process results give a deviation bound for bounded function classes with sub-Gaussian noise when their covering numbers grow at an exponent below 2.The bound follows from an entropy-integral argument and has a constant depending on the stated entropy and envelope parameters.Source: §S.7 · p. 78

S.7.1 Deviation bounds

  • Lemma S.7.3 assumes Assumptions 4–5 and β≥2τ+d, with 0≤s≤β; for ρ∈(0,ρ̄], A≥2 and n meeting (S.7.3) and n≥C1A log(n), its conclusion invokes a union bound over j=1,...,q.Source: §S.7.1 · pp. 79, 83

S.8 Neural network based approximation on Sobolev space

  • For f in W^ν,2((0,1)^d) and integer ν≥3, RePU(ν) networks achieve W^r,2 error at most Cν,d K^-(ν-r)∥f∥W^ν,2 for every r=0,…,ν.The construction has depth at most c1(1+⌈log2 d⌉), width at most c2(K+ν)d, and sparsity at most c3(K+ν)d; these constants do not depend on K.Source: §S.8 · p. 83
  • The full tensor-product spline feature map is computed exactly by a RePU(ν) network of depth at most 1 + ⌈log2 d⌉, with width and active parameters scaling as O((K + ν)^d).The construction combines univariate spline values using balanced-tree multipliers; its architecture bounds hold up to constants depending on ν and d.Source: §S.8 · pp. 86–87
  • For targets in W^β,2 with β < ν, the same RePU(ν) architecture has approximation order governed by β, and choosing ν > β does not improve convergence order.The depth, width, and sparsity bounds continue to hold up to constants depending on (β, ν, d).Source: §S.8 · p. 87

S.9 Additional technical lemmas and their proofs

  • The truncation function T_m,p,δ is C^p, odd, nondecreasing, and piecewise polynomial of degree at most 2p + 2; it equals x for |x| ≤ m − δ and saturates at m sign(x) for |x| ≥ m.The transition region provides a smooth connection between the identity and the saturated values.Source: §S.9 · p. 88
  • Under Assumptions 1, 4, and 5 with compact Θ, H_jk,n and A_jk,n are uniformly bounded and P-Donsker uniformly in n.The lemma establishes this for the deterministic sets U_n used in the proof of Theorem 3.8.Source: §S.9 · p. 89
Loading 2609.12301v1…