PaperlayerView exact revision on arXiv

Source-linked AI summary

Bernstein--von Mises theorems for Bayesian probabilistic numerics

Sascha Gaudlitz, Sven Wang

arXiv:2609.04124v1math.STcs.NAmath.NAstat.TH

TL;DR

Noisy random collocation raises the question of the full posterior’s asymptotic shape for nonlinear PDE solutions. The paper uses a truncated Gaussian series prior to establish contraction and function-space Gaussian approximations, including strong-topology results enabled by differential-operator regularity.

  • Problem

    For noisy-collocation GP methods, the asymptotic shape and frequentist calibration of the full posterior remain largely open beyond deterministic or MAP approximation and neural-network posterior contraction.

    Source: §1 · p. 1

  • Method

    The paper places a truncated Gaussian series prior on the PDE solution and studies the posterior under noisy random evaluations using Bayesian nonparametric and function-space BvM methods.

    Source: §1 · p. 1, §1.1 · p. 2

  • Results

    The posterior admits Gaussian approximations in positive-order Sobolev spaces and, under suitable conditions, in the uniform topology; estimation in H^γ(X) has a parametric N^(-1/2) rate.

    Source: §1 · p. 2, §3.2 · p. 11

  • Takeaways & Limitations

    Differential-operator inversion gains regularity, permitting posterior Gaussian approximations in stronger topologies than those typically available for smoothing inverse problems.

    Source: §1 · p. 2

    The stated setting assumes random design points drawn from a density bounded above and away from zero, with independent measurement errors.

    Source: §1.1 · p. 2

Abstract

from arXiv · show

We study probabilistic numerical methods for solving nonlinear PDEs from a Bayesian nonparametric perspective. Given noisy evaluations at random collocation points, we place a truncated Gaussian series prior on the unknown solution and establish contraction at the minimax nonparametric rate, up to a logarithmic factor. Our main results give Gaussian approximations of the posterior in positive-order Sobolev spaces and, under suitable conditions, in the uniform topology. This contrasts with classical ill-posed inverse problems, where Bernstein--von Mises theorems typically require substantially weaker topologies. Here, the observation operator is differential rather than smoothing, and inversion of its linearisation gains regularity, making these strong-topology results possible. The posterior may be centred at either the posterior mean or the posterior mode. We further prove that the Gaussian Laplace approximation is asymptotically equivalent to the true posterior at a $\sqrt{N}$-scale.

1. Introduction.

The paper develops Bayesian nonparametric guarantees for PDE solution methods, addressing the open question of the full posterior’s asymptotic shape under noisy collocation. It establishes contraction and function-space Gaussian approximations, with stronger topologies made possible by the regularising effect of differential-operator inversion.

1.1. Main contributions.

  • The differential operator’s linearisation gains regularity on inversion, enabling control in stronger Sobolev norms than is typical for smoothing inverse problems.The paper verifies its assumptions for four representative PDE classes.Source: §1.1 · pp. 3–4
  • A truncated Gaussian series prior on nested approximation spaces serves as both a numerical discretisation and a statistical regularisation.The spaces are assumed to satisfy suitable Jackson and Bernstein estimates.Source: §1.1 · p. 3
  • For u0 in H^(m+β)(X), the posterior contracts in H^m(X) at N^(-β/(2β+d)) log(N), up to logarithmic factors the minimax rate.The posterior mean and MAP attain the same convergence rate.Source: §1.1 · p. 3
  • The full posterior’s joint fluctuations are asymptotically Gaussian at the N^(-1/2) scale in H^γ(X), with L2 and, under stronger conditions, L∞ convergence.The limiting Gaussian is tight for 0 < γ < m − d/2; the L∞ result applies when d/2 < m − d/2.Source: §1.1 · p. 3
  • Under second-order regularity, the BvM theorem also holds when centred at the MAP, and the Gaussian Laplace approximation is asymptotically valid for the full posterior.The Laplace approximation is centred at the MAP and uses the inverse observed posterior Hessian as covariance.Source: §1.1 · p. 4

1.2. Outline.

  • The paper proceeds from the statistical model and prior to contraction and BvM results, then verifies assumptions for four PDE classes and gives a numerical illustration.Source: §1.2 · p. 4

2. Statistical model, prior and posterior.

This section sets out the function-space notation, statistical setting, approximation spaces, and prior used for inference on the unknown PDE solution.

2.2. Statistical setting.

  • The model uses a smooth bounded domain X and a closed subspace V of H^m(X) to encode homogeneous boundary conditions.Known sufficiently smooth inhomogeneous boundary conditions can be handled by a lifting argument.Source: §2.2 · p. 5
  • The design points and measurement errors are assumed mutually independent, with the design density bounded above and away from zero.The target solution is sought in V ∩ H^(m+β)(X).Source: §2.2 · p. 5

2.3. A truncated Gaussian series prior and the posterior distribution.

  • Nested finite-dimensional approximation spaces have dimension proportional to 2^(Jd) and satisfy stability, Jackson, and Bernstein conditions.They are equipped with a multiscale orthonormal basis.Source: §2.3 · p. 6
  • The prior is a truncated Gaussian series on the multiscale approximation space, with an N-dependent cutoff; the posterior is defined from this prior and the likelihood.The posterior expectation is denoted E_ΠN[· | YN].Source: §2.3 · pp. 6–7

3. Main results.

The paper establishes contraction and convergence guarantees for Bayesian PDE solution posteriors, then characterizes their Gaussian shape in positive-order Sobolev spaces. It also extends the BvM result to MAP centring and proves convergence of a Gaussian Laplace approximation.

3.1. Posterior contraction rates.

  • Global stability transfers posterior contraction for the observed field L[u] in L2(X) to recovery of u0 in Hm(X), with the posterior concentrating at rate εN.The posterior mean converges at the minimax-optimal rate up to logarithmic factors.Source: §3.1 · pp. 7–8

3.2. Nonparametric BvM theorems.

  • The BvM theorem describes joint fluctuations of the full posterior function, which are asymptotically Gaussian in positive-order Sobolev norms and, under stronger conditions, in the uniform topology.The limiting Gaussian is tight in Hγ(X) for 0 < γ < m − d/2; when d/2 < γ < m − d/2, the result also holds in L∞(X).Source: §3.2 · pp. 9–10
  • A BvM result transfers to continuously Fréchet-differentiable functionals under the theorem’s stated growth conditions.The paper identifies bounded linear projections, Nemytskii operators, and nonlinear integral functionals as examples.Source: §3.2 · p. 11
  • Under second-order regularity, the BvM theorem also holds when centred at the MAP, which the authors establish as an efficient estimator.The proof controls the empirical Hessian using the second-order regularity condition.Source: §3.2 · p. 12

3.12 to streamline the proofs since controlling the empirical score Hessian is required for establishing the convergence of the Laplace approximation in Theorem 3.16 (below).

  • The paper also establishes convergence of a Gaussian Laplace approximation as a probability measure on a fixed Sobolev space.The approximation is defined when the scaled negative Hessian at the MAP is invertible and is Gaussian on VJ conditional on the data.Source: §3.12 · pp. 12–13

4. Examples.

The examples verify the paper’s assumptions for four PDE classes and illustrate how differential order affects the strength of the available posterior limits. For semilinear elliptic equations, the authors establish stability and BvM results under the stated regularity conditions.

4. Examples.

  • Four examples cover second- and fourth-order elliptic equations, one-dimensional transport–reaction, and elliptic equations with nonlinear gradient dependence.The examples illustrate how regularity estimates establish global and linearised stability.Source: §4 · p. 13
  • Higher differential order permits stronger BvM topologies: the fourth-order elliptic example gives uniform-norm conclusions for d ≤ 3, while the one-dimensional transport example gives Hγ limits only for γ < 1/2.The second-order elliptic examples give uniform-norm conclusions in dimension one.Source: §4 · p. 13

4.1. Semilinear elliptic equation.

  • For the semilinear elliptic model, coercivity and elliptic H2 regularity establish global stability, so the contraction and convergence results apply for d ≥ 1 with m = 2.The model uses τ with 0 ≤ τ′(z) ≤ K and homogeneous Dirichlet, Neumann, or periodic boundary conditions.Source: §4.1 · pp. 13–14
  • For the semilinear elliptic model, the BvM results apply for d ≤ 3; in dimension one, posterior fluctuations are of parametric order N^-1/2 in L∞(X).If the projection-bias condition holds, both the posterior mean and MAP converge at a parametric rate in L∞(X).Source: §4.1 · p. 14

Numerical illustration.

The numerical illustration shows Gaussian behavior emerging in centered, standardized posterior marginals and examines MAP recovery across Sobolev norms. Three PDE examples verify contraction and Bernstein–von Mises conditions, with dimension- and bias-dependent uniform-topology results.

Numerical illustration.

  • At N = 250, the MAP reconstruction of u0 is visibly closer to truth than its forward-field reconstruction, illustrating stability’s link between prediction error and H^2 solution error.Source: Numerical illustration · p. 14
  • Empirical coefficient marginals approach the standard Gaussian as N grows from 150 to 1,000; MAP error slopes deteriorate in stronger Sobolev norms.The MAP mean squared error is estimated from 100 Monte Carlo samples in H^1, H^2, and H^3; jumps at N = 1,000 reflect an added resolution level.Source: Numerical illustration · p. 15

4.2. A fourth-order elliptic equation.

  • Fourth-order elliptic regularity gives contraction and convergence for d ≥ 1, m = 4; BvM results hold for d ≤ 7, with parametric L∞ rates for d ≤ 3 under the bias condition.The BvM range requires 0 < γ < 4 − d/2.Source: §4.2 · pp. 15–16

4.3. A stationary transport-reaction equation.

  • For the stationary transport-reaction equation, stability yields contraction and convergence for m = 1, while BvM results apply for d = 1 and 0 < γ < 1/2.Source: §4.3 · p. 16

4.4. A gradient-nonlinear elliptic equation.

  • For the gradient-nonlinear elliptic equation, contraction and convergence hold for m = 2; BvM results apply for d ≤ 3, with parametric L∞ rates for d = 1 under the bias condition.The BvM range requires 0 < γ < 2 − d/2.Source: §4.4 · p. 17

APPENDIX A: PROOF FOR SECTION 3.1

The appendix develops contraction arguments using Gaussian-prior concentration and stability, then transfers forward-field control to the solution and bounds the MAP estimator.

APPENDIX A: PROOF FOR SECTION 3.1

  • With cutoff 2^J ∼ N^(1/(2β+d)), the Gaussian prior uses εN = N^(-β/(2β+d)) log(N); the proof establishes the needed small-ball lower bound.Source: Appendix A · pp. 20–21
  • The contraction proof moves from Hellinger control of the forward prediction to L2(X) control, then uses stability to obtain contraction for u in H^m(X).Source: Appendix A · p. 22
  • The MAP proof proceeds from an empirical-norm rate to an L2(P_X)-norm rate, controlling shells around the projected truth.Source: Appendix A · pp. 25–26

APPENDIX B: PROOF FOR SECTION 3.2

This appendix passage sets up posterior notation and recalls the operator and pairing framework used in the proof.

APPENDIX B: PROOF FOR SECTION 3.2

  • The posterior notation is abbreviated, and the prior is localized to the set A_N.Source: Appendix B · p. 28
  • The map Iu0 from V to V* is recalled as a topological isomorphism.Source: Appendix B · p. 28
  • The notation (v|V) denotes restricting v in H^-m(X) to V, and the passage records a pairing identity for u in V and ψ in V*.Source: Appendix B · p. 28

B.1. Uniform LAN-expansion and convergence of the Laplace-transformation.

The appendix states a uniform LAN expansion under smoothness and prior-cutoff conditions, then develops the uniform remainder control and projection arguments used in its proof.

B.1. Uniform LAN-expansion and convergence of the Laplace-transformation.

  • The uniform LAN expansion is stated for β > 2d and d/2 < β0 < β + d/4, with the prior cutoff satisfying 2^J ∼ N^(1/(2β+d)).Source: §B.1 · p. 28
  • The proof controls the LAN remainders uniformly over the localized set using empirical-process chaining and bounds on the linearization remainder.Source: §B.1 · pp. 29–30
  • The LAN-orthogonal projection makes the second summand vanish, supporting uniform control of the projected expression over the localized set.Source: §B.1 · pp. 34–35

B.2. Proofs for the posterior mean centring.

The proof transfers contraction from the PDE output to the solution and establishes the posterior-mean Gaussian approximation through localization and convergence arguments.

B.2. Proofs for the posterior mean centring.

  • The BvM proof first uses an alternative recentering, then replaces it with the localized posterior mean.The proof states that it suffices to establish the result for the alternative centre before controlling the change of centre.Source: §B.2 · pp. 36, 38
  • The scaled alternative recentering has convergent finite-dimensional distributions, and the proof strengthens weak convergence to Wasserstein-1.The argument uses the Cramér–Wold device and establishes uniform integrability by bounding second moments.Source: §B.2 · pp. 37–39
  • The localization result assumes β>2d and d/2<β0<β+d/4, with the prior cutoff satisfying 2J ∼ N^(1/(2β+d)).Under these conditions, the proof obtains the stated localization bound.Source: §B.2 · pp. 39, 42

B.3. Proofs for the MAP centring.

The MAP-centred proof reduces the Gaussian approximation to controlling the MAP centre and establishing operator and covariance bounds. Its arguments use local regularity, approximation estimates, and fixed-dimensional Gaussian convergence.

B.3. Proofs for the MAP centring.

  • The proof controls linearization errors using first- and second-derivative bounds together with Lipschitz and approximation estimates.The cited steps use the local assumptions to bound terms in the error decomposition.Source: §B.3 · pp. 44–45
  • The MAP-centred BvM proof reduces to controlling the difference between the alternative centre and the MAP estimator.The proof states that the BvM result for the alternative centre has already been established.Source: §B.3 · p. 48
  • With probability tending to one, the segment from the projected truth to the MAP estimator lies in the local neighbourhood, where the relevant operator is invertible.The proof applies the fundamental theorem of calculus along this segment.Source: §B.3 · p. 48
  • For each fixed K, the covariance matrices converge, which yields Wasserstein-1 convergence of the finite-dimensional Gaussian measures.The remaining Gaussian-measure term is controlled by Lemma B.5.Source: §B.3 · p. 49

APPENDIX C: FURTHER TECHNICAL RESULTS

The appendix gives distance comparisons for probability measures and approximation properties of the LAN-orthogonal projection, alongside a concentration result for empirical norms.

APPENDIX C: FURTHER TECHNICAL RESULTS

  • The appendix states comparisons involving Hellinger distance and Kullback–Leibler divergence, including an expression for the KL divergence.It also gives a Hellinger-distance bound for the probability measures under consideration.Source: Appendix C · p. 50
  • The empirical-norm concentration lemma assumes iid design points and a uniformly bounded class of real-valued functions.Its setting also requires an open subset of Rd with finite Lebesgue measure.Source: Appendix C · p. 50
  • The LAN-orthogonal projection has boundedness and density properties, and satisfies Jackson and Bernstein estimates.The estimates apply under the stated Sobolev regularity conditions.Source: Appendix C · p. 51
Loading 2609.04124v1…