PaperlayerView exact revision on arXiv

Source-linked AI summary

Bayesian Operator Learning: Posterior Existence and Convergence of Point Estimates for Gaussian Priors

Niklas Reinhardt, Jakob Zech

arXiv:2610.03666v1math.STcs.NAmath.NAstat.TH

TL;DR

Learning nonlinear operators between infinite-dimensional spaces is motivated by scientific settings such as PDEs, but Bayesian theory for these operator-valued parameters is limited. This paper establishes posterior existence for continuous-operator priors and analyzes Gaussian-prior estimators, obtaining algebraic convergence rates and minimax optimality up to logarithmic factors under matched smoothness.

  • Problem

    Learning nonlinear maps between infinite-dimensional spaces is important for scientific applications, while Bayesian learning of operators has received limited theoretical study.

    Source: §1 · p. 2, §1.2 · p. 5

  • Method

    The paper constructs a Bayesian framework with posterior characterization for operator-valued priors and analyzes Gaussian-prior estimation through its posterior mean.

    Source: §1 · p. 3, §5 · p. 24

  • Results

    The posterior mean has algebraic convergence rates, is minimax optimal up to logarithmic factors over hyperrectangles when prior smoothness matches the truth, and coincides with the MAP estimator and ridge-regularized kernel regression.

    Source: §1.1 · p. 4, §3.5 · p. 20, §5 · p. 24

  • Takeaways & Limitations

    Holomorphic operators meet the theorem's assumptions, and the paper derives explicit learning rates for the Darcy flow solution operator.

    Source: §1 · p. 2

    The derived C0(K)-norm rate is suboptimal relative to the minimax rate, and sharp supremum-norm bounds are left for future work.

    Source: Then the posterior expectation · p. 16

Abstract

from arXiv · show

We develop a Bayesian framework for learning nonlinear operators between infinite-dimensional spaces. Given a map $g_0:\mathcal{X}\to\mathcal{Y}$ between separable Hilbert spaces, we study the recovery of $g_0$ from $n\in\mathbb{N}$ noisy input-output pairs $(\boldsymbol{X},\boldsymbol{Z})=(X_i,Z_i)_{i=1}^n$ with $Z_i= g_0 (X_i ) + E_i$. Here the $X_i\in\mathcal{X}$ are randomly drawn 'design' points in a compact subset of $\mathcal X$, and the $E_i$ are assumed to be i.i.d. draws from a Gaussian white noise process indexed by $\mathcal{Y}$. For any 'operator-valued' prior $\mathbb{P}_G$ supported on the space of continuous operators, we show existence of the posterior $\mathbb P_{G|(\boldsymbol{X},\boldsymbol{Z})}$ as a regular conditional distribution, and provide a characterization of its Radon-Nikodym derivative. For Gaussian priors, we establish algebraic (in the sample size $n$) convergence rates for the posterior mean towards the ground truth; this corresponds to a ridge regularized kernel estimator. Moreover, we show that the posterior mean is minimax optimal (up to logarithmic factors) over hyperrectangles when the smoothness of the prior matches that of the ground truth. To illustrate the applicability of our analysis, we derive explicit learning rates for the Darcy flow solution operator.

1 Introduction

The paper motivates Bayesian learning of nonlinear maps between infinite-dimensional spaces, including PDE solution operators, and outlines a framework for learning such operators from noisy observations. It establishes posterior existence for broad operator-valued priors and studies convergence for Gaussian priors, with applications to holomorphic operators such as Darcy flow.

1 Introduction

  • Nonlinear maps between infinite-dimensional spaces arise naturally in scientific applications, including PDE solution operators and dynamical-system evolution.The introduction also notes data-driven operator-learning approaches as fast-to-evaluate surrogates when the physical process lacks a mathematical description.Source: §1 · p. 2
  • Bayesian nonparametric learning places a prior on an unknown parameter, conditions it on measurements to obtain a posterior, and derives point estimators from that posterior.The paper frames the operator problem around choosing a prior, establishing posterior existence, and studying estimator convergence.Source: §1 · pp. 2–3

1.1 Outline and Contributions

  • The paper establishes posterior existence for a general class of priors on continuous operators, while its convergence-rate analysis uses Gaussian priors.It leaves posterior contraction in distribution and Bernstein–von Mises-type results for future work.Source: §1 · p. 3, §1.1 · p. 3
  • The posterior mean has an algebraic convergence rate under coefficient-decay assumptions, with the best rate when prior smoothness matches the ground truth.The paper also reports minimax optimality up to logarithmic factors over hyperrectangles in the matched-smoothness setting.Source: §1.1 · p. 4
  • Holomorphic operators satisfy the convergence theorem's assumptions, yielding explicit learning rates for the Darcy flow solution operator.The paper identifies the Darcy solution operator as holomorphic.Source: §1.1 · p. 4

1.2 Existing Results

  • Prior operator-learning analyses largely lack Bayesian guarantees; this work places a prior on the operator itself and allows misspecification in its vector-valued kernel setting.The cited prior work on PDE models is described as finite-dimensional and without theoretical guarantees.Source: §1.2 · pp. 4–5

1.3 Notation

  • The notation section sets asymptotic comparison conventions and defines embeddings and Bochner-space notation, including that L2 is a Hilbert space.It also introduces conventions for pushforward measures and absolute continuity.Source: §1.3 · pp. 5–6

2 Problem Formulation

This section formulates operator recovery from randomly sampled input-output pairs under additive white noise, including a rigorous Hilbert-space noise model and conditions on priors for continuous operators. It establishes posterior existence as a regular conditional distribution and describes the compact-set setting, including when compactness may be weakened.

2.1 Measurement Model

  • The model seeks a Borel measurable operator g0 between separable Hilbert spaces from random design points Xi and observations Zi = g0(Xi) + Ei, with independent white-noise errors.The noise is centered Gaussian with covariance Cov(WY(y), WY(y′)) = ⟨y, y′⟩Y.Source: §2.1 · p. 6, §2 · p. 6

2.1.1 Gaussian Measures

  • A Gaussian measure on a separable Hilbert space is characterized by its mean and covariance, whose operator is self-adjoint, positive semidefinite, and trace class.The section also introduces the Cameron–Martin space as a central object for Gaussian measures.Source: §2.1.1 · p. 7

2.1.2 White Noise

  • White noise cannot generally be treated as a Y-valued random variable, so the construction realizes it as a centered Gaussian random variable in a larger dual space Ỹ∗.Its Cameron–Martin space is isometrically isomorphic to Y, and the white-noise process is recovered through the dual-space observation.Source: §2.1.1 · p. 7, §2.1.2 · p. 8

The three maps (interpreted in the sequence spaces)

  • For white noise, the noise Cameron–Martin space is isomorphic to Y, allowing shifts across all of Y while retaining the nondegenerate-likelihood condition.The section places the spaces in a Gelfand triplet and notes that Gaussian-measure noise can be treated by replacing the dual observation space with Y.Source: §2.1.2 · p. 7, The three maps (interpreted in the… · p. 10

2.2.1 General Existence Theory

  • The framework puts a prior on the operator and conditions on both the random inputs and outputs, rather than placing a prior on inputs and conditioning only on outputs.This is the section’s stated distinction from inverse-problem formulations discussed in.Source: §2.2.1 · p. 10
  • Under Assumptions 1–3, the posterior over operators conditioned on (X, Z) exists as a regular conditional distribution, with its characterization based on the joint data law and Radon–Nikodym derivatives.The assumptions include measurability and absolute continuity of shifted noise laws.Source: §2.2.1 · p. 12
  • Even without absolute continuity of shifted noise, a posterior can exist when the product space is Polish, but it need not admit a Radon–Nikodym density with respect to the prior.This distinguishes existence of a conditional distribution from the density representation used under the stronger assumptions.Source: §2.2.1 · p. 12

2.2.2 Additive White Noise for Continuous Operators on Compact Sets

  • For the specified application setting, the inputs lie in compact subsets and continuous operators map into a dual space, with observations Zi = d ◦ G(Xi) + Ei.This setting combines additive white noise with continuous operators on compact sets to apply the posterior-existence result.Source: §2.2.2 · p. 12
  • The model uses design points drawn from a probability measure supported on compact K and an operator prior on C0(K, Y), with independent Gaussian white-noise errors.The operator space is endowed with the supremum norm; continuity ensures the translation map is continuous and Borel measurable.Source: §2.2.2 · pp. 12–13
  • Under Setting 1, the posterior given the design and observations is well-defined and has the characterization in (2.14), with a specified negative log-likelihood.The result holds for P(X,Z)-almost every data pair.Source: §2.2.2 · p. 13
  • Compactness of K can be weakened to closedness when a bounded continuous ONB of L2(K) makes the relevant space separable, and Theorem 2.11 still applies.In that case, the operator prior space is replaced by Gψ.Source: §2.2.2 · p. 13

2.3 Prior Distribution

  • Gaussian and uniform priors are examples of priors that can encode a priori knowledge of the ground truth.The uniform construction defines a Borel prior when the coefficient sequence κ belongs to ℓ2(N).Source: §2.3 · p. 13
  • For the Gaussian and uniform constructions, continuous basis functions and sufficient coefficient decay ensure that prior draws are almost surely bounded and continuous.Under the stated decay and positivity conditions, the prior is a Borel probability measure on C0(K).Source: §2.3 · p. 14

3 Posterior Analysis for Gaussian Model

This section analyzes posterior-mean, MAP, and maximum-likelihood estimation under the Gaussian white-noise model. It establishes concentration and convergence results for the posterior mean, identifies its relation to MAP and kernel ridge regression, and states conditions for minimax optimality.

3.1 Posterior Mean Estimator

  • The posterior expectation exists almost surely as a G-valued Bochner integral and is Borel measurable; its Gaussian-prior expression generalizes the conjugate linear-model formula.The expression is given for the Gaussian prior, while convergence results are not established for the uniform prior.Source: Then the posterior expectation · p. 14, §3 · p. 14

Then the posterior expectation

  • The posterior mean concentrates around g0 in three deviation regimes, with an algebraic rate governed by truth and prior regularity and basis fluctuations.The effective prior smoothness is r′ = r − 2ρ; choosing r′ = q maximizes the convergence rate.Source: Then the posterior expectation · pp. 15–16
  • Integrating the concentration inequality yields convergence in expectation for the posterior mean.Corollary 3.5 gives the expectation result; its rate is minimax optimal up to logarithmic factors under ρ = 0 and r = q.Source: Then the posterior expectation · p. 16

3.2 Maximum a Posteriori Estimator

  • For the Gaussian posterior, the posterior mean is the unique MAP estimator, so the posterior-mean algebraic convergence rates also apply to MAP.The MAP characterization uses the posterior’s Tikhonov Onsager–Machlup functional.Source: §3.2 · p. 17

3.3 Maximum Likelihood Estimator

  • For compact operator classes, existing maximum-likelihood expectation rates depend on class entropy and bias toward g0; convergence results for noncompact classes remain ongoing work.Source: §3.3 · p. 18

3.4 Vector-Valued Kernel Methods

  • The posterior mean and MAP equal vector-valued kernel ridge regression over Hprior with regularization λ = 1/n.The kernel is trace class; the comparison with notes sharper upper bounds there in the well-specified case, where its regularization can depend on q and r.Source: §3.4 · p. 19
  • The kernel has a multiplicative structure when both the basis and its weights factorize; the present prior covariance is trace class.For infinite-dimensional Y, the identity operator choice allowed in [68] [67] [103] is not admissible in this setting.Source: §3.4 · p. 19

3.5 Minimax Optimality

  • With ρ = 0 and r = q, the posterior-mean rate is minimax optimal over Θq up to logarithmic factors.The prior decay must match the decay of the ground-truth expansion coefficients.Source: §3.5 · pp. 19–20

4 Applications

The applications derive posterior-mean convergence for holomorphic operators and apply the framework to diffusion and Darcy flow solution operators. The results distinguish priors informed by the ground truth from priors depending only on the operator class.

4 Applications

  • The applications include learning holomorphic operators and the solution operator of the diffusion equation.The diffusion example defines its input and output spaces using Sobolev spaces on the two-dimensional torus.Source: §4 · p. 20, §4.2 · p. 22

4.1 Bayesian Learning of Holomorphic Operators

  • The holomorphic-operator application uses decay of expansion coefficients to establish the conditions needed for posterior-mean convergence.The section obtains coefficient and basis-weight bounds under its holomorphy assumption, then uses an enumeration to verify the convergence assumptions.Source: §4.1 · p. 21
  • Holomorphic operators admit two Gaussian-prior convergence results: the faster uses explicit information about g0, while the other depends only on the operator class.The stated conditions are t > 0 for the ground-truth-dependent prior and t > 1/2 for the class-based prior.Source: §4.1 · p. 22

4.2 Diffusion Equation on the Torus

  • For the g0-dependent prior, the posterior-mean rate matches the maximum-likelihood rate for β > 4; for β ∈ (3, 4), r > 1 makes it suboptimal relative to [89, Theorem 27], while the general prior yields a worse rate.Source: §4.2 · p. 23

5 Conclusion and Outlook

The paper develops Bayesian operator learning and derives Gaussian-prior convergence results. Its outlook identifies limits in the norm, posterior, and prior settings covered by the analysis.

5 Conclusion and Outlook

  • For Gaussian priors and white-noise measurements, the posterior mean has an explicit representation, equals the MAP estimator and ridge-regularized kernel regression, and satisfies concentration bounds toward the ground truth.The framework is applied to the solution operator of the standard parametric diffusion equation.Source: §5 · p. 24
  • The convergence results cover posterior-mean concentration in L2(K); the C0(K) estimates are not known to be sharp, and posterior-distribution concentration remains future work.The paper identifies Bernstein–von Mises results as an example of the unaddressed full Bayesian analysis.Source: §5 · p. 24
  • The convergence analysis covers linear priors of the form PG = T#µ, while nonlinear priors, including neural-network priors, remain ongoing research.Source: §5 · p. 24

LLM Usage Declaration

The declaration reports LLM assistance in manuscript polishing and proofreading; the accompanying appendix passages outline white-noise construction and posterior-existence arguments.

LLM Usage Declaration

  • LLMs assisted with language polishing and proofreading, identified minor technical errors in several results, and the authors independently checked the corrections.The named results include Theorem 2.11, Theorem 3.8, Lemma 4.3, Corollary 4.4, and Theorem 3.11.Source: LLM Usage Declaration · p. 24

A.1 White Noise Construction

  • The white-noise construction builds a Hilbert space from the noise covariance weights and establishes an isometric relationship used in the construction.The proof verifies completeness and dense embeddings for the constructed space.Source: §A.1 · pp. 30–31

A.1.1 Proof of Lemma 2.2

  • The white-noise proof constructs the process as an L2 limit and identifies its action through the duality pairing.It also identifies the Cameron–Martin space with a space isometric to Y.Source: §A.1.1 · pp. 31–33

A.2.1 Proof of Lemma 2.7

  • The posterior-kernel proof uses joint measurability and independence of the noise variables to establish a probability measure for each input.Source: §A.2.1 · pp. 33–34

A.2.2 Proof of Theorem 2.8

  • The posterior-existence proof identifies the conditional distribution and derives its Radon–Nikodym formula.Source: §A.2.2 · p. 35

A.2.3 Proof of Theorem 2.11

  • For Gaussian white noise, the proof invokes the Cameron–Martin theorem to verify absolute continuity of shifted noise laws.The required shift belongs to the Cameron–Martin space for continuous operators evaluated at supported inputs.Source: §A.2.3 · p. 35

A.3.1 Proof of Lemma 2.15

The proof shows that the Gaussian and uniform priors considered induce measures supported on continuous operators by establishing almost-sure continuity of their draws.

A.3.1 Proof of Lemma 2.15

  • For the Gaussian and uniform priors, summability of expected weighted basis coefficients implies that the weighted coefficient sum is finite almost surely.Source: §A.3.1 · p. 36
  • Finite weighted coefficient sums make the operator-series approximations converge uniformly, so the prior draw is continuous almost surely.Source: §A.3.1 · p. 36
  • The resulting prior measure is supported on C0(K), with its construction justified through a measurable full-probability set.Source: §A.3.1 · p. 37

B.1 Proof of Lemma 3.1

These proofs establish the integrability and measurability needed for the posterior-mean construction, then verify the finite-dimensional Gaussian calculations underlying the posterior formula.

B.1 Proof of Lemma 3.1

  • The integrand defining the posterior mean is Bochner integrable almost surely, and its parametric integral is Borel measurable.Source: §B.1 · pp. 37, 39
  • For both uniform and Gaussian priors, the likelihood-side bounds establish almost-sure finiteness under the corresponding prior assumptions.The proof treats the uniform case separately from the Gaussian case.Source: §B.1 · pp. 37–39

B.2 Proof of Lemma 3.2

  • The normalization factor is almost surely positive and finite, while the finite-dimensional terms used in the posterior calculation are Borel measurable and finite.Source: §B.2 · pp. 40–41

B.3 Proof of Theorem 3.3

The proof controls posterior-mean error over several deviation regimes by combining concentration bounds with matrix estimates. It also establishes the likelihood-functional properties used in the analysis.

B.3 Proof of Theorem 3.3

  • For δn ≲ δ ≤ n^3/2, the proof bounds ||ĝn − g0||²_L2(K) by ≲ δ² on Sδ, then combines the regimes to finish the theorem.Source: §B.3 · pp. 42, 49, 50
  • The error analysis splits the matrix contribution into off-diagonal and diagonal terms, controlling the former with a Neumann series and the latter using the linear-analysis approach.Source: §B.3 · p. 42
  • The constants in the final bound depend on g0 through its sup norm and its coefficient-decay bound.Source: §B.3 · p. 50

B.4 Proof of Lemma 3.7

  • Almost surely, the likelihood functional is bounded on Hprior, and the set Λx,z has full prior measure.Source: §B.4 · p. 51

B.5 Proof of Theorem 3.8

The proof identifies the posterior mean with a Riesz representer and establishes its approximation by finite-dimensional projections. It then shows that this mean is the unique MAP estimator.

B.5 Proof of Theorem 3.8

  • The posterior is Gaussian with Cameron–Martin space Hpost, which equals Hprior as a set.Source: §B.5 · p. 51
  • The posterior mean ĝn is the Hpost limit of projections of the Riesz representer m, and the projections also converge in C0(K).Source: §B.5 · p. 52
  • The posterior mean maximizes Jδ for every δ > 0 and is the unique MAP estimator.Source: §B.5 · p. 52

B.6 Proof of Theorem 3.11

The proof establishes uniform upper control over the stated function class and a separate lower-bound argument. The accompanying appendix proofs use an envelope construction and L2-space reductions.

B.6 Proof of Theorem 3.11

  • For q > 1 and ρ = 0, the theorem's bound holds uniformly over Θq by applying Corollary 3.5 with r = q.Source: §B.6 · p. 52
  • The lower-bound proof compares nearby gτ through KL divergence and converts an arbitrary estimator into Hamming-coded decisions.Source: §B.6 · pp. 52–53

B.7 Envelope Lemma

  • Borel–Cantelli yields an almost-sure eventual index J(x,z), which the proof uses to construct a measurable envelope M(x,z).Source: §B.7 · pp. 53–54

C.1 Proof of Lemma 4.1

  • The proof of Lemma 4.1 reduces its claim to an L2 expansion in an orthonormal basis after an isometric identification.Source: §C.1 · p. 54

C.2 Proof of Theorem 4.2

  • Theorem 4.2's proof uses the pushforward identity for one part and cites [48, Theorem 5 (iii)] for the other.Source: §C.2 · p. 54

C.3 Proof of Lemma 4.3

The proof constructs a decreasing enumeration of weighted coefficients and uses it to establish the required decay bounds. The resulting sequence satisfies ω_j ≲ j^ρ for every ρ > 0.

C.3 Proof of Lemma 4.3

  • The enumeration is built by rearranging the coefficients c_ν,k in decreasing order, with bounds derived from the sequence a_ν.For part (ii), the proof chooses the enumeration independently of g_0.Source: §C.3 · pp. 55–56
  • A decreasing rearrangement of the coefficients is used to bound their jth term by c_j ≲ j^-1/p.The proof obtains ℓp summability before applying the decreasing-sequence bound.Source: §C.3 · p. 55
  • The proof bounds the auxiliary quantity d_β(j) by 1 for all sufficiently large j.This follows as τ_j tends to infinity and once τ_j ≥ 3β/2.Source: §C.3 · p. 55
  • For every ρ > 0, the construction yields ω_j ≲ j^ρ, establishing the decay property (3.5).The proof combines bounds on the rearranged coefficients and weights, with the exponent ρ chosen arbitrarily.Source: §C.3 · p. 56

C.4 Proof of Corollary 4.4

The proof applies Corollary 3.5 after choosing parameters that meet its assumptions. It derives an n-dependent bound and obtains the second claim by the same argument with a different smoothness parameter.

C.4 Proof of Corollary 4.4

  • The selected q and ρ satisfy Lemma 4.3’s conditions, allowing the proof to establish (3.5) and (3.6).The argument also checks compactness of R(X) and the required support and continuity conditions.Source: §C.4 · p. 56
  • The proof derives an n-dependent bound for the estimation error after applying Corollary 3.5.The displayed rate depends on M and r, with an ε term.Source: §C.4 · p. 56
  • Claim (ii) follows by the same proof after setting m := min{s, t} > 1/2.Claim (i) follows from Corollary 3.5.Source: §C.4 · p. 56

D Posterior Mean Convergence in C0(K)

The appendix extends the posterior-mean convergence analysis to the C0(K) norm and states a concentration bound under additional conditions on q and r. Its proof follows the strategy for Theorem 3.3, adapting the error bounds to this norm.

D Posterior Mean Convergence in C0(K)

Proof of Theorem D.1

  • The proof follows Theorem 3.3’s steps and splits the error bound into separate terms for estimation.It adapts the earlier argument to C0(K), bounding the terms before concluding the theorem.Source: Proof of Theorem D.1 · p. 57
  • The proof bounds the intermediate error terms and concludes the stated result (D.2).The final steps combine the preceding bounds to finish the proof.Source: Proof of Theorem D.1 · p. 59

Related papers

Loading 2610.03666v1…