PaperlayerView exact revision on arXiv

Source-linked AI summary

Approximating Measures on Function Spaces: Transport and Truncation

Ricardo Baptista, Bamdad Hosseini, Alexander W. Hsu

arXiv:2609.17802v1math.PRcs.NAmath.NAstat.ML

TL;DR

Approximating measures on function spaces is challenging when targets are high- or infinite-dimensional, but differ from a tractable reference through low-dimensional structure. The paper approximates targets by changing a finite-dimensional marginal while preserving reference conditionals, and shows that optimal-transport corrections need not retain finite rank even when covariance perturbations do.

  • Problem

    The paper addresses efficient sampling from high- or infinite-dimensional target measures when a known, computable reference is available and the target's departure from it may be low dimensional.

    Source: §1 · pp. 1–2

  • Method

    It represents approximations through a finite-dimensional ψ-marginal while preserving reference conditionals on fibers, and fits the marginal using generative or variational methods.

    Source: §1 · pp. 2, 4

  • Results

    For Gaussian references, structured transports into Pψ(μ) are finite-rank perturbations of identity, while optimal-transport corrections can be infinite rank even for finite-rank covariance perturbations.

    Source: §1 · pp. 3–4

  • Takeaways & Limitations

    The fitting error's marginal term is governed by the dimension of ψ rather than the ambient discretization, while reference-conditionals carry the infinite-dimensional remainder.

    Source: §6.4 · p. 38

    One KL-bound strategy requires two-sided control of Φ − ˜ϕ∘ψ, including exponential integrability of fiber oscillations for the direction opposite Jensen's inequality.

    Source: §6.2 · p. 33

Abstract

from arXiv · show

Measures on function spaces arise throughout Bayesian inverse problems and generative modeling, often with low-dimensional structure relative to a tractable reference measure. We introduce the class $\mathcal{P}_\psi(\mu)$ of measures that differ from a reference measure $\mu$ only through a finite-dimensional map $\psi$ while preserving the reference conditionals on its fibers. Class members are determined by their $d$-dimensional pushforwards under $\psi$ and admit convenient block-triangular transport map representations. Draws are taken from this $d$-dimensional distribution and then completed to function space through sampling of the reference conditionals. For Gaussian references, these transport maps are finite rank perturbations of the identity. In contrast, optimal transport maps do not preserve this low-dimensional structure. For covariance perturbations that are trace class in the Cameron-Martin geometry of the reference, we show the optimal map is a trace class perturbation of the identity. Even finite rank perturbations yield corrections whose rank, governed by an invariant subspace, is typically infinite. We develop approximation theory for $\mathcal{P}_\psi(\mu)$ and error analysis for fitting within it, splitting the total error into an irreducible class error and a finite-dimensional marginal term set by the dimension of $\psi$ rather than the ambient discretization. We present numerical results including inference from low-dimensional nonlinear observation maps, deconvolution under a jump process prior, and state estimation for Navier-Stokes flows.

1 Introduction

The paper studies approximations of function-space measures using a tractable reference and finite-dimensional information. It defines a class that changes only the reduced marginal while preserving reference conditionals, and contrasts its structured transports with optimal transport.

1 Introduction

  • The target is to build efficient samplers for measures on high- or infinite-dimensional spaces using a known, computable reference measure.The motivating settings include Bayesian inverse problems, spatial statistics, imaging, and generative modeling.Source: §1 · pp. 1–2
  • Sampling first draws the d-dimensional ψ-marginal, then completes the function-space sample using the reference conditional distribution.For a Gaussian-process reference and point evaluations, the conditional draws retain the reference structure after sampling the observed values.Source: §1 · pp. 2–3
  • Pψ(μ) consists of measures that modify the ψ-marginal while preserving the reference conditional law on each fiber.Its members can be represented through finite-dimensional densities and transport maps, and the paper characterizes best forward-KL approximations within the class.Source: §1 · pp. 2–4
  • The dimensionality-reduction map may be a projection or a non-projection map, including point evaluations, coarse-grid summaries, and sensor-derived observations.The paper considers maps continuous on a subset where the reference measure is concentrated, while errors may be measured in a larger ambient space.Source: §1 · p. 2
  • For Gaussian references, transports into Pψ(μ) are finite-rank perturbations of identity, whereas Gaussian optimal-transport corrections can be infinite rank even for finite-rank covariance perturbations.When covariance perturbations are trace class in the reference Cameron–Martin geometry, the optimal map is a trace-class perturbation of identity.Source: §1 · pp. 3–4

2 Background and related work

The related work situates the paper among dimension-reduction methods, conditional transport and generative modeling, and function-space variational inference and sampling. It distinguishes the paper's broader function-space class and learned reduced marginal from approaches centered on projections, whole-measure learning, or explicit marginalization.

2.1 Dimension reduction in Bayesian inference

  • Bayesian dimension-reduction methods retain prior conditionals on a complement and perform inference on a reduced marginal; this paper extends that structure to function spaces and maps beyond projections.In finite dimensions with linear maps, Pψ(μ) recovers the approximation class used by likelihood-informed subspaces, while the paper fits its marginal with a generative model or transport without explicit likelihood marginalization.Source: §2.1 · pp. 5–6

2.2 Transport maps and generative models

  • Triangular transports encode conditioning, and the paper's block-triangular construction uses reference conditionals to complete samples from the reduced marginal.The paper also places its conditional generative approach in simulation-based inference, where joint data-parameter draws train a model to sample the posterior directly.Source: §2.2 · p. 6
  • Function-space generative models learn the entire measure, whereas this paper assigns the infinite-dimensional structure to reference conditionals and learns a finite-dimensional marginal, with an irreducible class error.Normalizing flows and diffusion models are noted as possible marginal models when their Wasserstein or KL guarantees transfer through the paper's results.Source: §2.2 · pp. 6–7

2.3 Variational inference and sampling on function spaces

  • Function-space variational inference must remain compatible with the reference measure for its divergence to be finite; this paper performs variational inference over Pψ(μ).Its Algorithm 4.3 minimizes reverse divergence within the class, while reference-preserving MCMC proposals are described as stable under mesh refinement.Source: §2.3 · p. 7

3 Low-dimensional modifications of a reference measure

This section characterizes measures that preserve the reference conditionals along ψ’s fibers and gives explicit finite-rank transports for Gaussian references. It also identifies KL-optimal approximations within Pψ(µ), including η* as an approximation to ν that attains a lower bound across the class.

3.1 Three characterizations of Pψ(µ)

  • Members of Pψ(µ) are determined by their ψ-marginal, can be sampled by drawing that marginal and then the reference conditional on each fiber, and admit a block-triangular transport representation.With absolute continuity of the ψ-marginal, membership is also characterized by a density with respect to µ that depends only on ψ(u).Source: §3.1 · pp. 8–9

3.2 Explicit transports for Gaussian references

  • For Gaussian references and linear ψ, every class measure is reachable by an identity-plus-finite-rank transport moving mass only along Range(Σuψ); layered maps concatenate their conditioning dimensions.A flow of k planar layers produces a measure in Pψ(µ) for a ψ of dimension at most 2k.Source: §3.2 · p. 10
  • Generic OT maps do not preserve the low-dimensional structure, but the structured form is recovered when Range(ψ*) is Σ-invariant or the cost uses the reference Cameron–Martin norm.For members of Pψ(µ) with finite second moment, the OT map restricts to the identity on K⊥ and displaces within K; the Cameron–Martin-cost case has the structured form as well.Source: §3.2 · pp. 10–13
  • For covariance changes trace class in the reference Cameron–Martin geometry, the Gaussian OT correction is trace class; for finite-rank changes, its rank is governed by the smallest Σ-invariant subspace K containing Range(ψ*).When C is definite, the correction’s rank equals dim K.Source: §3.2 · pp. 11–12
  • In a two-dimensional example, the structured transport has a rank-one correction, while OT contracts both directions with correction eigenvalues −0.757 and −0.588.The example illustrates that the optimal correction need not be low rank even for a finite-rank covariance perturbation.Source: §3.2 · p. 12

3.3 Best approximation in KL divergence by measures in Pψ(µ)

  • When the stated finite-KL existence conditions hold, the forward-KL best approximation matches ν’s ψ-marginal, while the reverse-KL minimizer weights reference conditionals according to their fiberwise KL divergence from ν’s conditionals.The forward projection combines ψ♯ν with the reference conditionals; the reverse projection uses DKL(µw, νw) to define its marginal weighting.Source: §3.3 · pp. 13–14
  • Every η in Pψ(µ) satisfies DKL(η, ν) ≥ −log Z*.Source: §3.3 · p. 14
  • η* attains the minimum KL divergence to ν over all measures in Pψ(µ).Source: §3.3 · p. 14

4 Computational methods

The methods reduce approximation in Pψ(µ) to fitting a finite-dimensional marginal and then sampling reference conditionals. They offer sample-based and likelihood-based fitting, with conditioned-sampling strategies for Gaussian, independent-increment, and learned-reference settings.

4.1 Generative approximation from samples

  • Sample-based fitting reduces generative modeling to approximating νψ from projected target samples, including conditional marginals when joint data samples are available.For inverse problems, joint samples (y,u) can be projected to (y,ψ(u)) to fit the conditional marginal.Source: §4.1 · p. 15

4.2 Variational inference with likelihood access

  • With likelihood access, the method fits the finite-dimensional marginal by minimizing reverse KL over Pψ(µ), using a triangular transport parameterized by F.The transport’s conditional component can be replaced by sampling from the reference conditional.Source: §4.2 · p. 16

4.3 Conditioned sampling of reference measures

  • Both fitting routes complete a modeled marginal by sampling the reference conditional, preserving its infinite-dimensional structure.The computational framework identifies conditional sampling as the shared step across its algorithms.Source: §4 · p. 15, §4.3 · p. 17

4.3.1 Linearly conditioned Gaussian reference measures

  • For Gaussian processes, linear conditioning uses Matheron’s update, while point-evaluation samples can be extended exactly to finer resolutions as needed.For a Brownian motion, recursive bridge refinement samples each new point with one univariate Gaussian draw at O(1) cost, plus O(log n) neighbor lookup.Source: §4.3.1 · pp. 17–18

4.3.2 Processes with independent increments

  • For a Brownian Poisson jump process, bridge sampling conditions on the jump count, samples ordered jump times, and then samples jump sizes.The jump-count probabilities are cheap to compute, and their sum may be safely truncated because Poisson weights decay exponentially.Source: §4.3.2 · p. 18

4.3.3 Conditioned sampling via an auxiliary generative model

  • When exact reference-conditionals are intractable, a learned conditional model can supply fine-scale structure while a task-specific model fits the coarse marginal.The auxiliary conditional model may be reused across targets sharing a reference, so task-specific sample complexity depends on dim(ψ(u)) rather than ambient dimension.Source: §4.3.3 · p. 20

5 Numerical results

The numerical examples apply the framework to nonlinear Gaussian-process inference, jump-prior deconvolution, and Navier–Stokes filtering, with exact, approximate, or learned dimension reduction and conditional sampling. Results illustrate multimodal inference and how coarse-grid resolution affects posterior approximation.

5 Numerical results

  • The three examples span exact dimension reduction and conditional sampling, approximate likelihood dependence with exact conditional sampling, and learned marginal and reference-conditionals.The first is nonlinear Gaussian-process inference, the second jump-prior deconvolution, and the third Navier–Stokes filtering.Source: §5 · p. 20

5.1 Nonlinear inference on a Gaussian process

  • In nonlinear Gaussian-process inference, the model captures all 32 sign-pattern modes and achieves good agreement with numerically computed mode probabilities.The likelihood depends exactly on the five observed point values, and the Gaussian-process prior is conditioned on generated values.Source: §5.1 · p. 21

5.2 Nonlinear deconvolution with a jump prior

  • For jump-prior deconvolution, prior-normalized squared kernel Stein discrepancy decreases until about d = 30, then stops improving as marginal fitting becomes harder.At larger d, the increased difficulty of marginal fitting dominates the reduction in approximation class error.Source: §5.2 · p. 23

5.3 Navier–Stokes filtering

  • For Navier–Stokes filtering, separate models learn the coarse-grid marginal conditioned on sensor history and the fine-grid reference conditional conditioned on coarse values.The sampler draws coarse values from the marginal model, then generates the fine field without giving the conditional model access to sensor measurements.Source: §5.3 · p. 25
  • The Navier–Stokes two-stage sampler achieves fine-grid CRPS 0.36, mean squared error 0.42, and variance reduction of 59% on 200 test trajectories.The coarse marginal model has CRPS 0.30 and 66% variance reduction relative to the prior; empirical pixelwise credible-interval coverage deviates from nominal by 4.2%.Source: §5.3 · p. 25
  • End-to-end CRPS improves steadily as coarse resolution increases to d = 16, with only a small marginal gain at d = 32.Both models are retrained at each resolution.Source: §5.3 · p. 26

6 Error analysis

The section analyzes approximation error by separating the finite-dimensional marginal-model term from the irreducible error within Pψ(µ), deriving Wasserstein and KL bounds using conditional moments or functional inequalities to guide the choice of ψ. With exact conditioned sampling, the remaining statistical difficulty depends on ψ’s dimension rather than the ambient discretization; reported CRPS gains are small when coarse resolution exceeds d = 16.

6 Error analysis

  • The error splits into the fitted d-dimensional marginal error and an irreducible error within Pψ(µ), so generative-model convergence guarantees transfer to the function-space approximation.The infinite-dimensional structure remains in the reference conditionals; the marginal term is a finite-dimensional generative-modeling problem.Source: §6 · p. 26
  • KL divergence and Pψ(µ) do not depend on the ambient norm, but the upper bounds generally do; weighted inequalities absorb this dependence into a reference-measure operator Γ.The bounds hold for norms satisfying their hypotheses, while weaker norms require more likelihood smoothness and less from the reference.Source: §6 · pp. 26, 28
  • CRPS decreases from d = 4 to d = 16 with only a small further gain at d = 32; explained variance falls near zero by k ≈ 8, indicating a data-driven cutoff.At d = 32, the lack of high-wavenumber gain despite resolving to k = 16 suggests the cutoff is due to the data rather than the choice of ψ.Source: Figure 9 · p. 27, §6 · p. 27

6.1 Wasserstein bounds

  • Wasserstein error is bounded through a nested transport problem coupling ψ-marginals externally and conditionals internally; Lipschitz variation of reference conditionals controls how marginal fitting error propagates through the lift.When the marginals match, the bound reduces to conditional transport; for the marginalized measure, the identity marginal coupling yields an average of diagonal conditional transport costs.Source: §6.1 · pp. 28–29

6.2 Kullback–Leibler bounds

  • KL error decomposes into marginal-model error and irreducible conditional error; the irreducible term is bounded using expected symmetrized conditional KL, equal to a difference of conditional expectations of Φ.The reverse KL admits an analogous decomposition, and the symmetrized bound controls both the forward irreducible error and the reverse suboptimality term.Source: §6.2 · pp. 31–32
  • Under fiber-Lipschitz likelihoods, second-moment bounds are quadratic in the Lipschitz constant and sharper when CσW ≲ 1; for Matérn-ν references, the KL approximation rate improves from h^ν to h^(2ν).The first-moment rate requires only first moments, while the second-moment bound uses conditional variances and also controls the reverse divergence.Source: §6.2 · pp. 32–33
  • Projected-gradient growth yields explicit KL error rates from conditional moments; fiber-critical sections remove the first term, while conditional-mean sections minimize the remaining term to an averaged conditional variance.The bound requires conditional moments, not Poincaré or log-Sobolev constants; projected-gradient Lipschitz continuity along fibers is one sufficient growth condition.Source: §6.2 · pp. 33, 35

6.3 Error bounds assuming functional inequalities

  • Conditional log-Sobolev inequalities bound KL error by averaged squared fiber-projected gradient norms, while the weaker conditional Poincaré inequality controls squared Hellinger error.The conditional log-Sobolev inequality implies a Poincaré inequality with the same constant, but not conversely.Source: §6.3 · pp. 35–36
  • For Gaussian references, the conditional covariance sets the functional-inequality constant and can improve as ψ grows; Matérn point observations yield conditional variance bounded by Ch^(2ν).The Matérn example assumes ν > n/2 and point evaluations on discrete sets with mesh norm h_d.Source: §6.3 · p. 36
  • Choosing ψ to minimize the residual-trace bound selects the leading prior-weighted eigenvectors, equivalently solving Mνφ = λΓ^-1φ; for Lipschitz potentials, the Matérn bound matches the h_d^(2ν) rate.The eigenproblem uses Mν = Eν[∇Φ∇Φ*]; the cited rate matches Corollary 6.9.Source: §6.3 · p. 37
  • Talagrand inequalities on the conditionals convert KL bounds into Wasserstein bounds; for Gaussian conditionals, the Poincaré, log-Sobolev, and Talagrand constants coincide, adding a factor of 2κT.The shared Gaussian constant is the operator norm of the conditional covariance and does not depend on w.Source: §6.3 · p. 37

6.4 Discussion

  • With exact conditioned sampling, marginal-fitting difficulty is governed by dim(ψ), not the discretization dimension; low-dimensional likelihood information can therefore avoid errors growing with ambient dimension.The ψ-marginal is fit at the observed resolution, while reference conditionals supply the function-space remainder without statistical cost.Source: §6.4 · p. 38

Appendix A. Proofs for the Gaussian optimal transport results

The appendix proves the Gaussian optimal-transport results by constructing the bounded positive map and establishing trace-class control of its identity correction; it then characterizes finite-rank covariance perturbations through an invariant subspace.

Theorem 3.8: Gaussian optimal transport

  • The integral representation establishes S = A − I and convergence in trace class, yielding a trace-norm bound for the correction.The proof uses an integrable majorant for the integrands.Source: Appendix A · p. 41
  • Theorem 3.8 gives a unique bounded positive map A with AΣA = Σ′, whose optimal transport correction S = A − I is trace class.The proof also provides an integral representation for S.Source: Appendix A · p. 42

Theorem 3.9: finite-rank perturbations

  • For a finite-rank covariance perturbation, each integral-representation integrand has rank at most d, while the correction’s range closure is K when C is definite.K is the smallest closed Σ-invariant subspace containing Range(ψ∗).Source: Appendix A · pp. 41–42
  • The finite-rank perturbation result also bounds rank C by twice rank S.The range of the relevant operator is contained in Range(S) + ΣRange(S).Source: Appendix A · p. 42
  • When Range(ψ∗) is Σ-invariant, the optimal map has the form TF(u) = u + ψ∗(F(ψ(u)) − ψ(u)), with F the marginal optimal map.Under the stated normalization, the two Gaussian measures share a common factor on the orthogonal complement.Source: Appendix A · p. 42

Use of large language models

The authors report using language models for numerical implementation, analysis, formatting, and proofreading, while taking responsibility for the article’s content and correctness.

Use of large language models

  • Language models aided numerical-experiment implementation, and interactive use of Fable 5 and GPT-5.6 Sol refined and partly developed the Section 6 analysis and Theorems 3.8 and 3.9.Fable 5 through Claude Code was also used for formatting and proofreading; the authors take responsibility for the article’s content and correctness.Source: Use of large language models · p. 43

Related papers

Loading 2609.17802v1…