PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem

Yang Zhao, Junxiong Jia, Tao Zhou

arXiv:2609.03343v2math.NAcs.LGcs.NA

TL;DR

Infinite-dimensional Bayesian inversion needs valid priors and density-like quantities, but measures may be singular relative to a reference. The paper learns priors with functional continuous normalizing flows and supplies training and sampling procedures. Experiments across representative PDE inversions report benefits where measurement information is limited, while ODE computation remains costly.

  • Problem

    Infinite-dimensional Bayesian inversion needs reference-measure densities for prior learning and posterior sampling, but Lebesgue densities do not exist there and measures may be singular to the reference.

    Source: §1 · p. 2

  • Method

    The paper learns infinite-dimensional f-CNF priors from direct parameter samples or indirect observations and uses pCN and SMC-GM algorithms for posterior sampling.

    Source: §1 · p. 3, §3.3 · p. 9, §3.2 · p. 9

  • Results

    Across representative PDE inversions, experiments support the framework; in inverse scattering and heat conduction, learned priors improve inference where likelihood information is limited.

    Source: §5.2 · p. 19, §5.3 · p. 23, §6 · p. 24

  • Takeaways & Limitations

    In the tested settings, learned priors can reduce reconstruction error or uncertainty in regions with limited measurement information.

    Source: §5.2 · p. 19, §5.3 · p. 23

    Training f-CNFs requires heavy ODE computations, creating considerable computational overhead for sophisticated models.

    Source: §6 · p. 24

Abstract

from arXiv · show

This paper addresses infinite-dimensional Bayesian inference for inverse problem of partial differential equations with model parameters in infinite-dimensional Hilbert space. To effectively incorporate prior information, we propose a novel continuous normalizing flows based infinite-dimensional model. Specifically, by introducing a well-defined neural ordinary differential equation in infinite-dimensional space, a simple reference measure can be transformed into a more complex measure which encodes the prior information. A corresponding theoretical framework is established to ensure the well-posedness of our proposed Bayesian prior in infinite-dimensional space. We also provide training methods of the prior for two distinct data settings, along with two sampling algorithms for the resulting Bayesian posterior. The proposed framework is applied to three representative inverse problems: the simple smooth inverse problem, inverse scattering problem, and the inverse heat conduction problem. Numerical experiments support the theoretical analysis and demonstrate the efficiency of the proposed algorithms.

1. Introduction.

The introduction motivates learned priors for infinite-dimensional Bayesian inversion, where handcrafted priors may miss problem-specific structure and full parameter samples may be unavailable. It proposes f-CNFs to learn priors from direct or indirect data, with theoretical guarantees and tailored posterior samplers.

Motivation

  • Handcrafted priors can capture smoothness or geometric structure yet may miss problem-specific structure and historical information.Direct-data generative methods also depend on access to full parameter samples, which may be unavailable in PDE inverse problems.Source: §1 · pp. 1–2
  • Infinite-dimensional priors must support valid density ratios against a reference measure, but mutual singularity can break the absolute continuity those ratios require.This complicates both prior learning and posterior sampling.Source: §1 · p. 2

Proposed framework

  • The paper proposes infinite-dimensional functional continuous normalizing flows (f-CNFs) to construct Bayesian priors from historical PDE-model data.A parameterized neural ODE transforms a simple reference measure into a complex prior while preserving measure equivalence and enabling an explicit RN derivative.Source: §1 · pp. 2–3

Training and sampling

  • The framework supports prior training from direct parameter samples and indirect observational data, then pairs learned priors with pCN and SMC-GM posterior samplers.For indirect training, it uses a Wasserstein loss and trains the full f-CNF network.Source: §1 · p. 3

Contributions

  • The paper generalizes measure-equivalence theory, derives an infinite-dimensional direct-data loss using RN derivatives, and verifies posterior well-posedness under trained f-CNF priors.Numerical experiments cover smooth inversion, inverse scattering, and inverse heat conduction.Source: §1 · p. 3

2. Functional Continuous Normalizing Flows.

The section extends continuous normalizing flows to function-valued parameters by evolving a Gaussian reference sample with an infinite-dimensional neural ODE. Its central measure-theoretic result establishes conditions under which the resulting prior remains equivalent to the reference and has an explicit RN derivative.

2.1. Basic Setting.

  • The infinite-dimensional ODE evolves an initial Gaussian function through a vector field; global Lipschitz continuity and boundedness ensure a unique solution and define the pushforward map.The terminal function is distributed according to the pushforward of the initial measure.Source: §2.1 · p. 4

2.2. The Radon-Nikodym Derivative.

  • Reference-measure densities are essential for learning and sampling, but infinite-dimensional measures can be mutually singular, so ordinary shifts or scalings may violate absolute continuity.The section frames measure-equivalent flows as the needed construction.Source: §2.2 · pp. 4–5
  • The theory extends measure equivalence from discrete functional normalizing flows to continuous flows through a limit theorem for measures.The resulting framework carries measure-equivalence guarantees into the f-CNF setting.Source: §2.2 · pp. 5–6
  • Under the stated conditions on the ODE vector field, the f-CNF pushforward is equivalent to the Gaussian reference measure and has an explicit RN derivative.This supplies a density-like quantity for the learned prior in infinite dimensions.Source: §2.2 · p. 7

3. Inverse Problem and Bayesian Prior.

The section develops an f-CNF prior for infinite-dimensional Bayesian inversion, using historical parameter or measurement data to inform the prior while the likelihood accounts for new observations. It describes posterior sampling, two training strategies, and extensions beyond the settings presented.

3.1. Bayesian Approach for Inverse Problem.

  • The f-CNF pushforward serves as an infinite-dimensional prior that encodes historical information, while the likelihood accounts for measurements in the inverse problem.The prior is especially intended to improve the posterior when the likelihood provides limited information.Source: §3.2 · p. 8, §3.1 · p. 8

3.2. Functional Continuous Normalizing Flows as Prior.

  • SMC-GM tempers measurement information and uses Gaussian-mixture mutations, particle weighting, and resampling to transform f-CNF prior samples into posterior samples.The paper motivates this alternative to pCN by its long mixing time, particularly with expensive forward solvers.Source: §3.2 · p. 9

3.3. Training with Direct Dataset.

  • Direct-data training fits the f-CNF pushforward to historical parameter samples, with an infinite-dimensional negative log-likelihood based on a reference-measure RN derivative.The reference-measure formulation is used because infinite-dimensional spaces lack a Lebesgue measure.Source: §3.3 · p. 9

3.4. Training with Indirect Dataset.

  • Indirect-data training learns from historical observations by matching their measurement distribution with measurements simulated from f-CNF parameter samples.The section describes a distributional loss using L2 to compare the empirical measurement distributions.Source: §3.4 · pp. 10–11
  • Indirect prior learning becomes difficult when PDE information loss, too few measurement points, or excessive noise obscures information about the parameters.These conditions diminish the parameter information in the historical measurement dataset.Source: §3.4 · p. 11

3.5. Other Settings.

  • The f-CNF framework also connects to functional flow matching and diffusion probability-flow ODEs, while broader prior-learning integrations remain future work.The diffusion connection relies on a conjecture adapted from, and the explicit reverse-SDE drift requires further investigation.Source: §3.5 · p. 12

4. Well-Posedness of Bayesian Inverse Problems.

The section establishes well-posedness criteria for Bayesian inversion with an f-CNF prior. Under suitable flow and likelihood conditions, the posterior exists uniquely and is stable in several probability metrics.

4. Well-Posedness of Bayesian Inverse Problems.

  • Bayesian well-posedness requires a unique posterior and continuous dependence of that posterior on the observed data.The section distinguishes notions of stability by the probability-measure metric used.Source: §4 · p. 13
  • The section classifies well-posedness using Prokhorov, total variation, Hellinger, or Wasserstein(p) metrics.Each metric defines a corresponding notion of posterior stability.Source: §4 · p. 13
  • With an f-CNF ODE satisfying Theorem 5 conditions and a continuous likelihood bounded above and positive, the inverse problem is weakly, Hellinger, total variation, and Wasserstein(p) well-posed.The likelihood bound is uniform in the parameter and data.Source: §4 · p. 13

5. Numerical Experiments.

The numerical experiments evaluate the learned f-CNF prior on three canonical inverse problems, reporting inversion and sampling findings for the simple smooth, scattering, and heat-conduction problems. In the heat-conduction setting, they show samples closer to the background truth and compensation for baseline-prior uncertainty when observations cover only part of the parameter interval.

5. Numerical Experiments.

  • The experiments cover simple smooth inversion with direct and indirect datasets, scattering with direct data, and heat conduction with indirect data.The paper reports that incorporating the learned prior significantly improves inversion results compared with using µ0.Source: §5 · p. 14

5.1. Simple Smooth.

  • In both the simple smooth and scattering examples, samples from learned f-CNF priors resemble the background truth more closely than samples from µ0.The simple smooth example uses learned priors from direct and indirect datasets; the scattering prior is trained on direct data.Source: §5.1 · p. 15, Figure 1 · p. 16, Figure 3 · p. 19
  • With sparse measurements, f-CNF priors trained on direct or indirect data mitigate the large posterior-mean error under µ0 and yield narrower credible regions and smaller covariance values.The passages provide no numerical error values.Source: Table 1 · p. 16, §5.1 · pp. 16–17, Figure 2 · p. 17

5.2. Inverse Scattering Problem.

  • In the scattering problem, the learned-prior posterior improves predictive accuracy and reduces uncertainty, especially in the unobserved lower-left region where likelihood information is limited.The reference-prior posterior is more accurate and tightly constrained near the three upper-right observation points.Source: Table 2 · p. 19, §5.2 · p. 19, Figure 4 · p. 20

5.3. Inverse Heat Conduction Problem.

  • The heat-conduction example infers the heat flux on a rod’s left boundary from temperature measurements at a sensor on its right end.The right end is insulated, and measurements span t ∈ [0, T2].Source: §5.3 · p. 21
  • The experiment sets T1 = 1 and T2 = 2 because heat flux takes time to propagate, so recovering u(t) over [0,T1] typically requires observations through T2.The baseline reference prior is µ0 = N(0, C0), with C0 = (I − α∆)^−2 and α = 0.05.Source: §5.3 · p. 21
  • The f-CNF prior is trained with Algorithm 2 using indirect data formed from simulated measurements at 50 times and Gaussian white noise of variance 0.001.The indirect dataset uses simulated parameter functions, and the background truth is generated by the same sampling procedure.Source: §5.3 · pp. 21–22
  • Samples from the trained f-CNF prior are more similar to the background truth than samples from µ0.Figure 5 compares samples from both priors with the background truth.Source: §5.3 · p. 22, Figure 5 · p. 22
  • With full measurements, the baseline prior µ0 yields accurate inversion results.The experiment observes w(L,t) over [0,T2] with 20% Gaussian noise and uses SMC-GM with 20,000 samples.Source: §5.3 · p. 22, Figure 6 · p. 23
  • With measurements ending at t=0.9, the learned prior compensates for baseline posterior uncertainty and mean error in the unobserved interval [0.9, 1].The comparison used SMC-GM with 20,000 samples for each prior.Source: §5.3 · pp. 22–23, Figure 6 · p. 23
  • The infinite-dimensional f-CNF transformation preserves measure equivalence while retaining flexibility in the induced measure, supporting a rigorous Radon–Nikodym derivative framework.Source: §6 · p. 24

6. Conclusion.

The paper presents f-CNF priors for infinite-dimensional Bayesian inverse problems and evaluates them in three representative settings. It also identifies computational overhead and architecture design as areas for further work.

6. Conclusion.

  • f-CNF priors are evaluated on a smooth linear inverse problem, an inverse scattering problem, and an inverse heat conduction problem, using direct data for cases 1 and 2 and indirect data for cases 1 and 3.The paper frames the model as suited to infinite-dimensional Bayesian inverse problems.Source: §6 · p. 24
  • Training f-CNFs entails heavy ODE computations, and task-specific architectures may improve performance on complex inverse problems.Future directions include broader applications, more expressive architectures, and measure equivalence for infinite-dimensional diffusion models.Source: §6 · p. 24

SUPPLEMENTARY MATERIAL OF “LEARNING INFORMATIVE PRIOR WITH INFINITE-DIMENSIONAL CONTINUOUS NORMALIZING FLOW FOR BAYESIAN INVERSE PROBLEM”

The supplement provides proofs and additional models and experiments that were excluded from the main text.

SUPPLEMENTARY MATERIAL

  • The supplement contains full proofs for the manuscript’s lemmas and theorems, plus extra models and numerical experiments excluded from the main text.The material is organized according to the main article’s sections.Source: SUPPLEMENTARY MATERIAL OF “LEARNING… · p. 27

1. supplementary of section 2

This supplement develops determinant definitions and proofs supporting the infinite-dimensional flow framework. Its arguments establish measure equivalence and an explicit Radon–Nikodym derivative for the flow prior.

1. supplementary of section 2

  • The supplement covers functional determinant definitions and proofs for Lemma 1 and Theorems 2 and 5.These materials correspond to Section 2 of the main article.Source: §1 · p. 27

1.1. Definition of Functional Determinant.

  • The Fredholm–Carleman determinant is defined for trace-class operators, while the regularized determinant addresses Hilbert–Schmidt operators where the standard product diverges.The regularized definition adds exponential convergence factors.Source: §1.1 · p. 27

1.2. Proof of Lemma 1.

  • The Lemma 1 proof decomposes the composed map, establishes injectivity and mutual absolute continuity, then derives an explicit Radon–Nikodym density.The proof reports injectivity of the Fréchet derivative and measure equivalence.Source: §1.2 · p. 28

1.3. Proof of Theorem 2.

  • Theorem 2’s proof uses positivity of the weight and weak convergence to identify the flow pushforward measure with the weighted reference measure.The argument invokes dominated convergence and uniqueness of weak limits.Source: §1.3 · pp. 29–30

1.4. Proof of Theorems 5.

  • Theorem 5’s proof approximates the forward flow by discrete steps, establishes equivalence for the discrete pushforward, and passes to the continuous-flow measure.The discrete weight converges pointwise, and an integrable dominating function supports the measure-equivalence conclusion.Source: §1.4 · pp. 30–32

2. supplementary of section 3

The supplement details posterior sampling with f-CNF priors, including pCN and SMC-GM, an accelerated Radon–Nikodym evaluation, and a splitting strategy for pCN. The accelerations address computational cost and mixing challenges associated with the learned prior.

2. supplementary of section 3

  • The supplement describes integrating pCN and SMC-GM with the f-CNF prior, alongside RN-derivative and splitting accelerations.It also mentions optional acceleration methods for computing the RN derivative.Source: §2 · p. 33

2.1. pCN algorithm.

  • When informative observations make the posterior differ substantially from the prior, pCN needs smaller steps, which can correlate successive samples and hinder independent sampling.The supplement presents SMC-GM as an alternative for this setting.Source: §2.2 · p. 34

2.2. SMC-GM algorithm.

  • The supplement gives an SMC-GM procedure for posterior inference under the f-CNF prior, initialized with particles drawn from that prior and followed by mutation, reweighting, and resampling.The procedure returns particles approximating the posterior.Source: §2.2 · p. 34

2.3. Accelerate.

  • Theorem 2.1 computes the RN derivative without the inverse flow mapping, avoiding two ODE solves per sample and reducing computational cost by half.The original route requires an inverse ODE followed by a forward ODE; the alternative requires one ODE solve during training.Source: §2.3 · pp. 34–35

2.4. Splitting acceleration strategy.

  • The splitting strategy performs repeated low-cost energy-based local updates before one global data-fidelity acceptance test, mitigating slow mixing without approximation error.It applies to f-CNF because its prior shares the TV-Gaussian prior’s structural form.Source: §2.4 · pp. 36–37

3. supplementary of section 4

This supplementary section proves Theorem 9 by establishing integrability bounds for the posterior and then deriving well-posedness in several metrics.

3. supplementary of section 4

  • The supplementary section provides the proof of Theorem 9 from Section 4 of the main text.Source: §3 · p. 37

3.1. Proof of Theorem 9.

  • Theorem 9 concludes that the Bayesian inverse problem is well-posed in the weak, Hellinger, and total variation metrics.The proof establishes integrability of the likelihood-weighted prior density using a bound on the likelihood and an integrable dominating function.Source: §3.1 · p. 37
  • The posterior is also shown to be well-posed with respect to the p-Wasserstein metric.Source: §3.1 · p. 38

4. supplementary of section 5

The supplement proves that the functional continuous planar flow meets Theorem 2’s conditions and describes other f-CNF constructions beyond the model used in the main article.

4.1. Proof of functional continuous planar flow.

  • The flow’s parameterization gives h(·, t; θ) a finite-rank derivative continuous in its input, establishing the stated C1 regularity.The argument parameterizes ui(t) using functions ϕj in H(µ0).Source: §4.1 · pp. 38–39
  • The proof bounds rank-one derivative components using compact-time continuity and applies the mean value theorem to establish the Lipschitz condition.The component operators Ai are rank one, and the norms of ui(t) and wi(t) are bounded on [0, T].Source: §4.1 · pp. 39–40
  • The proof concludes that the functional continuous planar flow belongs to T(µ0, [0, T], H), establishing the conditions required by Theorem 2.The proof is organized into four parts.Source: §4.1 · pp. 38, 40

4.2. Other flow model.

  • The supplement defines continuous Householder, projected-transformation, and Sylvester flows by extending earlier models,, and and, respectively.It also describes a nonlinear transformation using A(t), B(t), b(t), and tanh, reported to approximate complex multimodal distributions more accurately than the Householder and projected-transformation flows.Source: §4.2 · pp. 40–41
  • The functional continuous Householder flow has a linear structure, so its transformed measure remains Gaussian and its representational capacity is limited.The passage describes it as a generalization of the functional Householder flow.Source: §4.2 · p. 40
  • The projected-transformation flow uses covariance-eigenfunction projection and reconstruction operators; its linearity limits representation, so it is not recommended for complex or multimodal targets.Its parameters are R(t) and b(t).Source: §4.2 · pp. 40–41
Loading 2609.03343v2…