Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems

1. Introduction

• MCMC requires sequential, correlated sampling and a new forward-model computation for each step and observation, making repeated inverse problems costly.A jointly trained transport map can instead provide approximate posterior samples for new observations at negligible online cost.Source: §1 · pp. 1–2
• The introduction situates flow matching among transport and generative approaches, noting its least-squares training and applicability when likelihoods are unknown or intractable.The paper also provides a unified account of the methodology for inverse-problems and uncertainty-quantification audiences.Source: §1 · p. 2
• The paper addresses the lack of a careful, MCMC-literate assessment of flow matching on increasingly difficult PDE-based inverse problems and develops computable posterior-accuracy estimates.It also proposes an asymptotically exact hybrid sampler using independent Metropolis–Hastings.Source: §1 · p. 2
• The flow’s tractable density and classical Monte Carlo theory yield computable lower and upper accuracy estimates in total variation distance and Kullback–Leibler divergence.The flow matching proposal can also be Metropolized to remove sample bias asymptotically.Source: §1 · pp. 2–3
• The numerical study checks accuracy estimates against quadrature, varies difficulty in a stationary diffusion problem, and examines amortization in EIT and likelihood-free Lotka–Volterra inference.In EIT, one offline training serves a stream of measurements.Source: §1 · p. 3
• Accuracy can degrade for concentrated posteriors and observations unlikely under the prior, and the paper leaves deeper mathematical analysis and remedies for future work.The authors characterize the work as computational and proof-of-concept.Source: §1 · p. 3

2. Posterior sampling via Markov chain Monte Carlo

• Metropolis–Hastings constructs a chain with the posterior as invariant distribution, and this section recalls the variants used in the paper’s comparisons.The paper uses RWMH, pCN, and DRAM as baselines, while IMH underlies its hybrid sampler.Source: §2 · p. 4, §2.2 · p. 5

2.1. The Metropolis–Hastings algorithm

• Each MH iteration proposes a candidate and accepts it by a probability based on posterior-density ratios; each step requires at least one forward solve.The chain is sequential, produces correlated output, and must be rerun for a new observation.Source: §2.1 · pp. 4–5
• Effective sample size measures how many independent draws would match the variance of a chain average, while poor mixing can make it far below chain length.The section cautions that ESS estimates are unreliable when a chain makes only a few dozen moves.Source: §2.1 · pp. 4–5

2.2. Common MH algorithms

• IMH draws proposals independently of the current state, so its performance depends on proposal–posterior overlap; using the flow approximation as proposal yields asymptotically exact sampling.A poor proxy can cause long rejection runs and highly correlated output.Source: §2.2 · p. 5
• RWMH uses Gaussian perturbations, pCN is prior-reversible and robust under mesh refinement, and DRAM adapts proposals and retries after rejection.These methods provide complementary baselines for the comparisons.Source: §2.2 · p. 5

3.1. Posterior sampling as a transport problem

• A block-triangular transport map trained on the joint parameter–observation distribution yields the posterior conditional on the observation for πY-almost every observation.The observation component is carried along unchanged, reducing posterior sampling to one transport problem on the product space.Source: §3.1 · p. 6

3.2. Neural transport maps and their training via Bayesian flow matching

• The ODE flow has a tractable pushforward density that supports diagnostics and Metropolization, but likelihood-based training requires costly nested ODE solves and derivatives in high dimensions.The density is computed by integrating backward and accumulating divergence along the trajectory.Source: §3.2 · p. 8
• Bayesian flow matching trains a conditional velocity field by regression on interpolated reference and joint samples, avoiding ODE solves and divergence evaluations during training.Each joint training sample requires a forward solve; training and sampling need only joint-distribution draws, so the method also applies when the likelihood is intractable but forward simulation is feasible. Finite capacity, data, optimization, and sampling discretization still introduce error.Source: §3.2 · pp. 8–9

3.3. Flow-based independent Metropolis–Hastings

• Using the flow approximation as an independent Metropolis–Hastings proposal removes its sampling bias and yields asymptotically exact posterior samples.Proposals are independent of the current chain state and can be generated in parallel before acceptance.Source: §3.3 · p. 10

3.4. Diagnostic metrics

• These Monte Carlo estimates are not guaranteed bounds, and finite-sample estimates of ρ tend to understate it when rare, large weights are missed, especially in higher dimensions.When the approximation is poor, the empirical KL upper bound may fall below the true divergence; it is reliable when weights behave well.Source: §3.4 · p. 11
• The flow’s tractable density enables computable estimates of lower and upper bounds on posterior error in total variation and Kullback–Leibler divergence.The stationary IMH acceptance rate informs total-variation bounds; importance-sampling and χ² relations support KL bounds.Source: §3.4 · pp. 10–11

4.1. Example with a two-dimensional parameter

• In the two-parameter problem, FM closely matched the true posterior for prior-likely observations, showed minor deviations for less likely ones, and FM-IMH removed them with acceptance rates of at least 84%.The posterior and its TV and KL distances could be computed by quadrature; prior predictive evidence varied over several orders of magnitude across the 20 observations.Source: §4.1 · p. 12, §4 · p. 12

4.2. Diffusion equation with distributed parameter

• For distributed diffusion, FM posterior means and 95% credible bands were visually indistinguishable from DRAM’s across four truths and two observation settings, though latent-coordinate errors grew as the posterior concentrated.The concentrated case constrained ξ2 to a posterior standard deviation about 180 times smaller than its prior; the MLP’s ξ2 marginal was about twice as wide as DRAM’s.Source: §4.2.1 · pp. 15–16, Figure 5 · p. 16, Figure 6 · p. 16

4.2.1. Comparison of FM and MCMC

• In the concentrated diffusion setting, weighting and a Gaussian velocity head improved FM-IMH acceptance for every truth; for x⋆ 3, rates rose from 5.97% to 25.47% and 14.67%, and to 33.02% together.The combined model also matched DRAM’s ξ2 and ξ4 marginal widths within 3% for three truths, although plotted posterior means and bands changed little.Source: §4.2.1 · pp. 16–17
• FM-IMH used 3 × 10^6 forward solves including training data, versus 2.1 × 10^7 for each DRAM chain, and reached ESS 87 703 versus 1 412 in one concentrated-posterior case.The reported per-1,000-effective-sample times were about 15 seconds for FM-IMH, or 26 seconds including training, versus about 85 seconds for DRAM; timings used different hardware.Source: §4.2.1 · pp. 17–18, Table 2 · p. 18

4.2.2. Observations outside the training support

• Observation typicality independently affected performance: the sine truth’s observation lay at 2.96 times the median training radius, with only 0.12% of training observations beyond it, and had the lowest acceptance.The radius ordering tracked acceptance more closely than posterior concentration; the sine observation was extreme despite its truth being a single Karhunen–Loève mode.Source: §4.2.2 · pp. 18–19

4.2.3. Parameter studies

• As diffusion posteriors concentrated, FM-IMH acceptance fell with observation count and latent dimension; its ESS stayed around 10^5 to 10^6 for three truths, while pCN ESS dropped by several orders with concentration.DRAM ESS was largely insensitive to observation count and noise, but declined with latent dimension; pCN ESS leveled off beyond nKL ≈32.Source: §4.2.3 · pp. 19–20

4.3. Electrical impedance tomography

• The EIT experiments use the Bayesian formulation of [DS16] with log-normal, level-set, and extended two-inclusion star-shaped priors; amortization is relevant because clinical devices produce new frames tens of times per second.A transport map is trained once for the shared prior and forward model, whereas a Markov chain must be restarted for each frame.Source: §4.3 · p. 20

4.3.1. The complete electrode model

• The EIT forward model is the complete electrode model, with conductivity on a bounded 2D domain, boundary electrodes on disjoint arcs, and positive contact impedances.The model maps a zero-sum current pattern to the interior potential and electrode voltages.Source: §4.3.1 · p. 20
• The EIT setup used 16 electrodes, 15 current patterns, and 195 voltage observations on a unit disk, with synthetic data and inversion solved on fine and coarse meshes.The noise standard deviation was γ = 2×10−4. The complete electrode model accounts for electrode shunting and contact impedances.Source: §4.3.1 · p. 20

4.3.2. Priors

• The three priors differ in representation, and the two-valued level-set and star-shaped priors have zero conductivity derivatives almost everywhere, precluding Gauss–Newton Hessians, Laplace approximations, and likelihood-informed subspaces.Only the star-shaped prior's support contains the two-inclusion truth; support alone does not ensure prior mass near the truth, which matters for amortized inference.Source: §4.3.2 · p. 21, §4.3.3 · p. 22

4.3.3. Results

• On y, star-shaped flow matched inclusion probabilities at correlation 0.9989 and area 0.182 versus 0.178 true; log-normal means correlated 0.995, while level-set probabilities correlated 0.975.The level-set flow was over-dispersed, with standard deviation 1.5 times the reference and inclusion area 0.222 versus 0.180. Log-normal means were far from truth because the prior poorly represented the sharp interfaces; this mismatch affected both samplers.Source: §4.3.3 · p. 22
• On y, IMH accepted 7.5% under the star-shaped prior but only 0.20% and 0.13% under log-normal and level-set priors, respectively.The observation was atypical under the latter priors, forcing flow extrapolation; the level-set latent posterior was also scale-invariant and thin in every direction. For prior-draw observations, acceptance averaged 36.7% under log-normal and 26.2% under star-shaped, but 0.11% under level-set.Source: §4.3.3 · p. 22

4.3.4. Dynamic Bayesian flow matching

• 21.7 ms per frame (46 frames per second): one flow tracked a 720-frame breathing phantom, with posterior-median RMSE 0.122 on average and inclusion-area correlation 0.997.The flow was trained once and conditioned on each frame; posterior area differed from the exact area by 0.002 on average and at most 0.043. Its uncertainty concentrated near inclusion boundaries, especially those farther from electrodes.Source: §4.3.4 · pp. 24–25, Figure 10 · p. 25

4.4. Stochastic Lotka–Volterra

• The example is a continuous-time predator–prey Markov jump process; its intractable likelihood does not prevent flow matching from using simulator-generated joint samples.Trajectories are simulated with Gillespie’s algorithm, and observations use time-series summary statistics.Source: §4.4 · pp. 25–26
• 25.5 minutes of training falls to approximately 90 seconds with Bayesian flow matching in the reported hardware and architecture setup.Source: §4.4 · p. 26
• The 50,000-sample model does not learn the posterior sufficiently well, which the authors attribute to the low number of training samples.Source: §4.4 · p. 26
• With 500,000 training samples, flow matching agrees well with SMC-ABC and matches it more closely than COT-Flow.Figure 11 compares models trained with 50,000 samples (top) and 500,000 (bottom).Source: §4.4 · pp. 26–27, Figure 11 · p. 27

5. Discussion

• Computable lower and upper bounds on TV distance and KL divergence are based mainly on the hybrid sampler's acceptance rate and were verified against quadrature values.The verification used the example in Section 4.1, where reference values could be computed by quadrature.Source: §5 · p. 27
• For prior-covered observations, flow matching closely matches MCMC posterior means, standard deviations, and quantiles at a fraction of the forward solves.In one setting, it reproduces the posterior mean and 95% band to about 1% in relative L2-norm, despite FM-IMH acceptance of only 5% to 11%.Source: §5 · pp. 27–28
• Highly informed, low-variance parameter marginals can be poorly fit, affecting TV distance more than posterior means or spread; many or highly accurate observations may also challenge flow matching.Source: §5 · p. 28
• Two proposed remedies for uneven posterior spread are inverse-linearized-standard-deviation loss weighting and an affine velocity term reproducing a Gaussian posterior approximation.Source: §5 · p. 28
• Flow matching can lose accuracy for prior-predictive-tail observations, where the authors prefer MCMC; normalized radius can be compared with training observations to check coverage.Source: §5 · p. 28
• EIT demonstrates amortization and Lotka–Volterra demonstrates likelihood-free inference; the IMH hybrid restores asymptotic exactness but mixes poorly for prior-tail observations.For typical prior observations, the IMH chain mixed well; its independent proposals can be evaluated in parallel.Source: §5 · p. 28
• TV and KL can degenerate in high- or infinite-dimensional function spaces, and the study leaves matched-budget total-cost comparison unattempted because outcomes depend strongly on hardware.Source: §5 · p. 28

A. The Metropolis–Hastings algorithm

• The samplers share a generic Metropolis–Hastings iteration and differ only in their proposal density.Source: A. The Metropolis–Hastings algorithm · p. 31
• Random walk Metropolis, pCN, and DRAM propose locally around the current state.Source: A. The Metropolis–Hastings algorithm · p. 31
• IMH instead draws proposals independently of the current state, using the flow as its proposal.Source: A. The Metropolis–Hastings algorithm · p. 31

B. Proof of Proposition 3.2

• For an independent proposal, writing the target and proposal densities as f and g gives f(x)g(x̃)α(x,x̃) = min{f(x)g(x̃), f(x̃)g(x)}.Source: B. Proof of Proposition 3.2 · p. 31
• The generic Metropolis–Hastings iteration takes a target density, proposal density, initial state, and chain length, then accepts or rejects each sampled candidate.Source: B. Proof of Proposition 3.2 · p. 32
• The proof establishes dTV(q, πy) ≤ D ≤ 2 dTV(q, πy).Source: B. Proof of Proposition 3.2 · p. 32

C. Flow matching in practice

• Training-set generation draws parameters from the prior, adds noise, and pairs each parameter with its forward-map observation.Source: C. Flow matching in practice · p. 32
• Flow-matching training fits the velocity field using minibatches of parameter–observation pairs and updates its parameters with Adam.Source: C. Flow matching in practice · pp. 32–33
• Approximate posterior sampling initializes draws from a standard Gaussian, with samples generated independently and batched in practice.Source: C. Flow matching in practice · pp. 32–33

• Both the MLP and gauss-and-weights flows reproduce the widely spread marginals of ξ1 and ξ3.Source: D. Marginals of the leading… · p. 33
• For strongly constrained ξ2 and ξ4, the MLP marginals are too wide or shifted.Source: D. Marginals of the leading… · p. 33
• The gauss-and-weights combination matches DRAM in all four coefficients apart from ξ1.Source: D. Marginals of the leading… · p. 33

• Log-normal draws are smooth, low-contrast fluctuations around the background value.Source: E. Draws from the EIT priors · p. 33
• Level-set draws are two-valued and cover about half the disk in many irregular components, several touching the boundary.Source: E. Draws from the EIT priors · p. 33
• Star-shaped draws contain two inclusions that can overlap or extend beyond the disk, and are the only prior draws resembling the two-disk ground truth.Source: E. Draws from the EIT priors · p. 33

Effective sample size

• Per forward solve, flow-proposal IMH leads the reference by about 35× in ξ and 11× in mean conductivity for the star-shaped prior.Under the log-normal and level-set priors, IMH makes only 19 and 12 moves, respectively, so those ESS estimates are rough; the one-time cost of 2.2 × 10^6 training and validation samples is excluded.Source: F. Flow matching against MCMC on the… · p. 35

Star-shaped

• For the star-shaped prior, both methods place uncertainty in interface annuli, with posterior means differing only in those regions.Source: F. Flow matching against MCMC on the… · p. 35, Figure 14 · p. 36

Log-normal

• For the log-normal prior, both methods produce similarly blurred means that are far from the piecewise-constant truth, reflecting prior mismatch rather than flow error.Source: F. Flow matching against MCMC on the… · p. 36, Figure 15 · p. 37

Level set

• For the level-set prior, the flow locates both inclusions but is over-dispersed, shifting inclusion probability from the inclusions into the background.Its estimated inclusion area is 0.222 against 0.180 for the reference.Source: F. Flow matching against MCMC on the… · p. 36, Figure 16 · p. 37

• With 10,000 proposals, flow-proposal IMH accepts 52.8% for star-shaped and 38.6% for log-normal, but visits only 9 states for level-set.The level-set comparison instead uses pCN, whose reference ESS is about 360 per element and 285 for inclusion area.Source: G. A prior draw as truth · p. 38
• Under the star-shaped prior, both means recover the draw’s two overlapping inclusions, blurred only at the interface.Source: G. A prior draw as truth · p. 38
• Under the log-normal prior, both means are nearly featureless because the measurements do not inform the draw’s fine-scale fluctuations.Source: G. A prior draw as truth · p. 38
• Under the level-set prior, flow and reference agree on structure near the boundary, while the interior remains uncertain.Source: G. A prior draw as truth · p. 38

H. Moving and merging inclusions

• 0.109 average RMSE (0.061–0.157 across frames) measures the posterior median’s deviation from truth over the 72-frame sequence.The inclusion area differs from the exact area by 0.002 on average and at most 0.029, with a 0.961 correlation over frames.Source: H. Moving and merging inclusions · p. 38
• Two inclusions orbit clockwise for 48 frames, then move to the center over 24 frames and merge; the flow follows every stage.The sequence uses 1,000 samples per frame, and the prior represents the merge by placing the two inclusions on top of each other.Source: H. Moving and merging inclusions · p. 38, Figure 18 · p. 40
• RK4 integration at 10 steps matches 25 steps within 0.002 per-frame RMS difference in the mean, while running at 46 versus 29 frames per second.The figure uses 10 steps; timings are reported on one RTX 4070 Ti SUPER.Source: H. Moving and merging inclusions · p. 38