Source-linked AI summary
A non-parametric ensemble transform method for Bayesian inference
Sebastian Reich
TL;DR
Estimating Bayesian posterior expectations with finite Monte Carlo ensembles is challenging because ensemble Kalman filters can produce biased estimators. The paper introduces a deterministic optimal-transport ensemble transform and provides theoretical and numerical evidence that it yields consistent posterior estimators.
Problem
Ensemble Kalman filters provide robust finite-ensemble updates but remain biased even as ensemble size grows, motivating consistent posterior estimation methods.
Method
The paper constructs a deterministic linear ensemble transformation by solving a finite-dimensional optimal transportation problem through linear programming.
Results
The proposed ensemble transform produces consistent posterior estimators, with numerical and theoretical convergence demonstrated as ensemble size increases.
Takeaways & Limitations
The method supports a novel ensemble transform filter for applying the approach to intermittent data assimilation and sequential Monte Carlo methods.
Takeaways & Limitations
The analysis assumes independent and identically distributed prior ensembles and observations available at discrete times.
Abstract
from arXiv · showhide
Many applications, such as intermittent data assimilation, lead to a recursive application of Bayesian inference within a Monte Carlo context. Popular data assimilation algorithms include sequential Monte Carlo methods and ensemble Kalman filters (EnKFs). These methods differ in the way Bayesian inference is implemented. Sequential Monte Carlo methods rely on importance sampling combined with a resampling step while EnKFs utilize a linear transformation of Monte Carlo samples based on the classic Kalman filter. While EnKFs have proven to be quite robust even for small ensemble sizes, they are not consistent since their derivation relies on a linear regression ansatz. In this paper, we propose another transform method, which does not rely on any a prior assumptions on the underlying prior and posterior distributions. The new method is based on solving an optimal transportation problem for discrete random variables.
1 Introduction
The paper develops a deterministic, non-parametric ensemble transform for Bayesian inference from finite prior ensembles. It uses finite-dimensional optimal transport and is designed to produce consistent posterior estimators for filtering and intermittent data assimilation.
- Problem setting: The paper estimates posterior expectations from a finite ensemble drawn from a generally unknown prior distribution.This setting targets Monte Carlo implementations of Bayesian inference in filtering and intermittent data assimilation.
- Existing methods: Importance sampling uses likelihood-based weights, but recursive applications require a resampling step.The introduction cites monomial and systematic resampling as examples.
- Existing methods: The EnKF applies a classic Kalman linear update but remains biased even as M →∞.Its derivation transforms the prior ensemble through linear filtering.
- Contribution: The proposed method is a non-random ensemble transform based on finite-dimensional optimal transportation formulated as linear programming.The paper reports numerical and theoretical evidence for consistent posterior estimators.
- Paper organization: The paper develops a correlation-maximizing resampling method and modifies it into a deterministic linear transformation.The subsequent sections address importance sampling, its linear-programming formulation, and convergence of the proposed transform.
2 Bayesian inference and importance sampling
The section formulates Bayesian inference through Bayes’ theorem and explains how importance sampling uses a forecast ensemble to estimate posterior expectations. The resulting estimator is consistent but loses finite-ensemble unbiasedness.
- Bayesian inference: Bayes’ theorem defines the analyzed posterior PDF from the forecast prior PDF and the observation likelihood.The forecast random variable is Xf, the observation is y0, and the likelihood is πY(y|x).
- Monte Carlo approximation: Forecast ensembles provide realizations of the prior random variable for Monte Carlo estimation of expectations.The prior expectation of a function g is estimated from M ensemble members.
- Monte Carlo approximation: The prior Monte Carlo estimator is unbiased for any M > 0, and its variance vanishes as M →∞ when g has finite variance.This establishes the standard sampling properties of the prior expectation estimator.
- Importance sampling: Importance sampling estimates posterior expectations using the forecast ensemble.The method applies the importance-sampling idea to the posterior PDF πXa(x|y0).
- Importance sampling: The importance-sampling estimator is no longer unbiased for finite M but remains consistent as M →∞.Consistency means the root mean square error between the estimator and exact posterior expectation vanishes.
3 An ensemble transform method based on linear programming
The section develops an ensemble transform based on optimal transport and linear programming that avoids explicit PDFs and extends naturally to multivariate variables. It produces equally weighted posterior ensembles, supports a deterministic transformation, and converges to an optimal transport map as ensemble size grows.
- Optimal-transport formulation: The method constructs an optimal coupling between prior and posterior samples by solving a linear programming problem that minimizes expected squared Euclidean distance.The formulation requires no explicit knowledge of the underlying PDFs and generalizes to multivariate random variables.
- Posterior ensemble construction: Sampling one realization from each induced random variable yields an equally weighted ensemble that approximates posterior expectation values.The resulting resampling minimizes the expected distance between prior and posterior samples, unlike standard resampling choices.
- Deterministic transformation: A deterministic transformation replaces random resampling and generally avoids multiple copies in the analyzed ensemble while preserving a consistent estimator as M →∞.For g(x) = x, the method directly verifies the corresponding posterior-mean consistency property.
- Theoretical convergence: Under independent and identically distributed prior samples, the finite-ensemble maps converge weakly to a map whose pushforward has the posterior distribution and minimizes expected transport distance.The limiting map is characterized through cyclical monotonicity and is the gradient of a convex potential.
4 Application to sequential data assimilation
The proposed ensemble transform (ET) method is applied to sequential state estimation for ordinary differential equation models by alternating model propagation with data assimilation. Unlike the ensemble square root filter (ESRF), ET obtains its transform matrix through an associated linear programming problem without assuming Gaussian probability distributions.
- Sequential assimilation: ET propagates an initial ensemble under the model dynamics and applies the transformation when each observation becomes available.The analysed ensemble is then reused as new initial conditions, repeating propagation and assimilation for subsequent observation times.
- Sequential assimilation: The ET posterior ensemble is generated using a Markov-chain transform matrix P obtained from an associated linear programming problem.The transformation acts on the forecast ensemble after observations are incorporated.
- Comparison with ESRF: ET differs from ESRF because its transform matrix is obtained differently and does not rely on Gaussian probability density assumptions.The passage also identifies Lei and Bickel (2011) as an alternative approach for making EnKFs consistent with non-Gaussian distributions.
- Numerical example: The Lorenz-63 experiment observes all three state variables every ∆tobs = 0.12 time units with measurement error variance R = 8 and compares ESRF and ET for M = 10, 20, 40, 60, 80, 100.The equations use the implicit midpoint rule with step-size ∆t = 0.01.
5 Conclusions
The paper applies linear programming and optimal transportation to Bayesian inference and particle filters. It shows theoretically and numerically that the proposed ET method recovers posterior expectations as M →∞ and converges to the associated continuum optimal transport problem.
- The study explores linear programming and optimal transportation for Bayesian inference and particle filters.
- M →∞: the proposed ET method reproduces posterior expectation values in the limit.
- The ET method converges to the associated continuum optimal transport problem.