PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative models for simulation based filtering: Formulations and Empirical Comparisons

Mohammad Al-Jarrah, Wei Deng, Bamdad Hosseini, Amirhossein Taghvaei

arXiv:2609.16317v1cs.LG

TL;DR

Nonlinear filtering must represent a state posterior from noisy observations with finite samples produced online as observations arrive. This paper develops generative filters within a common transport formulation and compares them with established filters. The comparisons show that no single generative framework dominates, with preferences depending on online budget and ensemble size, while particle trajectory regularity also differs.

  • Problem

    Nonlinear filtering requires recursively representing the state posterior from noisy observations with a finite sample collection produced online.

    Source: §I · p. 1

  • Method

    The paper formulates filtering updates as transport from forecast distributions to posteriors and derives stochastic-interpolant, flow-matching, and Schrödinger-bridge filters.

    Source: §I · p. 1

  • Results

    All generative methods except SBF recover a bimodal reference posterior in the quadratic observation example, while SIR collapses to one mode and EnKF misses the bimodality.

    Source: C. The quadratic observation model · p. 4

  • Takeaways & Limitations

    OTF is preferable at small online budgets or ensembles, whereas SIF-SDE is preferable otherwise; OTF and KRF trajectories remain regular while the other generative methods’ nearby-particle paths crisscross.

    Source: §V · p. 6

    OTF is constrained by adversarial-training convergence, while KRF is constrained by per-coordinate bisection and back substitution at inference.

    Source: §V · p. 6

Abstract

from arXiv · show

This letter presents a unified formulation and a controlled numerical comparison of generative-model approaches to the nonlinear filtering problem. Under this formulation the analysis step is realized by a transport of the forecast distribution to the posterior, the approaches differing only in how that transport is selected and learned. We derive three new filters, based on stochastic interpolants, their deterministic flow-matching limit, and Schr\"odinger bridges realized through forward--backward SDEs. We develop a two-stage tuning procedure that separates the training of the generative model from its online refinement. The resulting methods are compared against the optimal transport filter (OTF), the Knothe--Rosenblatt filter (KRF), the sequential importance resampling (SIR) particle filter and the ensemble Kalman filter (EnKF), in terms of accuracy, computational time, and sensitivity to ensemble size and state dimension. The results indicate that every generative filter resolves multimodal posteriors that the EnKF and SIR do not, that no single generative framework dominates, the preferred method being set by the available online budget and ensemble size, and that the filters differ in the regularity of the particle trajectories they produce.

I. INTRODUCTION

Nonlinear filtering recursively updates a finite sample representation of a state posterior as noisy observations arrive. This paper develops a shared transport formulation for generative filters and compares methods under a controlled empirical study.

I. INTRODUCTION

  • Generative filtering applies conditional transport to move the forecast distribution to the posterior, extending coupling-based filters such as KRF and OTF,.These approaches share a triangular map structure but differ in how the map is represented and estimated.Source: §I · p. 1
  • The paper addresses the lack of a controlled comparison of generative filters under a common formulation, identical ensembles, and matched computational budgets.It also compares accuracy, computational cost, robustness, and tuning and implementation effort.Source: §I · p. 1
  • The paper derives SIF, deterministic FMF, and SBF, proposes two-stage tuning, and compares them with OTF, KRF, EnKF, and SIR on two benchmarks.It also studies cost–accuracy trade-offs and sensitivity to dimension and ensemble size.Source: §I · p. 1

II. PROBLEM FORMULATION

  • At each observation, filtering propagates the state distribution through the dynamics and then applies Bayes’ rule to condition on the observation.The target is the state posterior given the observation sequence.Source: §II · p. 2
  • A transport-based implementation represents the posterior with an ensemble of particles, forecasting them through the dynamics before applying an update map.The update map may depend on auxiliary randomness, and the resulting ensemble is uniformly weighted.Source: §II · p. 2
  • The central analysis problem is to learn a map that reproduces the conditioning operator from forecast samples paired with simulated observations.The forecast step requires only simulation of the dynamics; the difficulty lies in identifying an accurate update map.Source: §II · p. 2

III. GENERATIVE FILTERS

The section introduces a shared triangular-transport structure for generative filters and describes the characteristics of the individual methods.

III. GENERATIVE FILTERS

  • The section presents the common triangular structure of the generative filters and specifies the characteristics of individual methods.Source: §III · p. 2

A. Conditioning with triangular maps

Generative filters realize the analysis update with a triangular map learned to transform a source distribution into the joint distribution of observations and states. The map’s conditioning property does not determine a unique solution, leaving methods to differ in how they select and learn it.

A. Conditioning with triangular maps

B. Optimal Transport Filter (OTF)

OTF selects a conditional optimal-transport map, with quadratic cost yielding a max-min formulation. Its numerical implementation parameterizes the map and critic with residual networks and trains them adversarially.

B. Optimal Transport Filter (OTF)

C. Flow Matching Filter (FMF)

FMF realizes the transport map as a learned ODE flow, keeping the observation fixed along the flow so the map is triangular. Its vector field is trained by regression, and filtering applies the learned flow to forecast particles using the realized observation.

C. Flow Matching Filter (FMF)

  • FMF realizes the transport map T as the time-one flow of an ODE with a learned vector field.The artificial flow time τ is distinct from filtering time.Source: C. Flow Matching Filter (FMF) · p. 2
  • Holding the observation fixed along the flow makes the associated map triangular by construction.Source: C. Flow Matching Filter (FMF) · p. 3
  • The vector field is learned by regression, and analysis numerically integrates the flow from each forecast particle with the realized observation fixed.The regression expectation uses a uniform flow time, training pairs, and an independent forecast-state sample.Source: C. Flow Matching Filter (FMF) · p. 3

D. Stochastic Interpolant Filter (SIF)

SIF extends FMF by allowing noise in the source-to-target interpolant and stochastic generation. Its ODE and SDE variants use a score correction so that the SDE retains the ODE's time marginals.

D. Stochastic Interpolant Filter (SIF)

  • SIF generalizes FMF through a noisy interpolant and the option to generate with an SDE rather than an ODE.The noise level ε = 0 recovers the linear interpolant.Source: D. Stochastic Interpolant Filter (SIF) · p. 3
  • The score correction compensates for injected noise, so the SDE and ODE share time-τ marginals for every λ.Setting λ = 0 recovers the ODE.Source: D. Stochastic Interpolant Filter (SIF) · p. 3
  • The three implemented filters are FMF, SIF-ODE, and SIF-SDE, distinguished by deterministic or stochastic interpolation and ODE or SDE generation.FMF uses ε = 0, λ = 0; SIF-ODE uses ε > 0, λ = 0; SIF-SDE uses ε > 0, λ = 1.Source: D. Stochastic Interpolant Filter (SIF) · p. 3

E. Schr¨odinger Bridge Filter (SBF)

SBF selects a minimum-energy drift subject to source and target marginal constraints, whereas KRF learns a monotone triangular map to a Gaussian reference and composes it with its inverse at the realized observation.

E. Schr¨odinger Bridge Filter (SBF)

  • SBF uses a stochastic flow whose initial state is independent of the observation and whose endpoint has the joint observation-state distribution.Among drifts meeting these marginal constraints, it selects the minimum-energy drift.Source: E. Schr¨odinger Bridge Filter (SBF) · p. 3
  • Unlike FMF and SIF, SBF must determine its coupling through an optimality condition; its entropic regularization g tends to the OT map as g approaches zero.The problem has no closed-form solution and must be solved iteratively.Source: E. Schr¨odinger Bridge Filter (SBF) · p. 3
  • SBF training alternates forward- and backward-drift neural networks using likelihood objectives and simulated trajectories, unlike simulation-free FMF and SIF training.Source: E. Schr¨odinger Bridge Filter (SBF) · p. 3

F. Knothe–Rosenblatt rearrangement Filter (KRF)

  • KRF learns a monotone triangular map that sends the joint observation-state distribution to a Gaussian reference, then composes the map with its inverse at the realized observation.Each component depends on the full observation and only the first k state coordinates through component k.Source: F. Knothe–Rosenblatt rearrangement… · p. 3
  • KRF evaluates the inverse coordinate by coordinate using bisection and backward substitution.Source: F. Knothe–Rosenblatt rearrangement… · p. 3

IV. NUMERICAL RESULTS

The numerical-results section compares the generative filters on a quadratic observation model and the Lorenz-63 system.

IV. NUMERICAL RESULTS

  • The generative filters are compared on a quadratic observation model and the Lorenz-63 system.Source: §IV · p. 4

A. Tuning procedure

The tuning procedure separates initial network training from online refinement, while the error metric measures distance to a reference posterior and accounts for finite-ensemble sampling error.

A. Tuning procedure

  • The two-stage Bayesian-optimization procedure tunes from-scratch training at the first analysis step, then warm-started online refinement at later steps.Stage 1 selects and retrains a configuration whose weights initialize Stage 2; Stage 2 evaluates consecutive analysis steps at several optimizer-iteration budgets.Source: A. Tuning procedure · p. 4

B. Error metric and sampling floor baseline

  • Accuracy is measured by axis-aligned sliced Wasserstein-2 (aa-SW2), computed from coordinate-wise empirical quantile functions.Each summand is a one-dimensional optimal-transport problem between the filter ensemble and reference posterior.Source: B. Error metric and sampling floor… · p. 4
  • The reference posterior uses large-particle SIR for Lorenz-63 and independent two-dimensional SIR filters for each quadratic-model block.The quadratic-model reference stacks block marginals because full-dimensional SIR is unreliable there.Source: B. Error metric and sampling floor… · p. 4
  • The sampling floor estimates aa-SW2 error from an independent finite sample of the reference posterior, and methods are not expected to fall below it.It isolates error attributable solely to finite ensemble size.Source: B. Error metric and sampling floor… · p. 4

C. The quadratic observation model

In the quadratic-observation benchmark, scaling sweeps show that generative filters outperform EnKF and SIR across dimensions, while learned-map quality—not ensemble size—dominates residual error.

C. The quadratic observation model

  • The quadratic observation leaves each observed coordinate's sign unresolved, producing a bimodal posterior throughout the trajectory.The model has linear dynamics and a quadratic observation of one coordinate per two-dimensional block.Source: C. The quadratic observation model · p. 4
  • Generative methods except SBF recover the reference bimodality, whereas SIR collapses to one mode and EnKF's Gaussian approximation misses it.The comparison concerns posterior densities in the quadratic-observation benchmark.Source: C. The quadratic observation model · p. 4

D. Scaling studies

  • In the dimension sweep, all generative filters have lower error than EnKF and SIR at every dimension, degrade only mildly with dimension, and SBF is weakest.SIR degrades quickly, while EnKF error remains high and nearly independent of dimension.Source: D. Scaling studies · p. 5
  • Across ensemble-size sweeps, errors depend only weakly on N and remain above the sampling floor, implicating learned-map approximation quality as the dominant residual error.The ensemble-size trend is reported for the sweeps described in the section.Source: D. Scaling studies · p. 5

E. The Lorenz-63 benchmark

On Lorenz-63, transport filters perform best at small ensemble sizes and preserve particle modes more consistently; broader comparisons show that preferred methods depend on budget and ensemble size.

E. The Lorenz-63 benchmark

  • Without retuning quadratic-benchmark configurations, several generative filters struggle at smaller ensemble size, while OTF and KRF more consistently resolve both modes.The benchmark observes only the third component; sign symmetry makes the first two components' posterior bimodal.Source: E. The Lorenz-63 benchmark · p. 5
  • In Lorenz-63's small-ensemble sweep, OTF and KRF achieve lower aa-SW2 errors, with OTF showing the strongest improvement over the other methods.Figure 2d reports the ensemble-size comparison for Lorenz-63.Source: Figure 2 · p. 6, E. The Lorenz-63 benchmark · p. 6
  • OTF and KRF tend to preserve the previous mode across analysis steps, while nontransport methods frequently switch modes.The possible value of this temporal persistence for time-series inference remains for future investigation.Source: E. The Lorenz-63 benchmark · p. 6

V. DISCUSSION

  • FMF and SIF variants have nearly the same cost–accuracy trade-off, with SIF-SDE's slight advantage widening as ensemble size grows.SIF-SDE's noise makes it more sample-hungry, so its advantage appears only with larger ensembles.Source: §V · p. 6
  • SBF is more expensive because it lacks a simple regression loss and requires SDE simulations inside the training loop.Source: §V · p. 6
  • OTF is constrained by adversarial-training convergence, while KRF is constrained at inference by coordinatewise bisection and back substitution.Source: §V · p. 6
  • OTF is preferable at small online budget or ensemble size, while SIF-SDE is preferable otherwise.Source: §V · p. 6
  • OTF and KRF produce regular particle trajectories, whereas nearby particle paths crisscross under FMF, SIF, and SBF.The implications for downstream control and inference tasks remain future work.Source: §V · p. 6

Related papers

Loading 2609.16317v1…