PaperlayerView exact revision on arXiv

Source-linked AI summary

Closing the Approximation Gap in Simulation-free Latent SDEs

Henry D. Smith, Brian L. Trippe, Scott W. Linderman

arXiv:2606.16138v2stat.MLcs.LG

TL;DR

Existing simulation-free VI restricts the approximate posterior to a subset of the path laws available to simulation-based VI, limiting inference when posterior uncertainty is high. Helmholtz-SDE optimizes over compatible path laws by adding a marginal-preserving correction. On OU spiral data, it achieves a posterior approximation orders of magnitude closer than the square root gauge, with the largest gains for sparse observations and high noise.

  • Problem

    Simulation-free VI optimizes over a strictly smaller family of approximate posteriors than simulation-based VI, and fixing one compatible drift can degrade inference and learned dynamics under high posterior uncertainty.

    Source: §1 · pp. 1–2, §4 · p. 4

  • Method

    Helmholtz-SDE optimizes over path laws sharing prescribed one-time marginals by adding a q-divergence-free correction to a reference drift.

    Source: §1 · p. 2, §4 · p. 4, §4.1 · p. 4

  • Results

    On strong-spiral OU data, Helmholtz-SDE achieves a symmetric KL distance to the true posterior orders of magnitude smaller than the square root gauge, with the largest gains at sparse observations and high noise.

    Source: §6.1 · p. 8

  • Takeaways & Limitations

    Helmholtz-SDE closes the expressivity gap to simulation-based VI while retaining a simulation-free algorithm.

    Source: §7 · p. 10

    The analytical correction depends on state-independent diffusion and Gaussian marginals, and higher-order polynomial corrections scale unfavorably with latent dimension.

    Source: §7 · p. 10

Abstract

from arXiv · show

Recovering dynamical systems from noisy observations is a recurring challenge across scientific domains, including neuroscience and physics. Latent stochastic differential equations (SDEs) address this by modeling the system as an unobserved state that evolves according to a learnable SDE and generates the observations. Variational inference (VI) provides a tractable objective for fitting latent SDEs. Traditional VI algorithms evaluate this objective by numerical simulation over a time discretization, trading fidelity for computational cost. A recent class of algorithms, simulation-free VI, sidesteps this tradeoff by parameterizing the posterior through its instantaneous marginals rather than its drift. In this work, we show that the efficiency of existing simulation-free VI algorithms comes at a price: their parameterizations restrict the approximate posterior to a subset of the SDEs available to simulation-based methods, degrading posterior inference and parameter learning. We propose Helmholtz-SDE, a simulation-free VI algorithm that closes this gap by optimizing over path laws compatible with a prescribed collection of marginals. Helmholtz-SDE recovers dynamics more faithfully than prior simulation-free methods, with the largest gains under high posterior uncertainty. It further matches the performance of simulation-based VI at a fraction of the runtime. Code is available at https://github.com/lindermanlab/helmholtz-sde .

1 Introduction

Latent SDEs model noisy observations through an unobserved continuous-time state, and variational inference offers a tractable way to fit them. Existing simulation-free VI restricts the approximate posterior to a smaller family than simulation-based VI; Helmholtz-SDE removes this restriction by optimizing over path laws compatible with prescribed marginals.

1 Introduction

  • Latent SDEs represent an unobserved state evolving continuously in time and generating observations at finite time points.They support unsupervised discovery from noisy, irregularly sampled time series and have been applied across scientific domains.Source: §1 · p. 1
  • Simulation-free VI optimizes over a strictly smaller family of approximate posteriors than simulation-based VI.One-time marginals do not determine a path law, so choosing a single compatible drift restricts the available posterior family.Source: §1 · pp. 1–2
  • When observations are sparse or noisy, the arbitrary drift choice can substantially degrade posterior inference and learned dynamics.Source: §1 · p. 2
  • Helmholtz-SDE optimizes over path laws compatible with prescribed marginals by adding a correction that preserves those marginals.This retains the simulation-free framework while expanding the posterior family.Source: §1 · p. 2

2 Problem Statement

The problem statement sets out latent SDE models, their learning and inference goals, and why both are generally intractable. In nonlinear or non-Gaussian settings, the posterior drift is unavailable in closed form and solving its associated PDE can be computationally prohibitive.

2 Problem Statement

  • Latent SDE models use a K-dimensional latent state evolving over time to generate D-dimensional observations at finite time points.The prior state follows an SDE with drift and diffusion, with the exposition restricting notation to time-independent drift and state- and time-independent diffusion.Source: §2 · p. 2
  • The two goals are to learn parameters maximizing log p(y|θ) and infer an approximate posterior over the latent trajectory given data and parameters.With multiple trials, parameters are learned across trials while the posterior factorizes over trials.Source: §2 · p. 2
  • Learning is intractable because evaluating the marginal likelihood requires marginalizing over latent trajectories, while posterior inference requires solving for the posterior drift.In nonlinear or non-Gaussian settings the drift is unavailable in closed form, and solving the associated PDE can be computationally prohibitive even for small K.Source: §2 · p. 2

3 Background

The background frames variational inference as a tractable approach to joint learning and inference in latent SDEs, then contrasts simulation-based and simulation-free ways to evaluate its objective.

3.1 Variational inference for latent SDE models

  • Variational inference approximates the intractable posterior with a tractable family and minimizes the negative ELBO to fit the model.The negative ELBO upper-bounds the negative marginal log likelihood, with the gap equal to the KL divergence from the exact posterior.Source: §3.1 · p. 3

3.2 Simulation-based VI

  • Simulation-based VI evaluates the path KL over a time discretization at every gradient step, trading approximation fidelity against computational cost.It computes marginals by integrating an ODE for tractable families or simulates trajectories for more flexible drifts.Source: §3.2 · p. 3

3.3 Simulation-free VI

  • Simulation-free VI estimates the objective stochastically by sampling a random observation, time, and posterior states rather than evaluating full paths.This avoids the full observation sum, trajectory integral, and expectations over the variational family, while introducing variance managed by Monte Carlo averaging.Source: §3.3 · p. 3
  • Simulation-free methods prescribe one-time marginals and choose a compatible drift, allowing states to be sampled without simulating the preceding trajectory.SDE Matching and SVISE use Gaussian marginals with affine drifts, while choosing different drift constraints.Source: §3.3 · pp. 3–4

4 Helmholtz-SDE: Augmented simulation-free variational inference

Helmholtz-SDE resolves the ambiguity left by marginal-based parameterizations by optimizing over path laws that share prescribed one-time marginals. It characterizes compatible drifts, identifies the ELBO-optimal correction, and provides a tractable Gaussian-marginal algorithm with guarantees in the conjugate setting.

4 Helmholtz-SDE: Augmented simulation-free variational inference

  • Helmholtz-SDE expands simulation-free VI to optimize over path laws sharing the same one-time marginals.The paper characterizes the ambiguity, derives an optimal drift, and turns the result into an algorithm for Gaussian marginals and state-independent diffusion.Source: §4 · p. 4

4.1 The Fokker–Planck equation and divergence-free vector fields

  • Two diffusions with a common diffusion coefficient have the same one-time marginals if and only if their drifts differ by a q-divergence-free vector field.Such a field can be added to a compatible drift without changing the marginals.Source: §4.1 · p. 4

4.2 Optimizing over path laws at fixed marginals

  • At fixed marginals, minimizing the negative ELBO adds the q-divergence-free component of the prior-reference drift residual to the reference drift.The q-weighted Helmholtz decomposition separates the component that alters marginals from the component that leaves them invariant.Source: §4.2 · p. 5

4.3 A variational inference algorithm for Gaussian one-time marginals

  • For Gaussian one-time marginals and state-independent diffusion, Helmholtz-SDE approximates the optimal correction using a finite-dimensional polynomial subspace.It expands the residual in a Hermite basis; coefficients can be approximated by quadrature or Monte Carlo, and corrections of degree ℓ≥2 can yield non-Gaussian multi-time joints.Source: §4.3 · pp. 5–6
  • In the linear-prior, linear-Gaussian setting, a degree-one Helmholtz-SDE correction can attain zero KL gap to the true posterior.Existing simulation-free methods can incur an unbounded KL gap in this setting; two-node-per-dimension Gauss–Hermite quadrature computes the degree-one coefficients exactly.Source: §4.3 · p. 6
  • Outside the conjugate setting, Gaussian one-time marginals generally incur a nonzero KL gap to the true posterior.When Hermite coefficients are evaluated exactly, Helmholtz-SDE is guaranteed to perform at least as well as using the reference drift alone.Source: §4.3 · pp. 6–7

5 Related Work

Related work spans simulation-free methods in observation space, fractional-Brownian latent SDEs, mean-field inference, and latent ODEs, but does not address path-law non-uniqueness. This paper’s use of Helmholtz decomposition differs from approaches that impose divergence-free dynamics.

Dynamical systems models

  • Existing adjacent approaches do not address path-law non-uniqueness in marginal-based posterior parameterizations.These include observation-space simulation-free methods, fractional-Brownian latent SDEs, mean-field variational inference, and latent ODEs.Source: §5 · p. 7

Divergence-free constraints

  • Unlike prior uses of Helmholtz decomposition, this method characterizes compatible path laws and their optimal drift correction without imposing divergence-free dynamics.Source: §5 · p. 7

6 Experiments

Across conjugate, nonlinear synthetic, and real-world systems, Helmholtz-SDE improves dynamics learning over prior simulation-free methods and narrows the gap to simulation-based VI. The experiments examine posterior fit, learned dynamics, forecasting, and computational cost.

6 Experiments

  • The experiments progress from a tractable conjugate setting to nonlinear synthetic dynamics, predator-prey cycles, and reduced-order fluid dynamics.Methods used matched implementation choices and shared amortization schemes whenever supported; Helmholtz-SDE used degree ℓ=1 polynomial approximation throughout.Source: §6 · pp. 7–8

6.1 Ornstein-Uhlenbeck (OU) spiral

  • With strong prior rotation, Helmholtz-SDE achieved symmetric KL orders of magnitude closer to the true posterior than the square root gauge, especially with sparse observations and large noise.When ω = 0, the square root gauge could represent the posterior exactly because its drift had no divergence-free component.Source: §6.1 · p. 8
  • The regimes with the worst square root gauge posterior approximations were also those with the greatest prior-learning degradation, although better variational approximations did not improve learning monotonically.The divergence-free correction was often the difference between recovering the correct drift and learning an incorrect one.Source: §6.1 · p. 8

6.2 Noisy Lorenz attractor

  • 123.7 ± 3.4 s versus 3695.6 ± 503.8 s: Helmholtz-SDE reached per-trial nELBO below 50 nat, while SING was over 25× slower.It achieved a ∼20 nat smaller negative ELBO than other simulation-free methods and matched or exceeded simulation-based methods in dynamics recovery.Source: Figure 3 · p. 8, §6.2 · p. 9

6.3 Predator-prey system

  • 6.92 days and 1.66 days: Helmholtz-SDE’s median cycle period and predator-prey lag were close to estimates, while SDE Matching learned 13.33 days and 3.23 days.Helmholtz-SDE learned a stable limit cycle; SDE Matching decayed to a fixed point.Source: §6.3 · p. 9, Figure 4 · p. 9

6.4 Reduced order models of fluid dynamics

  • Approximately 10% lower integrated MSE than SDE Matching and SVISE: Helmholtz-SDE better recovered traveling-wave vortex advection over the 10-unit forecast window.The improvement came at approximately 3× larger per-iteration cost on an H100 GPU.Source: §6.4 · p. 10, Figure 5 · p. 10

7 Discussion

The discussion presents Helmholtz-SDE as removing a restriction on simulation-free posterior path laws while preserving prescribed marginals, yielding more faithful recovery of underlying dynamics. It also identifies assumptions and scaling costs that limit applicability.

7 Discussion

  • Helmholtz-SDE adds a closed-form, marginal-preserving correction to optimize over a larger family of path laws and close the expressivity gap to simulation-based VI.The discussion says this yields more faithful recovery of the underlying dynamics.Source: §7 · p. 10
  • The analytical correction assumes state-independent diffusion and Gaussian marginals, while higher-order polynomial corrections scale poorly with latent dimension.The authors report no experimental gains beyond linear and quadratic approximations.Source: §7 · p. 10

A Background on variational inference for latent SDEs

  • Appendix A derives the ELBO, discusses divergence-free fields, characterizes affine divergence-free fields for Gaussian marginals, and reviews SVISE and SDE Matching.It relates those algorithms’ posterior-drift choices to Helmholtz-SDE.Source: A Background on variational inference… · p. 17

A.1 Technical Assumptions

  • A.1 states formal assumptions for a well-posed model and VI objective, including unique strong SDE solutions, marginal densities, and finite relative entropy.The assumptions are verified for the Helmholtz-SDE variational family in Appendix B.4.Source: §A.1 · p. 17

A.2 ELBO derivation

This appendix derives the latent-SDE ELBO and characterizes how posterior path laws can differ despite sharing one-time marginals, relating existing simulation-free methods’ fixed drift choices to Helmholtz-SDE’s divergence-free corrections. It also describes computing the Helmholtz decomposition by solving a Poisson equation with an auxiliary diffusion, yielding a q-divergence-free residual.

A.2 ELBO derivation

  • The negative ELBO follows from Jensen’s inequality and Girsanov’s theorem, and each term depends on both generative-model parameters θ and variational-posterior parameters ϕ.Source: §A.2 · p. 19

A.3 Divergence-free vector fields

  • With a shared diffusion coefficient and initial density, two posterior SDEs have the same one-time marginals exactly when their drift difference is q-divergence-free.For a fixed prior SDE, the posterior diffusion coefficient must match the prior’s because differing diffusion coefficients are generally not mutually absolutely continuous under Girsanov’s theorem.Source: §A.3 · pp. 19–20

A.4 Gaussian one-time marginals

  • For Gaussian marginals, compatible affine drifts are characterized by a Lyapunov equation, while a skew-symmetric matrix parameterizes path-law freedom without changing those marginals.Nonlinear q-divergence-free fields can also preserve Gaussian one-time marginals while inducing non-Gaussian multi-time distributions.Source: §A.4 · pp. 20, 22

A.5 SVISE (Course and Nair [6]) and SDE Matching (Bartosh et al. [8])

  • SVISE [7] and SDE Matching [8] select single drift gauges for Gaussian marginals: respectively, a symmetric gauge and a square-root gauge.The symmetric gauge does not naturally extend to state-dependent diffusion; extending SDE Matching with a learned skew-symmetric correction could span compatible linear drifts, but loses its closed-form structure and remains within Gaussian-process path laws for state-independent diffusion. Helmholtz-SDE computes the optimal correction in closed form and can use higher-order polynomial approximations to produce non-Gaussian-process posteriors.Source: §A.5 · pp. 22–23

B Helmholtz-SDE

  • Appendix B outlines Helmholtz-SDE’s analysis, polynomial approximation of the optimal divergence-free correction, validity conditions, optimality discussion, and algorithm pseudocode.Source: B Helmholtz-SDE · p. 24

B.1 Computing the Helmholtz decomposition

  • For smooth, strictly positive marginals, the q-weighted Helmholtz decomposition splits the residual drift into a gradient and a q-divergence-free field, computed by inverting a diffusion Stein operator and approximable by Monte Carlo.The construction also applies with state-dependent diffusion, although the exposition assumes state-independent diffusion for notational simplicity.Source: §B.1 · p. 24
  • For fixed t, an auxiliary time-homogeneous diffusion with coefficient Σ(t)^1/2 yields a Poisson equation for the residual η; when Σ(t) = I_K, the diffusion is overdamped Langevin with invariant density q(x, t).The Poisson equation solution can be represented using a forward run of the diffusion initialized at x, under the regularity conditions of Gorham et al. [66, Theorem 5].Source: §B.1 · p. 25
  • The generator identity holds for twice continuously differentiable u satisfying integrability conditions, and the Poisson equation is identified as such in [68, 69].Source: §B.1 · p. 25
  • The decomposition defines h(x, t) = r(x, t) − Σ(t)∇xψ(x, t), whose q-weighted divergence vanishes, so h is the q-divergence-free component of the residual field.The construction first solves the equation for ψ.Source: §B.1 · p. 25

B.2 The Helmholtz decomposition for Gaussian marginals

For Gaussian one-time marginals, the appendix establishes a weak weighted Helmholtz decomposition and then gives conditions for its pointwise representation and regularity. The resulting split separates a gradient component from a q-divergence-free component.

B.2 The Helmholtz decomposition for Gaussian marginals

  • The weak decomposition exists uniquely in the weighted L2 space, with h the orthogonal projection onto the subspace of weakly q-divergence-free vector fields.The gradient component is orthogonal to that subspace.Source: §B.2 · p. 25
  • Theorem 3’s weak result alone does not establish pointwise divergence freedom, a gradient representation, or regularity; the subsequent Gaussian-marginal result supplies these properties.The appendix specializes to Gaussian one-time marginals and state-independent diffusion for this existence, uniqueness, and regularity result.Source: §B.2 · pp. 25–26
  • Under the stated regularity assumptions, the decomposition has a pointwise form with a strongly q-divergence-free component, and its potential and vector field have specified smoothness and uniqueness properties.The q-divergence-free component is unique pointwise, while the potential is unique up to a time-dependent additive constant.Source: §B.2 · p. 26
  • The proof constructs the gradient component through a Poisson equation for the Langevin generator, which becomes a system of one-dimensional Ornstein–Uhlenbeck processes in transformed coordinates.The pointwise equation and the resulting decomposition are established after this coordinate transformation.Source: §B.2 · pp. 26–27

B.3 Polynomial approximation of the optimal correction

This section approximates the optimal q-divergence-free correction by restricting it to polynomial vector fields and solving for the coefficients in a Hermite basis. The resulting finite-dimensional solution is unique and preserves the divergence-free constraint.

B.3 Polynomial approximation of the optimal correction

  • Restricting the correction to degree-ℓ polynomials yields the optimal correction within that subspace, computed coefficient by coefficient in a Hermite basis.The expansion is truncated at degree ℓ, and the resulting optimal correction is transformed back to the original coordinates.Source: §B.3 · pp. 31–32
  • The coefficient solution is the unique minimizer because the objective is strictly convex when every λ_i(t) is positive.The displayed coefficient conditions provide a feasible stationary point.Source: §B.3 · p. 32
  • At ℓ=0 the compatible correction cannot change the prescribed mean path; at ℓ=1 it is linear and skew-symmetric in whitened coordinates.When Λ(t) is proportional to the identity, the linear correction is the antisymmetric part of the averaged Jacobian.Source: §B.3 · p. 33
  • Estimated Hermite coefficients preserve the q-divergence-free constraint exactly, although the resulting field is generally not the optimal degree-ℓ correction.Stein estimands can reduce coefficient-estimation variance by replacing high-degree Hermite factors with residual derivatives, but this reduction is not guaranteed.Source: §B.3 · pp. 33–34

B.4 Verification of assumptions for the Helmholtz-SDE variational family

This section verifies that Helmholtz-SDE defines a valid variational family for correction degrees 0–3 under the stated prior assumptions, characterizes its optimality in conjugate and nonconjugate settings, and summarizes the Hermite correction and training algorithms. It describes estimating reconstruction and path-space KL terms from sampled times and states, then using these terms with the initial term to update model and variational parameters.

B.4 Verification of assumptions for the Helmholtz-SDE variational family

  • The posterior SDE has a pathwise-unique strong solution on all of [0,T]; the prior and posterior path laws are mutually absolutely continuous, and the path-space KL identity (11) is finite.Source: §B.4 · pp. 35, 37

B.5.1 Conjugate setting

  • In the conjugate setting, degree-1 Helmholtz-SDE can approximate the posterior path law arbitrarily closely in KL, because its projection captures all q-divergence-free fields at fixed Gaussian marginals.The posterior marginals may have derivative jumps at observation times, so the proof smooths them before taking the KL limit.Source: §B.5.1 · pp. 37–38

B.5.2 Nonconjugate setting

  • In the nonconjugate setting, increasing polynomial correction degree reduces negative ELBO at fixed marginals, and the resulting bound quantifies improvement over the reference-drift families SVISE [7] and SDE Matching [8].The fixed-marginal comparison follows from nested divergence-free projections and Hermite coefficients; the bound extends to optimizing within each variational family.Source: §B.5.2 · pp. 38–40

B.6 Algorithm summary

  • HelmCorrection estimates Hermite coefficients for the polynomial divergence-free correction, using coefficient samples independent of the state samples used for the path-space KL.Source: §B.6 · pp. 40–41
  • For the reconstruction estimate, it samples times from {t1, . . . , tN} and states from Gaussian distributions parameterized by m(m) and S(m).Source: §B.6 · p. 42
  • For the path-space KL estimate, it samples times uniformly on [0, T], applies HelmCorrection, and independently samples states.Source: §B.6 · p. 42
  • The algorithm combines the reconstruction, initial, and path-space terms, then takes a gradient step on model parameters θ and variational parameters ϕ.Source: §B.6 · p. 42

C Linear SDE experiments: the Ornstein-Uhlenbeck spiral

The OU spiral is a linear SDE with rotational drift and Gaussian observations. The section uses this model to examine whether existing simulation-free methods recover the posterior.

C Linear SDE experiments: the Ornstein-Uhlenbeck spiral

  • The OU spiral uses a two-dimensional linear SDE prior and a linear Gaussian observation model.The prior drift is (−αI2 + ωJ)x, and observation noise is R = σ2CC⊤.Source: C Linear SDE experiments: the… · p. 43
  • The prior has drift combining a contracting gradient field with a divergence-free rotational field.Its stationary distribution is N(0, 1/(2α)I2).Source: C Linear SDE experiments: the… · p. 43
  • Existing simulation-free methods cannot be expected to recover the posterior in this model, even though exact inference is possible.Additional simulations validate the theoretical claim.Source: C Linear SDE experiments: the… · p. 43

C.1 Theoretical results for OU spiral

For the OU spiral, the section theoretically characterizes why the symmetric and square root gauges are suboptimal. Their KL suboptimality grows with rotational speed, while the posterior-mean path remains optimally represented.

C.1 Theoretical results for OU spiral

  • The symmetric and square root gauge families’ KL suboptimality grows linearly with rotational speed ω.The analysis concerns the OU spiral with fixed prior and likelihood models.Source: §C.1 · p. 43
  • At each time, either eigenvalue separation creates rotational mismatch or keeping the smaller eigenvalue low requires costly negative control.The proof uses this alternative to establish a lower bound on covariance-path suboptimality.Source: §C.1 · p. 43
  • Under the stationary prior, the observed-data log likelihood is lower-bounded by a constant independent of ω.The proof uses uniform spectral bounds on the observation covariance.Source: §C.1 · p. 44
  • In the Gaussian OU setting, the KL separates into mean and covariance terms; the mean term is uniquely minimized by the posterior mean, leaving covariance to control suboptimality.The posterior is a Gaussian process because the prior and likelihood are linear Gaussian.Source: §C.1 · pp. 43–44
  • The covariance analysis separates intervals with unequal eigenvalues from times when the covariance is isotropic.The eigenvalue cases are combined to obtain an almost-everywhere bound.Source: §C.1 · pp. 44–47

C.2 Simulation experiment details

The OU spiral experiments test whether Helmholtz-SDE’s larger variational family improves posterior inference and learned dynamics in practice. The nonlinear experiments extend evaluation to Lorenz, predator-prey, and fluid-dynamics systems.

C.2 Simulation experiment details

  • The OU experiments test theoretical separation in practice by varying rotation speed, observation noise, and observation count, while learning prior and likelihood parameters jointly from 1024 trials.The reported results average five replicates; the theoretical variational-family results do not guarantee practical optima under finite-capacity parameterizations and stochastic optimization.Source: §C.2 · pp. 49–50
  • For OU inference, the true posterior is computed with a Kalman–Bucy smoother, while the variational posterior uses a 1001-point grid and a q-divergence-free correction.The path-KL estimates use Girsanov’s formula and discretize [0, 5] at stepsize 5 · 10−4.Source: §C.2 · p. 50
  • Learning experiments use a kernel-GLM posterior with a smoothness inductive bias, 200 inducing values, and a Matérn 5/2 kernel.A grid posterior converged very slowly, while a neural-network posterior was unstable and required aggressive gradient clipping.Source: §C.2 · pp. 50–51
  • When posterior uncertainty is high, Helmholtz-SDE improves dynamics recovery; the regimes where the other gauges poorly approximate the posterior are also where they poorly recover prior dynamics.The OU experiments vary observation noise, observation count, and rotational speed; with strongly informative observations, a single posterior drift can suffice even at high rotation speed.Source: §C.2 · pp. 50–51

D Nonlinear SDE experiments

  • The nonlinear SDE experiments cover the stochastic Lorenz attractor, rotifer-algae predator-prey dynamics, and reduced-order fluid-dynamics models.The experiments use Gaussian posterior approximations with dense covariance matrices.Source: D Nonlinear SDE experiments · p. 55

D.1 Noisy Lorenz attractor

The noisy Lorenz experiments examine posterior inference and prior-dynamics learning under nontrivial uncertainty. Helmholtz-SDE more faithfully recovers the attractor’s outward spirals than the simulation-free baselines, while the standard low-noise benchmark cannot distinguish methods by posterior uncertainty.

Noisy Lorenz attractor

  • The modified Lorenz setting uses larger process and measurement noise and less frequent observations, producing frequent lobe switching and nontrivial posterior uncertainty.The standard setup instead uses small process noise and dense, low-noise observations.Source: §D.1 · pp. 55–56
  • The comparison includes simulation-free ARCTA and SDE Matching and simulation-based SING and Latent SDE; Helmholtz-SDE uses an amortized posterior shared across trials and time.Its observation sequence is encoded by a GRU and mapped to posterior parameters by an MLP.Source: §D.1 · p. 56
  • Helmholtz-SDE more faithfully learns the Lorenz dynamics than SDE Matching and ARCTA/SVISE, particularly the outward spirals within each attractor lobe.In the standard benchmark, the posterior collapses to the true trajectory, making methods with different representations of posterior uncertainty indistinguishable.Source: §D.1 · p. 56, Figure 12 · p. 60
  • The q-divergence-free correction’s Stein estimand generally outperforms the original expectation, especially with few Monte Carlo samples; Gauss–Hermite quadrature is less sensitive to estimand degree.Increasing polynomial degree from ℓ=1 to ℓ=2 reduces negative ELBO by approximately 0.7 nats, with no significant improvement from ℓ=2 to ℓ=3.Source: §D.1 · pp. 60–61, Table 1 · p. 61
  • The estimated lower bound on negative-ELBO improvement is nearly 12 nats from degree ℓ=0 to ℓ=1, versus 0.4 nats from ℓ=1 to ℓ=2.These bounds are smaller than the observed 20-nat improvement from ℓ=0 to ℓ=1 and 0.7-nat improvement from ℓ=1 to ℓ=2.Source: §D.1 · p. 61, Table 2 · p. 61

D.2 Experimental predator-prey dynamics

This section details preprocessing, model setup, and evaluation for experimental predator-prey and reduced-order cylinder-flow dynamics, and examines how simulation-free VI captures temporal coherence in wake dynamics.

D.2 Experimental predator-prey dynamics

  • The predator-prey analysis uses one 374-day trial, restricts it to four coherent-oscillation subtrials, and log-transforms populations to model multiplicative noise with state-independent diffusion.With Gaussian one-time marginals in log coordinates, the population marginals are log-normal.Source: §D.2 · p. 62
  • Prior-dynamics evaluation draws 1,024 samples and reports median and IQR for peak period, predator-prey phase lag, and rotational coherence across 10 fitting replicates.Phase lag uses times meeting a coherence threshold of 0.83, while rotational coherence is the fraction of time points retained for phase analysis.Source: §D.2 · pp. 63, 65

D.3 Reduced order models for fluid dynamics

  • For cylinder flow, the study drops initial transients, projects training data onto 38 POD modes capturing 90% of kinetic energy, and adds noise at 0.3 times each dimension’s standard deviation.Observations are spaced by 0.3 time units; the authors note that less noise and more frequent observations make learned-dynamics discrepancies less apparent.Source: §D.3 · p. 66
  • The cylinder-flow model is a 38-dimensional ROM with a neural-network prior drift and fixed observation noise, compared with kernel-GLM and state-independent-diffusion SDE Matching baselines [8].Source: §D.3 · p. 66

• Helmholtz-SDE:

  • The cylinder-flow Helmholtz-SDE configuration uses a kernel-GLM posterior with 500 temporal inducing points, learned diagonal state- and time-independent diffusion, and 512 Monte Carlo time samples per iteration.Its Helmholtz correction uses a linear approximation with ℓ=1 and the Stein estimand, with Nmc=1.Source: • Helmholtz-SDE · p. 66

Helmholtz-SDE

  • Prior trajectories are evaluated over a 10-unit forecast from the true initial condition, with analysis restricted to that window because some trajectories begin diverging around 15–20 time units.The analysis reconstructs the velocity field through POD modes and measures streamwise velocity autocorrelation along the near-wake strip.Source: • Helmholtz-SDE · p. 67
  • Simulation-free VI loses temporal coherence, because temporal correlations depend on the joint posterior path law, whereas spatial correlations exist within a single snapshot.The observed space–time autocorrelation patterns are streamwise oscillations from periodic vortices and diagonal temporal bands from vortices moving downstream.Source: • Helmholtz-SDE · p. 67
  • Time-integrated MSE is computed over a 10-unit forecast using 128 simulated trajectories and reported across 10 training replicates with ±2 SE.The reconstructed velocity is compared with the true velocity projected onto the top 38 POD modes, with error normalized by total grid size.Source: • Helmholtz-SDE · p. 67
  • Helmholtz-SDE reports nELBO −2.57 · 10^4 ± 0.00 · 10^4; Table 3 reports nELBO in units of 10^4 and time-integrated MSE over the 10-unit forecast.Source: Table 3 · p. 68

E Runtime comparisons

The section compares Helmholtz-SDE runtime with the considered baseline VI algorithms and describes the timing and convergence criteria used for that comparison.

E Runtime comparisons

  • Runtime comparisons include Helmholtz-SDE and all baseline VI algorithms considered.Algorithms are timed on one NVIDIA H100 GPU; JIT compilation and ELBO evaluation time are excluded.Source: E Runtime comparisons · p. 68
  • Convergence is assessed using a dense-grid ELBO for Helmholtz-SDE, SING, SDE Matching, and SVISE, but 10 variational-SDE samples per trial for Latent SDE.Source: E Runtime comparisons · p. 68
  • Table 4 reports per-iteration runtime and the time and number of iterations needed to reach specified nELBO thresholds.Values are means ± 2 standard errors over 10 replicates; a dash means not every replicate reached the threshold within the training budget.Source: Table 4 · p. 68
Loading 2606.16138v2…