Source-linked AI summary
Sparsifying Priors for Bayesian Uncertainty Quantification in Model Discovery
Seth M. Hirsh, David A. Barajas-Solano, J. Nathan Kutz
TL;DR
Noisy measurements complicate the discovery of governing equations, while existing Bayesian model-discovery methods generally produce non-sparse realizations. UQ-SINDy brings sparse Bayesian inference to SINDy, estimating coefficient uncertainty and term-inclusion probabilities. It produces accurate, meaningful models under significant noise and sparse sampling.
Problem
Noisy measurements complicate governing-equation discovery, while existing Bayesian model-discovery methods generally produce non-sparse realizations that do not identify relevant terms.
Source: §1 · pp. 1–2Method
UQ-SINDy applies sparse Bayesian inference to SINDy coefficients, estimating parameter uncertainty and term-inclusion probabilities with spike-and-slab and regularized-horseshoe priors.
Source: §1 · pp. 1–2Results
UQ-SINDy produced accurate, meaningful models under significant noise and sparse sampling; spike-and-slab and regularized-horseshoe posteriors had truly sparse draws, unlike Laplace.
Source: §1 · p. 2, §4 · p. 17Takeaways & Limitations
The framework represents uncertainty in coefficient values and in which candidate terms belong in a governing-equation model.
Source: §1 · p. 1Scalability to very large SINDy libraries is limited by the computational cost of MCMC sampling high-dimensional posterior distributions.
Source: §4 · p. 17
Abstract
from arXiv · showhide
We propose a probabilistic model discovery method for identifying ordinary differential equations (ODEs) governing the dynamics of observed multivariate data. Our method is based on the sparse identification of nonlinear dynamics (SINDy) framework, in which target ODE models are expressed as a sparse linear combinations of pre-specified candidate functions. Promoting parsimony through sparsity in SINDy leads to interpretable models that generalize to unknown data. Instead of targeting point estimates of the SINDy (linear combination) coefficients, in this work we estimate these coefficients via sparse Bayesian inference. The resulting method, uncertainty quantification SINDy (UQ-SINDy), quantifies not only the uncertainty in the values of the SINDy coefficients due to observation errors and limited data, but also the probability of inclusion of each candidate function in the linear combination. UQ-SINDy promotes robustness against observation noise and limited data, interpretability (in terms of model selection and inclusion probabilities), and generalization capacity for out-of-sample forecast. Sparse inference for UQ-SINDy employs Markov Chain Monte Carlo, and we explore two sparsifying priors: the spike-and-slab prior, and the regularized horseshoe prior. We apply UQ-SINDy to synthetic nonlinear data sets from a Lotka-Volterra model and a nonlinear oscillator, and to a real-world data set of lynx and hare populations. We find that UQ-SINDy is able to discover accurate and meaningful models even in the presence of noise and limited data samples.
1 Introduction
UQ-SINDy applies sparse Bayesian inference to governing-equation discovery, estimating both coefficient uncertainty and candidate-term inclusion probabilities. The introduction motivates this approach by the difficulty of identifying relevant terms with Bayesian models that are not sparse.
- The framework combines sparsity promotion with Bayesian inference to estimate parameter uncertainty and candidate-term inclusion probabilities.Source: §1 · p. 1
- Parsimonious representations are emphasized for interpretability and generalization, and SINDy has been applied across diverse fields and extended in several directions [9] [60] [63].Reported applications include chemistry, optics, engineered systems, epidemiology, and plasma physics; extensions address noise, PDEs, multiscale physics, and rational-function libraries.Source: §1 · pp. 1–2
- Existing Bayesian model-discovery methods quantify uncertainty but generally produce non-sparse realizations that cannot identify relevant model terms.Source: §1 · p. 2
- The spike-and-slab prior uses discrete indicators to select terms but makes high-dimensional inference expensive; smooth alternatives include the regularized horseshoe.The introduction notes that smooth priors can perform comparably to spike-and-slab and identifies the regularized horseshoe as the primary focus.Source: §1 · p. 2
- UQ-SINDy extracts accurate, meaningful Bayesian models even with significant noise and sparse samples.Source: §1 · p. 2
2 Background
The background frames SINDy as sparse equation discovery from time-series data and introduces Bayesian inference for uncertainty quantification. It reviews priors that shape sparsity in posterior coefficient distributions.
2.1 Sparse identification of nonlinear dynamics (SINDy)
- SINDy represents governing equations with a sparse combination of candidate library functions, estimating coefficients by sparse regression on observed derivatives.The coefficients encode the linear combination of library terms.Source: §2.1 · p. 4
2.2 Bayesian inference for data-driven discovery
- Bayesian inference estimates posterior distributions for model parameters from noisy observations, enabling posterior predictive reconstructions and forecasts.MCMC can approximate the posterior and the predictive distribution.Source: §2.2 · pp. 4–5
2.3 Sparsity promoting priors
- The section compares Laplace, spike-and-slab, and regularized horseshoe priors as approaches to sparsity in Bayesian regression.The priors differ in how they shrink coefficients and produce sparse posterior realizations.Source: §2.3 · p. 5, §2.3.1 · p. 6, §2.3.2 · p. 7
2.3.1 Laplace prior
- Laplace-prior posterior distributions can remain nonsparse, with irrelevant terms retaining substantial probability away from zero and obscuring relevance.The example’s posterior distributions are wide, and many peak at nonzero values.Source: §2.3.1 · p. 6
2.3.2 Spike and slab prior
- Spike-and-slab inference separates included coefficients from excluded ones, yielding sparse posterior draws and inclusion probabilities from the indicators’ means.The example shows broad distributions for three nonzero coefficients and a sharp spike at zero for seven zero coefficients.Source: §2.3.2 · pp. 6–7
2.3.3 Regularized horseshoe prior
- The regularized horseshoe combines global and local shrinkage with a slab cap; shrinkage factors serve as inclusion pseudo-probabilities rather than explicit probabilities.Large coefficients are regularized to the order of c, while smaller ones approximate the original horseshoe.Source: §2.3.3 · pp. 7–8
3 UQ-SINDY
UQ-SINDy combines sparse Bayesian inference with SINDy to estimate coefficient uncertainty and term-inclusion probabilities. Across synthetic and real-world examples, it identifies governing dynamics under noisy observations, while performance and ambiguity vary by dataset and prior.
3.1 Method
- The method constructs a symbolic library of candidate functions, specifies likelihoods and sparsity-promoting priors, then uses MCMC to sample coefficient and initial-condition posteriors and estimate predictive distributions.The priors include spike-and-slab and regularized horseshoe variants.Source: §3 · p. 8, §3.1 · pp. 8–9
3.2 Examples and applications
- UQ-SINDy significantly outperforms original SINDy in identifying underlying dynamics across three noisy examples.The examples comprise two synthetic systems and lynx–hare population data.Source: §3.2 · p. 10
3.2.1 Lotka-Volterra model
- For Lotka–Volterra data, both Bayesian priors distinguish the four true nonzero terms, while original SINDy misses the dynamics under observation noise.The Bayesian coefficient posterior modes are close to the true parameter values.Source: §3.2.1 · p. 11, Table 1 · p. 12
- Lotka–Volterra PPDs reconstruct the observed states and forecast held-out data with means close to truth and 90% intervals that capture uncertainty.Some test samples lie near the credibility bounds.Source: Figure 3 · p. 10, §3.2.1 · p. 11, Figure 4 · p. 12
3.2.2 Nonlinear oscillator and model indeterminacy
- For the nonlinear oscillator, rh-SINDy identifies all four governing terms, while ss-SINDy assigns uncertain inclusion to three terms and admits an alternative mode.The alternative mode reflects model indeterminacy; scaling priors for two small-coefficient terms removes the ambiguity.Source: §3.2.2 · pp. 13–15
3.2.3 Lynx-hare population model
- For lynx–hare data, ss-SINDy identifies the Lotka–Volterra model, rh-SINDy identifies three of its four terms, and original SINDy fails.Both Bayesian methods’ posterior modes are closer to the Lotka–Volterra maximum-likelihood estimates than the SINDy estimates.Source: §3.2.3 · p. 16
4 Conclusions and future work
UQ-SINDy discovers sparse probabilistic ODE models from noisy, sparse time series, quantifying coefficient uncertainty and term-inclusion probabilities. Its reported findings include robustness to noise and small datasets, with scalability and coordinate discovery identified as future-work boundaries.
4 Conclusions and future work
- UQ-SINDy quantifies coefficient uncertainty and term-inclusion probabilities; spike-and-slab and regularized-horseshoe priors yield truly sparse draws, unlike the Laplace prior.Source: §4 · pp. 16–17
- Across two synthetic examples and one real-world example, the approach is reported robust to observation noise and able to accommodate sparse samples and small datasets.Source: §4 · p. 17
- MCMC sampling of high-dimensional posteriors limits scalability to very large SINDy libraries; variational inference is proposed as a possible remedy.Source: §4 · p. 17
- The work focuses on settings where sparsity-inducing coordinates are known; learning unknown coordinates jointly is proposed as a future extension.Source: §4 · p. 18