Source-linked AI summary
Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
Urban Fasel, J. Nathan Kutz, Bingni W. Brunton, Steven L. Brunton
TL;DR
Sparse model discovery is vulnerable to noisy measurements and limited data. The paper introduces E-SINDy, which ensembles SINDy models and uses aggregate statistics for robust discovery and uncertainty-aware applications. Across PDE, active-learning, and control settings, E-SINDy improves robustness and data efficiency while retaining computational scaling similar to standard SINDy.
Problem
Limited data and noisy measurements make sparse nonlinear model discovery difficult, especially because derivative computation can amplify noise in PDE settings.
Method
E-SINDy identifies ensembles of SINDy models from bootstrapped data or library terms, then aggregates coefficients and computes inclusion probabilities.
Results
E-SINDy significantly improves model-discovery robustness and accuracy, including a 2.3-fold improvement in WSINDy noise robustness across investigated PDEs.
Takeaways & Limitations
Ensemble statistics support probabilistic forecasts, active learning, and improved nonlinear model predictive control in low-data settings.
Takeaways & Limitations
Model quality depends on choosing an appropriate candidate-function library, whose complexity and order must be increased until sparse and accurate models are obtained.
Abstract
from arXiv · showhide
Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world data sets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900-1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control.
1 Introduction
SINDy enables sparse discovery of nonlinear dynamical systems but remains sensitive to limited, noisy data, motivating the ensemble-based E-SINDy extension. E-SINDy aggregates models from bootstrapped data or library terms to improve robustness and support uncertainty-aware forecasting, active learning, and control.
- Motivation: Limited data and noisy measurements challenge data-driven model discovery, particularly when derivative computation amplifies noise.The problem is especially severe for PDEs involving higher-order spatial derivatives.
- Contribution: E-SINDy applies bootstrap aggregating to ensembles of SINDy models for ordinary and partial differential equation discovery.Sparse regression is performed on measured-data bootstraps or library-term subsets.
- Contribution: Low inclusion probabilities are thresholded after aggregating coefficient statistics, producing models with uncertainty information for candidate functions.The framework uses mean or median coefficient aggregation and inclusion probabilities for model selection.
- Applications: E-SINDy supports probabilistic forecasts, active learning, and nonlinear model predictive control through ensemble statistics.The ensemble statistics provide uncertainty estimates that can guide these downstream tasks.
- Efficiency: E-SINDy has similar computational scaling to standard SINDy while identifying models and summary statistics in seconds rather than hours for MCMC-based methods.The computational comparison is reported against probabilistic model identification methods based on Markov Chain Monte Carlo sampling.
- Evaluation: Experiments on synthetic and real-world data evaluate E-SINDy in challenging model-discovery, active-learning, and control problems.The paper reports applications across several datasets and emphasizes low-data, high-noise settings.
2 Background
SINDy identifies sparse nonlinear dynamical models by regressing measured derivatives onto a library of candidate functions. PDE-FIND extends this framework to partial differential equations, where derivative-induced noise amplification motivates weak and integral formulations.
- Sparse identification: SINDy assumes that dynamical systems contain relatively few active terms and uses sparse regression to select them from candidate libraries.Sparsity-promoting optimization balances model complexity with accuracy.
- Sparse identification: The method constructs a snapshot data matrix, computes a library of candidate nonlinear functions, and estimates time derivatives before regression.The library may include constants, powers, and functions such as sine terms.
- Sparse identification: The coefficient matrix Ξ records each library term's contribution to the dynamics, and sparse regression identifies a model with few active coefficients.Regularizers such as STLS and STRidge promote sparsity during coefficient estimation.
- PDE discovery: PDE-FIND generalizes SINDy by including spatial derivatives in the candidate library for partial differential equation discovery.Its implementation arranges spatial time-series data into a vectorized representation and evaluates derivative-containing terms.
- Noise robustness: Noise is especially problematic for PDE-FIND because computing high-order partial derivatives amplifies measurement noise.Weak and integral formulations address this issue by multiplying the PDE by weight functions and integrating over domains.
- Noise robustness: The weak formulation replaces pointwise derivatives and library evaluations with integrated quantities before sparse regression identifies Ξ.The paper uses this weak formulation as the baseline and basis for its ensemble models.
3 Ensemble SINDy
E-SINDy incorporates ensemble methods into sparse model discovery to improve robustness and quantify uncertainty. It uses data or library bootstraps, aggregates coefficient estimates, and thresholds terms using inclusion probabilities.
- Framework: E-SINDy combines multiple sparse models to improve model-discovery robustness and provide inclusion probabilities and uncertainty estimates.These statistics support probabilistic forecasting and active learning.
- Data ensembling: b(r)agging E-SINDy samples data bootstraps and aggregates model coefficients with the mean for bagging or median for bragging.Bootstraps are data samples drawn with replacement.
- Aggregation: Inclusion probabilities measure how often candidate coefficients are nonzero across the ensemble, and low-probability terms are thresholded.The resulting aggregated coefficient matrix can be used as the E-SINDy model.
- Library ensembling: Library E-SINDy samples l of D library terms without replacement before identifying an ensemble of sparse models.Discarding low-probability terms can reduce the library size before standard SINDy or b(r)agging E-SINDy.
- Efficiency: Library bagging can reduce regression cost because solving least-squares problems scales as O(ml^2) for smaller sampled libraries.This can offset the cost of solving multiple regression problems in the ensemble.
- Uncertainty: Ensemble coefficient distributions can be aggregated for forecasting or sampled as multiple SINDy models to generate probabilistic forecasts.The distributions form a posterior distribution over model coefficients.
4 Results
E-SINDy improves sparse model discovery across ecological data, noisy PDEs, active learning, and low-data control by combining ensemble statistics with sparse regression.
- Ordinary differential equations: E-SINDy identifies Lotka–Volterra dynamics from only 21 noisy lynx-and-hare observations collected between 1900 and 1920.Library bagging uses ensemble coefficient statistics and reconstruction uncertainty for the ecological data.
- Ordinary differential equations: E-SINDy identifies models and coefficient and inclusion probabilities in seconds, compared with several hours for Markov Chain Monte Carlo methods.Its computational scaling is similar to standard SINDy.
- Partial differential equations: For all tested PDEs, E-WSINDy reduces coefficient error and increases discovery success rate, including correct nonlinear reaction-diffusion structure missed by WSINDy.The PDE tests cover inviscid Burgers, Korteweg–de Vries, nonlinear Schrödinger, Kuramoto–Sivashinsky, and reaction-diffusion equations.
- Partial differential equations: 2.3-fold greater noise robustness is achieved by ensembling WSINDy across the investigated PDEs, enabling identification in the extreme-noise limit.The comparison uses the noise level at which success rate falls below 90%.
- Exploiting ensemble statistics for active learning: Active E-SINDy selects high-forecast-variance initial conditions, reducing coefficient variance, increasing structure-identification success, and lowering coefficient error versus standard E-SINDy.After 80 active-learning steps, coefficient distributions have significantly reduced variance and model discovery becomes more data-efficient.
- Ensemble SINDy model predictive control: E-SINDy significantly improves Lorenz-system model-predictive-control performance over SINDy in the low-data limit.The evaluation compares models trained with 50 and 150 time-step data points under noise level 0.01.
5 Discussion
E-SINDy robustifies sparse model discovery through ensembling, reducing data requirements and increasing noise tolerance while providing interpretable uncertainty estimates. These ensemble statistics also support probabilistic forecasting, active learning, and nonlinear control.
- E-SINDy uses bootstrap aggregating to identify ensembles of ordinary and partial differential equation models from noisy data.
- Aggregate model statistics generate candidate-function inclusion probabilities that promote interpretable model selection and probabilistic forecasts.
- Combining ensembling with integral SINDy identifies partial differential equation models from more than twice the previously reported measurement noise.
- E-SINDy produces probabilistic models in seconds, compared with several hours for existing Bayesian inference methods.
- Active E-SINDy reduces coefficient variance, increases correct-structure identification, and lowers coefficient error versus standard E-SINDy in the low-data limit.
- E-SINDy reduces training data requirements for nonlinear model predictive control with constraints in the very low-data limit.
A E-SINDy trends compared with standard SINDy
Across Lorenz-system experiments, ensembling improves SINDy’s model-discovery accuracy and robustness under higher noise and shorter data records. Library bagging further supports larger candidate libraries, while ensemble forecasts can track chaotic dynamics longer than standard SINDy.
- The Lorenz sensitivity analysis evaluates coefficient error and correct-structure success rate across noise levels and data lengths using 1000 noise realizations per case.
- Ensembling improves SINDy accuracy and robustness, enabling model identification with less data and more noise than standard SINDy.
- Bragging further robustifies bagging, while library bagging also improves accuracy and robustness.
- Library bagging permits larger libraries by discarding irrelevant terms before applying bagging E-SINDy.
- At 2.5% noise and T = 10s, library-bagging E-SINDy forecasts several chaotic lobe switches, whereas standard SINDy deviates earlier.
- Stability selection provides an alternative that samples data subsets, computes term importance, thresholds library terms, and performs final least-squares regression.