Source-linked AI summary
Sparse Identification of Nonlinear Dynamics for Rapid Model Recovery
Markus Quade, Markus Abel, J. Nathan Kutz, Steven L. Brunton
TL;DR
Abrupt changes in dynamical systems must be characterized rapidly from limited, noisy data, while many learning methods rely on large datasets and do not readily leverage prior models. The paper detects change through prediction divergence and updates an existing SINDy model with sparse term modifications. Across its examples, abrupt-SINDy requires less data and computation and is more noise-robust than identifying a new model from scratch, though its practical scope and change-detection robustness remain subjects for further study.
Problem
Rapid recovery after abrupt dynamical changes is difficult because available data may be limited and noisy, while many learning methods rely on large datasets and do not readily use prior models.
Method
Abrupt-SINDy detects model change from prediction divergence and applies sparse regression to update a previously identified model through parameter variations, term deletions, or term additions.
Results
Sparse updates require less data, less computation time, and provide greater noise robustness than identifying a new model from scratch.
Takeaways & Limitations
Abrupt-SINDy provides a framework for rapid recovery of parsimonious nonlinear models after abrupt system changes.
Takeaways & Limitations
The method depends on the system dynamics, time scales, and sampling rate, and further work is needed for robust Lyapunov-based detection and real engineering validation.
Abstract
from arXiv · showhide
Big data has become a critically enabling component of emerging mathematical methods aimed at the automated discovery of dynamical systems, where first principles modeling may be intractable. However, in many engineering systems, abrupt changes must be rapidly characterized based on limited, incomplete, and noisy data. Many leading automated learning techniques rely on unrealistically large data sets and it is unclear how to leverage prior knowledge effectively to re-identify a model after an abrupt change. In this work, we propose a conceptual framework to recover parsimonious models of a system in response to abrupt changes in the low-data limit. First, the abrupt change is detected by comparing the estimated Lyapunov time of the data with the model prediction. Next, we apply the sparse identification of nonlinear dynamics (SINDy) regression to update a previously identified model with the fewest changes, either by addition, deletion, or modification of existing model terms. We demonstrate this sparse model recovery on several examples for abrupt system change detection in periodic and chaotic dynamical systems. Our examples show that sparse updates to a previously identified model perform better with less data, have lower runtime complexity, and are less sensitive to noise than identifying an entirely new model. The proposed abrupt-SINDy architecture provides a new paradigm for the rapid and efficient recovery of a system model after abrupt changes.
I. INTRODUCTION
Data-driven dynamical-system discovery must support rapid model recovery after abrupt changes despite limited, noisy data. The paper motivates abrupt-SINDy as a sparse, adaptive alternative that updates existing models rather than rebuilding them from scratch.
- Motivation: Modern nonlinear system-identification methods often require large datasets, generalize poorly beyond observed attractors, and incorporate physical constraints only weakly.These limitations motivate approaches designed for generalizability, limited data, and physical structure.
- Motivation: Abrupt changes require controllers to characterize and compensate for new dynamics rapidly, leaving little time for recovery from limited data.The paper identifies data requirements, generalization, overfitting, and interpretability as central real-time challenges.
- State of the art: Existing regression approaches span linear identification, neural networks, symbolic regression, and sparse methods, but differ in generality, accuracy, data needs, and computational complexity.Linear models cannot capture nonlinear dynamics or structural changes, while neural-network approaches can be unrealistic for real-time applications.
- State of the art: SINDy is suited to abrupt recovery because it uses fast, sparse regression and has extensions for constraints, limited measurements, noisy data, and control inputs.These capabilities support adaptive model recovery while retaining a parsimonious representation.
- Proposed framework: Abrupt-SINDy detects changes from prediction divergence and updates an existing model through sparse parameter variations, term deletions, or term additions.The framework focuses on the fewest modifications needed to recover new dynamics.
II. STATE OF THE ART
SINDy identifies sparse nonlinear dynamical models from candidate-function libraries using regularized regression. The paper describes the library construction, derivative estimation, sparse optimization, normalization, and adaptive extensions relevant to abrupt model changes.
- SINDy formulation: SINDy uses sparse regression over candidate nonlinear functions to identify dynamical models without exhaustive brute-force search.Candidate functions can include polynomials, trigonometric functions, or physics-motivated forms.
- SINDy formulation: Candidate libraries may be poorly conditioned or overcomplete, and an unsuitable choice of functions can prevent correct model identification.Non-orthogonal or overdetermined libraries are specifically identified as failure conditions.
- Data preparation: Time-series measurements are organized into data matrices, and derivatives are estimated using a forward Euler finite-difference scheme.The derivative estimate divides differences between consecutive observations by their time difference.
- Data preparation: Noisy data can make finite-difference derivative estimation numerically ill-conditioned, although noise-robust derivative methods are available.The paper notes that such methods must be used correctly to handle noise effectively.
- Candidate library: The candidate library includes polynomial and trigonometric terms, with columns mean-subtracted and normalized before regression.Polynomial matrices contain all possible time-series of a specified degree, and library terms may encode physical knowledge.
- Sparse optimization: Sparse regression selects active terms by penalizing complexity, using regularization schemes such as LASSO, sequential thresholded least squares, and elastic net.The paper uses sequentially thresholded ridge regression, repeatedly removing coefficients below a threshold until convergence.
III. METHODS
Abrupt-SINDy extends SINDy to recover changed dynamics by identifying sparse additions, deletions, or parameter modifications rather than rebuilding a model from scratch. The approach can combine these changes and uses smaller regression problems for efficiency.
- Sparse model recovery: Sparse model changes can reduce the data needed for recovery because fewer altered terms require fewer observations to distinguish their effects.The framework treats this as parsimony of change and targets rapid recovery after abrupt dynamics changes.
- Types of model changes: Parameter variation is handled by least-squares regression on the known model structure to identify new parameter values.This is computationally fast and can be checked before exploring term additions or deletions.
- Types of model changes: Term deletions are identified by applying SINDy regression to sparse coefficients corresponding to existing model terms.
- Types of model changes: Term additions are identified by finding the sparsest inactive-term combination that explains the model error.Because least-squares regression scales as O(p^3), restricting the regression to relevant inactive terms is less expensive than fitting the entire library.
- Combined changes: Abrupt-SINDy can explore combined changes, including simultaneous term addition and deletion, within its adaptive update procedure.
A. Baseline model
The baseline model is identified once with SINDy, using grid search to select hyperparameters that remain fixed during subsequent updates.
- Baseline model: A grid search tests all hyperparameter combinations and selects the best-performing set for the baseline SINDy model.The resulting sparse coefficients are denoted Ξ0.
B. Detecting model divergence
Model divergence is detected by comparing observed and predicted trajectory separation over time. The method uses prediction horizons and local Lyapunov-time estimates, with settings that depend strongly on the dynamical system and sampling conditions.
- Divergence detection: A predictor-corrector scheme compares model predictions with measurements to detect divergence faster than expected from the system dynamics.The predicted state is generated over a prediction interval, then compared with the measured state.
- Lyapunov-time comparison: For chaotic systems, local divergence is characterized using the largest Lyapunov exponent and its inverse time scale.The exponent is computed from the model dynamics, including tangent-space evolution or local eigenvalues.
- Prediction horizon: The prediction horizon T(t) is the first passage time at which predicted and measured states differ by more than tolerance Δx.
- Divergence criterion: The detection algorithm flags divergence when the model-based and measurement-based time scales disagree, using a fluctuation tolerance larger than typical noise fluctuations.
- Scope: The method depends heavily on the system dynamics, time scales, and sampling rate, and its dynamical-systems formulation does not directly extend to settings such as artificial neural networks.
C. Adaptive model fitting
After detecting divergence, adaptive fitting updates the existing sparse model in stages: first parameters, then deletions, then additions. Restricting regressions to relevant subsets reduces data and computational requirements.
- Adaptive update sequence: New data are first regressed onto the existing sparse structure to identify parameter variations.
- Adaptive update sequence: The next step identifies deleted terms by sparse regression on columns corresponding to nonzero baseline coefficients.This searches only for removals from the existing model.
- Adaptive update sequence: If residual error remains, sparse regression on inactive library columns identifies terms that should be added.
- Stopping and efficiency: The procedure stops when residual error is sufficiently small or iterates until convergence, using restricted regressions that require less data and computation than fitting a new model.The deletion-addition procedure is applied to transient data collected after divergence detection.
IV. RESULTS
The results compare abrupt-SINDy with standard SINDy for recovering dynamical models after abrupt changes, evaluating data requirements, computation, and accuracy.
- The study compares abrupt-SINDy, which updates an existing model, with standard SINDy, which identifies a new model from scratch.
- The evaluation considers model accuracy, computational time, and the amount of post-change data required.
- The grid search selects α, γ, ndegree, and nfold before evaluating the recovery methods.
A. Lorenz system
The Lorenz example changes the parameter ρ twice, producing distinct dynamical regimes that the framework detects and models with sparse updates.
- The Lorenz parameters are ρ = 28, β = 8/3, and σ = 10, with ρ representing atmospheric heating.
- Changing ρ from 28 to 15 at t = 40 and back to 28 at t = 80 produces bifurcations detected quickly.
- The adapted Lorenz parameters are detected accurately up to two digits after the changes.
- Prediction-horizon changes detect the parameter switches, while the recovered model closely matches the true dynamics except during tupdate = 1.
1. Effects of noise and data volume
The Lorenz experiments examine recovery under changing data volume and noise, showing that sparse updates can recover accurate models with less data and greater noise tolerance than refitting.
- Limitations: The reported noise and data-volume trends are specific to this case, and required data also depends on sampling rate.
- Lorenz recovery: The Lorenz results report detection and update times alongside identified equations, with detection coinciding up to the second digit with the true switching time.
- Data volume: The model is evaluated over time and against the amount of data used for updating or refitting after the first system change.
- Data volume: At tupdate ≈ 0.1, abrupt-SINDy identifies more accurate models, whereas standard SINDy becomes data-insufficient at smaller update times.
- Noise: Abrupt-SINDy tolerates approximately an order of magnitude more additive derivative noise than standard SINDy before model identification fails.
B. Van der Pol
The Van der Pol example tests abrupt-SINDy on periodic dynamics with parameter and structural changes, including the addition and removal of quadratic nonlinearities.
- The modified Van der Pol oscillator adds quadratic nonlinearities αx2 and αy2 to test structural changes.
- The oscillator uses µ to control nonlinear damping and α to parameterize the additional quadratic nonlinearity.
- The experiment switches from a periodic orbit to a stable fixed point at t = 100, then removes the additional nonlinearity at t = 200.
- The Lyapunov-time estimate captures the changes, which coincide with peaks in structural model error.
- The first and third stages are identified correctly, while −1.25x substitutes for −x −0.25x2 near the fixed point.
- More transient data obtained through actuation could resolve the correlated-term misidentification, although the recovered model may suffice for control.
V. CONCLUSIONS
The adaptive abrupt-SINDy strategy recovers nonlinear models after abrupt changes using sparse updates to previously identified models. The paper also identifies implementation limitations and directions for improving fallback behavior, hyperparameter adaptation, change detection, and practical validation.
- The abrupt-SINDy method recovers nonlinear models after abrupt changes using fewer data, less computation time, and greater noise robustness than rebuilding models from scratch.
- Fallback models: After detecting a change, the current implementation retains the old model until enough data are available to identify a new one.Dynamic mode decomposition is proposed as a potentially faster fallback requiring even less data.
- Hyperparameterization: The prototype fixes Δx and tupdate, although future versions may learn and adapt these hyperparameters.
- Comprehensive Lyapunov time estimation: Lyapunov-time estimation currently uses a fixed Δx, while broader estimation across Δx values and further study of chaotic systems could improve change detection.The paper notes that numerical Lyapunov-exponent calculations may fail to reveal divergence because of simple time averaging.
- Advanced optimization and objectives: Future work includes advanced optimization objectives and demonstrations on more sophisticated engineering problems, including incorporation into controllers.