Source-linked AI summary
Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
Steven L. Brunton, Joshua L. Proctor, J. Nathan Kutz
TL;DR
Distilling governing models of dynamic processes from abundant data remains difficult. This paper uses sparse regression to identify nonlinear equations from measurements and demonstrates accurate recovery across canonical, chaotic, and fluid systems, including parameterized and forced cases.
Problem
Machine learning has advanced analysis of static data, but extracting physical models of dynamic processes from data remains limited despite its importance in science and engineering.
Method
The method uses sparse regression and compressed sensing to identify the few relevant terms in nonlinear governing equations from measured states and derivatives.
Results
The algorithm recovers governing dynamics across oscillators, the chaotic Lorenz system, and cylinder-wake fluid data, while generalizing to parameterized, time-varying, and externally forced systems.
Takeaways & Limitations
Sparse identification can extract parsimonious nonlinear dynamical models from data, including systems whose underlying fluid dynamics took nearly three decades to uncover.
Takeaways & Limitations
For cylinder-wake transients outside the training regime, the identified sparse model may fail when the slow-manifold approximation is invalid.
Abstract
from arXiv · showhide
The ability to discover physical laws and governing equations from data is one of humankind's greatest intellectual achievements. A quantitative understanding of dynamic constraints and balances in nature has facilitated rapid development of knowledge and enabled advanced technological achievements, including aircraft, combustion engines, satellites, and electrical power. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover governing physical equations from measurement data. The only assumption about the structure of the model is that there are only a few important terms that govern the dynamics, so that the equations are sparse in the space of possible functions; this assumption holds for many physical systems. In particular, we use sparse regression to determine the fewest terms in the dynamic governing equations required to accurately represent the data. The resulting models are parsimonious, balancing model complexity with descriptive ability while avoiding overfitting. We demonstrate the algorithm on a wide range of problems, from simple canonical systems, including linear and nonlinear oscillators and the chaotic Lorenz system, to the fluid vortex shedding behind an obstacle. The fluid example illustrates the ability of this method to discover the underlying dynamics of a system that took experts in the community nearly 30 years to resolve. We also show that this method generalizes to parameterized, time-varying, or externally forced systems.
1 Introduction
The paper addresses the challenge of extracting governing physical laws for dynamic processes from abundant data, where existing data-driven methods often capture statistical relationships without yielding extrapolative physical models. It proposes sparse regression and compressed sensing to discover parsimonious nonlinear dynamical equations from the assumption that physical dynamics contain only a few relevant terms.
- Motivation: Extracting physical laws from data is a central challenge across science and engineering, including cognition, climate, financial stability, disease spread, and turbulence control.The motivation is strengthened by the availability of abundant data and the difficulty of identifying underlying laws.
- Motivation: Machine learning has advanced analysis of complex static data, but progress remains slow in distilling physical models of dynamic processes from big data.This limitation reduces the ability of data-driven models to represent dynamics beyond the attractor where they were sampled.
- Relation to prior work: Symbolic regression previously demonstrated that nonlinear differential equations could be identified by balancing model complexity, measured in number of terms, against accuracy.The present work introduces a different perspective based on sparsity-promoting methods and compressed sensing.
- Contribution: The paper reframes dynamical-system discovery as sparse regression and compressed sensing in a high-dimensional nonlinear function space.The approach exploits the assumption that most physical systems have only a few relevant terms defining their dynamics.
- Relation to prior work: The resulting objective is to identify governing equations whose dynamics are sparse, rather than relying only on measurement-based, equation-free approaches such as dynamic mode decomposition.The paper positions its method relative to DMD, extended DMD, and Koopman-operator connections for nonlinear systems.
2 Background
This section situates the work within system identification, symbolic regression, and sparse representation. It motivates using sparsity to select important model terms while controlling complexity and avoiding overfitting.
- System identification: System identification has produced powerful data-driven dynamics models, widely applied across engineering and applied mathematics, especially for model-based feedback control.Its history includes modeling complex flexible structures such as the Hubble Space Telescope and International Space Station.
- Approach: The work builds on symbolic regression and sparse representation to identify nonlinear functions relating measured variables to their time derivatives.Sparse representation is used instead of traditional parsimony approaches such as the Pareto front to determine relevant model terms.
- Symbolic regression: Symbolic regression treats function discovery as machine learning, typically using genetic programming to evolve candidate functions from simple building blocks.Candidate functions are modified across generations until a predetermined accuracy is achieved.
- Symbolic regression: Symbolic regression has discovered ordinary differential equations from measurements, but overfitting requires a parsimony constraint such as accepting equations on the Pareto front.This limitation is associated with symbolic regression and genetic programming.
- Sparse regression: When only a few regression terms matter, sparse feature selection represents measurements as combinations of a feature-library matrix Θ ∈ R^m×p with coefficient vector ξ ∈ R^p.Standard regression generally produces nonzero contributions across coefficient entries, motivating sparse formulations.
- Sparse regression: The LASSO adds an L1 regularization term to encourage most entries of ξ to be zero, while λ weights the sparsity constraint.The formulation is closely related to compressive sensing, whose convex architectures scale well.
3 Sparse identification of nonlinear dynamics (SINDy)
SINDy discovers governing equations by expressing dynamics in a library of candidate nonlinear functions and using sparse regression to retain only active terms. The framework handles noisy measurements and extends to time-varying, parameterized, forced, discrete-time, and high-dimensional systems.
- Sparse identification: The algorithm samples states and derivatives, constructs data matrices, and builds a library of constant, polynomial, and trigonometric candidate functions.The library may include higher-order polynomial terms such as quadratic nonlinearities.
- Sparse identification: SINDy assumes governing dynamics contain only a few important terms, enabling sparse regression to identify active right-hand-side functions.The method seeks parsimonious equations that balance model complexity and descriptive ability.
- Sparse identification: Each equation is represented by a sparse coefficient vector, with sparse regression determining which library terms remain active.Separate optimization problems determine the sparse coefficients for each row equation.
- Sparse regression: The sequential thresholded least-squares algorithm repeatedly removes coefficients below cutoff λ and refits the remaining terms.Cross-validation helps select λ while balancing model accuracy against the number of nonzero coefficients.
- Sparse regression: The approach remains robust to noise, including cases where derivatives must be approximated from noisy state data.Derivative denoising can use total variation regularization before sparse regression.
- Extensions: Extensions generalize SINDy to discrete-time, high-dimensional, parameterized, time-varying, externally forced, and controlled systems.These extensions address large discretized PDE states and systems influenced by external forcing or feedback control.
4 Results
The methods are demonstrated across canonical dynamical systems, noisy chaotic measurements, fluid wakes, nonlinear PDEs, high-dimensional data, and bifurcation-parameter models.
- Canonical systems: The demonstrations span linear and nonlinear damped oscillators, noisy measurements of the fully chaotic Lorenz system, and the unsteady wake behind a circular cylinder.These examples cover progressively more complex dynamical measurements.
- Extensions: The fluid-wake example extends the method to nonlinear partial differential equations and high-dimensional data.The wake measurements are used to illustrate this extension.
- Parameterized models: Including bifurcation parameters in the sparse models recovers the correct normal forms.The paper presents this as a final demonstration of the method.
4.1 Example 1: Simple illustrative systems
Sparse identification recovers the correct linear and cubic dynamics of a damped harmonic oscillator and accurately reproduces its phase portraits. It also identifies a three-variable linear system, while higher-order libraries can introduce degeneracy that motivates hierarchical identification.
- 4.1 Example 1: Simple illustrative systems: The algorithm correctly identifies both linear and cubic dynamics in a two-dimensional damped harmonic oscillator.The nonlinear library includes polynomials in x up to fifth order.
- 4.1 Example 1: Simple illustrative systems: The sparse identified system accurately reproduces the oscillator phase portraits.
- 4.1 Example 1: Simple illustrative systems: For a three-variable linear system, sparse identification correctly identifies the dynamics using polynomials up to second or third order.The resulting sparse model is reported in Table 3.
- 4.1 Example 1: Simple illustrative systems: Including fourth- or fifth-order polynomial terms introduces degeneracy because combinations of e^λt powers can approximate other exponential rates.This degeneracy motivates a hierarchical identification approach.
4.2 Example 2: Lorenz system (Nonlinear ODE)
The study applies sparse regression to identify the Lorenz system’s governing terms from chaotic trajectory data using polynomial candidate functions up to fifth order. The method accurately reproduces attractor dynamics and identifies the correct linear and quadratic terms, including under noisy derivative measurements.
- Data and setup: The Lorenz system is reconstructed from state snapshots X and derivative snapshots ˙X collected from trajectory data.The standard parameters are σ = 10, β = 8/3, and ρ = 28; data are collected over t = 0 to t = 100 with ∆t = 0.001.
- Method: Sparse regression selects active terms from polynomial candidate functions in (x, y, z) up to fifth order.Each candidate function represents a possible right-hand-side term, and convex ℓ1-minimization determines the sparse model.
- Noise robustness: Derivative noise is tested at η = 0.01 and η = 10 over short-time t = 0 to t = 20 and long-time t = 0 to t = 250 reconstructions.The study also evaluates ℓ2 error over time for increasing noise η; the error increases at large noise values.
- Results: The sparse model accurately reproduces attractor dynamics from chaotic trajectory measurements, where attractor capture matters more than individual-trajectory prediction.Chaotic sensitivity causes small initial-condition or coefficient differences to diverge exponentially over time.
- Results: The algorithm identifies the correct linear and quadratic terms in the Lorenz dynamics.This result demonstrates that sparse identification can recover the governing structure despite the system’s rich chaotic behavior.
4.3 Example 3: Fluid wake behind a cylinder (Nonlinear PDE)
The study applies sparse identification to cylinder-wake data from two-dimensional Navier–Stokes simulations at Re = 100, recovering low-dimensional dynamics with quadratic nonlinearities. The method captures both slow-manifold and transient behavior, while revealing limitations when off-attractor conditions are absent or poorly represented in training data.
- Data and identification: At Re = 100, sparse identification recovers the nonlinear relationship between dominant cylinder-wake coherent structures directly from flow-field measurements.The data come from direct numerical simulations of the two-dimensional Navier–Stokes equations, without supplying the governing equations to the identification procedure.
- Results: The algorithm identifies the on-attractor and off-attractor dynamics using only quadratic nonlinearities and preserves the correct slow-manifold dynamics.This is notable because the underlying form of the cylinder-wake dynamics took nearly three decades to uncover.
- Results: When off-attractor trajectories are excluded from training, the identified model incorrectly represents the transient dynamics.Including off-attractor data enables qualitative capture of trajectories that rapidly approach the slow manifold.
- Model reduction: The first two POD modes capture substantial energy, while an additional shift mode represents the mean-flow correction and produces a three-dimensional reduced model.The steady vortex shedding appears as a limit cycle in the first two POD coordinates, and the shift mode supports the slow-manifold description.
- Limitations: For an unseen initial condition with twice the fluctuation energy in the x and y directions, the identified model does not exactly reproduce the transients.Relaxing sparsity can yield near-perfect agreement, but the resulting model includes higher-order nonlinearities.
4.4 Example 4: Bifurcations and Normal Forms
The method identifies parameterized normal forms by incorporating the bifurcation parameter into the dynamics and using sparse combinations of functions of the state and parameter. It correctly recovers both logistic-map dynamics across ten parameter values and the Hopf normal form, although noise affects estimated cubic coefficients.
- 4.4 Example 4: Bifurcations and Normal Forms: Bifurcation parameters can be suspended as dynamical variables to identify associated normal forms.This enables the parameterized right-hand side to be represented within the sparse dynamics framework.
- 4.4 Example 4: Bifurcations and Normal Forms: The parameterized right-hand side is identified as a sparse combination of functions of the state components and bifurcation parameter.The approach is illustrated using the one-dimensional logistic map and two-dimensional Hopf normal form.
- 4.4.1 Logistic map: The logistic map exhibits a cascade of bifurcations that leads to chaotic trajectories under stochastic forcing and parameter variation.The system includes stochastic forcing η_k and bifurcation parameter µ.
- 4.4.1 Logistic map: Sampling at ten parameter values, the algorithm correctly identifies the logistic map’s underlying parameterized dynamics.The identified dynamics are compared with the stochastically forced system in Fig. 11 and Table 6.
- 4.4.2 Hopf normal form: The Hopf normal form is identified from noisy data collected across various values of µ.Training data comprise trajectories inside and outside the fixed point or limit cycle, with total variation derivatives used for de-noising.
- 4.4.2 Hopf normal form: The identified Hopf model reconstructs the noise-free dynamics, while its cubic-term values are off by almost 8% with noisy training data.The model terms remain correctly identified; more training data or lower noise improves agreement.
5 Discussion
The discussion presents sparse regression as a scalable, assumption-light method for identifying nonlinear dynamical systems across chaotic, high-dimensional, and parameterized examples. It also emphasizes unresolved effects of symmetries and conserved quantities, alongside the broader value of parsimonious data-driven models.
- Contributions: The method identifies nonlinear dynamical systems from data without assuming the governing equations’ form and scales to high-dimensional systems through sparse regression.Demonstrations include chaotic systems, high-dimensional data with low-rank coherence, and parameterized dynamics.
- Open problems: Dynamical symmetries and conserved quantities may change the form of identified dynamics, leaving important dynamical-systems questions open.The discussion also highlights degenerate identification of linear systems within high-order polynomial nonlinearities as a possible connection to nearidentity transformations and dynamic similarity.
- Broader implications: Parsimonious data-discovery algorithms can help leverage existing data, guide new data collection, and reduce overfitting in inherently dynamic scientific and engineering problems.The discussion characterizes such algorithms as rare and desirable rather than universally applicable.
Appendix
The appendix tabulates sparse-representation results for canonical dynamical systems, including oscillators, linear and chaotic systems, the logistic map, Hopf normal form, and cylinder-wake modes.
- Tables 1 and 2 cover damped harmonic oscillators with linear terms and cubic nonlinearity, respectively.
- Table 3 presents a three-dimensional linear system identified using sparse representation.
- Table 4 reports sparse identification of the Lorenz system using η = 1.0.
- Table 5 examines cylinder-wake mode dynamics and identifies quadratic terms using sparse representation.
- Tables 6 and 7 address sparse identification of the logistic map and Hopf normal form, with u representing the bifurcation parameter µ.