Source-linked AI summary
Data-driven discovery of coordinates and governing equations
Kathleen Champion, Bethany Lusch, J. Nathan Kutz, Steven L. Brunton
TL;DR
Unknown governing equations and unknown effective coordinates limit data-driven discovery of interpretable dynamical models. The paper combines a custom autoencoder with SINDy to learn both simultaneously, demonstrating parsimonious models on several high-dimensional systems while requiring approximately noise-free measurements.
Problem
Sparse model discovery relies on coordinates where dynamics have a simple representation, but effective measurement coordinates are often unknown.
Method
A custom autoencoder and SINDy model are jointly optimized to discover reduced coordinates and associated sparse nonlinear governing equations.
Results
The method discovers interpretable low-dimensional dynamical models and associated coordinates across three high-dimensional example systems.
Takeaways & Limitations
Simultaneous coordinate and model discovery places both choices on equal footing for obtaining sparse, interpretable, and generalizable dynamical models.
Takeaways & Limitations
The approach requires clean, approximately noise-free measurement data because SINDy needs reasonable derivative estimates.
Abstract
from arXiv · showhide
The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam's razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom autoencoder to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional dynamical systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. It is the first method of its kind to place the discovery of coordinates and models on an equal footing.
1 Introduction
Data-driven model discovery addresses systems whose governing equations are unknown, but sparse identification depends on coordinates where dynamics are simple. This work jointly discovers such coordinates and parsimonious governing equations using an autoencoder–SINDy framework.
- Data-rich modern systems often lack governing equations that can be derived from first principles, motivating data-driven model discovery.
- Parsimonious models balance descriptive capability and model efficiency by retaining only the terms needed to capture essential interactions.
- SINDy uses sparse regression to select the fewest candidate terms needed to model nonlinear dynamics while allowing incorporation of partial physical knowledge.
- Model discovery assumes an effective coordinate system in which the dynamics admit a sparse representation, making coordinate discovery a central challenge.
- The proposed method combines a custom autoencoder with SINDy to jointly discover reduced coordinates and nonlinear governing equations on three high-dimensional dynamical systems.The examples include chaotic Lorenz dynamics, a reaction-diffusion spiral wave, and a nonlinear pendulum.
2 Background
SINDy formulates dynamical-system discovery as sparse regression over candidate basis functions, while neural networks offer flexible modeling but face interpretability and generalization challenges. The background motivates combining these capabilities under an appropriate coordinate representation.
- Sparse identification of nonlinear dynamics: SINDy uses time-series snapshots and their derivatives to identify a dynamical system with as few active terms as possible.
- Sparse identification of nonlinear dynamics: The method constructs a library Θ(X) of candidate basis functions and solves an overdetermined regression when the number of snapshots exceeds the library size.Transient dynamics and multiple initial conditions may improve the conditioning of Θ.
- Sparse identification of nonlinear dynamics: Sparsity-promoting regression makes the coefficient matrix Ξ sparse, selecting only a few library terms and enabling low-dimensional state discovery for high-dimensional systems.
- Sparse identification of nonlinear dynamics: SINDy has been applied across fluid, optical, chemical, plasma, and structural dynamical systems, with extensions for PDEs, multiscale, hybrid, and constrained models.
- Neural networks for dynamical systems: Neural networks provide broad tools for learning dynamical systems, including coordinate discovery for Koopman analysis and linearized representations.
- Neural networks for dynamical systems: Neural networks face challenges in generalization, extrapolation, and interpretation, particularly when forecasting requires behavior beyond the training data.
3 SINDy Autoencoders
SINDy autoencoders jointly learn reduced coordinates and sparse nonlinear dynamics by coupling an autoencoder with SINDy in a shared optimization. Reconstruction, derivative prediction, and sparsity terms constrain the learned representation and model.
- Joint coordinate and model discovery: The method seeks reduced coordinates z = ϕ(x) with d ≪ n and an associated sparse dynamical model, rather than modeling dense dynamics directly in measurement coordinates.
- Autoencoder architecture: The encoder maps measurement coordinates to intrinsic coordinates, while the decoder approximately reconstructs the original data from them.
- Autoencoder architecture: An autoencoder alone does not guarantee sparse dynamics, so the network simultaneously learns a SINDy model to regularize the intrinsic coordinates.
- SINDy dynamics: Encoder derivatives are computed as ˙z = ∇xϕ(x) ˙x, allowing the SINDy model to predict the dynamics of the learned coordinates.
- Joint optimization: The training loss combines autoencoder reconstruction, SINDy losses in z and x, and L1 regularization on Ξ to promote parsimonious dynamics.
- Joint optimization: Sequential thresholding further removes small SINDy coefficients during training as a proxy for L0 sparsity.
4 Results
The SINDy autoencoder discovers parsimonious governing dynamics and reduced coordinates from high-dimensional data across Lorenz, reaction-diffusion, and nonlinear-pendulum examples. The models recover sparse or low-dimensional dynamics while reconstructing the observed systems with low unexplained variance.
- Example systems: The method is demonstrated on a high-dimensional Lorenz system, a reaction-diffusion system, and a two-dimensional nonlinear-pendulum video.The examples contain low-dimensional behavior embedded in high-dimensional observations.
- Chaotic Lorenz system: The Lorenz data are generated by combining six fixed spatial modes with Lorenz variables and their cubes, producing a nonlinear combination of the true variables.The spatial modes are Legendre polynomials, and the resulting input has dimension 128.
- Chaotic Lorenz system: The discovered Lorenz model has 7 active terms, a two-lobe attractor, and the original sparsity pattern after a suitable variable transformation.Its coefficients are close to the original parameters up to arbitrary scaling.
- Chaotic Lorenz system: Less than 3 × 10^-5 of input variance is unexplained by decoder reconstruction, while derivative prediction leaves 2×10^-4 and 7×10^-4 unexplained variance.The reported derivative fractions correspond to x and z, respectively.
- Nonlinear pendulum: For nonlinear-pendulum video, the autoencoder discovers a one-dimensional representation and a second-order SINDy model based on first derivatives.Five of ten training instances correctly identify the pendulum equation; the best decoder reconstruction has 8×10^-4 unexplained variance.
5 Discussion
The method jointly discovers low-dimensional coordinates and interpretable dynamical models for high-dimensional systems. Its current scope includes clean measurement data and possible limits to coordinate generalization beyond the training regime.
- 5 Discussion: The method discovers interpretable, low-dimensional dynamical models and their associated coordinates for high-dimensional dynamical systems.It combines a flexible autoencoder with parsimonious nonlinear governing equations.
- 5 Discussion: The simultaneous discovery of coordinates and dynamics addresses the unknown measurement-coordinate problem that limits prior governing-equation discovery approaches.The learned models identify coordinates associated with parsimonious dynamical equations.
- 5 Discussion: The identified models are interpretable and support forecasting through extrapolation for the example systems studied.This result is reported for the three demonstrated example systems.
- 5 Discussion: The approach currently requires clean measurement data that is approximately noise-free for continuous-time SINDy fitting.Derivative estimation from noisy data can be difficult, although denoising or discrete-time alternatives are noted.
- 5 Discussion: The learned coordinate transformation may not generalize to data regimes far from the original training set.When the dynamics are known, the network can be retrained on new data with fixed latent-dynamics terms.
S1 Network Architecture and Training
The network and training procedure jointly optimize reconstruction, dynamical prediction, and coefficient sparsity. Sequential thresholding and regularization produce parsimonious models while model selection balances reconstruction, dynamical fidelity, and active-term count.
- S1 Network Architecture and Training: The autoencoder uses fully connected layers with sigmoid activations except at the encoder and decoder output layers.Rectified and exponential linear units are also reported as viable alternatives.
- S1 Network Architecture and Training: The weighted training loss combines reconstruction, input-derivative prediction, latent-derivative prediction, and SINDy coefficient regularization.These terms respectively support reconstruction, dynamical prediction, and sparsity.
- S1 Network Architecture and Training: Encoder derivatives are computed by propagating derivatives through the network, with second derivatives additionally required for the nonlinear pendulum example.The procedure assumes input derivatives are available or computable.
- S1 Network Architecture and Training: Sequential thresholding removes SINDy coefficients below magnitude 0.1 every 500 epochs, and removed terms cannot return.The number of active terms therefore decreases during training; L1 regularization further promotes parsimony.
- S1 Network Architecture and Training: L1 regularization creates a trade-off between sparse coefficients and accurate dynamical reconstruction, controlled by the relative loss weights.The resulting coefficients are typically slightly smaller than those that best reproduce the dynamics, motivating coefficient refinement.
- S1 Network Architecture and Training: Model assessment balances unexplained variance in inputs and derivatives against the number of active SINDy terms.Ten random initializations are compared, and the selected model has the lowest validation unexplained variance among models with fewest active coefficients.
S2 Example Systems
The method is evaluated on three high-dimensional systems: Lorenz dynamics, reaction-diffusion spiral waves, and synthetic-video nonlinear pendulum dynamics. Across these examples, discovered low-dimensional models reproduce key attractor or test-set behavior, while model structures vary in sparsity and form.
- Chaotic Lorenz system: The Lorenz data are generated from six spatial modes of Lorenz dynamics and evaluated using training, validation, and test trajectories.The data use 2,048 training, 20 validation, and 100 test initial conditions, with trajectories sampled from t = 0 to t = 5.
- Chaotic Lorenz system: Two Lorenz models use 7 and 10 active terms; the 7-term model can be transformed into the original Lorenz form.The 10-term model has lower test error, while the 7-term model is highlighted for having fewer active terms.
- Chaotic Lorenz system: The discovered Lorenz system reproduces true trajectories closely through t = 5 before chaotic divergence, while remaining on a similar attractor.The simulated trajectories remain on an attractor after diverging from the true trajectory.
- Reaction-diffusion system: The reaction-diffusion system produces a spiral wave, and two-term discovered models take either linear or nonlinear oscillatory forms.Both forms achieve similar test errors and predict test-set dynamics by simulating the low-dimensional system.
- Nonlinear pendulum: The pendulum experiment uses high-dimensional two-dimensional synthetic-video snapshots, with five of ten models correctly recovering the nonlinear pendulum equation.A linear-oscillator model has reasonably low acceleration prediction error but qualitatively different simulated dynamics; models with two active terms perform worse than the one-term models.