Source-linked AI summary
Deep learning for universal linear embeddings of nonlinear dynamics
Bethany Lusch, J. Nathan Kutz, Steven L. Brunton
TL;DR
Strongly nonlinear dynamics lack easily identifiable coordinate transformations that make them linear, although Koopman eigenfunctions provide such intrinsic coordinates. The paper uses constrained deep auto-encoders and an auxiliary frequency network to discover parsimonious Koopman representations, including continuous-spectrum systems. The resulting models globally linearize demonstrated nonlinear dynamics and represent continuous frequencies compactly at intrinsic rank.
Problem
Identifying and representing Koopman eigenfunctions is mathematically and computationally challenging, despite their role as intrinsic coordinates for globally linearizing nonlinear dynamics.
Method
A modified auto-encoder identifies reconstructive intrinsic coordinates with linear dynamics, while an auxiliary network parameterizes continuous Koopman frequencies.
Results
The approach identifies nonlinear coordinates yielding globally linear dynamics and correct Koopman eigenvalues, while accurately reconstructing and predicting nonlinear pendulum trajectories.
Takeaways & Limitations
Deep learning can produce compact, interpretable Koopman embeddings for strongly nonlinear systems, including systems with continuous spectra.
Takeaways & Limitations
Continuous-spectrum systems may require infinite Fourier sums for finite approximations, and Koopman eigenfunctions may not exist in some cases.
Abstract
from arXiv · showhide
Identifying coordinate transformations that make strongly nonlinear dynamics approximately linear is a central challenge in modern dynamical systems. These transformations have the potential to enable prediction, estimation, and control of nonlinear systems using standard linear theory. The Koopman operator has emerged as a leading data-driven embedding, as eigenfunctions of this operator provide intrinsic coordinates that globally linearize the dynamics. However, identifying and representing these eigenfunctions has proven to be mathematically and computationally challenging. This work leverages the power of deep learning to discover representations of Koopman eigenfunctions from trajectory data of dynamical systems. Our network is parsimonious and interpretable by construction, embedding the dynamics on a low-dimensional manifold that is of the intrinsic rank of the dynamics and parameterized by the Koopman eigenfunctions. In particular, we identify nonlinear coordinates on which the dynamics are globally linear using a modified auto-encoder. We also generalize Koopman representations to include a ubiquitous class of systems that exhibit continuous spectra, ranging from the simple pendulum to nonlinear optics and broadband turbulence. Our framework parametrizes the continuous frequency using an auxiliary network, enabling a compact and efficient embedding at the intrinsic rank, while connecting our models to half a century of asymptotics. In this way, we benefit from the power and generality of deep learning, while retaining the physical interpretability of Koopman embeddings.
1 Introduction
Nonlinear dynamics exhibit diverse behaviors, but no general mathematical framework solves them, motivating linear representations that retain linear theory’s analysis and control tools. The paper uses deep learning to discover parsimonious, interpretable Koopman-eigenfunction representations, including systems with continuous spectra.
- Nonlinear systems span physical, biological, and engineering applications but lack a general mathematical solution framework.
- Linear representations are appealing because powerful analysis and control techniques for linear systems do not readily generalize to nonlinear systems.
- Deep learning can fit functions without hand-designed features or user-selected bases, but identifying parsimonious, interpretable, transferable models remains challenging.
- The paper develops deep neural-network representations of Koopman eigenfunctions that match intrinsic low-rank dynamics while avoiding overfitting and preserving interpretability.
- For continuous spectra, an auxiliary network parameterizes linear dynamics on intrinsic coordinates, avoiding infinite asymptotic expansions in harmonic eigenfunctions.
2 Data-driven dynamical systems
Data-driven Koopman analysis seeks finite-dimensional linear representations of nonlinear dynamics from measurements, but obtaining compact representations is difficult, especially for continuous spectra. The paper motivates deep-learning architectures that identify interpretable eigenfunction coordinates while explicitly parameterizing continuously varying frequencies.
- Nonlinear dynamics are difficult to solve generally, motivating linear representations that retain linear systems’ analysis and control techniques.
- The Koopman operator advances measurement functions linearly, and its eigenfunctions provide invariant subspaces on which the operator becomes finite-dimensional.
- Finite-dimensional Koopman approximations remain challenging: DMD uses linear measurements, while nonlinear measurement models can suffer closure issues, high dimensionality, and poor interpretability.
- Deep-learning architecture: The core architecture combines an auto-encoder for intrinsic coordinates and reconstruction with a linear Koopman model that advances those coordinates through time.
- Koopman for systems with continuous spectra: Away from the linear regime, continuous spectra may require an infinite Fourier sum, and eigenfunctions may not exist in some cases.
- Koopman for systems with continuous spectra: The proposed approach uses an auxiliary network to model the parametric dependence K(λ), allowing nonlinear oscillators with continuous spectra to use a conjugate eigenfunction pair parameterized by frequency.
3 Deep learning to identify Koopman eigenfunctions
The paper uses a parsimonious deep-learning architecture to discover Koopman eigenfunctions as interpretable intrinsic coordinates with linear dynamics. An auxiliary network extends the representation to systems with continuous spectra by allowing eigenvalues to vary across phase space.
- Architecture: The network identifies low-dimensional intrinsic coordinates y = ϕ(x) and linear dynamics y_k+1 = Ky_k through Koopman eigenfunctions.The architecture is designed to match intrinsic rank while retaining interpretability.
- Architecture: An auto-encoder enforces reconstruction by encoding x into y = ϕ(x) and decoding it through ϕ^-1.The latent dimension p is a hyperparameter, and reconstruction uses the error ∥x − ϕ^-1(ϕ(x))∥.
- Training objectives: Linearity and future-state prediction are enforced by requiring Kϕ(x_k) to match ϕ(x_k+1) and ϕ^-1(Kϕ(x_k)) to match x_k+1.The losses generalize to prediction over m time steps using K^m.
- Continuous spectra: For continuous spectra, an auxiliary network learns λ = Λ(y), allowing eigenvalues and the block-diagonal K(µ, ω) to vary across phase space.This design facilitates compact eigenfunction representations despite continuous frequency variation.
- Training procedure: Training uses trajectory data divided into training, validation, and test sets, with validation-based early stopping to limit overfitting.Test-set accuracy is reported after model selection.
4 Results
The approach identifies globally linear Koopman coordinates for a discrete-spectrum nonlinear system and compact coordinates for continuous-spectrum systems. Demonstrations include the nonlinear pendulum and cylinder-flow model, where phase-space-dependent frequency or damping supports prediction and intrinsic representations.
- Simple model with discrete spectrum: For the simple discrete-spectrum system, the Koopman embedding flattens the inertial manifold and identifies the correct Koopman eigenvalues.The example uses stable eigenvalues µ = −0.05 and λ = −1.
- Nonlinear pendulum with continuous spectrum: For the nonlinear pendulum, a parameterized network represents the continuous frequency shift with two conjugate eigenfunctions as energy increases.The representation remains compact, whereas harmonic approximations would be overly bulky and less interpretable.
- Nonlinear pendulum with continuous spectrum: The pendulum network reconstructs trajectories, predicts their evolution, and produces nearly concentric circles showing linear dynamics in eigenfunction coordinates.Prediction is stopped when relative error reaches 10%, and the prediction horizon deteriorates as initial energy increases.
- High-dimensional nonlinear fluid flow: The cylinder-flow example uses a Reynolds-number-100 vortex-shedding model whose high-dimensional dynamics evolve on a low-dimensional attractor.The Koopman network is trained on trajectories from this model and produces eigenfunctions shown in Fig. 5.
- High-dimensional nonlinear fluid flow: For cylinder flow, an auxiliary network allows damping µ(R) and frequency ω(R) to vary with the radius in eigenfunction coordinates.The reported demonstration focuses on future-state prediction, while the network also functions as an auto-encoder.
5 Discussion
The paper uses deep learning to discover interpretable Koopman eigenfunctions that transform strongly nonlinear dynamics into globally linear coordinates. Its continuous-spectrum extension uses an auxiliary network to retain compactness while representing state-dependent frequencies.
- Deep auto-encoders identify Koopman eigenfunctions as coordinate transformations that recast strongly nonlinear dynamics into a globally linear framework.
- Additional constraints and loss functions enforce linear evolution in the learned eigenfunction coordinates.
- An auxiliary network parametrizes continuous frequencies, avoiding unwieldy higher-harmonic expansions while preserving parsimonious and interpretable Koopman models.
- The discussion emphasizes that interpretable machine-learning models can promote physical insight, including connections between conserved eigenfunctions and generalized Noether-type conservation laws.
Model problems and training datasets
The study evaluates Koopman embeddings on three systems: a discrete-spectrum benchmark, a nonlinear pendulum with continuous spectrum, and cylinder-flow dynamics with a slow manifold. Trajectories are generated from broad randomized initial-condition domains and divided into standard datasets.
- Model problems: Three examples comprise a discrete-spectrum fixed-point system, a nonlinear pendulum, and cylinder flow at Reynolds number 100.
- Model problems: The frictionless pendulum is conservative, with trajectories evolving on Hamiltonian energy level sets as its continuous spectrum emerges with increasing energy.
- Model problems: The cylinder-flow model has a slow manifold and includes trajectories initialized both on and off that manifold.
- Training datasets: Each system uses 5000 initial conditions for testing, 5000 for validation, and 5000–20000 for training.
- Training datasets: The datasets sample phase-space regions chosen to capture benchmark, near-full-phase-space pendulum, slow-manifold, and off-manifold dynamics.
Network architecture and training
The model combines encoder–decoder networks with an auxiliary eigenvalue network and losses for reconstruction, prediction, and linearity. Training uses regularization, Adam optimization, validation selection, and early stopping.
- Network architecture: The network uses ReLU hidden layers and linear output layers in the encoder, decoder, and auxiliary networks.
- Network architecture: The auxiliary network maps embedded coordinates to the real and imaginary parameters of Koopman eigenvalues.
- Network architecture: Separate auxiliary networks prevent coordinates associated with one eigenvalue component from influencing unrelated components.
- Training: The loss combines reconstruction, future-state prediction, and dynamical-linearity mean-squared errors with an L∞ penalty and ℓ2 weight regularization.
- Training: Eigenvalues may vary along trajectories through K(λ), while Hamiltonian structure would permit the specialized constraint K_m = K(λ)^m.
- Training: Models are trained with Adam, random-search initialization, validation-based model selection, and early stopping.
Results
Across the benchmark, pendulum, and fluid-flow examples, the learned Koopman coordinates support accurate linear prediction and recover physically meaningful eigenfunction and eigenvalue structure. The continuous-spectrum model captures smoothly varying frequencies or damping rates across the relevant state spaces.
- Discrete spectrum: The benchmark produces small prediction errors over many steps, while its learned eigenfunctions match analytically expected linear and quadratic forms.
- Discrete spectrum: The benchmark’s learned eigenvalues remain near the true values −0.05 and −1 when allowed to vary with embedded coordinates.
- Nonlinear pendulum: For the pendulum, frequency varies smoothly from around −0.95 to −0.4 with radius, while damping remains nearly µ = 0.
- Nonlinear pendulum: Pendulum eigenfunction magnitude traces Hamiltonian energy level sets, linking the learned representation to prior theoretical derivations.
- Fluid flow: For cylinder flow on the attracting manifold, the model identifies one conjugate eigenfunction pair with frequency near −1 and damping that switches stability outside the limit cycle.
- Fluid flow: For trajectories starting off the slow manifold, the Koopman model faithfully predicts evolution toward the limit cycle, with nearly constant frequency −1 and radius-dependent damping.
Miscellaneous notes
The passages connect extended dynamic-mode methods with variational approaches for assessing Koopman models, while documenting eigenfunction visualizations and parameter variations for fluid-flow trajectories.
- Related methods: eDMD is equivalent to the variational approach of conformation dynamics, which was developed for molecular dynamics with broadly separated timescales.The passage attributes the original derivation to Noé and Nüske in 2013.
- Model assessment: VAC provides a variational score for objectively assessing Koopman models through cross-validation.
- Fluid-flow eigenfunctions: For fluid-flow trajectories starting off the attractor, Figure 15 shows eigenfunctions associated with a complex-conjugate eigenvalue pair, including their magnitude and phase.
- Parameter variations: Figure 16 depicts parameter variations of the complex eigenvalues in terms of y1 and y2, extending an earlier figure limited to data on the bowl.
- Applications: Recent work has shown that eDMD can improve model predictive control performance in nonlinear systems.