Source-linked AI summary
Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems
Enoch Yeung, Soumya Kundu, Nathan Hodas
TL;DR
Extended dynamic mode decomposition can require computationally demanding, manually curated dictionaries for nonlinear systems. The paper trains Koopman operators with deep learning to automate dictionary generation, reporting improved multi-step prediction performance and applying the approach to complex and partially observed systems. Its demonstrated scope includes cases where better basis choices may still be possible for the IEEE 39 bus benchmark.
Problem
Extended dynamic mode decomposition is limited by computational complexity and dictionaries that depend on scientist knowledge or intuition.
Method
The paper uses deep learning to train Koopman operators from data while automatically generating and updating dictionaries.
Results
Deep Koopman models improve the performance of extended dynamic mode decomposition and multi-step prediction tasks.
Takeaways & Limitations
Deep learning provides an automated dictionary approach for Koopman-operator calculation across nonlinear dynamical-system applications.
Takeaways & Limitations
For the IEEE 39 bus benchmark system, the authors note that a better choice of bases may be possible, requiring different functions for different systems.
Abstract
from arXiv · showhide
The Koopman operator has recently garnered much attention for its value in dynamical systems analysis and data-driven model discovery. However, its application has been hindered by the computational complexity of extended dynamic mode decomposition; this requires a combinatorially large basis set to adequately describe many nonlinear systems of interest, e.g. cyber-physical infrastructure systems, biological networks, social systems, and fluid dynamics. Often the dictionaries generated for these problems are manually curated, requiring domain-specific knowledge and painstaking tuning. In this paper we introduce a deep learning framework for learning Koopman operators of nonlinear dynamical systems. We show that this novel method automatically selects efficient deep dictionaries, outperforming state-of-the-art methods. We benchmark this method on partially observed nonlinear systems, including the glycolytic oscillator and show it is able to predict quantitatively 100 steps into the future, using only a single timepoint, and qualitative oscillatory behavior 400 steps into the future.
1 Introduction
The Koopman operator enables linear analysis of nonlinear dynamics, but extended dynamic mode decomposition depends on manually selected dictionaries and can become computationally demanding. The paper introduces deep learning to automate dictionary generation and improve long-term Koopman-operator forecasting.
- Motivation: The Koopman operator represents nonlinear dynamical-system evolution through an infinite-dimensional linear operator.Its spectrum supports analysis of asymptotic system behavior.
- Motivation: Extended dynamic mode decomposition lifts observables into a higher-dimensional nonlinear manifold where trajectories evolve linearly.The nonlinear transformations used for lifting are called a dictionary.
- Limitations of existing methods: Dictionary functions are often chosen manually or from a priori physical knowledge, making Koopman learning dependent on scientist expertise and intuition.Insufficient dictionaries require engineering additional candidate functions and reviewing the results.
- Proposed approach: The paper introduces a deep learning approach that trains Koopman operators from data and uses neural networks to generate dictionaries while traversing function space.The framework is presented as applicable to larger problems than existing dynamic mode decomposition methods.
- Reported contribution: Deep Koopman models improve state-of-the-art multi-step prediction tasks by learning higher-fidelity Koopman operator models.The paper emphasizes long-term forecasting rather than relying only on single-step prediction accuracy.
- Implementation: The framework is implemented in TensorFlow, supporting varied activation functions, network architectures, and deep training algorithms.The listed options include RELUs, cRELUs, ELUs, AdaGrad, ADAM, and hierarchical dropout.
2 Deep Dynamic Mode Decomposition for Koopman Operator Learning
Extended dynamic mode decomposition lifts observables with a scientist-selected dictionary, but dictionary size, invariance, overfitting, and operator parameterization make nonlinear Koopman learning difficult. Deep Koopman learning instead represents the dictionary with a neural network and learns it jointly with the Koopman operator.
- Extended Dynamic Mode Decomposition: Extended dynamic mode decomposition lifts observables into a higher-dimensional space using a manually selected dictionary, then estimates a finite Koopman operator with dynamic mode decomposition.The dictionary may contain linear, polynomial, trigonometric, and scalar functions; unsuccessful models prompt further user-designed functions.
- Extended Dynamic Mode Decomposition: The unknown dictionary makes it difficult to determine which functions, how many functions, and what minimal set can ensure K-invariance.The observed data are finite trajectories, so selected functions must satisfy invariance relative to the available samples.
- Extended Dynamic Mode Decomposition: Expansive polynomial dictionaries scale poorly and can overfit as nonlinear interaction terms increase the dictionary matrix.Each of n state components is lifted to m polynomials, producing m^n distinct dictionary functions; cross terms further enlarge the matrix.
- Extended Dynamic Mode Decomposition: Even systems with 10-20 outputs can yield Koopman operators with thousands of potential entries, making regularization critical for minimizing overfit.The operator is generally treated as fully parameterized and gradually regularized during optimization.
- Deep Dynamic Mode Decomposition: Deep dynamic mode decomposition defines the dictionary as a deep neural network output and learns the dictionary and Koopman operator simultaneously.Prediction error supplies the loss used to refine the dictionary, while upstream architectures may include feedforward, convolutional, or recurrent networks.
- Deep Dynamic Mode Decomposition: The approach can use network memory or switching behavior to represent time-varying dictionaries or multiple invariant Koopman subspaces.Recurrent networks can make dictionary properties depend on a dynamic memory state, while hybrid networks may model multiple invariant subspaces.
3 Experimental Results
Experiments compare learned deep Koopman operators with E-DMD across partially observed systems, glycolysis, and a power-grid benchmark. The deep approach achieves low prediction error, supports multi-step forecasting, and adaptively learns nonlinear basis functions, while classical methods face scaling difficulties.
- Partially observed systems: 1% average one-step training and test error was achieved by the deep Koopman operator after 10,000 iterations, versus over 10% for E-DMD.Further iterations reduced the deep operator’s error to 0.1%.
- Computational scaling: E-DMD becomes difficult to execute on larger systems because quadratic and cubic terms rapidly expand the Koopman operator and exhaust solver memory.The reported scaling issue appears for moderate-sized networks with more than 10 variables.
- Emergence of Koopman basis functions: The learned basis functions evolved from flat constants into spikes, dips, or ramps as training progressed, with major profile changes occurring during the first 10,000 iterations.The dictionary is updated during training while the Koopman operator is optimized.
- Glycolysis pathway: Less than 0.1% error was achieved on all glycolysis test cases with 7, 5, or 3 measured nodes.The glycolysis experiment evaluates partially observed measurements with different numbers of observed states.
- Glycolysis pathway: Approximately 100 future timepoints were predicted quantitatively and the full 500-timepoint simulation qualitatively from a single root timepoint.The result addresses multi-step prediction despite compounded operator-estimation error.
4 Conclusion
The paper presents deep learning as an automated approach to calculating Koopman operators, reporting strong multistep prediction performance and a complementary alternative to extended dynamic mode decomposition for complex nonlinear systems.
- The paper presents a deep learning approach to calculating the Koopman operator.
- An automated dictionary approach enabled by deep learning improves Koopman-operator performance for longer-term forecasting.
- The method is compared with extended dynamic mode decomposition and performs relatively well on multistep prediction tasks in many cases.
- The deep Koopman operator is presented as a complementary and promising alternative to extended dynamic mode decomposition for data-driven modeling of complex nonlinear systems.