Source-linked AI summary
Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
Maziar Raissi, Paris Perdikaris, George Em Karniadakis
TL;DR
The paper addresses the open problem of automatically turning observations into predictive mathematical models of nonlinear physical systems. It combines multistep numerical time-stepping schemes with deep neural networks to identify dynamics from time-series data, then demonstrates accurate learning, forecasting, and basin identification across nonlinear benchmarks. The authors also show that chaotic forecasting has a finite practical horizon despite preservation of the Lorenz attractor’s structure.
Problem
Automated construction of meaningful predictive models from observations remains an open problem, especially for complex nonlinear dynamics and when appropriate basis functions are difficult to choose.
Method
The approach embeds a deep neural-network approximation of the unknown dynamics within classical multistep time-stepping schemes applied to temporal state data.
Results
The method accurately learns and reproduces nonlinear dynamics across oscillator, Lorenz, fluid-flow, and Hopf-bifurcation examples, supporting forecasting and basin identification.
Takeaways & Limitations
Multistep neural networks provide a data-driven route to forecasting and analyzing nonlinear dynamical systems, including high-dimensional and parameterized cases.
Takeaways & Limitations
For the chaotic Lorenz system, small modeling discrepancies produce large forecast errors after t > 4 even though the attractor structure remains well captured.
Abstract
from arXiv · showhide
The process of transforming observed data into predictive mathematical models of the physical world has always been paramount in science and engineering. Although data is currently being collected at an ever-increasing pace, devising meaningful models out of such observations in an automated fashion still remains an open problem. In this work, we put forth a machine learning approach for identifying nonlinear dynamical systems from data. Specifically, we blend classical tools from numerical analysis, namely the multi-step time-stepping schemes, with powerful nonlinear function approximators, namely deep neural networks, to distill the mechanisms that govern the evolution of a given data-set. We test the effectiveness of our approach for several benchmark problems involving the identification of complex, nonlinear and chaotic dynamics, and we demonstrate how this allows us to accurately learn the dynamics, forecast future states, and identify basins of attraction. In particular, we study the Lorenz system, the fluid flow behind a cylinder, the Hopf bifurcation, and the Glycoltic oscillator model as an example of complicated nonlinear dynamics typical of biological systems.
1. Introduction
The paper frames nonlinear systems identification as a way to learn predictive dynamical models from observations when first-principles modeling is incomplete. It introduces a multistep time-stepping and deep-neural-network approach that broadens the searched function space while supporting forecasting and basin identification.
- Complexity can prevent complete understanding of a system and make first-principles mathematical models unavailable or impractical.
- Systems identification seeks mathematical models that predict future states from previously observed or latent system states.
- Existing nonlinear identification methods learn mappings from present and past data, while symbolic and sparse regression aim to produce more interpretable dynamics.
- Sparse representations often require the nontrivial selection of appropriate basis functions, motivating broader function search spaces.
- The proposed approach combines multistep numerical time-stepping with deep neural networks to discover temporal dependencies without direct access to temporal gradients.
- The method is evaluated on damped oscillatory, chaotic, high-dimensional fluid, parameterized bifurcation, and biological oscillator dynamics.
2. Problem setup and solution methodology
The methodology identifies a nonlinear vector field from noisy state measurements by embedding a neural-network approximation within a multistep discretization of the governing differential equation. The resulting model is trained with mean squared error and can incorporate parameters, time dependence, and forcing.
- The target system is a nonlinear differential equation whose state vector evolves according to an unknown function f to be recovered from noisy observations.
- A general M-step linear multistep method discretizes the differential equation using weighted past states and evaluations of f.
- A neural network serves as the prior or approximation for f, with its parameters learned by minimizing mean squared error.
- The framework can extend the vector field to include parameterization, explicit time dependence, external forcing, and feedback control.
- The multistep formulation produces the training relation used to obtain the learned model from temporal data.
3. Results
The proposed multistep neural-network workflow identifies nonlinear dynamics across oscillator, chaotic, fluid, and parameterized systems. Results show accurate phase portraits and dynamics, while robustness depends on time-step, noise, and network architecture choices.
- Two-dimensional damped oscillator: Adams-Moulton schemes consistently achieve more accurate oscillator trajectories than Adams-Bashforth and BDF methods.The comparison reports relative L2 errors for different multistep families and step counts under fixed architecture and noise-free training data.
- Two-dimensional damped oscillator: The cubic damped harmonic oscillator’s learned dynamics reproduce the exact trajectory form and phase portrait.The experiment uses Adams-Moulton with M = 1, equivalent to the trapezoidal rule, and a one-hidden-layer network with 256 neurons.
- Two-dimensional damped oscillator: Larger time steps and noise levels produce inconsistent accuracy changes, sometimes degrading performance and sometimes improving it.The authors attribute improvements in some cases to input noise acting as a training regularizer, while excessive noise makes true dynamics harder to distinguish.
- Two-dimensional damped oscillator: Increasing network depth generally improves accuracy, whereas widths above 128 neurons per layer can reduce accuracy.The authors state that broader architecture sensitivity requires a more systematic study across multiple data sets.
- Lorenz system: The learned Lorenz system preserves the attractor’s form and bi-stable structure, but forecast trajectories diverge after t > 4.Because the Lorenz system has a positive Lyapunov exponent, small model discrepancies grow exponentially while the attractor remains intact.
- Fluid flow behind a cylinder: For cylinder flow, the learned model reproduces both the transient dynamics and the attained limit cycle in the phase portrait.The flow is reduced using proper orthogonal decomposition before learning the nonlinear dynamics.
- Hopf bifurcation: The parameterized Hopf model captures the transition from a fixed point for µ < 0 to a limit cycle for µ > 0 on unseen parameter values.Training uses data from multiple initial conditions and parameter values, while evaluation includes parameter values not used during training.
4. Summary and Discussion
The approach uses multistep time-stepping schemes with deep neural networks to learn nonlinear dynamical systems from time-series data. Across benchmark problems, it supports learning dynamics and forecasting, including glycolytic-oscillator trajectories from random initial conditions.
- The method combines Adams-Bashforth, Adams-Moulton, and BDF schemes with deep neural networks to learn nonlinear dynamical systems.
- Using M steps incorporates memory effects and addresses nonlinear, non-Markovian dynamics.The formulation reduces regression complexity from an (M × D)-dimensional problem to a simpler D-dimensional regression problem.
- The framework is evaluated on a diverse collection of benchmark problems involving nonlinear and chaotic dynamics.
- For the glycolytic oscillator, exact and learned dynamics are compared for random initial conditions drawn from specified parameter ranges.