Source-linked AI summary
Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
Jaideep Pathak, Zhixin Lu, Brian R. Hunt, Michelle Girvan, Edward Ott
TL;DR
The paper asks whether Lyapunov exponents can be estimated from limited time-series data without an accurate governing model. It trains a reservoir computer, converts it into an autonomous system, and computes its exponents from the known reservoir dynamics. Across Lorenz and Kuramoto–Sivashinsky examples, the method reproduces relevant ergodic properties and accurately estimates many Lyapunov exponents, especially in the higher-dimensional KS system.
Problem
Limited time-series data may be available when an accurate first-principles model is unavailable or unusable, motivating model-free estimation of the data-generating system’s Lyapunov exponents.
Method
The method trains a reservoir computer by linear regression, runs the trained system autonomously to reproduce the input attractor’s climate, and computes Lyapunov exponents from its known equations and output weights.
Results
The approach obtains excellent agreement for all positive and many negative Lyapunov exponents in a moderately high-dimensional system, while reproducing the positive and zero exponents of Lorenz accurately.
Takeaways & Limitations
A suitably trained reservoir can approximate the ergodic properties of a chaotic system and provide a model-free route to estimating many of its Lyapunov exponents from data.
Abstract
from arXiv · showhide
We use recent advances in the machine learning area known as 'reservoir computing' to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process. The technique uses a limited time series of measurements as input to a high-dimensional dynamical system called a 'reservoir'. After the reservoir's response to the data is recorded, linear regression is used to learn a large set of parameters, called the 'output weights'. The learned output weights are then used to form a modified autonomous reservoir designed to be capable of producing arbitrarily long time series whose ergodic properties approximate those of the input signal. When successful, we say that the autonomous reservoir reproduces the attractor's 'climate'. Since the reservoir equations and output weights are known, we can compute derivatives needed to determine the Lyapunov exponents of the autonomous reservoir, which we then use as estimates of the Lyapunov exponents for the original input generating system. We illustrate the effectiveness of our technique with two examples, the Lorenz system, and the Kuramoto-Sivashinsky (KS) equation. In particular, we use the Lorenz system to show that achieving climate reproduction may require tuning of the reservoir parameters. For the case of the KS equation, we note that as the system's spatial size is increased, the number of Lyapunov exponents increases, thus yielding a challenging test of our method, which we find the method successfully passes.
I. INTRODUCTION
The paper addresses model-free estimation of Lyapunov exponents when limited time-series data are available but an accurate first-principles model is not. It uses reservoir computing to replicate the input system’s ergodic properties and diagnose chaotic behavior.
- The method targets dynamical processes for which limited time-series data exist but no sufficiently accurate first-principles model is available.
- Reservoir computing provides an alternative model-free approach to delay-coordinate embedding for analyzing dynamical time series.
- The paper focuses on determining the Lyapunov exponents of the system generating the observed data.
- The approach requires the learned model to replicate the ergodic properties of the system generating the input.
- The authors report good agreement for all positive and many negative Lyapunov exponents in a moderately high-dimensional system.
II. RESERVOIR COMPUTERS, SHORT TERM PREDICTION AND ATTRACTOR CLIMATE
Reservoir computing trains a high-dimensional dynamical system on input-output time series and then runs it autonomously for prediction. The resulting autonomous reservoir can reproduce long-term climate properties even after short-term chaotic prediction breaks down.
- A reservoir receives an input vector through an input-to-reservoir coupler and produces an output vector through an output coupler, while its internal state provides memory.
- During training, reservoir states and inputs are recorded, and linear regression selects output parameters that minimize prediction error with optional regularization.
- For deterministic-system prediction, the desired output is set equal to the future input, vd(t + ∆t) = u(t + ∆t).
- At prediction time, the configuration switches to an autonomous reservoir whose output supplies the predicted input values for t > 0.
- Chaotic error amplification eventually limits short-term prediction, but the autonomous output can still approximate the input system’s long-term climate and support Lyapunov-exponent computation.
III. EXAMPLE 1: THE LORENZ SYSTEM AND THE QUESTION OF WHETHER THE CLIMATE IS REPLICATED
The Lorenz experiments show that reservoir climate reproduction depends on parameter tuning: R1 reproduces long-term attractor behavior, whereas R2 does not. The successful R1 reservoir estimates the positive and zero Lyapunov exponents well, but not the strongly negative exponent.
- Experimental setup: The Lorenz system is used to test whether trained reservoirs reproduce the attractor’s long-term climate.The experiment constructs reservoirs with three-dimensional Lorenz inputs and outputs, then evaluates autonomous dynamics and Lyapunov exponents.
- Climate reproduction: Both R1 and R2 produce correct short-term predictions before chaotic error growth causes divergence from the actual trajectory.R1 uses spectral radius ρ = 1.2, while R2 uses ρ = 1.45.
- Climate reproduction: R1 reproduces the Lorenz climate over 0 < t < 1000, while R2 settles near a fixed maximum zmax ≈30 instead of covering the actual return map.The successive-maxima return map shows R1 prediction points overlaying the actual points, whereas R2 leaves most actual points uncovered.
- Lyapunov estimation: The autonomous reservoir is treated as a known discrete-time dynamical system, whose tangent map yields Lyapunov exponents through QR decomposition.The reservoir’s known input, adjacency, and trained output parameters make the required derivatives computable.
- Lyapunov estimation: R1 approximates the Lorenz positive and zero Lyapunov exponents, but its third exponent is less negative than the true value.R2 fails to reproduce the positive exponent and has a largest exponent approximately zero, consistent with a periodic-orbit attractor.
- Interpretation: The negative-exponent mismatch arises because reproducing the visible return-map curve does not require reproducing its very thin transverse structure.That thin structure provides the orbital evidence for the strongly negative exponent Λ3.
IV. EXAMPLE 2: THE TASK OF DETERMINING A LARGE NUMBER OF LYAPUNOV EXPONENTS OF A HIGH DIMENSIONAL SPATIOTEMPORAL CHAOTIC SYSTEM FROM DATA
The study tests reservoir computing on the high-dimensional Kuramoto–Sivashinsky system, using data-trained autonomous reservoirs to reproduce climate and estimate many Lyapunov exponents. The method succeeds across the spectrum, while revealing sensitivity to parameter choice, training duration, and symmetry-related exponents.
- Data and setup: The KS field is discretized on an evenly spaced one-dimensional grid, producing Q coupled time series as the reservoir’s multivariate input.The input vector samples y(x,t) at Q grid points with Δx = L/Q.
- Data and setup: The reservoir is trained only on time-series data, then run autonomously to generate predictions and calculate its Lyapunov spectrum.The autonomous predictions are compared with the true KS dynamics and climate statistics.
- Climate reproduction: Accurate short-term prediction does not guarantee climate reproduction: one parameter set reproduces the KS climate, whereas another with ρ = 3.1 and Dr = 5000 does not.The successful and unsuccessful cases are distinguished by long-term behavior despite short-term prediction in both cases.
- Climate reproduction: The successful reservoir reproduces the training-data power spectrum, providing a quantitative indication that its long-term orbit captures the KS climate.The comparison uses power spectra computed from the training data and autonomous reservoir dynamics.
- Lyapunov-spectrum estimation: For L = 60, the reservoir matches the positive KS Lyapunov exponents very well and matches negative exponents after excluding the two near-zero exponents Λ7 and Λ8.The standard KS system has three zero exponents associated with continuous symmetries, while the reservoir does not reproduce all of them.
- Lyapunov-spectrum estimation: For the symmetry-broken case, the reservoir reproduces the KS Lyapunov spectrum accurately and remains reasonable beyond the attractor’s information dimension DKY ≈ 15.The reported agreement extends to exponents with index k > DKY.
- Lyapunov-spectrum estimation: Training-data length significantly affects spectrum accuracy, with negative Lyapunov exponents more sensitive than positive ones and about 20000 time steps needed for reasonable negative-exponent estimates.The cited training duration corresponds to about 400 Lyapunov times for the KS system.
V. DISCUSSION AND CONCLUSION
A suitably trained reservoir can approximate a chaotic system’s ergodic properties and estimate many Lyapunov exponents from data. Performance is strong for high-dimensional KS dynamics, while the Lorenz system’s large-magnitude negative exponent is less accurately recovered.
- The method accurately calculates many positive and negative Lyapunov exponents for a high-dimensional spatiotemporal chaotic system.The paper reports good accuracy for a large number of exponents in the KS equation.
- For the Lorenz equations, the positive and zero Lyapunov exponents are calculated with good accuracy.
- The Lorenz system’s high-magnitude negative Lyapunov exponent is less accurately estimated, although its substantially larger magnitude is captured.
- The trained reservoir approximates the ergodic properties of the system used for training.
- The paper identifies model-free machine-learning analysis of measured chaotic-system data as a fruitful subject for further research.