PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-driven spectral decomposition and forecasting of ergodic dynamical systems

Dimitrios Giannakis

arXiv:1507.02338v4math.DSphysics.data-an

TL;DR

The paper tackles dimension reduction, mode decomposition, and nonparametric forecasting from time-ordered data generated by ergodic dynamical systems. It constructs a diffusion-maps basis for Koopman and Perron-Frobenius analysis, uses regularized eigenfunctions for smooth reductions and forecasting, and applies time change to complex spectra. Its scope is chiefly demonstrated on pure-point and torus-focused systems, with broader extensions left open.

  • Problem

    The problem is how to perform dimension reduction, mode decomposition, and forecasting from ergodic-system data without equations of motion when Koopman spectra are difficult to eigendecompose.

  • Method

    The paper constructs a smooth orthonormal L2 basis from time-ordered data using diffusion maps and applies it to regularized Koopman analysis, vector-field reconstruction, delay coordinates, and forecasting.

  • Results

    The framework provides projectible smooth dimension reduction, observation-aware vector-field decomposition, and nonparametric forecasts using Koopman eigenfunctions whose coefficients evolve as uncoupled harmonic oscillators.

  • Takeaways & Limitations

    The approach supplies a data-driven operator framework for analyzing and forecasting ergodic systems from a single time series, including systems with more complex spectral behavior through time change.

  • Takeaways & Limitations

    The paper's applications are heavily focused on dynamical systems on tori, and extensions to mixed spectra and other settings remain to be investigated.

Abstract

from arXiv · show

We develop a framework for dimension reduction, mode decomposition, and nonparametric forecasting of data generated by ergodic dynamical systems. This framework is based on a representation of the Koopman and Perron-Frobenius groups of unitary operators in a smooth orthonormal basis of the L2 space of the dynamical system, acquired from time-ordered data through the diffusion maps algorithm. Using this representation, we compute Koopman eigenfunctions through a regularized advection-diffusion operator, and employ these eigenfunctions in dimension reduction maps with projectible dynamics and high smoothness for the given observation modality. In systems with pure point spectra, we construct a decomposition of the generator of the Koopman group into mutually commuting vector fields that transform naturally under changes of observation modality, which we reconstruct in data space through a representation of the pushforward map in the Koopman eigenfunction basis. We also establish a correspondence between Koopman operators and Laplace-Beltrami operators constructed from data in Takens delay-coordinate space, and use this correspondence to provide an interpretation of diffusion-mapped delay coordinates for this class of systems. Moreover, we take advantage of a special property of the Koopman eigenfunction basis, namely that the basis elements evolve as simple harmonic oscillators, to build nonparametric forecast models for probability densities and observables. In systems with more complex spectral behavior, including mixing systems, we develop a method inspired from time change in dynamical systems to transform the generator to a new operator with potentially improved spectral properties, and use that operator for vector field decomposition and nonparametric forecasting.

1. Introduction

The paper addresses data-driven dimension reduction, mode decomposition, and forecasting for ergodic dynamical systems using operator-theoretic methods and time-ordered observations. It motivates a diffusion-maps basis adapted to the intrinsic geometry of data and develops regularization for systems whose Koopman spectra are difficult to decompose.

  • Motivation: Ergodicity lets long-time observations sample the invariant measure, enabling nonparametric dimension reduction and forecasting without prior equations of motion.The framework targets high-dimensional time series generated by ergodic systems or ergodic components.
  • Operator-theoretic setting: Koopman and Perron-Frobenius operators provide linear operator descriptions of nonlinear dynamics on observables and measures.They are dual descriptions, although they act on fundamentally different mathematical objects.
  • Challenges: Existing Koopman eigendecomposition becomes problematic for high-complexity systems, including weak-mixing systems with continuous spectra.Weak mixing is characterized by the absence of nonconstant Koopman eigenfunctions, making numerical eigenvectors difficult to interpret.
  • Challenges: A basis adapted to intrinsic data geometry avoids the cost and poor representation associated with ambient-space dictionaries when data lie on low-dimensional or measure-zero subsets.Diffusion-maps basis functions are supported on the evolving data subset and can converge to a complete orthonormal basis of the dynamical system's L2 space.
  • Contributions: The paper develops a diffusion-maps-based orthonormal basis for dimension reduction, mode decomposition, forecasting, and regularized Koopman analysis.The basis is paired with variable-bandwidth kernels and supports spectral and Galerkin methods from a single time series.
  • Contributions: For systems without pure point spectra, the paper develops a time-change strategy intended to produce improved spectral properties for eigendecomposition and forecasting.The method targets weak-mixing systems with no nonconstant eigenfunctions while preserving the original system's orbits.

2. Notation and basic results from ergodic theory

The paper formulates Koopman and Perron-Frobenius dynamics on L2 spaces for smooth ergodic flows observed through an embedding. It uses Koopman eigenfunctions as Fourier-like coordinates whose coefficients evolve as uncoupled oscillators, while addressing spectral and regularity issues through diffusion and time-change regularization.

  • System and observations: The system is a smooth flow on a compact manifold with an invariant probability measure, observed at fixed intervals through a smooth map into data space.The observation map is assumed to be an embedding, or replaced by delay coordinates when necessary.
  • Koopman framework: Koopman operators act on observables by composition with the flow, while their skew-adjoint generator gives directional derivatives along the dynamical vector field.The Koopman group is unitary on L2, and the generator satisfies the continuous-time structure used later in reconstruction and regularization.
  • Operator framework: Perron-Frobenius operators act on probability densities and form an adjoint pair with Koopman operators in the L2 setting.Both operator families can therefore be characterized through the Koopman generator and its spectral properties.
  • Spectral structure: Koopman eigenvalues lie on the imaginary axis, and eigenfunctions evolve under the flow by multiplication with harmonic phase factors.Along typical trajectories, eigenfunction values behave like simple harmonic oscillators with their corresponding intrinsic frequencies.
  • Regularization: Continuous spectra and dense eigenvalues can make numerical eigendecomposition and regularity control difficult, motivating diffusion and time-change regularization.Highly oscillatory eigenfunctions may have large Dirichlet energy despite frequencies near those of low-roughness eigenfunctions.
  • Spectral structure: In the Koopman eigenfunction basis, Fourier coefficients satisfy uncoupled equations d f_hat_k/dt = iω_k f_hat_k and can be advanced independently.This representation supports forecasting and requires only generating eigenfunctions and their basic frequencies to evaluate the operator action.
  • Pure-point systems: Pure-point ergodic systems with smooth eigenfunctions admit a torus-like representation with canonical angle coordinates and finitely many basic frequencies.For smooth flows on an m-dimensional manifold in the stated setting, the number of basic frequencies is necessarily m.

3. Dimension reduction and forecasting in systems with pure point spectra

For systems with pure point spectra, Koopman eigenfunctions define smooth dimension-reduction maps whose coordinates evolve as harmonic oscillators. They also support a commuting vector-field decomposition, observation-modality-independent reconstruction, and nonparametric forecasting.

  • Dimension reduction: Generating Koopman eigenfunctions define projection maps onto circle-valued coordinates and, jointly, a torus-valued dimension-reduction map.Under rationally independent eigenfrequencies, the composite map is a diffeomorphism onto the m-torus.
  • Eigenfunction selection: Smooth generating eigenfunctions are selected using low Dirichlet energy and rationally independent frequencies, while remaining consistent across suitable observation modalities.The resulting projection maps provide a unified parameterization for data from different sensors observing the same dynamical system.
  • Dimension reduction: The projected dynamics in each eigenfunction coordinate are autonomous simple harmonic oscillations with frequency Ωi, avoiding closure issues.The vector field is projectible under each projection map, so the reduced dynamics are well defined on the image space.
  • Vector field decomposition: The dynamical vector field decomposes into linearly independent, nowhere-vanishing, mutually commuting components that preserve the invariant measure.Their flows are non-ergodic components whose composition recovers the full evolution, with order-independent composition.
  • Vector field decomposition: Pushforward representations reconstruct the component vector fields in data space and transform them naturally as tensors under changes of observation modality.The reconstructed fields are tangent to the embedded data manifold and obey the pushforward transformation rule.
  • Nonparametric forecasting: Forecasting evolves the initial generating eigenfunction values, reconstructs other eigenfunctions through their group structure, and then recovers observables from expansion coefficients.Out-of-sample extension supplies initial eigenfunction values for observed data, including points affected by model error.
  • Limitations and conditioning: In irrational torus flows, eigenfunctions can have arbitrarily small frequencies but arbitrarily large Dirichlet energy, complicating numerical identification of smooth slow observables.The paper therefore treats smoothness as distinct from slowness when selecting eigenfunctions.

4. Galerkin approximation in a data-driven orthonormal basis

The method uses diffusion-map eigenfunctions as a smooth, sampling-robust basis for a regularized Koopman generator, with convergence and error sources characterized. For pure point systems, commuting dynamics improve eigenfunction recovery and enable data-space vector-field reconstruction, while noncommutativity and weak mixing limit interpretation and spectral behavior.

  • Basis construction: A conformal metric change reduces sampling-density effects, improving the robustness of the approximated Laplace–Beltrami spectrum.The associated error bound is independent of the sampling density σ.
  • Galerkin method: Equal Dirichlet energies and sampling-robust eigenfunctions make the diffusion basis suitable for a well-conditioned Galerkin approximation of the Koopman generator.The basis elements have unit Dirichlet energy for nonconstant modes.
  • Basis construction: Diffusion maps approximate Laplace–Beltrami eigenvalues and eigenfunctions, with normalized estimates converging under increasing sample size and suitable kernel scaling.The approximation also involves finite-difference, Galerkin, and diffusion-regularization errors.
  • Limitations and extensions: Noncommutativity makes the regularized generator nonnormal, while weak-mixing systems can exhibit complicated spectra that undermine the physical interpretation of its eigenfunctions.The framework therefore considers time change rather than diffusion alone for such systems.
  • Spectral properties: When the generator and Laplace–Beltrami operator commute, joint smooth eigenfunctions yield exact Koopman eigenfunctions and an efficient Galerkin method.Commuting operators also guarantee orthogonality and completeness of the regularized-generator eigenfunctions.

5. Galerkin method with delay-coordinate maps for systems with pure point spectra

Delay-coordinate maps produce a diffusion operator that commutes with the Koopman group in pure-point systems, yielding an efficient Galerkin basis and a smooth invariant metric. This correspondence also supports denoising and clarifies when delay-coordinate diffusion maps provide effective Koopman eigenfunction approximations.

  • Delay-coordinate diffusion operators commute with the Koopman group in pure-point systems, enabling an efficient Galerkin scheme for Koopman eigenfunctions.The construction also provides a natural way to denoise data corrupted by i.i.d. observational noise.
  • For systems with nonzero Lyapunov exponents, the induced delay-coordinate metric generally fails to converge to a smooth tensor as the number of delays increases.Stable and unstable Oseledets subspaces acquire opposing exponential scalings under the induced metric.
  • In pure-point systems with smooth Koopman eigenfunctions, the limiting delay-coordinate metric is smooth, flow-invariant, flat, and has a volume form uniformly related to the invariant measure.The metric is invariant under the flows generated by the dynamically independent vector fields, and its volume form satisfies dvol¯g/dµ = Γ.
  • The associated vector fields commute with the Laplace-Beltrami operator, so Koopman eigenfunctions are also Laplace-Beltrami eigenfunctions and can be efficiently approximated in its Galerkin space.The leading generating eigenfunction lies entirely in the eigenspace associated with the smallest nonzero Laplace-Beltrami eigenvalue, while each Koopman eigenfunction is a finite linear combination of Laplace-Beltrami eigenfunctions.
  • Delay-coordinate kernels can denoise i.i.d.-corrupted observations because increasing the number of delays averages the noise-induced bias in pairwise squared distances.The cited result follows from the law of large numbers under a suitable scaling of the delay count.

6. Regularization by time change

Time change regularizes Koopman analysis for systems with mixing or otherwise poor spectral structure by transforming the generator to an orbit-equivalent system with more usable eigenfunctions. The resulting eigenfunctions support vector-field decomposition and forecasting, while mixing imposes finite-model limitations at late times.

  • 6. Regularization by time change: Time change transforms the generator into an orbit-equivalent system with potentially improved spectral properties for dimension reduction and forecasting.The approach uses eigenfunctions of the time-changed system to represent the original dynamics through non-autonomous oscillators with variable frequency.
  • 6. Regularization by time change: The method extends Koopman eigenfunction techniques to systems lacking nonconstant eigenfunctions when they are related to pure-point-spectrum systems by time change.An empirically accessible time-change function can recover the relevant transformation in special cases.
  • 6.4.1. Mixing flow on the 3-torus: For the mixing 3-torus, time-changed generating eigenfunctions lie approximately on the unit circle and exhibit phase-modulated wavetrains with variable frequency.Their behavior matches the expected structure for eigenfunctions associated with the time-changed system.
  • 6.4.1. Mixing flow on the 3-torus: The mixing-system forecast tracks ensemble mean and standard deviation before equilibrium relaxation but develops spurious oscillations at later times.Finite deterministic oscillator models cannot reproduce the arbitrarily small density lengthscales generated by mixing indefinitely.
  • 6.5. Ergodic flow on the 2-torus with a fixed point: For the fixed-point 2-torus flow, the time-change method produces approximate eigenfunctions with phase-modulated waves and observable timescale separation.Using ε = 0.02, the eigenfunctions lie on a narrow annulus; their basic frequencies are {0.735, 0.165} and Dirichlet energies are {1.54, 2.42}.

7. Concluding remarks

The paper develops data-driven tools for dimension reduction, vector-field decomposition, and nonparametric forecasting from ergodic-system data. Pure-point systems support commuting oscillator decompositions, while time change extends the framework to mixing systems.

  • The framework uses Koopman and Perron-Frobenius operator formalisms to analyze ergodic dynamical-system data.
  • A smooth complete L2 basis enables projectible Koopman-eigenfunction dimension reduction, vector-field decomposition, noise-robust eigenvalue computation, and forecasting of densities and observables.
  • In pure-point systems, the methods decompose nonlinear dynamics into uncoupled simple harmonic oscillators.
  • For mixing systems, rescaling the generator by the ambient-data-space vector-field norm preserves orbits while changing time along them and supports decomposition and forecasting.
  • The methods and applications are heavily focused on dynamical systems on tori and require investigation in more general systems.

Appendix A. Algorithms

The appendix summarizes algorithms for constructing diffusion-map bases, computing Koopman frequencies and eigenfunctions, decomposing vector fields, and forecasting densities and observables.

  • Algorithm 1 constructs a data-driven orthonormal L2 basis using diffusion maps, kernel density estimation, bandwidth tuning, and Laplace-Beltrami eigenfunctions.
  • Algorithm 2 computes regularized Koopman eigenfunctions, orders them by Dirichlet energy, and selects rationally independent generating frequencies.
  • Algorithm 3 forms products of generating eigenfunctions, computes their weighted Gram matrix, and reconstructs vector-field components from observation-map coefficients.
  • Algorithm 4 expands observables and the initial density, advances density coefficients by e^-iω_jt, and reconstructs forecast densities.

Appendix B. Proof of Theorem 16

The proof constructs a metric from commuting vector fields and establishes its constant volume ratio, isometry under the flow, and flatness.

  • Because the vector fields are nowhere-vanishing and linearly independent, verifying the metric on each vector-field pair establishes the desired metric expression.
  • The flow preserves the constructed metric because it preserves the pairwise metric products of the vector fields.
  • Mutually commuting, linearly independent Killing vector fields imply flatness locally, and their global definition gives global flatness.
  • In coordinates generated by the vector fields, the metric has constant components, while zero µ-divergence makes the volume-form ratio globally constant.

Appendix C. Treatment of i.i.d. noise through delay embeddings and diffusion maps

The appendix analyzes i.i.d. observational noise in delay-coordinate diffusion maps. With sufficiently many delays, noise contributes a controlled bias while preserving the accuracy order of the diffusion approximation.

  • The analysis considers small bandwidth, infinitely many delays, and sample growth satisfying N(s) ≫ s for noisy delay embeddings.
  • The noisy Markov approximation produces an O(ϵ2) bias, matching the diffusion-operator approximation error.
  • For non-overlapping delay sequences, noisy squared distances acquire an additive expected term 2R2.
  • The fixed-bandwidth kernel’s multiplicative noise bias cancels between numerator and denominator, unlike the variable-bandwidth density-estimation kernel.
  • With infinitely many delays, the density estimate converges up to O(ϵ2) to a constant on the manifold and estimates the sampling density to that accuracy up to proportionality.
Loading 1507.02338v4…