PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition

Naoya Takeishi, Yoshinobu Kawahara, Takehisa Yairi

arXiv:1710.04340v2cs.LGmath.DSstat.ML

TL;DR

DMD often requires manually chosen nonlinear observables, which is problematic when the underlying dynamics are unknown. The paper learns Koopman-invariant subspaces from observed data by minimizing linear-regression RSS, implements the approach with neural networks, and evaluates it on nonlinear dynamics and applications. LKIS-DMD identifies target-subspace eigenvalues under observation noise, while neural-network optimization may encounter local optima.

  • Problem

    DMD often requires manually chosen nonlinear observables, which is problematic when the underlying dynamics are unknown.

  • Method

    LKIS learns parametric functions from observed data by minimizing the residual sum of squares of linear least-squares regression.

  • Results

    LKIS-DMD identifies target-subspace eigenvalues under white Gaussian observation noise, while basis-expanded DMD is directly affected by that noise.

  • Takeaways & Limitations

    The framework provides a fully data-driven numerical algorithm for Koopman spectral analysis with empirical evaluations on nonlinear dynamics and applications.

  • Takeaways & Limitations

    The multi-layer-perceptron implementation may converge to local optima, making the adequacy of obtained results difficult to assess.

Abstract

from arXiv · show

Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynamics, which is not always possible since we may not have any a priori knowledge about them. In this paper, we propose a fully data-driven method for Koopman spectral analysis based on the principle of learning Koopman invariant subspaces from observed data. To this end, we propose minimization of the residual sum of squares of linear least-squares regression to estimate a set of functions that transforms data into a form in which the linear regression fits well. We introduce an implementation with neural networks and evaluate performance empirically using nonlinear dynamical systems and applications.

1 Introduction

Nonlinear dynamics are difficult to analyze with state-space models, while Koopman methods lift the problem to linear evolution of observables. DMD is effective when observables span a Koopman invariant subspace, but those observables often must be selected manually; LKIS learns them from data.

  • Koopman analysis lifts nonlinear dynamics into a linear, though infinite-dimensional, regime by modeling observable evolution.
  • DMD relies on observables spanning a Koopman invariant subspace, but original-state data often require manually prepared nonlinear observables.
  • LKIS proposes learning Koopman invariant subspaces from observed data through a fully data-driven modal-decomposition method.
  • The method estimates parametric functions by minimizing linear-regression residual sum of squares so transformed data fit a linear model.

2 Background

Koopman spectral analysis represents nonlinear dynamics through linear operator evolution of observables and their modal decomposition. DMD approximates this analysis from snapshot data, but its validity depends on access to observables spanning a finite-dimensional Koopman invariant subspace.

  • 2.1 Koopman spectral analysis: The Koopman operator is an infinite-dimensional linear operator acting on observables rather than state vectors.
  • 2.1 Koopman spectral analysis: Koopman eigenfunctions evolve by their eigenvalues, whose angle and magnitude encode modal frequency and decay rate.
  • 2.1 Koopman spectral analysis: Koopman modes and eigenvalues support analysis of dominant characteristics in complex nonlinear phenomena and provide features or representations for data-analysis tasks.
  • 2.2 Dynamic mode decomposition and Koopman invariant subspace: DMD computes the eigendecomposition of A = Y1Y0† from observable snapshot matrices, with eigenvectors serving as dynamic modes.
  • 2.2 Dynamic mode decomposition and Koopman invariant subspace: A Koopman invariant subspace G satisfies Kg ∈ G for every g ∈ G; finite-dimensional restrictions make Koopman analysis computationally tractable.
  • 2.2 Dynamic mode decomposition and Koopman invariant subspace: DMD approximates the restricted Koopman operator asymptotically when the chosen observables span the invariant subspace.
  • 2.2 Dynamic mode decomposition and Koopman invariant subspace: Without access to such observables, nonlinear systems require manually prepared functions or an appropriately selected dictionary or kernel.

3 Learning Koopman invariant subspaces

The LKIS framework learns observables spanning an approximately Koopman-invariant subspace by making linear least-squares regression fit transformed data. It extends this principle to partial measurements through a learned delay embedding and reconstruction, and implements the framework with neural networks.

  • Learning Koopman invariant subspaces: The method learns observables from measurement sequences so they span a Koopman invariant subspace.
  • Learning Koopman invariant subspaces: For observables g, Koopman invariance is equivalent to a finite matrix G satisfying g(f(x)) = Gg(x).
  • Minimizing residual sum of squares: Because finite data cannot directly optimize the functional equation, the framework uses empirical risk minimization based on ergodicity and empirical-matrix convergence.
  • Learning Koopman invariant subspaces: Under ergodicity and sufficiently excited modes, the DMD matrix A = Y1Y0† almost surely converges to the matrix representation of the linear operator G.
  • Minimizing residual sum of squares: The learning objective minimizes the residual sum of squares of linear least-squares regression, which becomes zero when g spans a Koopman invariant subspace.
  • Linear delay embedder for state space reconstruction: With partial measurements y = ψ(x), a learned linear delay embedder reconstructs an input ˜x, whose parameters are optimized jointly with g using the RSS loss.
  • Reconstruction of original measurements: The objective adds a reconstruction loss to prevent trivial observables and support back-projection to the original measurements.
  • Implementation using neural networks: The neural implementation models g and h with multilayer perceptrons and tunes maximum delay k, embedded dimension p, and observable dimension n.

4 Related work

Related work contrasts generative state-space modeling with Koopman-oriented data transformations and identifies uncertainty handling and method comparisons as open directions.

  • Generative nonlinear dynamical-system models include Bayesian filtering and smoothing via expectation-maximization, as well as structured VAE-based inference methods.
  • Other related approaches estimate nonlinear state-space models more consistently or learn transformations for optimal control.
  • Because generative models account for process and observation noise, integrating uncertainty-aware methodologies with operator-theoretic analysis remains an open challenge.
  • Several similar Koopman data-transformation methods appeared after the manuscript’s first submission, leaving their relationships and relative advantages for future work.

5 Numerical examples

LKIS-DMD is evaluated on nonlinear systems with fixed-point, limit-cycle, and multiple-basin attractors, including noisy observations. Across these examples, it recovers Koopman spectral structure and qualitatively identifies basin boundaries.

  • Fixed-point attractor: LKIS-DMD identifies the target Koopman eigenvalues for the fixed-point system, including λ, µ, λ2, and λ0µ0 = 1.With white Gaussian observation noise of σ = 0.1, LKIS-DMD estimates coincide with the true values, while extended DMD is directly affected.
  • Fixed-point attractor: Under white Gaussian observation noise, LKIS-DMD still identifies the fixed-point system’s eigenvalues.The observation noise has σ = 0.1.
  • Limit-cycle attractor: RSS decreases rapidly and remains small under mini-batch SGD updates in the limit-cycle experiment.The full-batch loss is evaluated with batch size 2,000 under mini-batch size 200 updates.
  • Limit-cycle attractor: For the FitzHugh–Nagumo limit cycle, LKIS-DMD eigenvalues agree well with kernel DMDs, whereas linear Hankel DMD yields rapidly decaying modes.The hyperparameters were set to produce 16 eigenvalues for each DMD method.
  • Multiple basins of attraction: For the Duffing system, LKIS-DMD estimates decaying continuous-time eigenvalues and an eigenfunction whose level sets qualitatively agree with the true basin boundary.The experiment uses n = 100 learned observables, and the nearly zero eigenvalue identifies the basin-related eigenfunction.

6 Applications

The applications use LKIS-DMD for chaotic time-series prediction and unstable-phenomena detection. LKIS-DMD achieves the smallest 30-step RMS prediction errors and represents instability through rapidly decaying modes whose activations can be inspected.

  • Chaotic time-series prediction: DMD predicts the next step by evolving the current observable values through the learned linear operator and back-projecting LKIS-DMD values to y.The back-projection uses the learned reconstructor h.
  • Chaotic time-series prediction: LKIS-DMD achieves the smallest root-mean-square errors in 30-step predictions of Lorenz-x and Rossler-x series.The comparison includes a simple LSTM network and linear Hankel DMD, with hyperparameters tuned on validation data.
  • Unstable phenomena detection: Unstable-phenomena detection uses eigenfunctions associated with discrete-time eigenvalues whose magnitudes are considerably smaller than one.Such eigenvalues indicate rapidly decaying, unstable modes, and peaks in the detection output should correspond to unstable-phenomenon occurrences.

7 Conclusion

The paper presents LKIS as a fully data-driven framework for learning approximate Koopman invariant subspaces from available data, and reports empirical evaluations on nonlinear dynamics and applications. It also identifies local optima as a drawback of the multilayer-perceptron implementation and outlines alternatives for future work.

  • Conclusion: LKIS learns an approximately Koopman-invariant subspace entirely from available data by minimizing residual-sum-of-squares loss.The framework is proposed as a fully data-driven numerical algorithm for Koopman spectral analysis.
  • Conclusion: The reported evaluation covers several typical nonlinear dynamics and application examples.
  • Limitations and future work: The multilayer-perceptron implementation may converge to local optima, making the adequacy of obtained results difficult to assess.The paper suggests sparse basis-function observables and approximate Bayesian inference as possible alternatives or extensions.
  • DMD pipeline: The DMD workflow computes a compact SVD, eigendecomposition, left eigenvectors, normalized modes, and corresponding eigenvalues.The compact SVD uses rank r of Y0, and the returned eigenvalues are discrete-time quantities.
  • DMD pipeline: Continuous-time eigenvalues are obtained from discrete-time eigenvalues using λc = log(λ)/∆t.Here, ∆t is the discrete-time interval.
  • DMD pipeline: DMD is run after the proposed framework learns the observable g from observed data, rather than presuming access to an appropriate observable.The standard DMD output coincides with Koopman spectral analysis when g spans a Koopman invariant subspace.

B.2 Fixed-point attractor experiment

The fixed-point attractor experiments use episodes of length 30, with and without observation noise, while setting LKIS-DMD parameters to cover a minimal three-dimensional Koopman invariant subspace.

  • Data configuration: The fixed-point attractor data were generated from four initial values.The passage introduces the initial values but does not fully preserve them.
  • Data configuration: Each fixed-point attractor episode has length 30, and noisy data use observation-noise standard deviation 0.1.
  • LKIS-DMD settings: Both noisy and noise-free experiments use k = 2 and n = 4.These settings were chosen to cover the minimal three-dimensional Koopman invariant subspace.

B.3 Limit-cycle attractor experiment

The limit-cycle experiment integrates data for 2,000 steps at ∆t = 0.1 and configures the compared methods to produce 16 eigenvalues.

  • Data generation: The data were generated with MATLAB’s ode45 function using time-step ∆t = 0.1 for 2,000 steps.
  • Method settings: LKIS-DMD uses k = 8 and n = 16 in the limit-cycle experiment.These settings produce 16 eigenvalues.
  • Method settings: LKIS-DMD, linear Hankel DMD, and kernel DMD are configured to produce 16 eigenvalues.Kernel DMD disposes of POD modes below ε, using ε = 0.0001 for the polynomial kernel and ε = 0.05 for the RBF kernel.

B.4 Multiple basins of attraction experiment

The multiple-basins experiment samples 1,000 initial values over [−2, 2] × [−2, 2], evolves each for 11 steps, and uses LKIS-DMD with k = 1 and n = 100.

  • Data generation: The experiment draws 1,000 initial values uniformly from [−2, 2] × [−2, 2].
  • Data generation: Each initial value is advanced for 11 steps with ∆t = 0.25 using MATLAB’s ode45 numerical integration.
  • LKIS-DMD settings: LKIS-DMD is configured with k = 1 and n = 100, and estimated eigenfunction values are evaluated and plotted for each data point.

B.5 Chaotic time-series prediction experiment

The experiment evaluates methods on chaotic attractor data and a Santa Fe time-series dataset under fixed delay settings, with limited hyperparameter tuning in the qualitative analysis.

  • Experimental setup: Lorenz and Rossler attractors provided 25,000 generated steps each, divided into training, validation, and test sets.The Lorenz parameters were β = 8/3, σ = 10, ρ = 28; the Rossler parameters were a = 0.2, b = 0.2, c = 5.7.
  • Experimental setup: A delay dimension of 7 was fixed across LKIS-DMD, linear Hankel DMD, and truncated LSTM backpropagation.The passage states k = 7 for LKIS-DMD and linear Hankel DMD, with LSTM backpropagation truncated to length 7.
  • Compared methods: LSTM and Hankel DMD were included because prior work had used them for nonlinear time series and chaotic-system analysis, respectively.The stated rationale is methodological precedent rather than a reported performance result.
  • Additional dataset: The Santa Fe Time Series Competition Data supplied another dataset, while the delay or sliding-window length was fixed at 10 for all methods.The source webpage was unavailable on the manuscript’s May 2017 submission date, although the dataset remained available online.
  • Additional dataset: The qualitative experiment used limited hyperparameter tuning, default libsvm settings for OC-SVM except ν = 0.05, and default author-provided RuLSIF settings.OC-SVM was selected as a de facto standard for novelty or change-point detection, while RuLSIF was selected for its reported performance among density-ratio methods.
Loading 1710.04340v2…