PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-driven Koopman mode approximation: A neural power iteration algorithm

Guillaume O. Berger, Raphaël M. Jungers

arXiv:2608.26943v1eess.SYcs.LGmath.OC

TL;DR

Finite-dimensional projection is needed for Koopman-based analysis, but projection can perturb the spectrum. The paper uses neural-network power iteration with sampled transitions and reports accurate dominant-mode approximations, including bistable systems, while noting asymptotic and projection-related limits.

  • Problem

    Koopman observables are typically infinite-dimensional, so practical methods require finite-dimensional projection, which can perturb the operator spectrum and produce spectral pollution.

  • Method

    The method applies power iteration directly to the projected Koopman operator, replacing exact projections with empirical projections from sampled transitions and implementing them through neural-network regression.

  • Results

    The approach accurately captures dominant Koopman modes, including discontinuous eigenfunctions and basins of attraction in bistable systems, while producing smaller approximation errors than polynomial EDMD in the nonlinear 1D experiment.

  • Takeaways & Limitations

    Direct power iteration avoids explicitly constructing the projected Koopman operator and does not require predefined dictionaries or anti-collapse regularization mechanisms.

  • Takeaways & Limitations

    The method inherits projection-related spectral pollution and dependence on the approximation space; its consistency result is asymptotic, without a finite-sample error bound or universal data requirement.

Abstract

from arXiv · show

This paper proposes a novel data-driven algorithm to approximate the dominant eigenfunctions (aka.~modes) of the Koopman operator of nonlinear dynamical systems using neural networks. The relevance of learning the dominant Koopman modes is to approximate nonlinear dynamics by linear ones in a lifted space, thereby enabling simplified control and analysis. To fight the curse of dimensionality arising from using expressive templates (here neural networks) for the mode approximation, the proposed method leverages a power-iteration scheme that directly learns the dominant Koopman modes without explicitly constructing the projection of the Koopman operator on the template of functions. Our approach connects to other approaches in the literature that avoid the curse of dimensionality by learning small dictionaries of functions, but differs from them in that we do not require ``anti-collapse mechanisms'' to ensure that the learned dictionary is expressive enough to approximate the Koopman operator since our power-iteration scheme is designed to converge toward the dominant modes of the projected Koopman operator. The approach is fully data-driven, requiring only sampled state transitions. Theoretical guarantees are provided, showing convergence under increasing sample size and network width (in connection with the neural tangent kernel theorem). Numerical experiments demonstrate that the method achieves accurate and smooth approximations of dominant modes while avoiding the limitations of traditional techniques such as extended dynamic mode decomposition.

I. INTRODUCTION

Koopman analysis represents nonlinear dynamics through a linear operator acting on observables, but practical finite-dimensional projections must balance computational efficiency and approximation accuracy. Existing dictionary- and neural-network-based approaches face expressivity, collapse, and scalability challenges.

  • Koopman theory studies observables under system evolution, allowing nonlinear dynamics to be analyzed with linear operator and spectral techniques.Applications include convergence analysis, chaos, model order reduction, and control.
  • Data-driven Koopman approximations use sampled one-step trajectories to approximate the operator’s action on observables.
  • Finite-dimensional projection is necessary because the observable space is typically infinite-dimensional, but selecting the projection subspace introduces an efficiency–accuracy trade-off.Projection also introduces approximation errors that must be considered in analysis.
  • DMD, EDMD, and Hankel DMD rely on linear observables or user-provided function dictionaries to represent the approximation space.EDMD can use polynomial dictionaries, while Hankel DMD repeatedly applies the Koopman operator to a supplied function.
  • Expressive dictionaries may be required for accurate modes, as bistable systems can have discontinuous dominant eigenmodes that high-degree polynomials approximate poorly.
  • Neural observables avoid predefined dictionaries but create an unclear basis and a collapse problem in which learned observables can become trivial or linearly dependent.State-retrieval constraints address collapse but require observables scaling with system dimension; decoder approaches also add training cost and data sensitivity.

Contributions

The paper introduces power iteration to learn dominant modes of a projected Koopman operator without explicitly constructing that operator or relying on predefined dictionaries and anti-collapse mechanisms. It combines data-based operator approximation with neural projections and provides theoretical and empirical support while clarifying the scope of projected-operator analysis.

  • Contributions: A power-iteration method learns the m dominant modes of the projected Koopman operator, where dominance is determined by eigenvalue modulus.The method targets a user-specified number m of modes.
  • Contributions: Recursive application of the projected operator to independent observables converges toward the subspace spanned by dominant modes under suitable spectral-separation assumptions.Periodic normalization is used to reduce ill-conditioning.
  • Contributions: The algorithm uses sampled data to approximate Koopman action and gradient descent to project iterates back onto neural-network observables.
  • Contributions: The approach avoids explicit projected-operator construction and the need for large user-provided dictionaries or anti-collapse mechanisms.Its neural implementation is theoretically connected to the neural tangent kernel theorem.
  • Contributions: Dominant modes govern long-term dynamics when eigenfunction expansions are available, while eigenfunctions with eigenvalue one encode basins of attraction and subunitary dominant eigenvalues encode convergence rates.
  • Contributions: Projected Koopman methods can suffer spectral pollution because eigenfunctions of the projected operator may poorly approximate eigenfunctions of the original operator.The paper leaves control for open-loop systems to future work.

Other related work

The paper formulates Koopman analysis on a Hilbert space of square-integrable observables and defines eigenfunctions through the operator’s linear action. These objects connect spectral quantities to stability and attraction properties of the dynamical system.

  • Other related work: Related latent-space learning methods use neural predictive models and require separate anti-collapse techniques such as contrastive samples, entropy maximization, or decoders.The paper instead uses linear models encoded by the projected Koopman matrix and power iteration to address collapse.
  • Other related work: The observable space is modeled as L2(X,Σ,µ), a Hilbert space whose inner product is defined through expectation under µ.
  • Other related work: A feed-forward neural network ϕθ maps R^n to C and is parameterized by weights and biases across hidden layers.
  • Other related work: For a forward-invariant continuous discrete-time system, the Koopman operator maps an observable f to f∘F while remaining within the observable space.
  • Other related work: An eigenfunction is a non-zero observable g satisfying Kg=λg, with λ as its associated eigenvalue.
  • Other related work: For asymptotically stable equilibria, eigenfunctions with eigenvalue one are constant on the equilibrium’s basin, while continuous eigenfunctions vanishing at the equilibrium have subunitary eigenvalues and semi-Lyapunov magnitudes.

Projected Koopman operator

The paper restricts Koopman analysis to a finite-dimensional subspace and targets dominant modes of its projected operator without explicitly constructing that operator. This projection-based approach remains fully data-driven but can introduce spectral pollution.

  • Projected Koopman operator: The projected operator is formed by applying the Koopman operator and projecting back onto a finite-dimensional subspace D.D is assumed finite-dimensional, closed, and such that KD remains within the observable space.
  • Projected Koopman operator: The method approximates the m dominant eigenvalues and invariant subspace of KD, where m is much smaller than dim(D), without explicitly computing KD.The target is the projected operator rather than the infinite-dimensional Koopman operator itself.
  • Projected Koopman operator: The power-iteration algorithm avoids an explicit matrix representation or basis of D and can use only sampled state transitions.This permits neural regression to implement projection without constructing the projected operator.
  • Projected Koopman operator: Projection may cause spectral pollution, so eigenfunctions of KD may not accurately approximate eigenfunctions of the full Koopman operator K.The paper treats this as a limitation shared by finite-dimensional projection-based Koopman methods.

A. Idealized algorithm

The idealized algorithm applies subspace power iteration directly to the projected Koopman operator, periodically normalizing iterates to avoid ill-conditioning. Under a spectral-gap condition, the resulting subspace converges toward the dominant modes.

  • A. Idealized algorithm: The idealized method repeatedly applies KD to m initially linearly independent functions in D.A normalization step may be inserted periodically, and Algorithm 1 presents the procedure.
  • A. Idealized algorithm: Power iteration converges toward the dominant modes when the modulus of the mth eigenvalue exceeds that of the (m+1)st.The required condition is |λm| > |λm+1|.
  • A. Idealized algorithm: For almost all initial choices, the span of the iterates converges to the invariant subspace associated with the first m eigenvalues.The convergence statement concerns the subspace rather than necessarily individual eigenfunctions.
  • A. Idealized algorithm: The neural implementation replaces exact Koopman evaluations and projections with regression and data-driven projection steps.These operations avoid requiring function values everywhere or explicit Gram-matrix-based projections.
  • A. Idealized algorithm: Neural basis functions are orthonormalized from sampled evaluations, with fitting error controlling approximate orthonormality.If the function class forms a linear subspace, the fitting error can achieve zero.

C. Analysis of neural data-driven v.s. idealized

The analysis separates sampling and neural-approximation limits: empirical projections converge to ideal projections as sample size grows, while infinite-width neural training is justified through the neural tangent kernel.

  • C. Analysis of neural data-driven v.s. idealized: Monte–Carlo inner products and norms become sound as the number of i.i.d. samples increases.This supplies the sampling component of the approximation analysis.
  • C. Analysis of neural data-driven v.s. idealized: For any fixed iteration count, the neural data-driven iterates converge almost surely to the idealized iterates as N tends to infinity.The result assumes the empirical neural projection is linear with image set D and the relevant observables remain in F.
  • C. Analysis of neural data-driven v.s. idealized: At the population level, idealized subspace iteration converges toward the dominant invariant subspace under a spectral gap and almost every initialization.The analysis distinguishes this operator-level limit from the finite-sample and neural limits.
  • C. Analysis of neural data-driven v.s. idealized: In the infinite-width regime, neural tangent kernel training becomes kernel regression in the reproducing kernel Hilbert space associated with a deterministic kernel.The kernel depends on network depth, activation, and parameter-variance distribution.
  • C. Analysis of neural data-driven v.s. idealized: Under the NTK justification, the neural empirical projection can be identified with the empirical linear projection, subject to initialization and optimization qualifications.Finite-width neural parametrizations themselves are not generally linear function classes.

IV. NUMERICAL EXPERIMENTS

The experiments use two-hidden-layer tanh networks trained with AdamW and compare the proposed method with extended dynamic mode decomposition under a fixed iteration budget.

  • IV. NUMERICAL EXPERIMENTS: All experiments use neural networks with L = 2 hidden layers and tanh activation.The widths are denoted c1 and c2.
  • IV. NUMERICAL EXPERIMENTS: Networks are trained for 800 epochs with AdamW, learning rate 10^-3, no weight decay, and full-batch training.These settings are applied uniformly in the reported experiments.
  • IV. NUMERICAL EXPERIMENTS: The maximum number of neural power-iteration steps is fixed at kmax = 50, although stopping by subspace or residual tolerance is also suggested.The fixed budget is used uniformly across the reported experiments.
  • IV. NUMERICAL EXPERIMENTS: The proposed method is compared with extended dynamic mode decomposition.Implementation code and experiment code are publicly available.

A. Nonlinear 1D system

Algorithm 2 accurately approximates the dominant Koopman modes for the nonlinear 1D system, with overall error at most 0.05. Compared with polynomial EDMD, it achieves smaller approximation errors but requires substantially more computation time.

  • A. Nonlinear 1D system: Algorithm 2 uses N = 2500 Gaussian samples and neural networks with hidden widths c1 = c2 = 8 to approximate m = 3 Koopman modes.The computed modes are normalized before evaluating their approximation errors.
  • A. Nonlinear 1D system: The approximation error is overall small, with ∥λ_i^-1 K g_i − g_i∥ ≤ 0.05.The error criterion compares each normalized mode with its Koopman-transformed and eigenvalue-scaled version.
  • A. Nonlinear 1D system: The neural approach takes 30 secs, compared with 2 secs for EDMD.The additional cost arises because power iteration fits several neural networks, whereas EDMD solves one finite-dimensional regression problem.
  • A. Nonlinear 1D system: For polynomial degrees > 6, numerical errors and spectral pollution make EDMD approximations worse.Thus, increasing the polynomial degree beyond 6 does not necessarily improve the approximation.

B. Bistable system

On the bistable Duffing system, Algorithm 2 learns dominant Koopman modes whose flat components qualitatively recover the attraction basins, while polynomial EDMD fails to represent the discontinuous structure well. The learned-mode approximation error remains at most 0.1, with computation taking about 8 min.

  • B. Bistable system: The Duffing system has two asymptotically stable equilibria and one saddle, motivating estimation of both attraction basins through dominant Koopman eigenfunctions.Good approximations are nontrivial because the relevant approximation functions are discontinuous.
  • B. Bistable system: Algorithm 2 uses m = 2 modes with N = 10^4 samples uniformly distributed on [−4, 4]^2 and hidden widths c1 = c2 = 64.The system is first time-discretized with sampling period T = 1.
  • B. Bistable system: The flat components of the learned modes qualitatively recover the system’s basins of attraction.This experiment illustrates structure encoded by the eigenfunctions but is not a certified region-of-attraction computation.
  • B. Bistable system: The approximation error is overall small, with ∥λ_i^-1 K g_i − g_i∥ ≤ 0.1.The corresponding contour plots compare normalized modes with their eigenvalue-scaled Koopman transforms.
  • B. Bistable system: The bistable-system computation takes approximately 8 min.The reported runtime accompanies the mode-approximation experiment.
  • B. Bistable system: Polynomial EDMD fails to approximate the dominant modes and attraction basins well because polynomials cannot represent discontinuous functions.The comparison uses polynomial dictionaries of degree up to 3 or 6.

V. SCOPE AND LIMITATIONS

The method targets dominant spectral components of a projected Koopman operator, so it retains limitations associated with finite-dimensional approximations and spectral separation. Its guarantees and practical costs are also constrained by asymptotic assumptions, finite-network optimization, and repeated regressions.

  • 14 Numerical errors and spectral pollution made approximations worse for degrees > 6.
  • The method inherits possible spectral pollution and dependence on the chosen approximation space from finite-dimensional Koopman approximations.
  • Increasing the requested mode count m requires additional regressions and suitable spectral separation between retained and discarded components.
  • The consistency result is asymptotic and provides neither finite-sample error bounds nor a universal data-size prescription.
  • The NTK justification is infinite-width, whereas experiments use finite networks and numerical optimization.
  • Repeated neural regressions make the method more computationally expensive than EDMD in the reported examples.

VI. CONCLUSIONS

The paper introduces a data-driven neural power-iteration algorithm for approximating dominant eigenfunctions of a projected Koopman operator. Its analysis provides asymptotic convergence and consistency justification, while experiments show accurate mode recovery, including discontinuous bistable cases, with improved accuracy over polynomial EDMD when expressive representations are needed.

  • The proposed algorithm uses neural-network power iteration to approximate dominant eigenfunctions of a projected Koopman operator.
  • It avoids explicitly constructing the projected Koopman operator and does not require predefined dictionaries or anti-collapse regularization mechanisms.
  • The analysis separates ideal subspace-iteration convergence from data-driven consistency as sample size increases, with NTK-based asymptotic justification.
  • Experiments accurately capture dominant Koopman modes, including discontinuous eigenfunctions in bistable systems and basin-of-attraction structure.
  • Compared with polynomial EDMD, the method achieves improved accuracy when expressive representations are required.
  • Future work includes controlled systems, computational-efficiency improvements, and robustness to noise and finite-sample effects.
Loading 2608.26943v1…