Source-linked AI summary
Data-driven model reduction and transfer operator approximation
Stefan Klus, Feliks Nüske, Péter Koltai, Hao Wu, Ioannis Kevrekidis, Christof Schütte, Frank Noé
TL;DR
High-dimensional dynamical systems motivate data-driven methods that reduce dimension and approximate transfer operators from data. The review connects TICA, DMD, VAC, EDMD, and related approaches through their shared operator-theoretic structure. It concludes that these methods have similar algorithms despite different derivations, while accuracy and feasibility depend on basis complexity, lag time, and problem structure.
Problem
Transfer-operator analysis of complex high-dimensional systems can be computationally infeasible because of the curse of dimensionality.
Method
The review synthesizes data-driven model-reduction and transfer-operator approximation methods, including TICA, DMD, VAC, and EDMD, and relates their derivations and algorithms.
Results
The reviewed methods have different derivations but produce algorithms with many similarities and can compute transfer-operator eigenvalues, eigenfunctions, and eigenmodes.
Takeaways & Limitations
Extensions, generalizations, and algorithms developed for one method can be carried over to related methods, yielding a unified theory and set of tools.
Takeaways & Limitations
VAC and EDMD can require multiple basis functions across coordinates and their combinations for accurate eigenfunction approximations, which may be infeasible.
Abstract
from arXiv · showhide
In this review paper, we will present different data-driven dimension reduction techniques for dynamical systems that are based on transfer operator theory as well as methods to approximate transfer operators and their eigenvalues, eigenfunctions, and eigenmodes. The goal is to point out similarities and differences between methods developed independently by the dynamical systems, fluid dynamics, and molecular dynamics communities such as time-lagged independent component analysis (TICA), dynamic mode decomposition (DMD), and their respective generalizations. As a result, extensions and best practices developed for one particular method can be carried over to other related methods.
1 Introduction
The review addresses computational challenges in analyzing high-dimensional, multiscale dynamical systems by connecting data-driven reduction and transfer-operator approximation methods. It seeks a unified theory showing how related methods can share algorithms, extensions, and best practices.
- Motivation: High-dimensional, multiscale systems make transfer-operator analysis computationally infeasible or prohibitively expensive because of the curse of dimensionality.The paper motivates projecting high-dimensional dynamics onto lower-dimensional spaces.
- Data-driven methods: TICA and DMD identify dominant dynamics using simulation or experimental data.
- Method relationships: TICA, DMD, VAC, and EDMD are related to Koopman-operator approximation and can compute transfer-operator spectral quantities.VAC and EDMD target eigenfunctions, eigenvalues, and eigenmodes of Koopman and adjoint Perron–Frobenius operators.
- Method relationships: Despite different derivations and applications, these methods produce algorithms with many similarities.
- Paper scope: The paper’s organization covers transfer operators and reversibility, data-driven spectral approximation, theoretical derivations, and open problems.
2 Transfer operators and reversibility
Transfer operators describe how densities and observables evolve in stochastic or deterministic dynamical systems, while their invariant densities and spectral components characterize long-term behavior. The section introduces these operators through the Ornstein–Uhlenbeck process and emphasizes equilibrium, adjointness, and lag-time-dependent spectra.
- Guiding example: The Ornstein–Uhlenbeck process illustrates relaxation from localized distributions toward a stationary Gaussian equilibrium density.
- Guiding example: The transition density pτ(x,y) is the conditional density of finding X_t+τ = y given X_t = x.
- Operator properties: Pτ and Kτ are adjoint linear operators and satisfy semigroup relations such as Pτ+σ = PτPσ and Kτ+σ = KτKσ.
- Transfer operators: The Perron–Frobenius operator evolves probability densities, whereas the Koopman operator evolves observables.
- Equilibrium: A density π is invariant when Pτπ = π, making it an eigenfunction of the Perron–Frobenius operator with eigenvalue 1.
- Spectral decomposition: Transfer-operator eigenvalues depend on lag time τ, and eigenvalues and eigenfunctions encode metastable sets and fast or slow processes.
3 Data-driven approximation of transfer operators
The data-driven methods use time-lagged data pairs to approximate transfer-operator spectra when the underlying dynamics may be unknown. VAC and EDMD additionally represent observables in a chosen basis whose suitability remains system-dependent.
- Overview: TICA, DMD, VAC, and EDMD identify dominant dynamics and approximate transfer-operator eigenfunctions from data.
- Time-lagged data: Data consist of paired vectors x_i = X_ti and y_i = X_ti+τ, which may come from measurements or black-box simulations.
- Time-lagged data: For a single trajectory, Y is X shifted by lag time τ; data from multiple trajectories can be concatenated.
- Basis representation: VAC and EDMD require uniformly bounded basis functions, including possible monomials, indicator functions, radial basis functions, or trigonometric functions.
- Basis representation: The optimal basis-function choice remains open and depends strongly on the dynamical system.
- Basis representation: The approximation goal is to find the best representation of a transfer operator in the span of the selected basis functions.
3.1 Time-lagged independent component analysis
TICA extracts slow dynamical directions from time-lagged correlations and is a linear special case of the variational approach. Its practical estimation uses covariance-based eigenproblems, with symmetrization improving real-valued coordinates but potentially introducing bias away from equilibrium.
- Concept: TICA was introduced for blind source separation and later understood as a VAC special case for approximating slowest relaxation processes.
- Estimation: Time-lagged independent components solve a covariance-based eigenvalue problem involving estimators C0 and Cτ.
- Estimation: Symmetrized covariance estimators make TICA coordinates real-valued by averaging over a trajectory and its time reversal.
- Caveat: Symmetrization can strongly bias the dominant spectrum for nonstationary data or distributions far from equilibrium.Reweighting can reduce this bias when the data distribution is far from equilibrium.
- TICA versus PCA: TICA uses time-lagged covariance information to identify a slow direction that PCA can miss.In the illustrated system, projection onto x preserves the slow process while eliminating fast stochastic noise.
- Algorithm: The AMUSE algorithm computes TICA through reduced SVD, whitening, and an eigenproblem for the transformed time-lagged covariance matrix.
3.2 Dynamic mode decomposition
DMD identifies coherent structures in fluid flows and can be interpreted through Koopman operator theory. Its close algebraic relationship with TICA becomes clear through their shared projections and transposed eigenproblems.
- DMD was developed to identify coherent structures in fluid flows and can be viewed as combining spatial PCA with frequency-domain Fourier analysis.
- DMD eigenvalues and modes are obtained from the eigenvalues and eigenvectors of the finite-dimensional matrix M_DMD.
- TICA coordinates are left eigenvectors of the DMD matrix, whereas DMD modes are left eigenvectors of the TICA matrix.
- The standard DMD algorithm computes a compact SVD of X before forming and eigendecomposing the reduced matrix.
- TICA and standard DMD use the same projection apart from the scaling Σ^-1, but compute eigenvectors of transposed matrices.
3.3 Variational approach of conformation dynamics
VAC generalizes Markov state modeling by allowing arbitrary basis functions and approximates transfer-operator quantities through a finite-dimensional matrix. It applies only to reversible systems, unlike EDMD.
- VAC generalizes Markov state modeling by allowing arbitrary basis functions, also called a dictionary.
- VAC and EDMD require data together with a chosen set of basis functions ψ.
- VAC is defined only for reversible systems, whereas EDMD does not require reversibility.
- The matrix M_VAC is a finite-dimensional approximation of the Koopman operator, or the Perron–Frobenius operator in reversible systems.
- VAC approximates operator eigenfunctions using the eigenvectors of M_VAC.
3.4 Extended dynamic mode decomposition
EDMD generalizes DMD by fitting linear relations in a basis-function-transformed space and approximating Koopman and Perron–Frobenius operator quantities. Its computed eigenfunctions coincide with VAC’s for the same data and basis.
- EDMD generalizes DMD to compute finite-dimensional approximations of Koopman eigenvalues, eigenfunctions, and eigenmodes.
- EDMD fits a linear relationship between transformed data matrices ΨX and ΨY rather than between the original data matrices X and Y.
- EDMD Koopman eigenfunctions are represented by the left eigenvectors of M_EDMD.
- VAC and EDMD produce exactly the same eigenvalue and eigenfunction approximations for a given dataset and basis set.
- EDMD can also approximate Perron–Frobenius eigenfunctions, whose interpretation depends on the density underlying the data.Uniformly sampled data yield eigenfunctions relative to Pτ, while a long trajectory approaches the equilibrium-density operator Tτ.
- For the Ornstein–Uhlenbeck example, EDMD used 10^5 uniform test points and monomials through order 10 to compute dominant operator eigenfunctions.The reported eigenvalues include λ1(τ) = 1.00, λ2(τ) = 0.37, λ3(τ) = 0.13, and λ4(τ) = 0.049.
3.5 Relationships with other methods
VAC and EDMD connect several established data-driven methods through their choice of basis functions. Linear bases recover TICA and DMD, while indicator bases recover Ulam’s method and Markov state models.
- VAC and EDMD become TICA and DMD, respectively, when the basis functions are chosen as ψ(x) = x.
- With linear basis functions, TICA coordinates approximate projected Koopman eigenfunctions, while DMD modes are the corresponding Koopman modes.
- For reversible processes, subtracting the mean removes the stationary eigenpair λ1 = 1, leaving the largest eigenpair to approximate the slowest dynamics.
- Indicator-function bases make VAC and EDMD equivalent to Ulam’s method and therefore to a Markov state model.
- These relationships are summarized in Figure 4, with detailed VAC and EDMD examples reported elsewhere.
3.6 Examples
The examples apply transfer-operator approximations to the double gyre and molecular-dynamics data, using EDMD and a multistage TICA/MSM workflow to expose dominant and slow dynamics.
- Double gyre: For the double gyre, diffusion creates small transport between nearly invariant subdomains, yielding a second eigenvalue λ2 = 1 − O(ε).At ε = 0, the two subdomains are invariant and produce a two-dimensional eigenspace at eigenvalue 1; for ε > 0, this splits into λ1 = 1 and λ2 = 1 − O(ε).
- Double gyre: EDMD computes leading Koopman eigenfunctions that clearly reveal the double gyre’s two almost invariant sets.The calculation uses radial basis functions centered on a 50 × 25 grid and lag time τ = 3.
- Deca alanine: In the deca-alanine workflow, TICA first projects internal coordinates onto leading eigenvectors, retaining the smallest M explaining more than 95 per cent of total kinetic variance.The selected coordinates are all 16 backbone dihedral angles.
- Deca alanine: The reduced data are clustered into 50 states, followed by an MSM whose implied-timescale estimates improve over TICA and converge for lag times τ ≥ 4 ns.The MSM is equivalent to Ulam’s method in this workflow.
- Deca alanine: At τ = 4 ns, PCCA+ coarse-grains the MSM into two macrostates associated with the slow process of helix formation.The assignment shown is by maximal fuzzy-membership value, although hidden Markov models are an alternative.
4 Derivations
The paper derives data-driven approximations of transfer-operator spectra by projecting infinite-dimensional operators onto finite-dimensional basis spaces. In reversible systems, a variational principle selects functions maximizing the Rayleigh trace, while EDMD provides a Petrov–Galerkin Koopman approximation without requiring reversibility.
- VAC and EDMD, including TICA and DMD as special cases, connect data-driven methods to transfer-operator eigenvalues, eigenfunctions, and eigenmodes.
- EDMD approximates the Koopman operator by minimizing prediction error over linear mappings on a finite-dimensional ansatz space.
- The derivation assumes a Feller-continuous process, piecewise continuous basis functions, weak convergence of empirical data to ρ, and independent sampling and noise processes.
- The EDMD matrix approximates the Koopman operator, whereas the corresponding Perron–Frobenius matrix is not generally its transpose unless the basis functions are density-orthonormal.
- For reversible systems, the sum of the first M Koopman eigenvalues maximizes the Rayleigh trace over M orthonormal functions, attained by the first M eigenfunctions.
- The variational approach restricts this optimization to a fixed dictionary of linearly independent ansatz functions and solves the resulting generalized eigenvalue problem.
5 Conclusion
The review connects data-driven model-reduction and transfer-operator approximation methods developed across several communities, showing that their differing derivations yield similar algorithms. It also identifies accuracy–tractability limits and iterative, tensor, and sparsity-based strategies for addressing them.
- The review establishes connections among methods from dynamical systems, fluid dynamics, machine learning, and molecular dynamics, whose algorithms share many similarities despite different derivations.
- DMD, TICA, and MSMs provide relatively rough approximations of high-dimensional dynamics, while larger lag times can control errors when only a few spectral components are needed.
- VAC and EDMD better approximate eigenfunctions but require many basis functions across coordinates and combinations, making accurate high-dimensional estimation infeasible and prone to overfitting.
- Iterative dimension reduction, low-rank tensor approximations, and sparsity-promoting methods are proposed to mitigate the computational and statistical difficulties of rich basis representations.