Source-linked AI summary
Nonlinear Discovery of Slow Molecular Modes using State-Free Reversible VAMPnets
Wei Chen, Hythem Sidky, Andrew L Ferguson
TL;DR
Recovering slow collective variables from molecular trajectories is difficult, although they are central to describing long-time dynamics and enhanced sampling. SRVs use neural networks within VAC to learn nonlinear approximations to leading slow eigenfunctions. Across the reported comparisons, SRVs agree with theoretical eigenfunctions and match MSM timescales where kTICA underestimates the slowest mode.
Problem
Slow collective variables are difficult to intuit for complex molecular systems, despite their importance for representing long-time dynamics and enhanced sampling.
Source: §I · p. 1Method
SRVs use neural networks within VAC to learn nonlinear input representations for approximating leading transfer-operator eigenfunctions.
Source: §I · p. 2, §IV · p. 16Results
SRVs agree with theoretical eigenfunctions more closely than kTICA in the ring-potential example, and nearly match MSM implied timescales for alanine dipeptide while kTICA underestimates its slowest mode.
Source: B. Ring potential · p. 10, C. Alanine dipeptide · p. 12Takeaways & Limitations
SRVs provide continuous, explicit eigenfunction approximations that can be used as differentiable collective variables for enhanced sampling.
Source: §I · p. 2, §IV · p. 16
Abstract
from arXiv · showhide
The success of enhanced sampling molecular simulations that accelerate along collective variables (CVs) is predicated on the availability of variables coincident with the slow collective motions governing the long-time conformational dynamics of a system. It is challenging to intuit these slow CVs for all but the simplest molecular systems, and their data-driven discovery directly from molecular simulation trajectories has been a central focus of the molecular simulation community to both unveil the important physical mechanisms and to drive enhanced sampling. In this work, we introduce state-free reversible VAMPnets (SRV) as a deep learning architecture that learns nonlinear CV approximants to the leading slow eigenfunctions of the spectral decomposition of the transfer operator that evolves equilibrium-scaled probability distributions through time. Orthogonality of the learned CVs is naturally imposed within network training without added regularization. The CVs are inherently explicit and differentiable functions of the input coordinates making them well-suited to use in enhanced sampling calculations. We demonstrate the utility of SRVs in capturing parsimonious nonlinear representations of complex system dynamics in applications to 1D and 2D toy systems where the true eigenfunctions are exactly calculable and to molecular dynamics simulations of alanine dipeptide and the WW domain protein.
I. INTRODUCTION
Molecular simulations need low-dimensional representations of slow dynamics, but common approaches face limits in nonlinearity, scalability, or differentiable CVs. SRVs use neural networks within VAC to estimate leading transfer-operator eigenfunctions, while the transfer-operator framework describes how equilibrium-scaled distributions evolve over time.
I. INTRODUCTION
- TICA estimates linearly optimal slow modes from mean-free coordinates, while kTICA extends this approach nonlinearly but has costly scaling, sensitive hyperparameters, and gradient limitations.For kTICA, the stated costs are O(N^2) memory and O(N^3) computation; exact gradients require kernels against training points.Source: §I · p. 1
- Neural networks offer nonlinear representations, training that scales linearly with dataset size, and exact derivatives through automatic differentiation, while VAE approaches described here estimate only leading eigenfunctions.The cited limitation of estimating only leading eigenfunctions restricts those approaches to a single learned coordinate for enhanced sampling.Source: §I · pp. 1–2
- SRVs learn nonlinear input featurizations for VAC estimation of leading transfer-operator eigenfunctions, rather than directly partitioning state space.The learned functions are intended as continuous representations of the slow modes.Source: §I · p. 2
- Unlike VAMPnets’ softmax state assignments, SRVs use linear or nonlinear output activations and target reversible VAC eigenfunctions, with clustering available afterward.The passage reports successful recovery in 70% of SRV runs versus 29% of VAMPnet runs in 100-network alanine-dipeptide experiments.Source: §I · p. 2
A. Transfer operator theory
- The transfer operator evolves equilibrium-scaled distributions through a transition density; under the Markovian assumption, that density is time-homogeneous, and equilibrium detailed balance makes the operator self-adjoint.Its eigenvalues characterize temporal autocorrelation, with smaller positive eigenvalues decaying faster.Source: A. Transfer operator theory · p. 3
B. Variational approach to conformational dynamics (VAC)
VAC approximates leading transfer-operator eigenfunctions by optimizing a variational objective over candidate functions while enforcing orthogonality. With an arbitrary input basis, the resulting approximations can be expressed as linear expansions and solved as a generalized eigenvalue problem.
B. Variational approach to conformational dynamics (VAC)
- VAC approximates the first non-trivial eigenfunction by maximizing its estimated eigenvalue subject to orthogonality to the trivial equilibrium eigenfunction.The variational principle bounds the estimate from above by λ1.Source: B. Variational approach to… · p. 4
- Higher-order eigenfunctions are approximated by extending the maximization procedure under orthogonality constraints, yielding an orthonormal set of approximations.The leading eigenfunctions correspond to the largest eigenvalues and longest implied timescales.Source: B. Variational approach to… · pp. 3–4
- For an arbitrary input basis, VAC represents eigenfunction approximations as linear expansions and leads to a generalized eigenvalue problem whose solutions give the best linear estimates in that basis.The expansion coefficients are eigenvectors, and the associated solutions can be obtained with standard techniques.Source: B. Variational approach to… · p. 4
C. Estimation of VAC equations from trajectory data
The section describes estimating the central VAC quantities from empirical trajectory data. It also estimates the components of the generalized eigenvalue problem from those data.
C. Estimation of VAC equations from trajectory data
- The numerator and denominator of the VAC objective are estimated from a trajectory, producing an empirical form of the objective.The supplied passages identify the two components and their trajectory-based estimation but do not provide the expressions.Source: C. Estimation of VAC equations from… · p. 4
- The generalized eigenvalue problem’s components are likewise estimated from trajectory data.This connects empirical trajectory observations to the VAC eigenvalue calculation.Source: C. Estimation of VAC equations from… · p. 4
D. Time-lagged independent component analysis (TICA)
TICA restricts each approximate eigenfunction to a linear combination of molecular coordinates. Centering those coordinates yields the mean-free representation, and the resulting objective and orthogonality constraints are those of TICA.
D. Time-lagged independent component analysis (TICA)
- TICA represents each approximate eigenfunction as a linear combination of molecular coordinates with an additive constant.Centering removes the constant and gives a linear function of mean-free coordinates.Source: D. Time-lagged independent component… · p. 5
- With mean-free coordinates, the VAC objective and orthogonality constraints become exactly those of TICA.The orthogonality relation is expressed through expectations of products of projected mean-free coordinates.Source: D. Time-lagged independent component… · p. 5
E. Kernel TICA (kTICA)
Kernel TICA (kTICA) extends TICA to nonlinear features by replacing coordinate inner products with a kernel. Its scalability, hyperparameter sensitivity, and lack of an explicit differentiable mapping limit its use, while landmark kTICA reduces computational costs.
E. Kernel TICA (kTICA)
- kTICA replaces linear inner products in TICA’s objective and constraints with a symmetric nonlinear kernel, yielding a generalized eigenvalue problem.The kernel defines feature-space similarity without explicitly constructing the nonlinear feature map.Source: E. Kernel TICA (kTICA) · pp. 5–6
- The full kTICA Gram matrix requires O(N^2) time and memory, and its generalized eigenvalue solve takes O(N^3) time.Results also depend on kernel choice and hyperparameters; kTICA does not provide an explicit differentiable mapping for enhanced sampling.Source: E. Kernel TICA (kTICA) · p. 6
- Landmark kTICA uses m << N landmarks to compute an m-by-m Gram matrix and applies the Nyström approximation to estimate the full matrix.This reduces computation time and memory usage relative to the full Gram matrix.Source: E. Kernel TICA (kTICA) · p. 6
F. State-free reversible VAMPnets (SRVs)
SRVs use a neural network to learn nonlinear features and apply linear VAC to estimate orthonormal approximations of the slow transfer-operator eigenfunctions. The resulting mapping is explicit and differentiable, while the training approach avoids kernel selection and is reported to be efficient and robust.
F. State-free reversible VAMPnets (SRVs)
- SRVs learn nonlinear features f(x), then apply linear VAC to obtain an orthonormal basis of approximated slow eigenfunctions.The eigenfunctions are linear combinations of network outputs, rather than direct network outputs.Source: F. State-free reversible VAMPnets (SRVs) · pp. 6–7
- The method trains network parameters and linear expansion coefficients using gradient descent, with automatic differentiation used to calculate gradients.Training uses paired configurations and validation-based early stopping.Source: F. State-free reversible VAMPnets (SRVs) · pp. 6–7
- The learned eigenfunction mapping is explicit and differentiable, supporting its use in enhanced sampling.A new configuration passes through the network and then a learned linear transformation.Source: F. State-free reversible VAMPnets (SRVs) · p. 7
- Training minimizes a loss based on learned eigenvalues, with the choice g(λ̃) = −λ̃^2 corresponding to the VAMP-2 score.The VAMP-2 score measures kinetic variance and reaches its maximum when the eigenfunction approximations are exact.Source: F. State-free reversible VAMPnets (SRVs) · pp. 6–8
- The SRV protocol uses O(N) memory and computation and does not require kernel selection or its associated hyperparameter tuning.The authors report that training is largely insensitive to network architecture, hyperparameters, and activation functions.Source: F. State-free reversible VAMPnets (SRVs) · p. 8
- The same hidden-layer sizes and activation functions were used in all four applications, which the authors present as evidence of robustness and generalizability.The authors note that degenerate or nearly degenerate eigenvalues can create numerical difficulties in resolving eigenfunctions.Source: F. State-free reversible VAMPnets (SRVs) · p. 8
A. 1D 4-well potential
In the 1D four-well example, SRV and kTICA recover eigenfunctions in excellent agreement with theoretical results. The example uses a discretized transition model with exactly calculable eigenfunctions to assess the learned modes.
A. 1D 4-well potential
- The top four theoretical eigenfunctions include a stationary mode and three modes corresponding to transitions over the three energy barriers.They are calculated from the transition matrix at lag time τ = 100.Source: A. 1D 4-well potential · pp. 8–9, Figure 3 · p. 9
- The authors generate a 5,000,000-step trajectory and learn the top three non-trivial eigenfunctions with kTICA and SRV at lag time τ = 100.For kTICA, they use 200 K-means landmarks and a Gaussian kernel with σ = 0.05; SRV processes all data points.Source: A. 1D 4-well potential · p. 9
- Both SRV and kTICA agree excellently with theoretical eigenfunctions, with small timescale deviations attributed to noise in the simulated data.The authors describe the SRV result as quantitatively agreeing with theory and corresponding excellently with kTICA.Source: A. 1D 4-well potential · p. 9, Figure 4 · p. 9
B. Ring potential
On the 2D ring potential, SRV agrees excellently with theoretical eigenfunctions without tuning the network architecture, activations, or loss, and its eigenfunctions agree better with theory than kTICA’s. The kTICA approximation is poorer outside the densely populated ring and depends on landmark count and kernel bandwidth.
B. Ring potential
- The 2D ring potential has four barriers of 1.0, 1.3, 0.5, and 8.0 kBT, and the region of interest is discretized into 50-by-50 bins.The methods use lag time τ = 100 for theoretical analysis and model learning.Source: B. Ring potential · pp. 9–10
- kTICA approximates eigenfunctions reasonably within the densely populated ring but performs worse outside it, with results sensitive to landmark count and kernel bandwidth.The authors report that satisfactory results require many landmarks and a narrow range of bandwidth values, requiring substantial tuning.Source: B. Ring potential · p. 10
- SRV eigenfunctions agree better with theory than kTICA’s, and SRV shows excellent theoretical agreement without tuning architecture, activations, or loss.The SRV eigenvalues closely match those extracted by kTICA.Source: B. Ring potential · p. 10
C. Alanine dipeptide
For alanine dipeptide, SRV eigenfunctions agree with kTICA and MSM results, while SRV timescales more closely match the MSM benchmark. SRV timescales are also robust across lag times, and the Chapman–Kolmogorov test supports Markovian behavior at τ ≳10 ps.
C. Alanine dipeptide
- Using 45 heavy-atom pairwise distances, kTICA’s three leading modes capture transitions along φ, between α and (β, P//), and between αL and γ basins.Modes above the third could not be resolved at τ = 20 ps because their timescales were below the lag time.Source: C. Alanine dipeptide · p. 12
- SRV eigenfunctions agree with kTICA, but its leading-mode implied timescale is 17% slower.The SRV and kTICA analysis used the same lag time and pairwise-distance features.Source: C. Alanine dipeptide · p. 12
- SRV nearly matches MSM implied timescales, whereas kTICA substantially underestimates the slowest mode.The three methods’ eigenfunctions are in excellent accord.Source: C. Alanine dipeptide · p. 12
- Landmark kTICA can miss slow modes when feature-space Euclidean distances poorly track kinetic distances; substantially more than about 5,000 landmarks is computationally difficult.Backbone dihedrals improved the leading-mode timescale estimate to t1 = 1602 ps, but good features are generally not known a priori.Source: C. Alanine dipeptide · p. 12
- SRV implied timescales are robust across τ = 10, 40, and 100 ps, and predictions from τ = 20 ps satisfy the Chapman–Kolmogorov test, supporting Markovian behavior at τ ≳10 ps.The test compares predicted timescales against those directly computed from SRVs built at other lag times.Source: C. Alanine dipeptide · pp. 12, 14
D. WW domain
On the WW domain, SRV and TICA-based MSM agree closely on the two leading slow modes, while SRV timescales converge substantially faster with lag time, especially for the second mode.
D. WW domain
- SRV and TICA-based MSM eigenfunctions correlate at ρ = 0.99 and 0.98 for the top two modes, and their inferred implied timescales are also in good agreement.The comparison uses a 1137 µs trajectory of the 35-residue WW domain and focuses on the two slowest modes; higher-order modes near or below the 400 ns lag time are not reliably resolved.Source: D. WW domain · p. 14
- SRV implied timescales converge substantially faster with lag time than those of the TICA-based MSM, particularly for the second eigenfunction.The authors attribute the advantage to nonlinear relationships captured in a continuous state representation, rather than the MSM's linear TICA coordinates and discretized microstate transition matrix.Source: D. WW domain · p. 14
IV. CONCLUSIONS
The conclusions present SRV as a scalable neural-network framework for learning a hierarchy of nonlinear slow modes. Its explicit, differentiable coordinates and flexible resolution support molecular-dynamics analysis and enhanced-sampling applications.
IV. CONCLUSIONS
- SRV learns a hierarchy of nonlinear slow modes from trajectory data using a neural network to form an optimal nonlinear basis.The framework is built on transfer-operator theory.Source: §IV · p. 15
- SRV can learn an arbitrary number of eigenfunctions while guaranteeing orthogonality, with O(N) memory and computation.The authors report that the network does not require kernel selection or hyperparameter adjustment and that training is insensitive to batch size, learning rate, and architecture.Source: §IV · p. 16
- The parametric SRV gives an explicit, differentiable mapping from configurations to leading transfer-operator eigenvector approximations, making its slow CVs suitable for enhanced sampling.The authors identify metadynamics, umbrella sampling, and adaptive biasing force as applicable techniques.Source: §IV · p. 16
- The authors anticipate SRVs may help resolve long-time conformational changes and direct adaptive sampling toward slow conformational pathways.They also propose replacing TICA-based MSMs with SRV-based MSMs while retaining established MSM construction practices.Source: §IV · p. 16