Source-linked AI summary
Variational Encoding of Complex Dynamics
Carlos X. Hernández, Hannah K. Wayment-Steele, Mohammad M. Sultan, Brooke E. Husic, Vijay S. Pande
TL;DR
High-dimensional biophysical time series are difficult to interpret, and linearity assumptions can limit their compression of nonlinear dynamics. The paper introduces a time-lagged VAE, the VDE, with saliency mapping, and reports improved slow-dynamics representation across Brownian dynamics and protein folding while identifying important scope limitations.
Problem
High-dimensional molecular simulations require interpretable and statistically robust extraction of dynamics, while linear methods can limit representation of nonlinear processes.
Method
The VDE extends a variational autoencoder with a time lag, nonlinear one-dimensional encoding, autocorrelation training, and saliency mapping for feature interpretation.
Results
The VDE outperforms tICA in describing slow dynamics in the 2-D Müller-Brown potential and protein folding, with a Brownian-dynamics GMRQ of 1.8580±5×10−4 versus 1.8460±5×10−4 for tICA.
Takeaways & Limitations
VDE embeddings can combine nonlinear feature combinations with interpretability while capturing nontrivial Brownian and protein-folding dynamics.
Takeaways & Limitations
The implemented VDE cannot reliably capture thermodynamics at differing temperatures, and autocorrelation-based training depends noticeably on finite batch size.
Abstract
from arXiv · showhide
Often the analysis of time-dependent chemical and biophysical systems produces high-dimensional time-series data for which it can be difficult to interpret which individual features are most salient. While recent work from our group and others has demonstrated the utility of time-lagged co-variate models to study such systems, linearity assumptions can limit the compression of inherently nonlinear dynamics into just a few characteristic components. Recent work in the field of deep learning has led to the development of variational autoencoders (VAE), which are able to compress complex datasets into simpler manifolds. We present the use of a time-lagged VAE, or variational dynamics encoder (VDE), to reduce complex, nonlinear processes to a single embedding with high fidelity to the underlying dynamics. We demonstrate how the VDE is able to capture nontrivial dynamics in a variety of examples, including Brownian dynamics and atomistic protein folding. Additionally, we demonstrate a method for analyzing the VDE model, inspired by saliency mapping, to determine what features are selected by the VDE model to describe dynamics. The VDE presents an important step in applying techniques from deep learning to more accurately model and interpret complex biophysics.
1 Introduction
High-dimensional molecular simulations require robust ways to extract meaningful dynamics, but existing dimensionality-reduction methods trade nonlinear modeling capacity against interpretability. The VDE addresses this tension by combining time-lagged variational encoding with saliency analysis.
- Thousands of molecular degrees of freedom make meaningful and statistically robust extraction of underlying dynamical processes essential.
- Existing dynamical methods must balance nonlinear representation of complex dynamics with interpretable feature importance.
- Kernel methods capture more nonlinear effects than linear methods, but their implicit coordinates remain difficult to assign biophysical meaning.
- Autoencoders learn low-dimensional representations of high-dimensional data, while VAEs regularize the latent space with Gaussian noise.
- The VDE introduces a time lag into a VAE to approximate a time-series propagator and combines nonlinear modeling with saliency-based interpretability.
2 Model: Variational Dynamics Encoder (VDE)
The VDE predicts a future system state from current features through a nonlinear encoder, stochastic latent layer, and decoder. Its training combines reconstruction and autocorrelation objectives to preserve one-lag fidelity while emphasizing long-timescale kinetics.
- 2.1 VDE Architecture: The VDE feeds features at time t into a network that predicts the system state at a user-selected Markovian lag t + τ.
- 2.1 VDE Architecture: A nonlinear DNN encoder compresses inputs into a one-dimensional latent coordinate, whose Gaussian perturbation is decoded by a mirrored DNN into the predicted future state.
- 2.1 VDE Architecture: After training, the encoder maps features to latent coordinates for dimensionality reduction, while the full network can iteratively generate synthetic trajectories.
- 2.2 VDE Loss Function: The total VDE loss combines reconstruction and autocorrelation losses.
- 2.2 VDE Loss Function: Reconstruction loss measures prediction of the state at t + τ and balances fidelity against Gaussian-prior regularization through a Kullback–Leibler term.
- 2.2 VDE Loss Function: Autocorrelation loss targets a more complete representation of long-timescale kinetics, using latent-space autocorrelation within the variational training procedure.
3 Results
The VDE provides nonlinear low-dimensional representations that capture complex basins and folding states more effectively than linear methods, while saliency maps improve interpretability. Its generative behavior reproduces some thermodynamic trends but has important limitations in transition and poorly sampled regions.
- 3.1 A Non-Linear Encoding for Brownian Dynamics: The nonlinear VDE separates Müller-Brown basins and the intermediate state more clearly than the strictly linear tICA and PCA coordinates.tICA and PCA approximate diffusion from region 1 to region 3, whereas the VDE maps regions 1, 2, and 3 nonlinearly.
- 3.1 A Non-Linear Encoding for Brownian Dynamics: 1.8580±5×10−4 versus 1.8460±5×10−4 and 1.8472±5×10−4: VDE achieves the highest held-out GMRQ among VDE, tICA, and PCA.The comparison uses identical MSM hyperparameters and the GMRQ of the slowest process.
- 3.2 The VDE Does Not Behave as a True Propagator: Adjusting Λ-layer scaling α can mimic temperature-dependent barrier changes and generate synthetic trajectories, but transition and unobserved phase-space thermodynamics are reproduced poorly.The selected α values were evenly sampled rather than rigorously fitted, and α = 0 behaves essentially as a basin indicator function.
- 3.3 A Simple Encoding for Villin Headpiece Dynamics: The VDE distinguishes villin’s folded, unfolded, and misfolded states, with a narrower folding free-energy basin than the first tICA coordinate.The first two tICA coordinates are needed to capture folding and prominent misfolding, while the VDE latent coordinate discriminates all three states.
- 3.3 A Simple Encoding for Villin Headpiece Dynamics: Removing the autocorrelation loss produces minimal differentiation across the villin landscape, highlighting the loss’s necessity for the VDE model.The comparison trains villin models using reconstruction loss without autocorrelation loss.
- 3.3 A Simple Encoding for Villin Headpiece Dynamics: 1620±80 nanoseconds versus 770±40 nanoseconds: VDE-based MSMs identify a slower villin process than optimized tICA, consistent with closer modeling of true dynamics.The reported timescales are for the slowest processes in MSMs built from the VDE projection and optimized tICA model, respectively.
4 Discussion
The VDE combines time-lagged variational encoding, autocorrelation training, latent-space noise, and saliency mapping to model and interpret complex dynamics. It outperforms tICA in the studied systems, while batch-size dependence and similar-timescale processes limit replacement of established methods.
- The VDE combines a time lag, autocorrelation loss, latent Gaussian noise, and saliency mapping to encode dynamics and identify features contributing to the reaction coordinate.
- The VDE outperforms tICA in describing slow dynamics in both the 2-D Müller-Brown potential and protein folding.
- The model captures folded, unfolded, and misfolded villin states, and its MSM projection yields a significantly longer slowest timescale than optimized tICA data.
- For the Müller-Brown system, the VDE generates realistic thermodynamics and shows some ability to extrapolate dynamics to unobserved temperatures, although this analysis is not quantitatively rigorous.
- Finite batches can underestimate global equilibrium statistics, while similar-timescale processes can make autocorrelation-loss training volatile.
- The authors frame VDEs as a route toward combining neural-network expressiveness with Markov-state-model theory for biophysical analysis.
5 Methods
The methods generate Müller-Brown and villin trajectories, train VDEs under specified lagged-network settings, compare projections through MSM construction, and adapt saliency mapping to protein transitions.
- 5.1 Müller-Brown Potential: Ten independent Müller-Brown simulations use Euler-Maruyama integration with a 0.1 time step, 10^6 steps, saving every 100 steps.Initial positions were uniformly sampled from [−1.5,1.2]×[−0.2,2.0].
- 5.1 Müller-Brown Potential: Müller-Brown VDEs use a 10-step lag, three 256-node Swish layers, α=10^-3, batch size 100, 30% dropout, and learning rate 1×10^-4.Models were trained with Adam for 50 epochs until losses converged; parameters were not automated-hyperparameter optimized.
- 5.2 Markov State Models: The study compares PCA, tICA, and pretrained VDE one-dimensional projections before clustering each representation into twelve states for MSM construction.
- 5.1 Müller-Brown Potential: Synthetic VDE trajectories begin from uniformly sampled positions and are propagated for 1,000 VDE steps across five logarithmically spaced α values.The α sweep probes how the Λ-layer affects propagation.
- 5.2 Protein Folding: Villin is represented by Cα contacts, and its VDE uses a 44 ns lag selected to match the optimal tICA lag.The data derive from simulations of the 35-residue villin headpiece domain.
- 5.3 Protein Saliency Maps: Protein saliency maps back-propagate the residual from a faux misfolded-to-folded transition, then aggregate feature scores into residue-level importance rankings.The method differs from classical saliency because it propagates a transition residual rather than only a class-label score.
Disclosures
The disclosure identifies the author’s consulting, advisory, board, and investment relationships with several companies.
- The author reports consulting and scientific-advisory roles at Schrodinger, LLC and Globavir, alongside board memberships and a General Partner role at Andreessen Horowitz.