Projected and Hidden Markov Models for calculating kinetics and metastable states of complex molecules

Frank Noe; Hao Wu; Jan-Hendrik Prinz; Nuria Plattner

Source-linked AI summary

Projected and Hidden Markov Models for calculating kinetics and metastable states of complex molecules

Frank Noe, Hao Wu, Jan-Hendrik Prinz, Nuria Plattner

arXiv:1309.3220v1Sep 12, 2013physics.chem-phphysics.bio-phphysics.data-an

TL;DR

MSMs can be biased because projected cluster dynamics are not actually Markovian, limiting reliable long-timescale prediction. The paper introduces PMMs and approximates them with HMMs, which recover slow kinetics under metastability and timescale separation while supporting observables commonly computed from MSMs.

Problem
MSMs assume Markovian dynamics on discrete clusters, although projected molecular dynamics are generally not Markovian, making unbiased long-timescale prediction difficult.
Method
The paper formulates projected Markov models and approximates them using hidden Markov models with metastable hidden states, transition dynamics, stationary distributions, and emission probabilities.
Results
HMMs robustly estimate slow molecular kinetics in metastable systems and can compute many thermodynamic, kinetic, mechanistic, and experimental observables commonly obtained from MSMs.
Takeaways & Limitations
PMMs and HMMs provide a framework for analyzing projected molecular dynamics without imposing the MSM Markovianity assumption on discrete clusters.
Takeaways & Limitations
Using too many hidden states can introduce random eigenvectors when the data contain no corresponding relaxation process, and uncertainty estimates require future Bayesian extensions.

Abstract

from arXiv · show

Markov state models (MSMs) have been successful in computing metastable states, slow relaxation timescales and associated structural changes, and stationary or kinetic experimental observables of complex molecules from large amounts of molecular dynamics simulation data. However, MSMs approximate the true dynamics by assuming a Markov chain on a clusters discretization of the state space. This approximation is difficult to make for high-dimensional biomolecular systems, and the quality and reproducibility of MSMs has therefore been limited. Here, we discard the assumption that dynamics are Markovian on the discrete clusters. Instead, we only assume that the full phase- space molecular dynamics is Markovian, and a projection of this full dynamics is observed on the discrete states, leading to the concept of Projected Markov Models (PMMs). Robust estimation methods for PMMs are not yet available, but we derive a practically feasible approximation via Hidden Markov Models (HMMs). It is shown how various molecular observables of interest that are often computed from MSMs can be computed from HMMs / PMMs. The new framework is applicable to both, simulation and single-molecule experimental data. We demonstrate its versatility by applications to educative model systems, an 1 ms Anton MD simulation of the BPTI protein, and an optical tweezer force probe trajectory of an RNA hairpin.

I. PROJECTED MARKOV MODELS

Projected Markov Models describe cluster-observed dynamics by projecting an underlying Markovian full-phase-space process, avoiding the assumption that cluster dynamics themselves are Markovian. This framework explains why lag-time transition matrices from MSMs may fail to predict long-time behavior.

The molecular dynamics are modeled as an ergodic, reversible Markov process with a unique stationary distribution.The state may include positions and velocities, and the stationary distribution can be the Boltzmann distribution.
Projected dynamics retain contributions from the slow relaxation processes of the full system when a timescale gap separates them from faster processes.The framework assumes m slow processes and considers lag times significantly larger than the mth relaxation timescale.
A coarse partition of molecular state space into clusters produces an observed correlation matrix describing cluster transitions.The partition may use configurations without velocities and can be relatively coarse.
MSM transition matrices estimated at lag time τ cannot generally predict long-timescale behavior because cluster dynamics are not Markovian.The projected eigenvectors are not eigenvectors of the cluster transition matrix, so the relevant orthogonality relation fails.

II. APPROXIMATING PMMS VIA HIDDEN MARKOV MODELS

The paper approximates Projected Markov Models with Hidden Markov Models when dynamics are metastable and separated in timescale. Under these conditions, even a poor discretization can support exact description of metastable dynamics, subject to estimation quality.

HMMs replace difficult estimation of projected eigenfunction matrices with hidden metastable states, transition probabilities, and output distributions over observed clusters.The observed correlation matrix is constructed from hidden-state dynamics and emission probabilities.
A PMM can be represented as an HMM under metastability, timescale separation, and an approximately decomposed stationary distribution.The assumptions include m metastable states, tm+1 ≪ tm, and negligible stationary population in transition regions.
Even with a poor state-space discretization, an HMM can describe metastable dynamics exactly in the stated regime.The practical estimate still depends on available statistics and may depend on discretization quality.
HMM estimation is expected to perform very well in this setting and almost exclusively better than MSMs in the reported applications.

A. Initializing a hidden Markov model from a Markov model

The initialization procedure seeds HMM estimation from a reversible MSM and PCCA+ memberships, then constructs hidden-state quantities that preserve dominant kinetics while enforcing a meaningful starting structure.

A reversible MSM transition matrix T(τ) is first estimated from trajectories discretized into n observed states.The reversible estimator enforces detailed balance before HMM initialization.
PCCA+ supplies a membership matrix M assigning each observed state degrees of membership in m metastable states.Representative observed states are initially assigned membership one to one metastable state and zero to the others.
The PCCA+ initialization can produce negative memberships, which are set to zero and row-renormalized before use.The text identifies PCCA++ as a future alternative that generates non-negative memberships directly.
Memberships are interpreted probabilistically and transformed into hidden-state stationary and output probabilities using coarse-graining and Bayesian statistics.The membership entries represent probabilities of hidden metastable states conditional on observed clusters.
The initialized hidden transition matrix preserves the dominant m eigenvalues of the observed transition matrix through coarse-graining and overlap normalization.The resulting matrix may require symmetrization, removal of negative elements, and row renormalization before optimization.

B. Hidden Markov model estimation

HMM parameters are estimated by maximizing the trajectory likelihood with the Baum-Welch expectation-maximization algorithm. The implementation estimates hidden transition counts and then enforces detailed balance in the transition matrix.

The HMM likelihood sums over hidden trajectories for an observed trajectory.Direct computation is infeasible because the number of hidden paths grows combinatorially.
Baum-Welch expectation-maximization maximizes the HMM likelihood through alternating expectation and maximization steps.The expectation step is general, while the maximization step is adapted to the specific HMM implementation.
The maximization step estimates a hidden-state transition count matrix and a maximum-likelihood transition matrix satisfying detailed balance.The HMM is assumed to be in equilibrium and uses its stationary distribution.

C. Implied timescale plot

Implied timescale plots assess how closely discretized dynamics follow a Markov chain by testing whether relaxation timescales remain constant with lag time. PMM/HMM estimates should instead become lag-time invariant when unresolved faster processes have decayed and statistics are good.

Implied timescale plots assess MSM quality by testing whether discretized dynamics satisfy the Markov-chain Chapman–Kolmogorov condition.For a Markov chain, eigenvalues and relaxation timescales remain consistent across multiples of a base lag time.
Because discretized dynamics are generally non-Markovian, MSM-implied timescales vary with lag time and underestimate true relaxation timescales.The estimation error decays slowly, as τ −1.
PMM validation uses eigenvalues of the hidden transition matrix rather than a discretized-space transition matrix to compute implied timescales.
When τ ≫ t_m+1 and statistics are good, PMM/HMM implied timescales should remain constant across lag times.The condition means unresolved faster processes have decayed; the paper notes that a factor of 23 already gives the required separation.

D. Hidden Markov Model validation

HMM validation selects a lag time where hidden-model relaxation timescales stabilize, then tests whether the model predicts kinetics across other lag times. Agreement with directly estimated MSM kinetics is required for a valid metastable-kinetics description.

Validation predicts discretized transition matrices at multiple lag times and compares their relaxation timescales with directly estimated MSM values.
The relaxation-timescale test must succeed for the estimated HMM to validly describe metastable kinetics.
A more general validation can compare norms of predicted and directly estimated transition or correlation matrices.

III. QUANTITATIVE ANALYSIS

The framework uses hidden-state spectral structure to quantify metastable kinetics, thermodynamics, structural changes, and experimental observables. HMM-based quantities extend MSM analyses while retaining statistical treatment of metastable-state memberships.

Spectral and thermodynamic quantities: HMM eigenvalue decompositions provide hidden relaxation modes, eigenvectors, and stationary probabilities for analyzing metastable dynamics.The hidden stationary distribution gives the probability of observing each metastable state.
Spectral and thermodynamic quantities: Metastable-state free-energy differences remain associated with state weights even when the discretization is poor.The stationary distribution on microstates can be obtained by transforming metastable-state stationary probabilities through output probabilities.
Metastable-state definitions: PCCA and PCCA+ define metastable states from dominant MSM eigenvectors, while HMM memberships arise from the estimated output matrix and can receive statistical treatment.The HMM membership matrix avoids representative-state selection and reflects likelihood or posterior maximization.
Spectral and thermodynamic quantities: Relaxation rates and timescales are central kinetic observables, and PMMs can estimate them without the systematic error associated with MSMs.For metastable systems, the same claim is made for HMM estimates.
Structural interpretation: The framework connects eigenvector sign changes and amplitudes to structural changes occurring at associated rates or timescales.
Spectral and thermodynamic quantities: Projected eigenvectors and relaxation rates support calculations of kinetic experimental observables and can differ substantially from MSM eigenvectors.
Experimental observables: HMM spectral quantities model time-correlation and relaxation experiments, including single-molecule trajectories and ensemble perturbation experiments.Examples include fluorescence, pulling, temperature-jump, pressure-jump, pH-jump, and rapid-mixing experiments.
Experimental observables: Dynamical fingerprints enable simulation–experiment comparisons and the design of experiments targeting individual relaxations.

IV. APPLICATIONS

Applications across model systems, BPTI, and RNA hairpin data show that HMM/PMM estimates can recover slow kinetics more robustly than MSMs when discretization or projection is problematic, while requiring an appropriate number of hidden states.

Model systems: In a bistable potential, PMM/HMM estimates converge quickly to the true relaxation timescale and remain constant with lag time, unlike slowly converging MSM estimates under poor discretization.The PMM/HMM convergence speed depends exponentially on the greatest neglected timescale, whereas MSM convergence under good discretization has an error vanishing as τ^-1.
Model systems: For a three-well potential projected onto one coordinate, MSMs and HMMs can recover only the slow process visible in that projection, while a three-state HMM recovers both processes from the y-projection.The projection can completely hide one process; using too many hidden states when a process is absent can compromise estimation.
Scope and caveats: The applications support using HMMs for simulation and single-molecule data, but model selection must match the number of relaxation processes present and their timescale separation.A three-state HMM can degrade estimation when the data contain only two relevant processes.
BPTI: In the 1 ms BPTI simulation, HMMs produced nearly lag-time-constant timescales around 40 and 20 µs, whereas MSM estimates slowly approached around 30 and 15 µs.The HMM also directly yielded metastable-state structures and a three-state rate matrix, with exchanges on the 20 and 40 µs timescales.

V. CONCLUSIONS

The paper replaces the Markov assumption on observed clusters with Projected Markov Models and approximates them with Hidden Markov Models in metastable systems. HMMs can recover slow kinetics without the systematic bias induced by MSM Markovianity assumptions and reproduce many molecular observables.

Projected Markov Models describe the dynamics observed on discrete clusters, whereas MSMs approximate those dynamics as Markovian.
Direct estimation of PMMs is currently unavailable, but metastable PMMs with separated slow processes can be approximated by efficiently estimated HMMs.
HMM estimation uses an m × m hidden transition matrix and an n-element output probability vector for each hidden state.
HMMs can compute thermodynamic, kinetic, and mechanistic quantities commonly obtained from MSMs, including kinetic experimental observables.

ACKNOWLEGDEMENTS

The supplied passages contain formal derivation fragments rather than substantive acknowledgement content.

The passages present mathematical statements about Markov and hybrid processes rather than acknowledgement-specific content.

A.3 m-metastable PMM ≡m-state HMM

Projecting metastable Markov dynamics onto discrete clusters yields a PMM, while projecting an equivalent hybrid process yields an HMM with the same number of hidden states.

Projecting an m-timescale Markov process onto a discrete partition produces a PMM with m relaxation timescales.
Projecting an m-hybrid process onto discrete sets produces an m-state HMM with the hybrid transition matrix as its hidden transition matrix.
In the metastable case, the PMM and m-state HMM descriptions are equal.

APPENDIX B. ALGORITHMS AND DERIVATIONS

The PMM/HMM algorithm initializes an HMM from a reversible MSM, refines it with Baum-Welch expectation-maximization, and validates predictions against direct MSM quantities.

The algorithm takes discretized trajectories, a lag time τ, and the number m of slow relaxation processes as input.
It estimates a reversible transition matrix T(τ) from the discrete trajectories.
PCCA and Equations (5-6) decompose T(τ) into initial HMM output and hidden-transition matrices.
The model is validated by comparing predicted and observed correlation matrices or apparent relaxation timescales across lag times.
Baum-Welch iterations alternate hidden-path expectation calculations with maximum-likelihood updates of the transition and output matrices until likelihood improvement falls below a threshold.