PaperlayerView exact revision on arXiv

Source-linked AI summary

Transition Path Sampling Using Koopman Operators and Exit-Time Optimal Control

Boya Hou, Shane Wang, Siddharth Ambekar, Maxim Raginsky, Olgica Milenkovic

arXiv:2610.10054v1eess.SYcs.LGcs.SYstat.ML

TL;DR

High free-energy barriers make transitions between metastable states rare, and recent learned-control samplers require repeated biased rollouts. This paper uses Koopman eigenfunctions to construct an exit-time optimal control for transition-path sampling; the optimal controller has a closed form and its path measure minimizes free energy over all path measures on the stopped σ-algebra.

  • Problem

    High free-energy barriers make reactive transitions rare, while recent neural drift-bias samplers use fixed horizons and repeated biased rollouts.

    Source: §1 · pp. 1–2

  • Method

    The method uses Koopman eigenfunctions to identify metastable sets and estimate a committor, then formulates sampling as optimal stochastic control stopped at the first target-set hit.

    Source: §1 · pp. 2–3

  • Results

    The closed-form optimal controller generates a path measure that minimizes free energy over all path measures on the stopped σ-algebra.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Koopman eigenfunctions can be estimated without transitions between metastable sets, and the controller can be constructed without repeated rollouts through a quadratic program.

    Source: §1 · pp. 2–3

    The exit-time analysis assumes bounded, Lipschitz drift and diffusion coefficients.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Sampling transitions between metastable states is a central problem in dynamical systems theory and molecular dynamics in particular. A key challenge is the existence of high free-energy barriers that separate the states, making transitions extremely rare. Recent machine learning-based methods cast transition path sampling (TPS) as an optimal stochastic control (OSC) problem over a fixed time horizon, and parameterize the drift bias via a neural network trained by simulation-in-the-loop, requiring repeated biased rollouts. To address computational and performance guarantee issues of these models, we propose a new approach for the problem based on Koopman operators. Because Koopman operators are linear, their leading eigenfunctions reveal the metastable sets and provide an estimate of the committor function with no transition path information required. Furthermore, we formulate TPS as an OSC problem up to an exit time. Our time horizon is the first hitting time of the target set, and our running cost penalizes time spent in nonreactive regions by encoding the estimated committor function. We derive the optimal controller in closed form and approximate it in a reproducing kernel Hilbert space (RKHS). This reduces the problem of constructing the optimal controller to solving a single equality-constrained quadratic program, whose solution can be characterized by a linear Karush-Kuhn-Tucker (KKT) system. On the two-channel double well and alanine dipeptide, our controller increases the fraction of trajectories reaching the target from 0% to 99.8% within 1000 steps, and from 0% to 93% within 1ps, respectively.

1 Introduction

High free-energy barriers make reactive transitions rare, while existing sampling and learned-control approaches face path-initialization, sparse-training, and horizon-selection challenges. The paper combines Koopman spectral analysis with exit-time optimal stochastic control to construct transition paths without simulation-in-the-loop training.

1 Introduction

  • High free-energy barriers leave most trajectories within one metastable set, making reactive transitions rare despite their relevance to protein conformational change, folding, and drug discovery.The passage cites Grubmüller (1995) and Voter (1997) for the applications, and Vanden-Eijnden et al. (2006) for reactive paths.Source: §1 · p. 1
  • Classical TPS can require an initial reactive path, while neural drift-bias methods use fixed horizons and repeated biased rollouts with sparse early training signals.A fixed horizon must be chosen even though transition time varies by path; replay buffers, annealing, and kernel-based truncation only partially mitigate these drawbacks.Source: §1 · pp. 1–2
  • The framework uses the unbiased path measure as reference, penalizes nonreactive time through the committor, and stops control at the first target-set hit.The paper proves the resulting Gibbs measure is generated by a controlled diffusion and minimizes free energy over path measures on the stopped σ-algebra.Source: §1 · p. 2
  • Koopman generator eigenfunctions identify metastable sets and construct a committor without transition paths, and sampled paths can be reweighted to recover original-dynamics statistics.The approach is described as interpretable because its controller uses metastable sets and the committor.Source: §1 · pp. 2–3
  • Approximating the controller in an RKHS reduces its construction to a convex quadratic program with linear equality constraints, solved through one KKT system rather than repeated rollouts.The optimal controller is stated in closed form as u∗ = ∇log ρ.Source: §1 · pp. 2–3
  • The paper evaluates the approach on synthetic and real-world datasets and generates paths by simulating the controlled diffusion until it enters the target set.The introduction identifies these as evaluation and sampling components without reporting their results here.Source: §1 · p. 3

2 Preliminaries: Itô Diffusion and the Koopman Generator

The preliminaries describe the diffusion dynamics and use the Koopman generator’s spectrum to characterize metastability. The leading eigenfunctions then support the paper’s construction of metastable sets and a committor-based control cost.

2 Preliminaries: Itô Diffusion and the Koopman Generator

  • In Langevin dynamics, the drift and diffusion are specified by potential energy, friction, and bath temperature; deep wells separated by large barriers yield metastable behavior.In the underdamped case, momentum is also part of the state, and friction and noise act on momentum.Source: §2 · p. 3
  • Although the dynamics may be nonlinear, the Koopman operators are linear, and generator eigenfunctions evolve by the exponential of their eigenvalue.The constant function is an eigenfunction with eigenvalue zero; the generator spectrum can characterize global dynamical structure.Source: §2 · pp. 3–4
  • A reactive path crosses between metastable sets without returning, while the committor is near 0 by the source and near 1 by the target.The committor satisfies a boundary-value problem with the Koopman generator in the region outside the two sets.Source: §2 · p. 4
  • The leading Koopman generator eigenfunctions identify metastable sets and provide the spectral term used to construct the committor and define the control running cost.The eigenpairs can be approximated from data using DMD, EDMD, or kernel EDMD; the kernel approach leads to a generalized eigenproblem from Gram matrices.Source: §2 · p. 4

3 Transition Path Sampling via OSC up to an Exit Time

The section formulates transition-path sampling as an exit-time optimal control problem and identifies a Gibbs path measure that minimizes free energy. It derives and verifies an optimal controller, then connects its construction to the running cost and target-hitting time.

3.1 The Sampling Problem

  • The exit time is the first hitting time of the target, and the running cost measures time spent in nonreactive regions before reaching it.The parameter β trades faster paths against greater deviation from the original dynamics.Source: §3.1 · p. 5
  • The Gibbs path measure is the unique free-energy minimizer, with minimum value -log Z.Free energy balances low-energy paths against closeness to the original dynamics.Source: §3.1 · p. 5

3.2 A Candidate Optimal Controller

  • The function ρ coincides with the unique solution of an elliptic boundary-value problem and is positive throughout the domain.The problem uses the Koopman generator and boundary data g = e^-Φ.Source: §3.2 · p. 6

3.3 Verification Theorem of OSC up to an Exit Time

  • The exit-time verification requires only almost-sure finite exit, avoiding the exponential-moment condition that can be hard to verify for metastable systems.The paper contrasts this requirement with Novikov’s condition in the fixed-horizon setting.Source: §3.1 · p. 6, §3.3 · p. 7
  • The optimal controller’s cost equals -log ρ(x), and its path law uniquely minimizes free energy over all path measures on the stopped σ-algebra.The theorem equates the control and free-energy minima.Source: §3.3 · p. 8
  • The controller points toward states more likely to exit, with a stronger push from deep inside the source basin.The normalization by positive ρ makes the push more aggressive where the original dynamics is least likely to escape.Source: §3.3 · p. 8

4 Koopman-Based Construction of the Optimal Controller

The section uses Koopman-generator eigenfunctions to identify metastable cores and estimate the committor, then constructs an exit-time optimal controller from that information. The controller approximation reduces to a single equality-constrained quadratic program.

4 Koopman-Based Construction of the Optimal Controller

  • The optimal controller is constructed from the Koopman generator through metastable sets, a committor estimate, and a solution to an elliptic problem for ρ.The section describes these as the ingredients for computing the controller.Source: §4 · p. 8

4.1 Constructing Metastable Cores and Committor Functions

  • Leading eigenfunctions near zero with a spectral gap capture slow inter-basin dynamics and are approximately constant within metastable sets.The method estimates eigenpairs from data, clusters the eigenfunction coordinates, and defines cores around cluster values.Source: §4.1 · pp. 8–9
  • The committor estimate combines a spectral term from leading eigenfunctions with an RKHS remainder, without requiring a reaction coordinate in advance.The spectral term captures the transition’s global shape and slow coordinate; the remainder minimizes the PDE residual in the stated construction.Source: §4.1 · p. 9

4.2 Kernel-Based Approximation of the Optimal Controller

  • The approximation of ρ separates the transition region from the source basin, using a closed-form factor and an RKHS residual to represent their different behavior.Near the target boundary ρ is approximately 1, while in the source basin it is small as trajectories linger there.Source: §4.2 · pp. 9–10
  • The controller approximation reduces to a convex quadratic program with linear equality constraints, characterized by one linear KKT system.The algorithm then integrates the controlled dynamics from the source set until paths hit the target.Source: §4.2 · p. 10

5 Numerical Experiments

The experiments evaluate the method on two synthetic systems and alanine dipeptide, reporting target-hit percentage before a numerical censor time. The alanine dipeptide experiment is outside Assumption 2.

5 Numerical Experiments

  • The evaluation includes the two-channel double-well system, the Müller-Brown system, and alanine dipeptide.One additional example and experimental details are deferred to Appendix G.4.Source: §5 · p. 10
  • Target-hit percentage is the fraction of paths reaching the target before numerical censor Tmax; paths not hitting are truncated at Tmax.A path counts as a hit at its first entrance to the target.Source: §5 · p. 10
  • Alanine dipeptide does not satisfy Assumption 2, but the experiment demonstrates performance beyond the confines of the theoretical analysis.Source: §5 · p. 10

A.1 Auxiliary Result: A Localized Dynkin Formula

The localized Dynkin formula handles expectations at a possibly random exit time when the function is only twice differentiable inside the domain. It localizes the calculation to increasing interior sets before passing to the exit time.

A.1 Auxiliary Result: A Localized Dynkin Formula

  • For h that is C2 only on D, the proof seeks its expected value along the trajectory at the exit time τD, where ordinary Itô’s formula cannot be applied directly.The exit time is itself random, and the trajectory reaches the boundary at τD.Source: §A.1 · p. 14
  • The argument first stops at σk = k ∧ τDk for an increasing sequence of open subsets Dk inside D, where optional stopping applies.This localizes the calculation before taking the limit to τD.Source: §A.1 · pp. 14–15
  • The stopping times σk are nondecreasing, and their pathwise supremum equals τD.Each σk is bounded above by τD; any time strictly before τD is eventually inside some Dk.Source: §A.1 · p. 15
  • When the initial point lies outside the closure of Dk, continuity gives τDk = 0, so σk and the integrals over the stopped interval vanish.Source: §A.1 · p. 15
  • When the initial point lies in the closure of Dk, the stopped trajectory remains there, allowing h to be replaced locally by a smooth compactly supported extension.The extension agrees with h and its relevant derivatives on a neighborhood of the closure of Dk.Source: §A.1 · p. 15
  • Applying Itô’s product rule to the extended function and Zt up to t ∧ σk yields the localized identity in terms of h.The identity follows because the stopped trajectory stays within the region where the extension agrees with h.Source: §A.1 · p. 15
  • Bounded stopping times, bounded derivatives, and 0 < Zs ≤ 1 make optional stopping applicable; setting t = k and taking expectations gives the stated identity.The bound on Zs follows from c ≥ 0.Source: §A.1 · pp. 14–15

B.1 Proof of Lemma 1

Under the stated assumptions, the proof bounds the mean exit time using an elliptic boundary-value problem and a localized Dynkin formula. It then extends finite mean exit time to every admissible controlled process.

B.1 Proof of Lemma 1

  • An elliptic boundary-value problem Lφ = −1 in D with φ = 0 on ∂D supplies a nonnegative function used to bound the mean exit time.The proof states existence and uniqueness of φ under its regularity and boundary assumptions.Source: §B.1 · pp. 15–16
  • The maximum-principle argument rules out an interior minimum of φ, yielding φ ≥ 0 on the closure of D.At an interior minimum, uniform ellipticity and a positive-semidefinite Hessian would contradict Lφ = −1.Source: §B.1 · p. 16
  • Bounded Lipschitz drift and diffusion give a pathwise unique strong solution for the SDE.Source: §B.1 · p. 16
  • The localized Dynkin formula and nonnegativity of φ imply Ex[τD] ≤ φ(x) < ∞, so τD is finite almost surely.The proof passes from σk to τD by monotone convergence.Source: §B.1 · p. 16

B.2 Finite Mean Exit Time for Controlled Processes

  • For any admissible control u, the controlled drift b + au remains bounded and Lipschitz, so the finite-exit-time result applies to the controlled SDE.The corollary concludes that its exit time has finite expectation and is finite almost surely.Source: §B.2 · p. 16

B.3 Proof of Lemma 2

The proof identifies ρ with its Feynman–Kac representation on D and extends the equality to the boundary. It also establishes that ρ is strictly positive with a positive minimum on the compact closure of D.

B.3 Proof of Lemma 2

  • ρ(x) equals its Feynman–Kac representation throughout D by taking limits of stopped-process expectations.The proof uses convergence to the exit boundary and bounded convergence.Source: §B.3 · pp. 16–17
  • On ∂D, ρ agrees with the boundary value e^−Φ, so the equality extends from D to its closure.At boundary points the exit time is zero.Source: §B.3 · p. 17
  • ρ is strictly positive on ¯D and has a positive minimum there.Positivity follows from the boundary condition and the positive expectation in the interior; continuity on compact ¯D then gives the minimum.Source: §B.3 · p. 17

B.4 Proof of Lemma 3

The proof transforms the equation for ρ and shows that the resulting equation for v coincides with the exit-time OSC HJB equation. Its pointwise minimizer gives the candidate optimal controller.

B.4 Proof of Lemma 3

  • Using Lρ − fρ = 0 and ρ > 0, the proof derives a rearranged equation for the transformed function.The section introduces the gradient and second-order terms before rearranging.Source: §B.4 · pp. 17–18
  • Applying Itô’s formula to the controlled process and substituting the transformed equation establishes the HJB correspondence up to the exit time.The proof considers admissible controls u in the controlled SDE.Source: §B.4 · p. 18
  • The control-dependent expression is uniquely minimized at u∗ = −∇v, yielding the candidate optimal controller for the exit-time OSC.Uniqueness follows from a ≻ 0.Source: §B.4 · p. 18

B.5 Proof of Proposition 1

The proof checks the conditions needed to apply the cited proposition on the stopped-path probability space.

B.5 Proof of Proposition 1

  • Because H ≥ 0 and Z is positive, the proof verifies the proposition’s hypotheses on (Ω, F_τD) with reference measure P^x|F_τD and energy H.It also establishes −E^x[H e^−H] < ∞.Source: §B.5 · p. 18
  • The established integrability and positivity conditions allow the proposition to yield the claim.The passage does not specify the claim’s content.Source: §B.5 · p. 18
  • The argument applies the proposition on the stopped-path space rather than an unrestricted path space.The measure and filtration are specified as P^x|F_τD and F_τD.Source: §B.5 · p. 18

B.6 Proof of Lemma 4

The proof establishes the controlled process and measure-change representation for the candidate controller, then proves the exit-time OSC value is attained by that controller. It also characterizes when equality holds in the associated path-measure result.

B.6 Proof of Lemma 4

  • The candidate controller u∗ = −∇v belongs to the admissible class, and the corresponding controlled SDE has a pathwise unique strong solution.The proof uses bounded Lipschitz coefficients.Source: §B.6 · p. 18
  • The stopped stochastic exponential is a true martingale, enabling a probability measure change under which the process follows the controlled SDE up to τD.The measure change and uniqueness in law identify the resulting law with the optimally controlled process.Source: §B.6 · p. 19

B.7 Proof of Theorem 1

  • For any admissible control, the value is bounded below by v(x), with equality only when the control agrees with u∗ along the trajectory up to exit.The proof uses the nonnegative quadratic remainder and uniform ellipticity.Source: §B.7 · p. 21
  • The lower bound is attained by u∗, so J(u∗, x) = v(x).The proof notes that u∗ is admissible and the remainder term vanishes.Source: §B.7 · p. 21
  • Equality in the path-measure variational result holds exactly for the law of the optimally controlled process.The proof identifies this equality condition with ˜P = ˜P∗,x.Source: §B.7 · p. 21

C Proof of Identity (4) in Section 4.1

The appendix verifies the committor boundary-value characterization and details its spectral and kernel approximations, the construction of the control function, and path reweighting. It also reports evaluations on the double well, Müller-Brown potential, and alanine dipeptide.

C Proof of Identity (4) in Section 4.1

  • The committor is the unique solution of the boundary-value problem and equals the probability of reaching the target core before the source core.The proof identifies the exit event at the target boundary with reaching the target before the source.Source: C Proof of Identity (4) in Section 4.1 · pp. 21–22

D Detailed Computation of the Committor Function from Section 4.1

  • The committor approximation combines a rescaled Koopman-eigenfunction term with a kernel approximation of the remainder.The spectral term maps the source- and target-core centroids to 0 and 1; the remainder satisfies a boundary-value problem with corresponding boundary conditions.Source: D Detailed Computation of the… · p. 22

E Details Behind the Computation of ρ from Section 4.2

  • The control-function approximation uses a reactive-event probability and an assumed asymptotically exponential first-hitting time with mean 1/|λ1|.The appendix states that the metastability contribution enters through the committor; in practice it uses an approximation of that function.Source: E Details Behind the Computation of ρ… · pp. 23–24

F Reweighting the Generated Paths

G.1 The Double-Well System

  • 99.8% of controlled paths hit the target, versus 0% of uncontrolled paths, while retaining both channels in the two-channel double well.The experiment used 1000 integration steps, with paths initialized at the source-core minimum.Source: §G.1 · pp. 26–27

G.2 Muller-Brown Potential

  • At 200 K, the controller raises target-hit percentage from 1.8% for uncontrolled dynamics to 99.0% on the Müller-Brown potential.The target is the medium well, with a shallow well between it and the deep source well.Source: §G.2 · p. 27

G.3 Alanine Dipeptide

  • The alanine dipeptide pipeline constructs the controller once, using Koopman eigenfunctions followed by solutions for the committor and controller.The comparison baseline repeatedly simulates controlled trajectories and updates a neural network, incurring sampling cost per rollout.Source: §G.3 · p. 28

G.4 Additional Experiment: The Four-Well Dynamics

The four-well experiment targets a transition from the source core at (−1, −1) to the target core at (1, 1). At 200 K, the proposed method has a higher target-hit percentage than uncontrolled dynamics and TPS-DPS.

G.4 Additional Experiment: The Four-Well Dynamics

  • The four-well benchmark defines source and target cores centered at (−1, −1) and (1, 1), respectively.Each core has radius ε = 0.3; the reported paths at 200 K are shown in Figure 6.Source: §G.4 · p. 28
  • 94.7% of paths hit the target with the proposed method at β = 2, compared with 81.6% for TPS-DPS and 0.9% for uncontrolled paths.The experiment is reported for four-well dynamics at 200 K.Source: Figure 6 · p. 30
  • Table 4 covers the four-well benchmark at 200 K and 300 K.The supplied table caption specifies the temperatures but provides no metric values.Source: Table 4 · p. 30
Loading 2610.10054v1…