PaperlayerView exact revision on arXiv

Source-linked AI summary

Estimating entropy production by machine learning of short-time fluctuating currents

Shun Otsubo, Sosuke Ito, Andreas Dechant, Takahiro Sagawa

arXiv:2001.07460v2cond-mat.stat-mech

TL;DR

TURs estimate entropy production from current means and variances but ordinarily yield only a lower bound. This paper combines a short-time TUR with machine learning to estimate entropy production from trajectory data; it reports exact estimation for Langevin dynamics with an optimal current and sampling-interval robustness in one-dimensional jump processes.

  • Problem

    TUR-based entropy-production estimation uses current means and variances rather than full stochastic-dynamics details, but ordinarily supplies only a lower bound.

    Source: §I · p. 1

  • Method

    The paper combines the short-time TUR with machine learning, including gradient ascent, to estimate entropy production from trajectory data.

    Source: §I · p. 1, A. General idea and the gradient ascent · p. 6

  • Results

    The short-time TUR can give exact entropy production for Langevin dynamics with an optimal current, while jump-process estimates are robust to sampling interval in one-dimensional systems.

    Source: §I · pp. 1–2, §V · p. 14

  • Takeaways & Limitations

    Sampling-interval robustness in one-dimensional jump processes is relevant where biological detectors have finite time resolution.

    Source: §I · p. 2, §V · p. 14

    Exact estimation is not generally achievable for Markov jump processes, although it is achieved in equilibrium and Langevin limits in the studied model.

    Source: §I · p. 2

Abstract

from arXiv · show

Thermodynamic uncertainty relations (TURs) are the inequalities which give lower bounds on the entropy production rate using only the mean and the variance of fluctuating currents. Since the TURs do not refer to the full details of the stochastic dynamics, it would be promising to apply the TURs for estimating the entropy production rate from a limited set of trajectory data corresponding to the dynamics. Here we investigate a theoretical framework for estimation of the entropy production rate using the TURs along with machine learning techniques without prior knowledge of the parameters of the stochastic dynamics. Specifically, we derive a TUR for the short-time region and prove that it can provide the exact value, not only a lower bound, of the entropy production rate for Langevin dynamics, if the observed current is optimally chosen. This formulation naturally includes a generalization of the TURs with the partial entropy production of subsystems under autonomous interaction, which reveals the hierarchical structure of the estimation. We then construct estimators on the basis of the short-time TUR and machine learning techniques such as the gradient ascent. By performing numerical experiments, we demonstrate that our learning protocol performs well even in nonlinear Langevin dynamics. We also discuss the case of Markov jump processes, where the exact estimation is shown to be impossible in general. Our result provides a platform that can be applied to a broad class of stochastic dynamics out of equilibrium, including biological systems.

I. INTRODUCTION

TURs estimate entropy production from current means and variances, but ordinarily provide only a lower bound. This paper develops a short-time TUR and machine-learning estimators, showing exact estimation for Langevin dynamics with an optimal current and robustness to sampling intervals in a one-dimensional jump-process setting.

I. INTRODUCTION

  • TURs use the mean and variance of a current to bound entropy production without requiring its full cumulants, but give only a lower bound.The paper builds on earlier TUR-based estimation approaches [62].Source: §I · p. 1
  • The short-time TUR’s equality is achievable in Langevin dynamics with an optimally chosen generalized current, but not generally in Markov jump processes.The formulation also extends to partial entropy production for autonomously interacting subsystems.Source: §I · p. 2
  • The paper derives short-time TUR equality conditions for both Markov jump processes and Langevin dynamics.Its numerical evaluation covers linear and nonlinear Langevin equations and a one-dimensional jump process.Source: §I · p. 2
  • For nonlinear Langevin dynamics, the proposed estimator converges faster than the previously proposed estimator, while the methods are comparable for linear cases.The authors attribute the nonlinear-case advantage to not assuming a Gaussian distribution.Source: §I · p. 2
  • TUR-based estimates are robust to the sampling interval in one-dimensional Markov jump processes, with exact estimation in equilibrium and Langevin limits.The authors identify finite detector time resolution as relevant to biological applications.Source: §I · p. 2

II. THERMODYNAMIC UNCERTAINTY RELATION IN THE SHORT-TIME LIMIT

The section derives a short-time TUR and its equality conditions for Markov jump processes and Langevin dynamics. It also extends the bound to subsystem partial entropy production, revealing how restricted current access limits the estimate.

II. THERMODYNAMIC UNCERTAINTY RELATION IN THE SHORT-TIME LIMIT

  • The short-time formulation includes TUR bounds on partial entropy production for autonomously interacting subsystems.The subsystem transitions are assumed bipartite, occurring independently of transitions in the rest of the system.Source: §II · pp. 2–3
  • The analytical equality-condition results do not assume steady states, although later numerical estimates use trajectories sampled from steady states.The analysis considers the τ→0 short-time limit.Source: §II · p. 2
  • For jump processes, the short-time integrated edge current has mean proportional to net flux and variance proportional to total forward and reverse flux.These expressions are obtained in the τ→0 limit from the probabilities of forward and reverse jumps.Source: §II · pp. 2–3
  • For Markov jump processes, the short-time TUR equality is not generally achievable, though it is asymptotically satisfied in equilibrium and Langevin limits.The Langevin limit has vanishing relative forward-reverse transition imbalance while keeping the net flux finite.Source: §II · p. 4
  • In Langevin dynamics, the short-time TUR can attain the exact entropy production rate by choosing the optimal generalized current, even far from equilibrium.With the scale set to one, the optimal coefficient is the thermodynamic force and the generalized current is the entropy production rate itself.Source: §II · pp. 4–5
  • When accessible currents are confined to a subsystem, maximizing the lower bound can yield only its partial rate σ_i rather than the total rate σ.The section describes this ordering of bounds as a hierarchy.Source: §II · p. 5

III. ESTIMATORS OF ENTROPY PRODUCTION RATE

This section outlines a machine-learning framework for estimating entropy production from limited trajectory data using the short-time TUR. It introduces gradient ascent and estimators for Langevin and Markov jump processes.

III. ESTIMATORS OF ENTROPY PRODUCTION RATE

  • The section presents short-time-TUR machine-learning estimators for entropy production using limited trajectory data.It covers both Langevin dynamics and Markov jump processes.Source: §III · p. 5
  • Gradient ascent is introduced as the learning method for the estimators.The section first discusses the motivation for using machine learning in this setting.Source: §III · p. 5
  • The section introduces two Langevin estimators and a learning estimator and direct estimator for Markov jump processes.Their characteristics are compared with previously proposed methods [54].Source: §III · p. 5

A. General idea and the gradient ascent

The estimator learns current coefficients by maximizing a short-time TUR bound while using held-out data to limit overfitting. For Langevin dynamics, finite-parameter functions approximate the coefficient field, and the target bound is exact in the short-time limit.

A. General idea and the gradient ascent

  • The learned TUR estimator gives the exact entropy production rate for Langevin dynamics as τ→0.The optimized bound is used as the estimator for both Langevin and Markov jump dynamics.Source: A. General idea and the gradient ascent · p. 5
  • Finite trajectory data make direct optimization unreliable because realization-specific overfitting can inflate the estimated entropy production.The estimation task is to approach the optimal coefficient while avoiding overfitting.Source: A. General idea and the gradient ascent · p. 5
  • Training data update the current coefficient to maximize the estimated bound, while the test-set bound’s peak selects a coefficient with expected generalization performance.The test learning curve can peak as the coefficient becomes overfitted to the training data.Source: A. General idea and the gradient ascent · p. 6
  • For Langevin dynamics, the coefficient field is approximated by a finite-parameter analytic function, whereas jump-process coefficients already have finitely many parameters.Gradient ascent updates those parameters, but the approximated objective can have local maxima and need not reach the global maximum.Source: A. General idea and the gradient ascent · p. 6

B. Estimators for Langevin dynamics

The Langevin estimators optimize a current coefficient field using short-time trajectory data, with binned or Gaussian model functions. Their flexibility and computational trade-offs distinguish them from KDE and SFI.

B. Estimators for Langevin dynamics

  • The estimators use finite-length stationary trajectories to calculate the short-time current’s mean and variance, then optimize its coefficient field.They model the coefficient with either a histogram-like function over bins or a linear combination of Gaussian functions.Source: B. Estimators for Langevin dynamics · p. 6
  • Unlike KDE, the learning estimators do not assume a data-point distribution, supporting nonlinear dynamics with non-Gaussian stationary distributions.KDE smooths a directly estimated force field using a kernel bandwidth selected under a Gaussian data-point assumption.Source: B. Estimators for Langevin dynamics · p. 6
  • Regularization improves the binned estimator’s data efficiency, while Gaussian functions are more data-efficient but computationally costly in higher dimensions.The study uses the Gaussian estimator for two-dimensional data and the regularized binned estimator for higher-dimensional data.Source: B. Estimators for Langevin dynamics · p. 6
  • The learning estimators fit the thermodynamic force field directly and allow any model functions, whereas SFI fits the mean local velocity and diffusion matrix with restricted basis-function combinations.The learning approach addresses overfitting through data splitting, using an objective function to maximize.Source: B. Estimators for Langevin dynamics · pp. 6–7
  • The KDE-based TUR estimator substitutes the KDE force-field estimate into the short-time TUR here, whereas the earlier method used the finite-time TUR.The temporal-average KDE estimator is another comparator in the study.Source: B. Estimators for Langevin dynamics · p. 7

C. Estimators for Markov jump processes

For Markov jump processes, the method reconstructs unobserved transitions between sampled states before estimating the short-time current. It then learns the current coefficient directly from its definition.

C. Estimators for Markov jump processes

  • Discrete-time observations omit transitions between samples, so the method heuristically reconstructs the intervening jump sequence before calculating the generalized current.In one-dimensional systems, connecting successive observed states by the shortest path is one possible reconstruction; reconstruction is unnecessary when the sampling interval is sufficiently small.Source: C. Estimators for Markov jump processes · p. 7
  • Because the jump-process coefficient has finitely many parameters, the learning estimator can be built directly from the generalized-current definition when transition edges are tractable.The section also introduces a simple comparator based on estimating transition rates from a reconstructed jump sequence.Source: C. Estimators for Markov jump processes · p. 7

IV. NUMERICAL EXPERIMENTS

The numerical experiments compare learning estimators with KDE-based estimators for Langevin systems and compare two estimators for a Markov jump process. Results vary with model type and dimensionality.

IV. NUMERICAL EXPERIMENTS

  • For the linear two- and five-bead Langevin models, learning-estimator performance is comparable to that of the KDE estimators.The estimators compared include the regularized binned or Gaussian learning estimators and two KDE estimators.Source: §IV · p. 7
  • In the nonlinear Mexican-hat Langevin model, the Gaussian learning estimator outperforms both KDE estimators, while the short-time-TUR optimum converges to the true rate in both specified jump-model limits.The two limits are the Langevin limit and the equilibrium limit.Source: §IV · p. 7
  • In the one-dimensional hopping model, the learning estimator converges faster than the direct transition-rate method and is robust to the trajectory sampling interval.These results concern the Markov jump-process comparison.Source: §IV · p. 8

A. N-beads model

The N-bead experiments compare learning estimators with KDE-based estimators in linear Langevin systems of two and five beads. Performance depends on temperature ratio and dimensionality, while learning estimators have a computation-time advantage for large trajectories.

A. N-beads model

  • For the two-bead model, learning-estimator convergence is comparable to KDE, with the Gaussian estimator best but differences among estimators not significant.Convergence is worse at Tc/Th = 0.5 than at 0.1 for all estimators, as mean local velocities become small near equilibrium.Source: A. N-beads model · p. 8
  • The beads are coupled by springs and immersed in fluids at different temperatures, producing steady heat flow between fluids and beads.The model uses two temperatures for two beads and a temperature gradient across five beads.Source: A. N-beads model · p. 8
  • In the five-bead model, the KDE temporal-average estimator converges best at Tc/Th = 0.1, while the binned learning estimator seems better at 0.5; all estimators converge more poorly than in the two-bead case.The authors attribute the broader decline to higher dimensionality.Source: A. N-beads model · p. 8
  • For five beads, the KDE TUR estimator converges less well than the KDE temporal-average estimator, unlike the earlier report; the authors suggest finite-time or long-time TUR estimators may converge faster in high dimensions.They state that more exhaustive research is needed to clarify this possibility.Source: A. N-beads model · p. 8
  • Learning estimators scale as O(N) in sample size, versus O(N^2) for both KDE estimators, and become faster as trajectory length increases despite requiring additional hyperparameter tuning.The authors argue the scaling advantage makes learning estimators better in computation time for large trajectory data.Source: A. N-beads model · p. 8

B. Mexican-hat potential model

The Mexican-hat model tests the estimators in nonlinear Langevin dynamics whose stationary distribution can depart from Gaussian. Gaussian learning converges best across the tested nonlinearities, and the learned coefficient field is close to the optimal field.

B. Mexican-hat potential model

  • The model’s stationary distribution departs from Gaussian at finite nonlinearity because of the small hill at the potential’s center.At A = 0, the model reduces to the two-beads model.Source: B. Mexican-hat potential model · p. 10
  • The Gaussian learning estimator converges fastest, while KDE estimators perform poorly at larger nonlinearity.The comparisons span A = 10^-4, 1, and 100.Source: B. Mexican-hat potential model · pp. 10–11
  • The learned coefficient field resembles the optimal field, especially near the center where data are sufficient.The comparison uses the two-beads model and the Mexican-hat model at A = 100.Source: B. Mexican-hat potential model · p. 11, Figure 5 · p. 12
  • The learned thermodynamic force field could be used to investigate higher-order statistics of time-integrated entropy production, a future application.The proposed route is to calculate the integrated generalized current using the obtained field.Source: B. Mexican-hat potential model · p. 11

C. One-dimensional hopping model

The hopping model tests the estimators on a ring-shaped Markov jump process and examines equilibrium and Langevin limits. The optimal estimate approaches the true entropy production in both limits, while the short-time TUR-based estimate is generally only a lower value; the learning estimator converges faster and is less sensitive to sampling interval than the simple estimator.

C. One-dimensional hopping model

  • The model is a hopping process among Nstate states arranged on a ring, with a Langevin dynamics limit as the neighboring-state spacing tends to zero.The equilibrium limit is reached as A tends to zero.Source: C. One-dimensional hopping model · pp. 12–13
  • The short-time TUR-based estimator is generally only a lower value than the true entropy production, but the exact value is obtained in the equilibrium and Langevin limits.The optimal estimate converges to the true value in both limits.Source: C. One-dimensional hopping model · pp. 13–14
  • The learning and simple estimators are compared using trajectory data sampled at Δt = 10^-3.Half the data are used for training and half for estimation in the learning-estimator case.Source: Figure 6 · p. 13, C. One-dimensional hopping model · p. 14
  • The learning estimator converges faster than the simple estimator and is little affected by coarser sampling intervals.The simple estimator deviates from the true value as the sampling interval increases because it must detect back-and-forth transitions.Source: C. One-dimensional hopping model · pp. 12, 14

V. CONCLUSIONS

The conclusions establish exact short-time TUR estimation for Langevin dynamics under optimal current choice, while Markov jump processes generally do not attain equality. The appendices detail learning estimators and their evaluation, including their performance trade-offs and practical limits.

V. CONCLUSIONS

  • The short-time TUR reaches equality for Langevin dynamics, including far from equilibrium, but not generally for Markov jump processes; partial entropy production yields a hierarchy under limited trajectory availability.The framework also applies to stochastic dynamics relevant to biological systems.Source: §V · p. 14
  • The learning estimators perform well across several setups and suit finite trajectory data through good convergence, small computational cost, and independence from parameters such as the diffusion constant.The conclusions report exact estimates for the one-dimensional hopping model in equilibrium and the Langevin limit, and robustness to sampling interval.Source: §V · p. 14
  • TUR-based estimators are robust to the observation sampling interval in the one-dimensional Markov jump model, and become exact in its equilibrium and Langevin limits.The authors note that reconstructing transitions in more complex Markov jump processes at finite Δt remains challenging.Source: §V · p. 14
  • Long-time TURs can also give exact Langevin estimates, but require preparing a generalized-current ensemble and choosing a sufficient observation duration.The authors leave open whether the maximized short-time TUR lower value indicates dissipation in Markov jump processes when it is not exact.Source: §V · p. 14

1. Adam

  • Adam uses past update vectors to provide inertia and tunes each parameter’s step size automatically, reducing the need for careful step-size tuning.The study tunes α while keeping β1, β2, and ϵ at their suggested values.Source: §1 · p. 15

2. Data splitting and hyperparameter tuning

  • Splitting trajectory data at its midpoint helps limit leakage of spatial occurrence frequencies, but short trajectories can yield overestimates through train-test correlation and statistical outliers.The authors report that these effects soon vanish as trajectory length increases and describe peak test estimates as a hyperparameter-selection criterion.Source: §2 · p. 15

1. Learning estimators

  • The Gaussian estimator trains on all data points and is automatically smooth but costs more; regularization instead smooths the binned coefficient field to reduce overfitting.The Gaussian estimator’s train and test curves converge to the true rate in the illustrated example; across two-dimensional settings it converges faster, while the regularized binned estimator costs less. The authors use Gaussian estimation for two-dimensional data and regularized binning for higher dimensions.Source: §1 · pp. 16–17

2. Estimators with kernel density estimation

  • The KDE thermodynamic-force estimate gives an exact entropy-production estimate under the short-time TUR, but KDE estimators require the diffusion matrix and converge slowly for nonlinear Langevin dynamics.The force estimate uses nearby data-point displacements weighted by their distance from the position of interest.Source: §2 · p. 18

Appendix C: Extension of the Gaussian learning estimator for higher dimensional setups

Appendix C extends the Gaussian learning estimator to higher-dimensional data, where the original basis size grows exponentially with dimension. The supplemental results also examine hyperparameter tuning and estimator computation time.

Appendix C: Extension of the Gaussian learning estimator for higher dimensional setups

  • Optimizing Gaussian coefficients and function parameters by gradient ascent converges faster than coefficient-only deterministic optimization, which is faster in computation time.The comparison uses five-, 10-, and 15-beads models.Source: Figure 9 · p. 19, Appendix C · p. 20
  • The original Gaussian estimator uses O(N N_Gauss) computation with N_Gauss = N_bin^ndim, so its basis size grows exponentially with dimension.The extension treats Gaussian positions as variables to suppress the number of functions.Source: Appendix C · p. 20
  • The high-dimensional variant optimizes Gaussian weights, centers, and widths by gradient ascent.Source: Appendix C · p. 20

1. Hyperparameter tuning

  • For the Gaussian estimator, α matters more than Nbin: small α overfits training data, while large α impedes gradient ascent.The study tunes Nbin with α fixed, then tunes α for each trajectory length.Source: §1 · p. 22
  • For the binned estimator, peaks align along constant αλ because regularization enters the gradient with coefficient αλ.The study fixes α, then tunes Nbin and λ for each trajectory length.Source: §1 · p. 22
  • Hyperparameters were tuned for two- and five-beads models and Mexican-hat potentials, with selected values summarized in Table I.Source: §1 · p. 22

2. Computation time

The learning estimators have lower computational complexity than the compared alternatives and become more advantageous as trajectory length increases. Their computation-time advantage must be weighed against hyperparameter-tuning costs, which are excluded from the reported timing.

2. Computation time

  • Binned score evaluation scales as O(N), its gradient as O(max(N, N_bin^ndim)), Gaussian learning as O(NN_Gauss), and the compared ss/SFI estimators as O(N^2).Source: §2 · p. 22
  • For the two-beads and five-beads models, the Gaussian (binned) learning estimator is around 50 (1000) times faster than b˙S; learning estimators improve as trajectory length increases.Source: §2 · p. 22
  • The deterministic Gaussian estimator has complexity O(max(NN_Gauss, N_Gauss^3)) and is reported as around 200 times faster than Gaussian learning.It optimizes only the Gaussian coefficients, rather than using gradient-ascent iterations.Source: §2 · pp. 22–23
  • Reported computation times exclude hyperparameter tuning, whose cost depends on implementation and required precision and may be small enough not to erase long-trajectory gains.The authors suggest tuning on shorter trajectories; Gaussian-estimator optima were nearly independent of trajectory length.Source: §2 · p. 23
Loading 2001.07460v2…