PaperlayerView exact revision on arXiv

Source-linked AI summary

Thermodynamic Cyclic Processes with Markov Samplers in Bayesian Inference

Heinrich von Campe, Bjoern Malte Schaefer

arXiv:2609.07660v1stat.COcond-mat.stat-mechcs.AI

TL;DR

The paper examines whether Bayesian inference and MCMC can be treated as thermodynamic systems with cyclic processes. It develops adaptive schedulers to vary temperature and sources during sampling, and finds that non-zero net work is possible if and only if the posterior is non-Gaussian.

  • Problem

    The paper examines how to construct and study cyclic processes for Bayesian inference using the thermodynamic description of Bayesian inference and MCMC.

    Source: §1 · pp. 1–2

  • Method

    Adaptive ensemble schedulers tune temperature and sources during RMH and HMC sampling to realize MCMC cycles while keeping the algorithms close to thermal equilibrium.

    Source: §2 · p. 3, §5 · p. 13

  • Results

    MCMC cycles can produce non-zero net work if and only if the model is non-Gaussian, with a suitable cycle yielding non-zero work for a non-Gaussian posterior and every trajectory yielding zero for a Gaussian posterior.

    Source: §5 · p. 13

  • Takeaways & Limitations

    Cycle work can serve as a measure of posterior non-Gaussianity independent of its particular functional shape, although it is computationally expensive.

    Source: §4.3 · p. 11, §5 · p. 13

    The non-Gaussianity measure is computationally expensive relative to sampling and comparing the posterior with an empirical Gaussian.

    Source: §4.3 · p. 11

Abstract

from arXiv · show

The concept of Markov chain Monte Carlo (MCMC) cycles, an analogy to cyclic processes in heat engines, is presented in order to examine Bayesian inference problems. In this effort, we develop adaptive ensemble schedulers that allow the tuning of external parameters of a Bayesian canonical ensemble during an MCMC run, realising the MCMC cycles in practice. We run these cycles on different statistical models. As a fundamental insight, we find (both theoretically and in practice) that such systems can produce a non-zero net work output if and only if the considered model is non-Gaussian. As such, they may serve as a measure of non-Gaussianity in Bayesian inference, which we test on an example from supernova cosmology.

1 Introduction

The introduction frames Bayesian inference and MCMC through statistical physics, then proposes tunable MCMC cycles as a thermodynamic analogue whose work can diagnose non-Gaussianity.

1 Introduction

  • Bayesian inference can be treated as a canonical ensemble by extending the evidence integral with temperature and sources, linking thermodynamic tools to posterior properties.At T = 1 and J = 0, derivatives of the partition sum yield posterior moments and cumulants.Source: §1 · pp. 1–2
  • MCMC samplers generate posterior samples through random walks in sample space, while simulated annealing instead lowers temperature to concentrate motion around a posterior maximum.The thermodynamic perspective has also motivated MCMC convergence criteria and sampling algorithms.Source: §1 · p. 2
  • The paper constructs MCMC cycles by varying temperature and sources in a closed loop, using adaptive ensemble schedulers to study their thermodynamic behavior and non-Gaussianity diagnostic value.The authors test this diagnostic on Bayesian inference with type Ia supernovae.Source: §1 · p. 2
  • The paper situates its work among applications of MCMC in statistical physics and Bayesian inference, alongside prior studies of the thermodynamic properties of Bayesian methods.Related thermodynamic work includes Jaynes (1957), Röver et al. (2023a;b), Herzog et al. (2024), and Kuntz et al. (2024; 2025).Source: §1 · p. 2

2 A tunable RMH sampling algorithm

The section develops an RMH sampler that can tune temperature and sources during a run, with adaptive changes intended to keep sampling near thermodynamic equilibrium.

2 A tunable RMH sampling algorithm

  • The tunable RMH algorithm changes temperature and sources during sampling, extending the Bayesian canonical potential V(θ; J) = −log(Lπ) − J · θ.The schedulers provide the mechanism for changing both external parameters while the sampler runs.Source: §2 · p. 3
  • Changing external parameters too quickly can drive acceptance probability toward zero and leave the sampler stuck near a local minimum.The acceptance probability accounts for changes in position, temperature, and sources.Source: §2 · p. 3
  • Quasi-static scheduling changes external parameters on timescales longer than equilibration, while proposal and scheduler step scales are chosen so potential changes remain proportional to temperature.For a source change, the target potential change is approximately ϵT.Source: §2 · p. 3
  • The source scheduler uses an ensemble-averaged gradient to set step size, while temperature changes follow a multiplicative exponential schedule.The source step limits potential change to ϵT; the temperature schedule corresponds to exponential simulated annealing.Source: §2 · p. 3
  • These schedulers do not guarantee quasi-staticity, and low-temperature simulated-annealing convergence guarantees require a logarithmic rather than exponential schedule.The exponential schedule worked sufficiently well for the paper's examples, but more complicated systems may require a more robust scheduler.Source: §2 · p. 4

3 Thermodynamic Concepts for MCMC Cycles

The section maps thermodynamic state variables and cyclic processes onto MCMC, then defines how cycle work and heat are estimated and compared with theoretical predictions.

3 Thermodynamic Concepts for MCMC Cycles

  • The canonical partition function defines free energy, whose derivatives provide entropy and a state variable Λ corresponding to the posterior's first moment.At unit temperature, the entropy is the Shannon entropy of the posterior.Source: §3 · p. 4
  • Reversible quasi-static cycles obey δQ = T dS, and their dissipated entropy should be compatible with zero within tolerance.The paper argues that the proposed cycles are generally reversible because they include no mechanism analogous to friction.Source: §3 · pp. 4–5
  • An MCMC cycle varies temperature and sources through a closed loop, with sources J playing an analogous role to volume in the Stirling-cycle comparison.The source path is a straight line between its endpoints, so results depend on the chosen endpoints.Source: §3 · pp. 4–5, Figure 1 · p. 4
  • Cycle outputs include sample and parameter trajectories; finite differences estimate internal-energy changes, work, and heat, which are then integrated over time.Work is estimated from Λ · ΔJ, and heat follows from the first law as the internal-energy change minus work.Source: §3 · p. 5
  • The heat prediction combines temperature change weighted by heat capacity with source change weighted by h, allowing theoretical and numerical heat exchange to be compared.The prediction is Q̇ = T Ṡ = C Ṫ + h · J̇.Source: §3 · p. 5

4 Application

Across the applications, Gaussian models produce zero net work, while non-Gaussian models can produce nonzero work along suitable source paths. In the supernova example, the cycles distinguish the posterior from its Gaussian approximation, though the measurement is computationally expensive.

4.1 Gaussian Toy Model

  • For the Gaussian toy model, net heat and work are zero after a full cycle, as predicted by its temperature-independent source response.Theoretical heat exchanged in the iso-J phases cancels over the cycle.Source: §4.1 · pp. 6–7
  • Changing the Gaussian model's source path across rather than along its elongated direction reduces work in the isothermal phases.The lower resistance along the across-ellipse direction explains the smaller work in those phases.Source: §4.1 · p. 8

4.2 Rosenbrock Toy Model

  • In the Rosenbrock model, nonzero net work occurs when source component 1 varies, but not when only component 0 varies.Its entropy depends on both J and T, so heat exchanged in the two iso-J phases need not cancel.Source: §4.2 · pp. 8–9
  • In the Rosenbrock sweep, work output is zero within tolerance at c = 0 and increases as c grows and the model becomes more non-Gaussian.The sweep varies c from 0 to 8 on the nonzero-work cycle trajectory.Source: §4.2 · p. 9, Figure 7 · p. 10

4.3 General non-Gaussian model

  • For a smooth non-Gaussian model with a non-degenerate global minimum, some cycle trajectory can yield nonzero net work; Gaussian models yield zero for every trajectory.The Rosenbrock example has zero work on one chosen trajectory and nonzero work on another.Source: §4.3 · pp. 9, 11
  • Net work in a Stirling-type MCMC cycle serves as a measure of posterior non-Gaussianity without assuming a particular functional shape.The authors caution that this approach is not computationally competitive with established measurement methods.Source: §4.3 · p. 11

4.4 Non-Gaussian cosmological inference problem

  • The cosmological likelihood is profiled to a two-dimensional inference problem in (Ωm, w), with absolute magnitude and Hubble constant fixed.The likelihood is expensive to evaluate, so a Gaussian surrogate is used first to tune cycle hyperparameters.Source: §4.4 · p. 12
  • ∆Wnet = −0.073 ± 0.014 for the SN Ia posterior, while its Gaussian approximation produces no net work; dissipated entropy is compatible with zero.The authors interpret the posterior's nonzero work as reflecting non-Gaussianity and report ∆Sdiss = 0.1 ± 0.4.Source: §4.4 · p. 12, Figure 8 · p. 12

5 Conclusion

The paper establishes MCMC cycles as a practical thermodynamic framework and finds that nonzero net work is possible for non-Gaussian, but not Gaussian, posteriors. The cycles can quantify non-Gaussianity, subject to equilibrium and computational constraints.

5 Conclusion

  • Adaptive ensemble schedulers implement four-phase cycles by varying temperature T and sources J while keeping canonical samplers close to thermal equilibrium.The cycles apply to statistical models with finite relevant moments when the scheduler maintains sufficient proximity to equilibrium.Source: §5 · p. 13
  • Nonzero net work is possible along some cycle for non-Gaussian posteriors, whereas it is zero for every trajectory for Gaussian posteriors.The result is verified analytically for arbitrary non-Gaussian posteriors and tested on a supernova-cosmology model.Source: §5 · p. 13
  • Cycle work can measure posterior non-Gaussianity independently of functional form, but the paper describes this measurement as computationally very expensive.The approach generalizes simulated annealing by adding cyclic changes and varying sources J, which are used to determine posterior moments or cumulants.Source: §5 · p. 13
  • The authors identify broader model coverage and higher-dimensional inference, where cycle-trajectory choice becomes more important, as directions for further study.They also suggest studying different canonical samplers; noncanonical ensembles would require adapting the thermodynamic theory.Source: §5 · p. 13
  • The paper presents applying statistical-physics concepts to canonical MCMC methods as a source of technical and fundamental insights into their thermodynamic nature.The authors report that adaptive schedulers can be used with canonical methods beyond the samplers implemented here.Source: §5 · p. 13

A Einstein summation convention

The appendix sets out the index notation used in the paper, including Einstein summation, free and dummy indices, and the index positions used for vectors, covectors, and bilinear forms.

A Einstein summation convention

  • Vector components use upper indices, covector components use lower indices, and bilinear-form components use two lower indices.The inverse bilinear form has two upper indices.Source: A Einstein summation convention · p. 15
  • A repeated index occurring exactly twice, once upper and once lower, is implicitly summed; such a dummy index may be renamed freely.For example, Jiθi = Jkθk.Source: A Einstein summation convention · pp. 15–16
  • An index appearing once is free, and expressions with the same index more than twice are avoided in this convention.An expression with one free index represents vector or covector components.Source: A Einstein summation convention · p. 16
  • Differentiation with respect to a covariant component Ji produces an upper index, while differentiation with respect to a contravariant component θi produces a lower index.The appendix also illustrates translating matrix symmetry and inverse relations into index notation.Source: A Einstein summation convention · p. 16

B Gaussian Measure Identity

The appendix motivates a Gaussian-measure identity by expanding a function around the origin and comparing its expectation with an expanded differential operator.

B Gaussian Measure Identity

  • The derivation expands f around the origin for θ drawn from a centered Gaussian and uses the vanishing of all odd moments.The displayed expansion includes terms through fourth order and an O(θ^5) remainder.Source: B Gaussian Measure Identity · p. 17
  • The identity follows by expanding the differential operator and comparing coefficients with the expectation-side expansion.The appendix says identities of this kind are used extensively, including in quantum field theory.Source: B Gaussian Measure Identity · p. 17

C Toy-Model Thermodynamics

The appendix collects derivations for Gaussian and Rosenbrock likelihood thermodynamics, with constants independent of T or J omitted.

C Toy-Model Thermodynamics

  • For the Gaussian likelihood, the canonical partition sum is computed by completing the square, and the entropy is independent of J.The derivation omits likelihood constants that do not depend on T or J.Source: C Toy-Model Thermodynamics · p. 17
  • For the Rosenbrock likelihood, the derivation sets x = θ0 and u = θ1 − c2x2 before completing the squares.The coordinate change preserves the integration measure: dθ0dθ1 = dxdu.Source: C Toy-Model Thermodynamics · p. 18
  • For the Rosenbrock likelihood, derivatives entering Eq. (11) are used to obtain the heat rate.The appendix presents this as the result of the preceding derivative calculation.Source: C Toy-Model Thermodynamics · p. 18

D Experimental Hyperparameters

Table 1 summarizes the numerical experiments’ hyperparameters and explains how thinning, skip steps, and the Rosenbrock c sweep are handled.

D Experimental Hyperparameters

Loading 2609.07660v1…