PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics of Information Geometry - Part II: Small-Step Active Inference on the Probability Simplex

C. Emre Koksal, Deniz Sargun

arXiv:2609.11187v1cs.ITcond-mat.stat-mechmath.IT

TL;DR

The paper asks how an agent should move a belief distribution toward a preferred target through a sequence of actions. It develops a KL-constrained variational framework for that motion and finds that small locally optimal steps yield a least-action path with exponentially tilted updates.

  • Problem

    The paper asks what sequence of distributions optimally transports a system from an initial point to a desired terminal distribution.

    Source: §1 · p. 2

  • Method

    It formulates sequential distributional motion using relative free and potential energies and a per-step KL kinetic constraint, then derives a least-action principle and exponentially tilted updates.

    Source: §1 · pp. 2–3

  • Results

    Sufficiently small local moves are more efficient than large global jumps, and their optimal sequence yields a least-action path with updates that exponentially tilt the current distribution toward the preferred one.

    Source: §1 · p. 2, §7 · pp. 13–14

  • Takeaways & Limitations

    State-dependent or geodesic costs can be represented by exponentially tilting the preferred distribution without changing the least-action structure.

    Source: §7 · p. 14

    A direct thermodynamic energy interpretation requires identifying the preferred distribution with a Gibbs equilibrium distribution at temperature τ; otherwise the quantities are dimensionless.

    Source: §2 · p. 5

Abstract

from arXiv · show

This paper is the second in a two-part investigation of the physics of information geometry. While Part I develops a physical foundation for distributional motion on the probability simplex, the present paper studies how that framework manifests in active inference. The treatment is fully self-contained and does not require familiarity with Part I. We focus in particular on active inference through small distributional steps and the geometric structure induced by such local motion. Starting from an initial distribution, an agent evolves its belief state toward a final target distribution through a sequence of constrained updates. We define a relative free energy functional with respect to the preferred distribution and extend it to a relative potential energy analogous to the Helmholtz/Gibbs free-energy decomposition. The evolution is subject to a per-step kinetic constraint expressed through the Kullback-Leibler (KL) divergence between consecutive distributions, which serves as a discrete kinetic energy on the probability simplex. Using the information-geometric Pythagorean theorem on KL balls, we show that sufficiently small local moves dominate large direct jumps, and that greedy maximization of free-energy reduction is globally optimal under the kinetic constraint. This leads to a sequential variational principle in which the optimal trajectory minimizes the associated Lagrangian of the optimization problem. Similar to classical mechanics, the Lagrangian takes on the form as the difference between the kinetic and potential terms, establishing a least-action principle for distributional motion on the simplex. The resulting optimal update admits a closed form as an exponentially tilted version of the current distribution toward the preferred distribution, parametrized by an inverse-temperature-like multiplier. We further extend the framework to incorporate state-dependent geodesic...

1 Introduction

The paper asks how to choose a path of distributions toward a target, and develops a sequential variational framework that treats KL divergence as kinetic cost and yields a least-action principle. Its small-step analysis and exponential-tilting updates extend to state-dependent costs by shifting the preferred distribution.

1 Introduction

  • The framework defines relative free energy and potential energy, assigns each transition a KL-based kinetic cost, and minimizes accumulated action along the distributional path.The constrained sequential optimization maximizes total free-energy reduction, with a greedy optimal structure and exponentially tilted updates toward the preferred distribution.Source: §1 · pp. 2–3
  • Unlike geodesic information geometry, entropy-regularized transport, and stepwise active inference, this framework derives a variational principle for the distributional path itself.Free energy, entropy, and KL-based kinetic cost combine in an action functional that determines the path toward a preferred distribution.Source: §1 · pp. 2–3
  • Small local moves are more efficient than large global jumps under the KL constraint, and the small-step free-energy reduction approaches the KL cost.The information-geometric Pythagorean relation supports greedy local optimization, which yields a least-action formulation for the path.Source: §1 · p. 3
  • State-dependent or geodesic costs can be incorporated by exponentially tilting the preferred distribution, reducing the generalized problem to the original least-action formulation.This reveals an equivalence between external geometric costs and exponential tilting in information geometry.Source: §1 · pp. 3–4

2 Model

The model represents an agent’s belief evolution as sequential motion from an initial distribution toward a preferred distribution. It defines relative free and potential energies and uses KL divergence between successive distributions as a discrete kinetic cost, while qualifying the thermodynamic interpretation.

2 Model

  • The trajectory {qt}t≥0 describes sequential motion on the probability simplex from initial distribution π0 toward preferred distribution ppref.The paper treats π0 as arbitrary and does not require it to be a Gibbs equilibrium distribution.Source: §2 · p. 4
  • Relative free energy measures progress from the initial free-energy gap toward the preferred distribution, while relative potential energy tracks how much of the initial potential has been reduced.The relative free-energy definition is distinguished from the thermodynamic formulation by reversing the ordering of distributions in the KL divergence.Source: §2 · pp. 4–5
  • KL divergence between consecutive distributions is the discrete kinetic cost of informational motion, and for infinitesimal changes it reduces to the Fisher–Rao metric.The direction DKL(qt∥qt−1) measures the cost of selecting the new distribution relative to the previous belief state.Source: §2 · p. 5
  • The model’s information-geometric quantities are dimensionless; a physical energy interpretation requires scaling by kBτ and identifying ppref with a Gibbs equilibrium distribution.Under that condition, kBτ DKL(q∥ppref) is the nonequilibrium free-energy excess relative to the Gibbs state.Source: §2 · p. 5

3 Main Insight: Information-Geometric Pythagorean Theorem and Incremental Motion

The paper uses KL-constrained local moves toward a preferred distribution to establish an information-geometric efficiency result. In the infinitesimal limit, free-energy gain and kinetic cost become locally balanced, motivating a least-action path.

3.1 KL Balls and Optimal Incremental Motion

  • The optimal next distribution is the information projection of the preferred distribution onto the KL ball of admissible next states.The KL ball represents distributions reachable under the per-step kinetic-energy budget.Source: §3.1 · p. 6
  • The Pythagorean relation lower-bounds each optimal step’s free-energy gain by its informational step size, making sufficiently small local moves more efficient than large direct jumps.The preferred distribution is projected onto a KL ball centered at the current distribution.Source: §3.1 · pp. 6–7

3.2 Small-Step Limit and Symmetry of KL Divergence

  • Although kinetic energy uses the reverse KL divergence, it becomes asymptotically equivalent to free-energy gain in the infinitesimal regime, balancing informational motion and dissipation locally.This balance underpins the least-action path formulation.Source: §3.2 · pp. 7–8

4 Problem Statement

The problem is to move from an initial distribution toward a preferred distribution through updates subject to a per-step KL constraint. The paper formulates this as a sequence of locally optimal steps that terminates when the preferred distribution becomes reachable within one step.

4 Problem Statement

  • Each update is constrained by a maximum kinetic cost δ, geometrically defining a KL ball around the previous distribution.The trajectory begins at π0 and moves toward ppref through a sequence of constrained updates.Source: §4 · p. 8
  • The greedy least-action process continues until the preferred distribution lies within the current KL ball, then terminates with exact arrival at that target.The stopping rule preserves the local-step structure while ensuring arrival once the target is admissible.Source: §4 · p. 8
  • Repeated locally optimal steps maximize free-energy reduction under the kinetic constraint and yield the least cumulative action along the trajectory.The construction evolves forward from the initial distribution toward the preferred one.Source: §4 · pp. 8–9

5 Solution: Lagrangian and the Principle of Least Action

The paper casts constrained distributional updates as a least-action problem and derives an exponentially tilted update toward the preferred distribution. In a three-state illustration, sequential motion uses less kinetic energy than a direct jump, with savings above 94% at the smallest tested step size.

5.1 Lagrangian and Least Action

  • The instantaneous information-geometric action has a kinetic-minus-potential structure: it penalizes distributional motion while rewarding progress in the preferred-distribution landscape.The kinetic term resists abrupt belief changes, while the potential term drives the distribution toward states favored by ppref.Source: §5.1 · p. 9

5.2 Least-Action Path: Tilted Distribution Dynamics

  • The closed-form update multiplicatively tilts the current distribution by ppref(x)^(1/λ); λ controls tilt strength, with large λ yielding a conservative update.When ppref is outside the admissible KL ball, the kinetic constraint is active; the path stops at ppref once it becomes reachable.Source: §5.2 · p. 10

5.3 Numerical Illustration

  • At δ = 0.001, sequential kinetic cost saves more than 94% versus the direct-jump cost; from δ = 0.4 to 0.001, stopping time rises from 1 to 18 while total cost falls from 0.297 to 0.017 nats.The illustration uses π0 = (1/3, 1/3, 1/3) and ppref = (0.7, 0.2, 0.1).Source: Figure 2 · p. 11, §5.3 · pp. 11–12

6 Geodesic Cost on Variational Distribution

The framework incorporates costs for occupying states or traversing regions of the simplex into the least-action problem. These costs preserve its basic structure by exponentially tilting the preferred distribution, shifting the path’s effective destination.

6 Geodesic Cost on Variational Distribution

  • State-dependent costs can represent penalties for occupying particular states or traversing regions of the probability simplex.The paper gives energetic penalties, environmental constraints, risk preferences, and application-specific geodesic penalties as examples.Source: §6 · p. 12

6.1 Greedy Least-Action Problem with Geodesic Cost

  • The greedy least-action problem treats a nonnegative state-dependent cost as an expected cost and equivalently as a geodesic penalty on motion through regions of the simplex.The equivalence is expressed by making the state cost depend on the distribution at that point on the simplex.Source: §6.1 · p. 12

6.2 Reduction to the Original Problem

  • Adding a geodesic cost preserves the problem’s basic structure by modifying the preferred distribution through an exponential tilt.The paper describes the resulting tilt as suppressing states with large costs by a factor e^-γc(x).Source: §6.2 · p. 13, §6.3 · p. 13

6.3 Implications

  • The cost shifts the effective attracting point from the original preferred distribution to a cost-adjusted destination, bending the trajectory toward that destination without changing the least-action path structure.The paper relates this exponential-tilting structure to entropy-regularized optimal transport.Source: §6.3 · p. 13

7 Discussion and Conclusion

The paper formulates belief evolution on the probability simplex as a discrete least-action process, balancing attraction toward preferred states against resistance to abrupt informational motion. Its results characterize locally optimal updates, including how state-dependent costs reshape the effective destination while preserving the least-action structure.

Discussion and Conclusion

  • The least-action formulation balances attraction toward preferred states, represented by relative potential energy, against resistance to abrupt informational motion, represented by kinetic energy.The paper presents this as a thermodynamic and geometric structure for belief evolution on the simplex.Source: §7 · p. 13
  • Sufficiently small local moves are more efficient than large global jumps, and small-step KL symmetry locally balances free-energy reduction against kinetic expenditure.The paper connects this result to a least-action principle built from locally optimal variational steps.Source: §7 · p. 13
  • Each update has a closed form: it exponentially tilts the current distribution toward the preferred distribution rather than using Euclidean interpolation on the simplex.The paper describes the path as progressive information-geometric deformations of the belief state.Source: §7 · p. 14
  • State-dependent or geodesic costs preserve the least-action structure by exponentially tilting the preferred distribution and deforming the effective potential landscape.The paper states that the total action along a trajectory is bounded below by the free-energy difference between its endpoints, analogously to the Jarzynski equality and Crooks fluctuation theorem [7].Source: §7 · p. 14

A Proof of Lemma 1

The proof uses KL-divergence expansions to establish the stated lemma, with a Taylor remainder bound for the local expansion and Pinsker’s inequality under a positive minimum-probability condition.

A Proof of Lemma 1

  • The second part of the lemma assumes qmin = min_x qt−1(x) > 0 and invokes Pinsker’s inequality under the KL constraint.The supplied passage identifies these as the conditions and inequality used in that proof step.Source: A Proof of Lemma 1 · p. 16
  • The proof compares expansions of two KL divergences and subtracts them to obtain the lemma’s stated relation.The displayed proof also subtracts the linear term before expressing the expansion through ψ(u).Source: A Proof of Lemma 1 · pp. 16–17
  • For |u| ≤ 1/2, Taylor’s theorem bounds the remainder function ψ(u) by C|u|^3 for some constant C.The proof defines ψ(u) = (2 + u) log(1 + u) − 2u.Source: A Proof of Lemma 1 · p. 17

B Proof of Theorem 2

The proof rewrites the optimization using an unnormalized tilted measure and its normalizer. Since the remaining term is independent of qt, minimizing the objective reduces to minimizing KL divergence to the normalized tilted distribution.

B Proof of Theorem 2

  • The proof defines an unnormalized tilted measure from the current distribution and preferred distribution, together with its normalizing constant.The measure is written as eqt(x) ≜ qt−1(x) ppref(x)^(1/λt).Source: B Proof of Theorem 2 · p. 18
  • Because the second term is independent of qt, minimizing the objective over the simplex is equivalent to minimizing KL divergence to the normalized tilted distribution.The proof then uses nonnegativity of KL divergence to identify the minimizer.Source: B Proof of Theorem 2 · p. 18
  • The passage states that t = ¯qt, identifying this equality as proving (17), without describing either quantity as a minimizing or normalized tilted distribution.Source: B Proof of Theorem 2 · p. 18
Loading 2609.11187v1…