Compositional Information Geometry of Probabilistic Programs

1 Introduction

• The geometry must depend on a program’s implementation: random choices recorded in its trace affect geometry even when absent from the realised measure.This follows from the desiderata of compositionality, Fisher geometry for standard statistical models, and ordinary pullback for deterministic programs.Source: §1 · p. 3
• The program geometry contracts in a controlled way when execution information is partially or fully forgotten.The paper introduces statistics that retain less than the full trace, including the return value or a branch taken in a conditional.Source: §1 · p. 3

1.1 Structure

• For a program, the pullback metric combines a Fisher score term with an averaged pullback of the output metric along the program’s velocity.The resulting assignment is a contravariant lax monoidal functor; it recovers Fisher geometry for standard statistical models and ordinary pullback for deterministic programs.Source: §1 · p. 3, §1.1 · p. 4
• The functor’s geometry is also obtained by pulling back canonical geometry from a universal space of programs with a fixed trace space.The paper characterises this space and computes its curvature, Levi-Civita connection, and volume form on regular families.Source: §1.1 · p. 4
• A compositional weighted-measure version extends the construction to observe()/score(), with the original functor as its normalised case.The paper also studies how normalisation interacts with geometry.Source: §1.1 · p. 4
• The extension to continuous random variables uses diffeological quasi-Borel spaces and requires tail restrictions to ensure integrability of relevant quantities.The paper also relates its score-velocity geometry to Wasserstein-Fisher-Rao geometry, giving a compositional upper bound that is tight for regular finite output-measurable atomic families.Source: §1.1 · p. 4

2 Related work

• The paper bridges information geometry and probabilistic programming using category theory and programming-language semantics.It presents this as a connection between bodies of work with existing alternative approaches to program geometry.Source: §2 · p. 4

2.1 Information geometry

• Information geometry supplies intrinsic geometric structure for statistical families, while the Wasserstein-Fisher-Rao metric provides a comparison point for the paper’s construction [Chizat et al., 2015].The paper also places its work alongside non-parametric information geometry and warped-product metrics [Pistone and Sempi, 1995, Ay et al., 2017, Bauer et al., 2024; Fujitani, 2022].Source: §2.1 · pp. 4–5

2.2 Probabilistic and differentiable programming languages

• The paper’s probabilistic-programming semantics draw on work combining differential and probabilistic structure, while differing from work focused on differentiating expectations [Staton, 2020; Lew et al., 2023].PAP and ωPAP spaces could support a further generalisation to discontinuities, which this paper does not attempt [Huot et al., 2023].Source: §2.2 · p. 5

2.3 Categorical probability and categorical information geometry

• Categorical probability and information-geometry approaches offer related frameworks, with comparisons to divergence-enriched categories and non-commutative probability left for future work [Perrone, 2024; Ciaglia et al., 2026].The paper also notes synthetic differential geometry and concurrent Sample-Smooth Spaces as possible points of connection [Kostecki, 2009; Garðarsson and Perrone, 2026; Forré, 2026].Source: §2.3 · p. 5

2.4 Diffeological spaces, stratifolds, orbifolds

• Diffeological spaces provide a setting for the paper’s universal moduli space, alongside other singular-geometry approaches considered in earlier versions [Lê, 2020; Esser and Nielsen, 2021; Harms et al., 2023].The paper describes diffeological geometry as more permissive but more subtle than smooth-manifold geometry.Source: §2.4 · p. 5

2.5 Program geometry and inference over programs

• Other program-geometry work models programs as analytic singularities, studies singular model geometry, or applies tropical geometry [Clift et al., 2021; Plummer, 2026; Barbarossa and Pistone, 2025].Exponential families are not closed under composition, and an arrow-based formalism closing them under sequential composition did not describe the resulting geometry [Scherrer, 2017].Source: §2.5 · p. 6
• A geometric prior combining Jeffreys prior with minimum-description-length ideas is a motivation, but the proposed diffeological Hausdorff-Jeffrey measure is left for future work [Grünwald and Roos, 2019; Lê and Tuzhilin, 2020].The paper also points to a possible connection between observe() and sample() [Richardson and Bao, 2024].Source: §2.5 · p. 6

2.6 Natural gradient

• Natural-gradient methods approximate or exploit structure in Fisher information, but the cited approaches do not use the exact a-priori decomposition studied here [Kingma and Ba, 2017; Martens and Grosse, 2020; Sun and Nielsen, 2017].These approaches include Adam’s rough diagonal approximation and methods that exploit Kronecker or weak-subsystem structure.Source: §2.6 · p. 6

3 Language L and its jet trace semantics

• The language L is a first-order generative probabilistic language with algebraic Euclidean data types and finite random variables.Source: §3 · p. 6

3.1 Typing rules

• The language initially has a single randomness source, a Bernoulli variable parametrised by log-odds, with probability given by a smooth sigmoid.Source: §3.1 · p. 7

3.2 Types and pure terms

• Types denote algebraic Euclidean spaces, and pure terms denote functions between them.These spaces have standard measurable, smooth, and componentwise Riemannian structures; the real type carries Lebesgue measure.Source: §3.2 · p. 7

3.3 Denotational semantics in Markov kernels

• Programs denote Markov kernels, while pure terms are smooth and Boolean outputs are constant on each Euclidean input component.Consequently, a branch selected by a pure term remains locally constant along every tangent direction.Source: §3.3 · pp. 7–8

3.4 Dominated trace category

• A dominated trace presentation separates a program into a dominated stochastic trace kernel and a deterministic output map, so singular output laws can be handled through a fixed trace space.Every smooth deterministic map has a canonical finite trace presentation whose realisation is the corresponding Dirac measure.Source: §3.4 · pp. 8, 10
• Smoothness of trace densities alone does not ensure continuous Fisher information: in the example, the limit at zero is 4 but the directly computed value is 0.The text identifies smoothness of the square root of the density as a way to enforce smooth Fisher information.Source: §3.4 · pp. 8–9
• Finite trace presentations form a copy-discard category after quotienting by trace isomorphism, and realisation is a copy-discard-preserving functor to Stoch.The quotient resolves associativity up to trace-space isomorphism.Source: §3.4 · pp. 9–10

3.5 Dominated jet traces

• Jet traces annotate finite dominated traces with the density score and the output map’s derivative, called velocity; only input tangent directions are differentiated.This annotation adds no information to the trace presentation but facilitates later geometric decomposition.Source: §3.5 · p. 11
• Every dominated trace isomorphism induces a jet trace isomorphism, while the extra jet data is bookkeeping and J FinTrace has copy-discard structure.The assignment from a finite dominated trace to its jet trace is an isomorphism with the forgetful functor.Source: §3.5 · p. 11

3.6.1 Return

• Return is interpreted using the construction of Proposition 3.Source: §3.6.1 · p. 12

3.6.2 Bernoulli sampling

• Bernoulli sampling uses the sigmoid of its term as the flip probability, and its discrete output has zero tangent velocity.The two trace outcomes are 1 and 0, with probabilities given by the flip probability and its complement.Source: §3.6.2 · p. 12

3.6.3 Bind

• Bind combines trace spaces by product, adds the upstream and downstream scores, and uses downstream velocity evaluated at the extended tangent.Because the bind rule shares context, the downstream computation depends on the upstream output and its velocity.Source: §3.6.3 · pp. 12–13

3.6.4 If-then-else

• For a pure Boolean guard, the selected branch is locally constant, so the conditional takes that branch's score and velocity on the trace support.The score on the non-selected branch is irrelevant.Source: §3.6.4 · p. 13

3.6.5 Context rules

• Context rules implement variable reordering by coordinate permutation and weaken by projection, dropping unused tangent directions.Weakening preserves the program's score and velocity along the retained context tangent.Source: §3.6.5 · p. 13

3.6.6 Functorial semantics

• The interpretation T from program syntax to jet traces is a strong monoidal functor.Substitution precomposes the density and output maps with the pure term's semantics; the proof verifies the syntax quotient laws.Source: §3.6.6 · p. 14

4 Algebraic formulation and A

• Tangent pullback is contravariant: it preserves identities and reverses program composition when mapping metrics from outputs to inputs.The assignment A is also stated to be a functor and invariant under jet trace isomorphisms.Source: §4 · pp. 16, 18
• The program metric combines a Fisher contribution from the score with the expected pullback of output geometry along the velocity.Degenerate metrics accommodate tangent directions that are null, such as directions for ignored parameters.Source: §4 · pp. 15, 16, 18
• A is lax symmetric monoidal, with the metric product on input spaces given by the direct sum of component metrics.The positive-semidefinite metric cone also supports sums, i.i.d. sampling, hiding, weakening, and copying.Source: §4 · pp. 15, 18
• The score-velocity metric is presentation-sensitive: two presentations of the same constant-output kernel can have different energy.One presentation uses parameter-dependent trace probabilities despite returning the same constant output.Source: §4 · p. 19
• For a Bernoulli program, the metric in log-odds coordinates is p(1 − p) dx^2, while a fixed output map pulls back output metrics.The section also notes that deterministic programs have zero score contribution.Source: §4 · pp. 16, 19
• For stochastic bind, the metric adds the first program’s Fisher metric, a pulled-back Fisher metric for the continuation, and a pulled-back output-geometry term.The displayed decomposition uses context sharing for the bound variable.Source: §4 · p. 19

5 Geometric formulation and EΩ[Y ]

• A moduli space of programs must represent deterministic and stochastic families uniformly, with a metric that pulls back to program parameter spaces and recovers the earlier construction.The classifying space introduced here retains distinctions between presentations that differ only on zero-mass outputs; the paper postpones collapsing those distinctions until Section 7.Source: §5 · p. 20, §5.1 · p. 20

5.1 The decorated trace law

• For a fixed trace space, the classifying space represents dominated trace presentations by a trace law and output map, with tangent directions described by score-velocity pairs.The space does not yet identify presentations differing only on zero-mass outputs.Source: §5.1 · pp. 20–21

5.2 The universal metric on E◦Ω[Y ]

• The universal score-velocity metric combines a Fisher-Rao score term with a trace-law-weighted output transport term, which are orthogonal and form a warped product.When the output metric is positive-definite and κ > 0, the resulting metric is Riemannian; smooth output maps pull it back naturally, and output isometries induce isometries.Source: §5.2 · p. 22

5.2.1 Hessian structure on E◦ Ω[Y ] for Euclidean Y

• For Euclidean outputs, the score-velocity metric is the Hessian of a potential combining negative entropy in the trace law with a weighted squared norm of probability-weighted outputs.The potential is defined in probability-weighted coordinates.Source: §5.2.1 · p. 23

5.3 Square-root coordinates and cone geometry

• In square-root coordinates, the classifying space is isometric to a radius-2 hypersurface in a product of cones, and its curvature is generally nonzero, with flat cases limited to deterministic programs and a single coin flip.For connected Y, if each pointwise cone is flat, the classifying space has constant sectional curvature 1/4.Source: §5.3 · pp. 23–24

5.4 Pullback factorisation of A

• For strictly positive dominated trace presentations, the affine metric transformer factors as a pullback from the universal metric, locally after restricting to active trace support.The stated factorization fixes the trace space; the universal-trace-space version is deferred to the later DQBS theory.Source: §5.4 · p. 25

5.5 Intrinsic characterization of the score-velocity metric

• Under the stated score-independence, continuity, trace-invariance, additivity, and normalization conditions, the velocity term is forced to be trace-weighted κh, and the universal metric is uniquely Gκ,h.The authors note these assumptions are strong and leave a more compositional characterization as future work.Source: §5.5 · pp. 25–26
• Orthogonality makes the score-velocity metric decompose into separate score and velocity terms, with cross-terms disappearing.Source: §5.5 · p. 26
• The score and velocity parts are fixed by the stated hypotheses, completing the intrinsic characterization.Source: §5.5 · p. 26
• For a Riemannian metric h and κ > 0, Gκ,h is the unique Riemannian metric on E°Ω[Y] satisfying the stated projection characterization.Source: §5.5 · p. 27
• The projection is a Riemannian submersion onto the open Fisher simplex.Source: §5.5 · p. 27
• Its horizontal distribution is given by density tangents, and its fibre metric is κHh.Source: §5.5 · p. 27
• The degenerate Riemannian description applies in the interior; at the boundary, output values attached to zero-mass traces are collapsed.Source: §5.5 · p. 27
• Defining the metric on the whole EΩ[Y] requires quotienting by Q-almost-sure equality and using stratified spaces.The passage identifies this as one use of the DQBS machinery developed in Section 7.Source: §5.5 · p. 27

6 Weighted programs: observe(), score() and normalisation

• The weighted language interprets score as exponential weighting, allowing program denotations to be finite measures rather than probability measures.The trace weights need not sum to one; the ordinary finite trace presentation is the special case with total mass 1.Source: §6 · p. 27

6.1 Weighted affine functor A

• The weighted affine functor A averages output-metric pullbacks across execution traces and is contravariant and functorial.Its local construction follows the change along each possible execution branch.Source: §6.1 · pp. 28–29
• In the non-normalised setting, a nonzero expected score produces cross-terms that mix score and velocity contributions under composition.The construction restricts metrics to be independent of the accumulated-weight coordinate.Source: §6.1 · p. 28
• On the unit-mass slice, the weighted block transformer recovers the affine transformer from Section 4.For normalised traces the derivative of total mass vanishes, and the earlier construction's cross-terms do not arise from an initial block-diagonal form.Source: §6.1 · p. 29

6.2 Normalisation

• Normalization leaves the output map and velocity unchanged and commutes with composition when the second jet trace is standard.The section also states that normalization produces a standard jet trace and removes constant score weights.Source: §6.2 · p. 30
• Normalization is partial and nonlinear, and it fails to preserve composition: separately normalizing two stages gives a fair coin, while normalizing their composition gives a 1/3-biased coin.The discrepancy arises because separate normalization treats each input independently, while branches interact in the composed program.Source: §6.2 · pp. 30–31

6.3 Classifying spaces for Lscore

• For Lscore, the classifying-space metric is unchanged while its domain expands to a product of cones, and normalization acts as radial projection.Source: §6.3 · p. 31

7 Analytic formulation and LR

• LR adds control reals Rc, Gaussian introduction, and real-valued guards, while intentionally providing no conversion from smooth reals Rs to Rc.Rc is treated as having discrete topology and trivial smooth structure, while retaining the standard measurable structure.Source: §7 · pp. 31–32

7.1 Diffeological and quasi-Borel spaces

• DQBS is complete, cocomplete, and Cartesian closed, but expectation need not be smooth, so programs remain interpreted as dominated trace presentations rather than Kleisli morphisms.The failure of smoothness is witnessed by a Cauchy distribution and a bounded smooth function whose expectation is not differentiable at t = 0.Source: §7.1 · pp. 32–33

7.2 Trace spaces in DQBS

• Compatibility with the deterministic fragment forces Rc to have discrete diffeology and zero internal tangents, with coerce : Rc → Rs smooth but not its reverse.The D-topology on Rc is consequently discrete.Source: §7.2 · pp. 34–35

7.3 Polynomially bounded maps

• LR trace presentations use DQBS morphisms for smooth densities and output maps, and their jet-trace tangent domains are interpreted as internal tangent spaces.Discrete diffeology on trace spaces makes their tangent contribution trivial; integrability issues require a further restriction to a concrete subcategory.Source: §7.3 · p. 35

7.3.1 Integrability of score

• In the example, the score is in L2 for 0 < α < 1/4, and in L1 but not L2 for 1/4 ≤ α < 1/2.The example uses a finite program. Marginalization can turn a non-2-integrable score into an L2 output score, and considering programs intensionally can limit expressivity.Source: §7.3.1 · p. 36

7.4 Jet semantics for LR

• The DQBS jet semantics extends L to LR and preserves the property that pure Rc-valued terms depend only on control coordinates and coproduct tags, not smooth-real coordinates.Coercion is interpreted as identity on points.Source: §7.4 · p. 37
• Jet trace isomorphisms over DQBS form an equivalence relation and a congruence for jet-trace composition and tensor product.Source: §7.4 · p. 37
• With polynomially bounded primitives, finite LR semantics factors through a strong monoidal jet semantics.The construction interprets T as putting DQBS structure on types.Source: §7.4 · p. 37

7.5 Functor A and score-velocity metric G in LR

• Polynomial-growth restrictions on LR primitives and metrics address integrability on noncompact outputs, but exclude some primitives at a cost to compositionality.Extending LR to unnormalised measures would also require changing score(w) from multiplying the weight by exp(w) to another polynomially bounded function.Source: §7.5 · p. 38
• Gaussian averaging preserves polynomial boundedness, and Theorem 6 extends the functor A with centered scores and the conclusions of Theorem 2.A assigns polynomially bounded metrics to objects and maps h to g_F + F♯h; the proof uses Gaussian averaging to control the Gaussian-random-variable case.Source: §7.5 · pp. 38–40
• The score-velocity metrics Gκ,h and G are extended to LR.The construction retains properties of A from the finite case, including affinity, symmetry, and invariance under trace isomorphisms.Source: §7.5 · p. 40

7.6.1 Exponentials Y Ω

• For finite or countable trace spaces, DQBS maps from Ω to an algebraic Euclidean space Y are exactly measurable functions, and evaluation identifies mapping-space tangents with pointwise tangents.The construction moves from a small target to the full DQBS to form the spaces of output maps.Source: §7.6 · p. 40, §7.6.1 · p. 41

7.6.2 2-forms on DQBS and EΩ[Y]

• Restricted to plots satisfying the integrability conditions, Gκ,h defines a plotwise 2-form and a positive-semidefinite form on the internal tangent space of EΩ[Y].The restrictions address non-integrability that can vary across coordinate systems.Source: §7.6.2 · p. 42
• Every LR program with semantics F defines a geometric plot b_F : X → EΩ[Y].The classifying map is b_F = ⟨q, curry(o)⟩.Source: §7.6.2 · p. 42

7.6.3 Relation to diffeological information geometry

• The projection QY from EΩ[Y] to Dλ(Ω) forgets the output map, so each plot of EΩ[Y] determines a dominated diffeological statistical model.Source: §7.6.3 · p. 43
• The density component is exactly the canonical diffeological Fisher geometry, while the decorated space also includes a parameter-dependent output map and therefore requires a choice of output metric.Source: §7.6.3 · p. 43

7.6.4 Pullback factorisation and square-root coordinates for LR

• For the real-valued extension, the construction restricts metrics to those whose pullbacks are admissible; defining curvature, Levi-Civita connections, or infinite-dimensional Riemannian volume remains future work.A size structure on the large program space is not chart-compatible under the diffeomorphism condition used in DQBS.Source: §7.6.4 · p. 43

7.7 Collapsed moduli space EΩ[Y ]

• The collapsed moduli space identifies trace-density pairs when densities agree λ-a.s. and output maps agree almost surely under the induced measure.This quotient removes distinctions between output maps that differ only on zero-probability outcomes.Source: §7.7 · p. 44
• Programs map into a single stratum on each connected component, and outputs attached to zero-mass traces carry no geometric information.For a connected neighbourhood, the trace laws are mutually absolutely continuous; the associated metric defines a plotwise metric on the collapsed space.Source: §7.7 · p. 44

7.8.1 Universal trace spaces for L

• For finite random variables, the universal trace space is countable with a σ-finite counting measure, and compatible fixed-trace forms induce a global plotwise form on the universal moduli space.The universal space quotients by finite trace permutations and almost-sure equality of outputs; its metric need not arise from a norm on the internal tangent space.Source: §7.8.1 · pp. 45–46
• The universal moduli space gives a precise interpretation of program-family intersections as equality of points and supports statements about submodels and transverse intersections.Source: §7.8.1 · p. 46

7.8.2 Universal trace spaces for LR

• Hiding the trace contracts the full-trace metric to an output metric for Y ∼ N(a + b, 2), making direction a − b null; a universal moduli space for LR remains future work.Replacing finite permutations by Borel isomorphisms would identify the family N(θ, 1) with the single point N(0, 1).Source: §7.8.2 · p. 47

8.1 Forgetting the trace

• A statistic on the trace defines projected score-velocity geometry, but only observing the full trace preserves compositionality.The statistic can retain the full trace, forget it entirely, or observe something in between.Source: §8 · p. 47

8.2 Distribution theory and the Wasserstein-Fisher-Rao geometry

• WFR provides a presentation-independent comparison because it minimises over possible presentations, whereas output-conditioned geometry can still differ between programs realising the same statistical model.Different implementations can trade off score and velocity contributions.Source: §8.2 · p. 49

8.2.1 Distribution theory

• Distributional derivatives let measure families be differentiated without a common dominating measure, while reaction-transport representations express tangents through reaction rates and transport velocities.The framework includes derivatives such as those of Dirac measures, which need not themselves be probability measures.Source: §8.2.1 · pp. 49–50

8.2.2 Wasserstein-Fisher-Rao metric

• WFR assigns the least energy among reaction-transport representations of a distributional tangent, and assigns +∞ when no such representation exists.Program tangents always admit such a representation.Source: §8.2.2 · p. 51

8.2.3 Distributional tangents for probabilistic programs

• A program’s realised output-measure derivative is a distributional tangent represented by output-conditioned expectations of its trace score and velocity.The output score generally is not just the conditional expectation of the trace score; that formula holds in special cases, including discrete output domains.Source: §8.2.3 · pp. 51–52
• Theorem 10 orders the extensional local WFR energy below output-projected score-velocity, which is itself bounded by presentation-level score-velocity energy.The first inequality follows because WFR minimises over reaction-transport representations; the second is a trace data-processing inequality.Source: §8.2.3 · p. 52

8.2.4 Regular finite atomic strata

• On a regular finite atomic stratum, the WFR and output-conditioned score-velocity forms coincide, while the program-level form may be larger when the trace contains unused randomness.For two output-measurable traces realising the same measure, the corollary gives a uniqueness result for their forms.Source: §8.2.4 · pp. 52–53

8.3 WFR geometry of unnormalised measures

• WFR also applies to unnormalised measures, where its reaction term can change mass; for normalised measures, the reaction term is centred.The section aims to rederive the WFR, output, and standard-form inequalities in the unnormalised setting.Source: §8.3 · p. 53
• Normalising a weighted trace removes the mean reaction and rescales the studied metrics by the trace mass.Theorem 11 applies this construction to a weighted jet trace realising an unnormalised kernel and its normalised measure.Source: §8.3 · p. 55

8.4 WFR geometry for LR

• Two output-measurable traces can realise the same measures and trace densities, with equal scores but different velocities.This shows that the uniquely determined velocity property for regular finite atomic output-measurable families need not hold for continuous output distributions.Source: §8.4 · p. 55

8.5 Natural gradient

• The experiment geometry theorem gives scores, velocities, and an induced score-velocity form for n-observation experiments.The same formula handles fixed or learned covariates, shared hyperparameters, and computed inputs.Source: §8.5 · p. 56

8.5.1 Riemannian and degenerate Riemannian gradients

• The natural gradient for a possibly degenerate score-velocity form lies in the tangent space modulo the metric kernel, and finite outputs recover the usual trace Fisher natural gradient.The program supplies the metric and the objective supplies the covector; the objective need not be a log-likelihood, and the resulting vector field may not be globally smooth. For real outputs, likelihood evaluation requires choosing a representative of the score equivalence class.Source: §8.5.1 · p. 56

9.1 Loops

• A loop extension would use the score martingale’s predictable quadratic variation, but requires output-velocity and metric-growth tail estimates; nontermination raises subprobability-score issues.Conditioning on termination or adding a partial order structure are suggested alternatives.Source: §9.1 · p. 57

9.2 Higher-order functions and recursion

• Higher-order functions and recursion require solutions for nontermination, fixed points, differentiation through limits, admissible metric suprema, and probabilistic tail and score-martingale issues.For LR, polynomial bounds may fail under recursion even when each primitive and terminal metric is polynomially bounded.Source: §9.2 · p. 57

9.3 Solomonoff-Jeffreys prior and Bayesian non-parametric regression

• A Solomonoff-Jeffreys prior could combine a score-velocity metric within program strata with a prior over program skeletons, but the skeleton prior is non-canonical and summing derivations may be difficult.Further obstacles include improper or zero normalizers and unresolved requirements for Turing-complete languages.Source: §9.3 · pp. 57–58

9.4 Trans-dimensional Riemannian Manifold Hamiltonian Monte Carlo

• Trans-dimensional Riemannian HMC could run within program skeletons and occasionally propose reversible dimension changes, but the stratified space complicates smoothness and movement between strata.The authors suggest standard Metropolis-Hastings could provide a crude correctness route, followed by practical testing for program synthesis.Source: §9.4 · p. 58

9.5 Natural gradient and variational inference

• The natural gradient has a variational formulation whose score- and velocity-energy objective can be estimated from program executions without constructing or inverting G.Implementation and development for variational inference are left for future work.Source: §9.5 · p. 58

9.6 Extension to dualistic geometry; 9.7 Relationship to global divergence geometry

• The framework can pull back dualistic geometry, including a metric and cubic tensor, while pulling back a smooth output divergence also composes and locally induces the score-velocity metric.Extending these constructions and studying their computational consequences remain open directions.Source: §9.6 · pp. 58–59, §9.7 · p. 59

9.8 Chentsov’s theorems for program-generated families

• The current score-velocity characterization is weak; stronger results could target selected program families or identify classes of families where Chentsov-like theorems apply.The passage contrasts this with characterizations of Fisher information for finite-dimensional and arbitrary parametric families.Source: §9.8 · p. 59

9.9 Polynomially controlled maps

• A principled theory of polynomially controlled stochastic kernels and sized spaces could improve on the current implicit category of LR-generated programs and its sufficient, non-necessary primitive restrictions.The proposed direction may use Nash, quasi-Nash, or polynomially bounded definable geometry.Source: §9.9 · p. 59

A Typing rules of L and LR

• Structural typing rules apply to both languages; L uses Rs = R and omits rules involving Rc.The appendix lists rules for units, variables, pairs, and sum injections, among others.Source: A Typing rules of L and LR · p. 64
• The pure-term rules in L include typing judgments for unit, variables, pairs, and sum-type injections.The figure is identified as the typing rules for pure terms in L.Source: Figure 6 · p. 64, A Typing rules of L and LR · p. 64
• LR’s real-term rules use separate smooth and control real types, with polynomially bounded smooth primitives; only control terms cast to Bool, and conversions to smooth reals go one way.The rules are repeated for Rs and Rc.Source: Figure 7 · p. 64
• LR probabilistic computations include binding a sampled computation before continuing and conditional computations with same-typed branches.The figure is captioned as typing rules for probabilistic computations in LR.Source: A Typing rules of L and LR · p. 65, Figure 8 · p. 65

B Diffeological and quasi-Borel spaces

• Diffeological spaces support smooth parameterized families without requiring fixed-rank manifold charts, fitting program spaces built from varying-dimensional parameter manifolds and quotients.The quotient maps may have critical points.Source: B Diffeological and quasi-Borel spaces · p. 66
• Diffeology controls smooth parameter dependence, while quasi-Borel structure controls random elements and probability kernels; smooth plots are required to be measurable.Source: B Diffeological and quasi-Borel spaces · p. 66

B.1 Plots and smooth maps

• A diffeology selects maps from open Euclidean domains as plots, and a diffeological space is a set equipped with such a collection.The stated axioms include constants, smooth reparameterization, and local checking over open covers.Source: §B.1 · p. 66
• A smooth map between diffeological spaces sends every plot of its domain to a plot of its codomain.Identities and composites are smooth, so these spaces and maps form a category.Source: §B.1 · p. 66
• Standard diffeology recovers ordinary smooth maps between manifolds, whereas discrete diffeology has locally constant plots and coarse diffeology allows every map as a plot.Source: §B.1 · p. 66
• Every set map from a discrete diffeological space is smooth, but a smooth map from a connected manifold into one must be constant.More generally, such maps from manifolds are locally constant.Source: §B.1 · p. 66

B.2 Subspaces, products, coproducts, and quotients

• In the coproduct diffeology, smooth maps locally stay within one summand, so a program skeleton is locally constant along an ordinary smooth parameter family.Quotient plots locally lift to the space being quotiented, without a constant-rank or regular-value hypothesis.Source: §B.2 · p. 67

B.3 Topology, internal tangents, and plotwise forms

• Diffeologies carry smooth information through plots that cannot in general be recovered from the topology they induce.Every smooth map is continuous for the D-topologies, but distinct diffeologies can induce the same topology.Source: §B.3 · p. 67

B.4 Quasi-Borel spaces and probability

• Quasi-Borel spaces specify admissible random elements; their probability laws are represented by random elements and ordinary measures, identified by bounded quasi-Borel integrals.Their function spaces are cartesian closed, and the probability construction gives a Markov category that recovers ordinary measures and kernels on standard Borel spaces.Source: §B.4 · pp. 67–68

Bernoulli regression as the score-only case

• Whether covariates are fixed or unknown is determined by the experiment map, which selects θ alone or includes the covariates in the unknown state.The section describes both a fixed-covariate experiment and an errors-in-variables variant.Source: Bernoulli regression as the score-only… · p. 69
• The experiment metric is obtained for the specified fixed-covariate setup, while the errors-in-variables setup treats covariates as unknown.The supplied passages identify the metric and log-likelihood covector but do not give their expressions.Source: Bernoulli regression as the score-only… · p. 69
• The full-context score-transport form differentiates both the regression parameter θ and latent covariates x_i.Real covariates are differentiated like real parameters; holding x fixed later is a pullback of this tensor.Source: Bernoulli regression as the score-only… · p. 69

D Example: composition of Gaussian mixtures

• With Euclidean output tensor h = dy2, the trace-level score-transport metric has no transport summand.The programs are composed by the bind rule, with trace space isomorphic to R2 ⊔ R2.Source: D Example: composition of Gaussian… · p. 70
• The realised output reveals the branch and second Gaussian trace, but only the sign of z.The component supports are respectively (-∞, 0) and (0, ∞).Source: D Example: composition of Gaussian… · p. 70
• The first inequality need not be an equality because the projected score-transport lift may not be the minimum-energy reaction-transport representation.The second inequality reflects loss of the magnitude of z under observation.Source: D Example: composition of Gaussian… · p. 71

Parameter sharing

• The sharing map r sends (θ, m) to (θ, m, m), duplicating m across the two shared coordinates.Source: Parameter sharing · p. 71
• Parameter sharing is ordinary deterministic pullback of the chosen score-transport geometry.The two experiment metrics are pulled back along the sharing map.Source: Parameter sharing · p. 71

• With the same Euclidean output tensor, sampling u and v and returning −χ(a + coerce(u)) or χ(b + coerce(v)) yields the same realised output family.Source: Different implementation of the same… · p. 71
• In that presentation, a and b disappear from the score and enter the output velocity.Source: Different implementation of the same… · p. 71
• The realised kernels agree, but the presentation-level score-transport metrics need not, because score-transport is intensional.Source: Different implementation of the same… · p. 71
• Both presentation-level metrics dominate the same extensional WFR form.Source: Different implementation of the same… · p. 71