PaperlayerView exact revision on arXiv

Source-linked AI summary

When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning

Wang Ru

arXiv:2610.28849v1cs.LGcs.AI

TL;DR

The paper asks when the Gaussian prior commonly used in latent diffusion becomes limiting for RL fine-tuning of multimodal data. It links latent mode overlap to collapse behavior and reports a 71× empirical collapse-boundary gap in Exp7; the theory’s boundary predictor assumes separated modes.

  • Problem

    The paper asks when the Gaussian prior used in latent diffusion limits RL fine-tuning, since the VAE KL term can compress distinct modes into an overlapping latent.

    Source: §1 · p. 2

  • Method

    The paper analyzes latent mode separation as a mediator of RL collapse, examining saddle-region score error and moving geometry by changing KL weight under a fixed Gaussian prior.

    Source: §1 · p. 2, §4.2 · p. 11, §5 · p. 25

  • Results

    71× was the empirical βc gap in overlap-prone geometry in Exp7, and dyn/var amplification dominated the gap rather than the variational prediction.

    Source: §3.9 · p. 10

  • Takeaways & Limitations

    A mode-separating prior is necessary in the overlap regime when coverage matters, because switching the prior can move latent geometry out of the amplifier’s reach.

    Source: §1 · p. 3

    The variational bound assumes separated modes; when dmin < 4σ0, the variational βc is a degenerate zero lower bound and the crossover predictor fails.

    Source: §3.8 · p. 9

Abstract

from arXiv · show

Latent diffusion models almost universally place a Gaussian prior on the VAE latent space. For base generation this default is unproblematic; under reinforcement-learning fine-tuning, however, it carries a structural limitation. The standard VAE's KL term does not merely ignore mode structure - it actively taxes separation and tightness. When data are multimodal, the ELBO optimum compresses distinct modes into an overlapping latent. RL fine-tuning amplifies this overlap: the score network is under-resolved on the saddle region between modes, dynamic-to-variational amplification rises sharply, and the collapse boundary beta_c is pushed far above its variational scale. On the real alanine-dipeptide Boltzmann distribution, a Gaussian-prior VAE collapses at beta_c = 0.224 while a mode-tracked Gaussian-mixture prior collapses at 0.00316, a 71x gap from the same pipeline. We trace this gap to a single geometric mediator, d_min/(2 sigma_0), and verify the causal chain through controlled interventions: moving geometry at fixed prior, and switching prior at matched geometry (full mediation; prior-label coefficient 0.995x, 95% CI [0.79, 1.25]). A three-source decomposition localises the amplification to the learned policy (zero ODE-solver contribution, bounded anchor mismatch); the coverage plateau is an unconditional benefit of mode-separating priors that no amount of RL capacity recovers. The remedy is conditional: a non-Gaussian prior is necessary precisely when the latent geometry would fall into overlap and the task is coverage-sensitive. A machine-checked theory skeleton supports the KL pricing structure - a divergent compactness tax under a Gaussian prior versus a finite assignment fee under a mixture - and the position-versus-plateau distinction delimits what is finite-resource from what requires changing the prior.

1 Introduction

This section examines how Gaussian-prior compression of multimodal data into overlapping latents can make KL-anchored RL fine-tuning vulnerable to coverage collapse, and how that risk relates to latent geometry. It describes when mode-separating priors improve coverage or lower the collapse boundary, and notes that in overlap-prone settings dynamic error amplification dominates the boundary gap.

1 Introduction

  • The Gaussian ELBO taxes separation and tightness, compressing distinct modes into overlap; RL then under-resolves the saddle region and sharply amplifies dynamic-to-variational error.The paper frames the limitation as upstream of reward over-optimisation: it can arise even with a well-designed reward and generous KL anchoring, and the prior is needed when overlap coincides with a coverage-sensitive task.Source: §1 · p. 2
  • On alanine dipeptide, the Gaussian prior has d_min/(2 sigma_0) = 0.81 and beta_c = 0.224, versus 1.03 and 3.16 × 10^-3 for the mixture prior—a 71× gap.The variational beta_c ratio is only 0.69×; the paper attributes the much larger empirical gap to dynamic-to-variational amplification in the overlapping Gaussian geometry.Source: §1 · p. 2, §3.5 · p. 7
  • A mode-preserving mixture prior keeps latent modes separated and coverage higher: reported comparisons include 0.70→1.00 in 1D, 0.58→0.73 in crowded 2D, and about 0.67→1.0 on alanine dipeptide.The paper presents the prior change as necessary in the overlap regime because the Gaussian-prior geometry cannot be independently tuned to escape the amplifier.Source: §1 · pp. 2–3

2 Related Work

  • Prior choice has been studied for VAE reconstruction and disentanglement, while latent diffusion commonly defaults to Gaussian priors; prior geometry’s effect on RL collapse boundaries is the paper’s focus.The paper distinguishes this upstream mechanism from reward-side diversity loss through over-optimisation: the two failure modes are complementary.Source: §2.1 · p. 4, §2.2 · p. 4, §2.3 · p. 4

3 Theory

  • Under reverse-KL RL with quadratic reward, narrow priors remain rigid while wide priors collapse toward the target; the crossover scale is beta_cross = 2 lambda sigma_0^2.The theory identifies prior width as a first-order determinant of the critical beta_c.Source: §3 · p. 4, §3.2 · p. 5

3.3 ELBO origin of sigma_0

  • Gaussian tightness incurs an unbounded KL cost as width approaches zero, whereas matching a mixture component costs at most its assignment fee; training comparisons show the mixture can widen separation.In the controlled lattice experiment, Gaussian separation stalls at 1.14 ± 0.07 while mixture separation reaches 5.0 (3.3–6.6), a 4.4× gap; alanine’s Gaussian-to-mixture width ratio is about 36×.Source: §3.3 · p. 6, Figure 2 · p. 6

3.4 The crossover scale predicts the empirical beta_c

  • The crossover predicts the sign, not magnitude, of the empirical boundary difference: alpha is 0.12 for Gaussian and 2.24 for GMM, while its geometry-dependent scaling prevents a universal value.The Gaussian boundary can fall inside the wide-prior regime when modes overlap; the GMM boundary can lie in the narrow-prior regime, where additional KL anchoring restores coverage.Source: §3.4 · p. 7

3.5 The variational bound predicts only the sign, not the magnitude

  • The 0.69× variational boundary ratio cannot explain the empirical 71× gap; controlled analyses locate the residual amplification in the learned policy, not the ODE solver.The ODE solver contributes zero at the boundary from 25 to 400 steps, anchor mismatch is bounded, and policy-side shortfall is 15–78%; the variational result also does not predict the coverage plateau.Source: §1 · p. 3, §3.5 · pp. 7–8

3.6 What the crossover-scale predictor actually predicts

  • The crossover scale predicts which prior has the lower collapse boundary and provides an order-of-magnitude lower bound, but not the gap magnitude or coverage plateau.In overlap-prone geometry, the 71× empirical gap is dominated by dynamic-to-variational amplification; the variational prediction gives ≈0.69×.Source: §3.6 · p. 8, §3.9 · p. 10

3.7 Variational vs. dynamic: the 50× systematic gap

  • The solver contributes 0.0% boundary shift, while the learned policy carries the residual: β_dyn_c = 0.22–0.65 (median 0.56) and realizes only 15–78% of the analytic tilt boundary.The on-path anchor gap is 0.012–0.075 DDPO units; replacing the learned reference moves βc/βvar inconsistently and closes at most half the gap. Policy-side broadening makes collapse more likely; for Exp7 GMM, β_variational_c = 6.05 × 10^-5 versus empirical 3.16 × 10^-3.Source: §3.7 · p. 9

3.8 Mode overlap: when the theory breaks

  • When d_min < 4σ_0, latent Gaussian balls already overlap at β = 0, and coverage can saturate near 0.33, making the separated-mode variational bound degenerate and the crossover predictor fail.Exp7 seed 2 with a Gaussian prior has d_min/σ = 0.99; this control indicates the prior-width mechanism operates only in the separated-mode regime.Source: §3.8 · p. 9

3.9 Summary

  • Mode-separating priors raise the coverage plateau in every geometry, while reducing empirical βc only when d_min/(2σ_0) < 1; the boundary advantage disappears in well-separated geometries.The reported empirical gaps are 71× in Exp7 and 3–6× in Exp6, whereas well-separated cases show coincident collapse positions or only a 1.23× ratio.Source: §3.9 · pp. 9–10

4 Results

The results examine how latent geometry and prior choice shape collapse boundaries and coverage, validating predicted scaling in well-separated regimes and characterizing the geometry-dependent prior gap on alanine-dipeptide. Mechanism and robustness analyses locate the dynamic-to-variational gap in the learned policy and qualify how collapse and coverage are measured.

4.1 Theory validation: βc scales with λ across parameter space

  • The reverse-KL βc slope is 0.96 versus the theoretical 1.0 across three orders of magnitude in λ, with empirical values within 2.5× of theory.The four-point mean dyn/var is 2.1× in the well-separated three-basin alanine-torus regime.Source: §4.1 · p. 10
  • Forward-KL βc has a resolved λ-scaling slope of 1.64, a non-monotone gap versus reverse-KL, and a λ = 0.1 boundary censored below 10^-6.At λ = 1 its βc is 0.914, or 0.17× the reverse-KL value; at λ ≥10 it requires 2.7–3.5× larger βc. Treating the scan floor as a measurement would yield a spurious upper-bound slope of 2.77; theory predicts a forward/reverse ratio of 1.24.Source: §4.1 · pp. 10–11

4.2 Mechanism analyses

  • Saddle-to-mode score MSE reaches 603× for narrow modes but falls to 0.6–1.0× for broad modes, while density-weighted MSE stays within 1.4× of mode MSE.The gap concentrates in near-zero-mass saddle regions; dimensions 2–5 show no monotone dimension trend, and 5D models are too poorly trained for a dimension claim.Source: §4.2 · p. 11
  • Across 72 configurations, mean dyn/var is 2.4× and theory predicts βc within 3×; βc decreases with separation and increases with λ and K.The λ exponent is approximately 1.0 across dimensions, while the d/σ0 exponent rises from 2.0 to 2.7 to 3.3 across 1D, 2D, and 3D.Source: §4.2 · pp. 11–12
  • Trajectory tests reject saddle-corridor routing as a general carrier of collapse, and the three-source decomposition attributes the gap to the learned policy, with zero solver contribution.Collapsing policies differ from reference saddle-strip residence by a median 7%; the learned policy reaches only 15–78% of the analytic tilt boundary, while anchor mismatch is bounded and non-systematic.Source: §4.2 · pp. 13–14

4.3 Exp5A: prior comparison in a controlled 1D setting

  • At β = 10000, GMM recovers 100% of target modes versus 70% for Gaussian, although both priors have βc = 100.This controlled 1D experiment separates collapse position, which is identical, from the prior-dependent coverage plateau.Source: Table 4 · p. 14, §4.3 · pp. 14–15

4.4 Exp6: crowded latent space (K=30)

  • 3.83×: in the 3-seed Exp6 mean, the Gaussian prior’s decoded βc gap over the GMM prior is consistent in direction across all seeds.The Gaussian decoded plateau is 0.53–0.63 versus 0.70–0.77 for GMM; the mean Gaussian/GMM plateau ratio is 0.80 ± 0.10.Source: §4.4 · p. 16, Table 5 · p. 16

4.5 Exp7: real molecular dataset (alanine-dipeptide)

  • 70.8×: on alanine-dipeptide, Gaussian-prior βc is 0.224 while GMM-prior βc is 3.16 × 10^-3.The gap is preserved across seeds; the GMM value is pinned down by a high-resolution bisect after the initial scan floor.Source: Table 6 · p. 18, §4.5 · p. 18
  • At β = 5 × 10−3, alanine-dipeptide latent coverage recovers sharply to 0.67, while decoded coverage remains 1.0 at all β.The refined collapse boundary is 3.16 × 10−3; decoded coverage is the tighter real-data mode-preservation test.Source: §4.5 · p. 18

4.6 Consistency across experiments

  • Across well-separated seeds, the GMM prior has strictly smaller βc and higher decoded plateau coverage, with a 3-seed mean ratio of 0.80 ± 0.10.The βc gap scales with σlat: Exp7’s 36× width ratio accompanies gaps of at least 22× to 71×, while Exp6’s 4–6× ratio accompanies gaps of 3.3–6.6×; in Exp7 seed 2, overlap makes the gap vanish.Source: §4.6 · p. 18
  • On alanine-dipeptide, βc is 0.224 for the Gaussian prior versus 3.16 × 10−3 for the GMM prior, a 71× gap accompanying dmin/(2σ0) of 0.81 versus 1.03.Figure 8 shows the Gaussian prior’s overlapping basins and the GMM prior’s separated clusters; the 3.16 × 10−3 boundary is refined by bisection.Source: Figure 8 · p. 19, §4.6 · p. 21
  • Soft coverage places the Gaussian-prior βc below 0.1 versus hard coverage’s 0.224, while GMM soft coverage stays near 0.94 without a collapse–recovery transition.The hard metric overestimates βc by at least 2.2×; the Gaussian soft curve rises from 0.56 to 0.95, whereas the high-β plateaus are 0.95 for Gaussian and 0.94 for GMM.Source: §4.6 · p. 21, Figure 10 · p. 22
  • Across 20 ns and 60 ns trajectories, Gaussian-prior seeds 0/1 retain βc = 0.224 and GMM-prior seeds remain at βc ≤0.01, with the GMM boundary at least 22× smaller.The same three basins appear at matching torsion positions; the results support geometry, rather than MD sampling length, as the determinant of βc.Source: §4.6 · p. 22
  • The 71× gap is consistent with overlap-driven dynamic-to-variational amplification: dyn/var is ≈83× for separated Exp7 GMM geometry but much larger in overlapping regimes.The authors interpret this as mediation evidence, not a direct prior-type intervention, because geometry and prior type are confounded; well-separated Exp5A geometry leaves the boundary unchanged.Source: §4.6 · pp. 22, 24, Figure 11 · p. 24

5 Discussion and Open Problems

The discussion distinguishes the finite-resource collapse-position gap from the prior-dependent coverage plateau and makes the remedy conditional on overlap-prone geometry and coverage-sensitive tasks. It also identifies resolved mechanisms and remaining theoretical and scaling questions.

Boundary condition on the remedy

  • The position gap is finite-resource policy-side amplification under overlap, whereas the plateau advantage comes from the VAE representation and requires changing the prior or objective.A sufficiently expressive, fully optimised policy could close the position gap toward the variational prediction; larger RL capacity does not remove the plateau gap.Source: §5 · pp. 24–25

Implication 1: the advantage is geometric, not symbolic

  • The prior’s benefit is geometric, not symbolic: any prior that preserves mode separation should behave similarly, but the mixture label alone does not ensure tight geometry.VampPrior and GMVAE do not reach the tight GMM geometry under the studied budget.Source: §5 · p. 25

Implication 2: geometry is the mediator

  • At matched geometry, the prior label has no direct measured effect on the collapse boundary: 0.995×, 95% CI [0.79, 1.25].The fixed-Gaussian intervention moved geometry from g = 2.08 to 0.65 by KL weight alone; the reverse intervention supports full mediation.Source: §5 · p. 25

Implication 3: the crossover scale predicts the sign, not the magnitude

  • The crossover scale predicts the sign, not the magnitude, of the collapse-position gap and does not predict the coverage plateau.In well-separated geometry, boundaries coincide to resolution; overlap-regime gaps reflect policy-side amplification beyond the order-1 variational prediction.Source: §5 · p. 25

Open Problems

  • The amplification decomposition attributes the residual 15–78% to the learned policy, with zero solver contribution and bounded, non-systematic anchor mismatch.The functional form of dyn/var as a function of (dmin/(2σ0), K) remains open.Source: §5 · p. 25

Remaining gaps

  • Remaining gaps include a theory of the overlap regime, an explanation of forward- versus reverse-KL offsets, and scaling tests across longer MD, more basins, seeds, dimensions, molecules, and methods.Below dmin < 4σ0, the ball-coverage metric saturates and the variational bound degenerates; a continuous overlap-integral metric is missing.Source: §5 · p. 26

Outlook: Applications to molecular generative models

A rare-basin reward test on alanine dipeptide compares Gaussian-prior guidance with a tight label-tracked GMM. The GMM retains common basins while enriching the rare basin, whereas Gaussian guidance erases one common basin.

Outlook: Applications to molecular generative models

  • On alanine dipeptide, Gaussian-prior guidance reaches ≈4× rare-C5 enrichment but erases a common basin, while a tight label-tracked GMM reaches 2.9–3.6× and retains every common basin above threshold.The comparison uses three seeds; the preserving mechanism relies on a small 2D-latent decoder being non-injective on the GMM’s tight components.Source: Outlook: Applications to molecular… · p. 26

6 Conclusion

The Gaussian ELBO optimum compresses multimodal data into overlapping latents, and RL fine-tuning amplifies this geometry; the need for a non-Gaussian prior depends on overlap and task sensitivity.

6 Conclusion

  • The Gaussian ELBO optimum packs distinct data modes into an overlapping latent, and RL fine-tuning amplifies that overlap into a collapse boundary far above the variational scale.The separation-to-width ratio is the identified geometric mediator.Source: §6 · p. 26
  • A non-Gaussian prior is needed when latent geometry falls into the overlap regime and the downstream task is coverage-sensitive; outside that regime, the Gaussian default suffices.Source: §6 · pp. 26–27
  • The collapse-boundary position is a finite-resource pipeline effect, whereas the coverage plateau is a representational property of the VAE.Source: §6 · p. 27
  • Prior choice is a geometric lever that can be checked before training, and the KL term prices separation and tightness rather than merely ignoring mode structure.Source: §6 · p. 27

A Extended 60 ns MD basin analysis

The extended alanine-dipeptide analysis finds three stable physical basins in both MD trajectories, with only mild occupancy shifts and no additional populated conformation in the longer trajectory.

A Extended 60 ns MD basin analysis

  • The analysis uses periodic Gaussian KDE on the torsion torus, with automatically selected bandwidths of 0.185 rad for 20 ns and 0.175 rad for 60 ns.Frames are assigned to the nearest detected peak within a 0.8 rad geodesic cap; others remain unassigned.Source: A Extended 60 ns MD basin analysis · p. 27
  • K = 3 in both trajectories at every minimum-mass threshold down to 0.5%, with no fourth local maximum carrying even 0.5% of KDE mass.Source: Table 9 · p. 27, A Extended 60 ns MD basin analysis · p. 27
  • The same three peak positions occur in both trajectories, with a maximum shift of 1.8°; the rarest basin is the β/C5 conformation in both.It lies 8.1° from the canonical C5 reference, and both experiments target this basin.Source: Table 10 · p. 27, A Extended 60 ns MD basin analysis · p. 27
  • Tripling trajectory length shifts αR and β/C5 occupancies by only 1.3 percentage points in opposite directions and reveals no additional populated conformation.C7ax and αL remain unvisited within 0.6 rad in both trajectories; the fine-tuning boundary is unchanged despite more thorough basin sampling.Source: A Extended 60 ns MD basin analysis · p. 28

B Full per-seed Exp7 numbers (20 ns and 60 ns)

Across the reported Exp7 seeds and MD lengths, the GMM-prior collapse boundary remains substantially below the Gaussian-prior boundary, while latent geometry is stable within each prior.

B Full per-seed Exp7 numbers (20 ns and 60 ns)

C Forward-KL collapse boundary: full scaling record

The forward-KL scaling record reports a resolved slope below the reverse-KL slope and dimension-dependent empirical forward/reverse boundary ratios on synthetic grids.

C Forward-KL collapse boundary: full scaling record

  • The alanine-torus sweep measures both KL directions at λ ∈ {0.1, 1, 10, 100} using identical β scans and a 0.5 coverage criterion.The synthetic-grid power-law specification includes λ, d/σ, and K, fitted separately by dimension using resolved boundaries.Source: C Forward-KL collapse boundary: full… · pp. 28–29
  • 1.64 is the resolved forward-KL log-log slope across λ = 1, 10, 100; the 2.77 estimate relies on treating a censored λ = 0.1 scan-floor value as measured.The resolved data do not establish a single power law across the full range.Source: C Forward-KL collapse boundary: full… · p. 29
  • 0.016 [0.003, 0.090] in 1D, 0.10 [0.029, 1.96] in 2D, and 1.13 [0.109, 1.42] in 3D are the median empirical forward/reverse βc ratios over resolved configurations.The asymmetry shifts from much smaller forward-KL boundaries in 1D to slightly larger ones in 3D.Source: C Forward-KL collapse boundary: full… · p. 29

D Regional score-MSE decomposition across dimensions

The regional score-MSE gap is governed by mode spacing relative to width, not ambient dimension, and can be large in saddle regions with little stationary mass. High-dimensional estimates are limited by sparse saddle samples and failed model learning.

D Regional score-MSE decomposition across dimensions

  • Regional MSE is computed in mode, saddle, and valley regions at three diffusion times, alongside density-weighted MC MSE on forward-noised samples.The region assignments use mode-centre distance and nearest-neighbour-pair midpoint rules.Source: D Regional score-MSE decomposition… · p. 31
  • The saddle/mode gap is largest for narrow modes and closes as modes broaden, while density-weighted MC MSE stays within 1.4× of mode MSE.The table caption identifies the 2D width sweep at t = 0.02 and the gap’s dependence on σ/d.Source: Table 14 · p. 31
  • Across 2–5D lattice packs, saddle/mode ratios span 0.5–5.3×, with spacing/width—not ambient dimension—identified as the controlling variable.Per-dimension medians are 1.2×, 2.1×, 1.2×, and 1.4×; no monotone dimension trend is reported.Source: D Regional score-MSE decomposition… · p. 31
  • In 5D, saddle estimates use only 4–30 points and the models cover only 0.00–0.60 of target modes, so those results are estimator diagnostics rather than dimension evidence.The dimensional claim is restricted to trained 2–4D models.Source: D Regional score-MSE decomposition… · p. 31, Table 15 · p. 32, Table 16 · p. 32
  • The saddle excess persists across diffusion times but carries little stationary mass; density-weighted MSE remains close to mode MSE, and the width-sweep models retain full coverage.The table caption says the saddle/mode gap persists at all three times and MC MSE remains an order of magnitude below saddle MSE even at 600× mode error.Source: D Regional score-MSE decomposition… · p. 31, Table 17 · p. 32

E Score error along controlled PF-ODE trajectories

Controlled PF-ODE trajectory tests reject corridor-routing as a general mechanism for collapse: residence and path-weighted reference-score error are usually similar between reference and collapsing policies. A large error amplification appears only in one narrow, few-mode geometry.

E Score error along controlled PF-ODE trajectories

  • The protocol compares reference and fine-tuned policies from a shared terminal-state batch, evaluating both score errors and mode/saddle/valley residence along PF-ODE trajectories.The 24 geometries comprise a 2D width sweep and 2D/3D lattice packs.Source: E Score error along controlled PF-ODE… · p. 33
  • For the 2D and 3D lattice packs, late-window reference-error ratios have medians of 0.95 and 1.00, and no pack exceeds 2.2× in any window.Median saddle-residence ratios are 1.09 in 2D and 1.02 in 3D.Source: E Score error along controlled PF-ODE… · p. 33
  • In the sole outlier, the collapsing policy funnels endpoints onto the target and centre modes while abandoning the others, despite corridor crossings resembling the reference paths.The figure reports cover 0.20 and 74% of endpoints on the target basin.Source: E Score error along controlled PF-ODE… · p. 33, Figure 13 · p. 34
  • Across 24 geometries, error ratios show no early-time amplification; endpoint coverage is stable from 200 to 480 solver steps, and higher β reduces amplification while coverage recovers.At β = 3βc the median saddle-error ratio is 1.12 and coverage is 0.55–1.00.Source: E Score error along controlled PF-ODE… · p. 35
  • Corridor-routing is rejected as a general mechanism: saddle residence is unchanged and path-weighted reference error is near unity across time windows, apart from one narrow, few-mode geometry.In the outlier, the mid-window error ratio is 84×; the estimate is based on 5,118 versus 41,093 saddle states.Source: E Score error along controlled PF-ODE… · pp. 33, 35

F Three-source decomposition of the dynamic amplification

The three-source decomposition attributes the dynamic-boundary deficit to the learned policy, not solver discretisation or KL-anchor mismatch. The policy realises only part of the analytic tilt boundary under the tested training setup.

F Three-source decomposition of the dynamic amplification

  • ODE discretisation contributes zero at the boundary: solver-only βc is identical from 25 to 400 steps and agrees with the grid-numeric variational boundary within 3–7% in separated geometries.In overlapping packs, the solver-only boundary is 16–47% above βvar, consistent with the formula’s disjoint-mode-ball assumption.Source: F Three-source decomposition of the… · p. 35
  • The on-path KL-anchor gap is small (0.012–0.075 DDPO units), and replacing the learned reference with the exact data score does not recover the boundary.The largest gap occurs in the most separated narrow geometry, while the smallest occurs in the overlapping σ0 = 2 geometry.Source: F Three-source decomposition of the… · p. 35
  • The exact-anchor intervention closes at most half the gap and moves the boundary non-systematically; the paper notes a single-seed design and that its anchor penalty is a score-matching twin, not the DDPO KL term itself.A deeper policy and longer fine-tuning would further bound the representation-versus-optimisation split.Source: F Three-source decomposition of the… · pp. 35–36
  • The learned policy accounts for the residual boundary deficit: even with an exact-score anchor and 400 solver steps, it realises only 15–78% of the analytic tilt boundary.The reported dynamic boundary is βc = 0.22–0.65, with median 0.56; the policy is a 2-layer MLP after 1000 guidance steps.Source: F Three-source decomposition of the… · p. 36

G Fixed-prior intervention on latent geometry

With the Gaussian prior held fixed, changing KL pressure moves latent geometry and the collapse boundary together. The intervention supports a geometry-to-boundary chain in the overlapping range, while leaving the reverse counterfactual untested.

G Fixed-prior intervention on latent geometry

  • The intervention varies KL weight and encoder capacity while keeping the prior label fixed, then measures latent geometry and the downstream latent-pipeline boundary.The data are a K = 10, d = 8 lattice pack, with geometry snapshots at seven training steps.Source: G Fixed-prior intervention on latent… · p. 36
  • At fixed Gaussian prior, increasing KL weight moves dmin/(2σlat) from 2.08 to 0.65 and βvar c from 3.21 to 0.77, while measured βc spans 0.50–1.37.The wkl = 1 cell reproduces the Exp6 Gaussian endpoint, βc = 0.67 versus 0.71 in Exp6.Source: G Fixed-prior intervention on latent… · p. 36, Table 19 · p. 37
  • The highest KL-weight cell reaches dmin/(2σlat) = 0.65 and a decoded coverage plateau of 0.80, reproducing a collapsed-Gaussian geometry without changing the prior label.The geometry lies inside the Exp7 Gaussian regime cited in the passage.Source: G Fixed-prior intervention on latent… · pp. 36–37
  • Geometry develops during training and saturates at KL-weight-dependent plateaus; encoder width alone is inactive across 64–256 hidden units, making KL pressure the binding constraint in this range.The latent width σlat rises from 0.23 at w = 0.03 to 0.45 at w = 3.Source: G Fixed-prior intervention on latent… · p. 37
  • The experiment supports wkl → geometry → βc over dmin/(2σlat) = 0.65–2.08, but does not test a multimodal prior forced into compressed geometry; the strongest-separation cell is a single observation.The GMM endpoint at dmin/(2σlat) = 3.34 lies outside the geometry reached by the Gaussian-prior ELBO on this data.Source: G Fixed-prior intervention on latent… · p. 37

H Four-prior comparison: Gaussian, GMM, VampPrior, GMVAE

Across four prior families, collapse boundaries track achieved latent geometry rather than prior labels. Under the shared ELBO budget, only the label-tracked GMM achieves tight, well-separated geometry; all four preserve decoded coverage at large β.

H Four-prior comparison: Gaussian, GMM, VampPrior, GMVAE

  • The tested comparison uses four priors trained on the Exp6 pack, with three seeds per prior and a 30k-step VAE budget.The learned-mixture priors include VampPrior and GMVAE without labels, while the GMM tracks labelled aggregate-posterior means.Source: H Four-prior comparison: Gaussian,… · p. 38
  • The label-tracked GMM reaches the tight, well-separated geometry and reproduces Exp6 endpoints, while VampPrior and GMVAE do not under the same 30k-step ELBO budget.The GMM ratio is 3.3–7.2; VampPrior reaches 1.4–2.0 with nearly Gaussian width, and GMVAE does not separate in ratio terms (1.09–1.18).Source: H Four-prior comparison: Gaussian,… · pp. 38–39
  • After conditioning on achieved geometry, measured boundaries fall within 0.5–1.2 of each family’s own variational boundary, with no residual prior-label effect reported.The GMVAE’s larger boundary is attributed to geometry, including its 2.1× wider latent at matched ratio.Source: H Four-prior comparison: Gaussian,… · p. 39
  • All four prior families reach full decoded coverage at large β, so the learned priors show no decoded plateau penalty in this comparison.GMVAE latent coverage dips to 0.95 in two seeds, while decoded coverage remains full.Source: H Four-prior comparison: Gaussian,… · p. 39

I Separation scan: does α = βc/βcross tend to 1?

The separation scan finds that α generally grows with geometry rather than converging to 1, with its crossing location depending on layout. On alanine dipeptide, rare-basin enrichment trades off against retention, and the Gaussian and tight GMM priors occupy different points on that frontier.

I Separation scan: does α = βc/βcross tend to 1?

  • In the separation scan, αemp rises with g rather than tending to 1, reaching 14.5, 55.9, and 445 at g = 10 for K = 3, 5, and 10.The αemp = 1 crossing occurs near g ≈2.4 for K = 3, ≈1.0 for K = 5, and below 0.5 for K = 10.Source: I Separation scan: does α = βc/βcross… · pp. 40–41, Figure 19 · p. 41, Figure 22 · p. 43
  • At every non-saturated cell, αemp < αvar < αcl; the saddle expression overshoots by factors of 1.5–25, while the learned-pipeline gap widens at large g.For K = 10, αemp/αvar falls from 0.43 at g = 1 to 0.25 at g ≥2, consistent with finite-network and fine-tuning limits.Source: I Separation scan: does α = βc/βcross… · p. 41
  • The single-number guide dmin > 4σ0 applies only to the K = 3 marginal layout; the general regime depends on g and j, and the conjectured dyn/var →1 endpoint is not observed.The exact optimum itself departs as j²g², with the learned pipeline departing further as the saddle sharpens.Source: I Separation scan: does α = βc/βcross… · pp. 40, 42

J Rare-basin reward on alanine dipeptide: enrichment versus retention

  • Threefold enrichment is reachable only for the rare C5 basin; the common β and αR basins have observed maxima below 3× for either prior.The authors limit the alanine result to a 2D latent and K = 3, and say the preserving mechanism depends on decoder non-injective amplification.Source: J Rare-basin reward on alanine… · pp. 43–44
  • On C5, Gaussian policies reach 3.9–4.3× enrichment but retain only 0.5 at 3×, whereas GMM policies reach 3.5–3.6× with retention 1.0 in two seeds.The third GMM seed peaks at 2.91× with retention 1.0; soft AUC favors or matches GMM on every basin, including 0.97 vs. 0.80 on β.Source: J Rare-basin reward on alanine… · p. 44, Figure 23 · p. 44
  • The GMM’s diffuse latent target mass is spread across physical basins by decoder gain, while Gaussian collapse transmits target concentration and enables sharper enrichment at the cost of erasing a basin.For GMM, latent coverage stays at 1/3 while decoded coverage stays at 1.0 over the stated β range.Source: J Rare-basin reward on alanine… · p. 44

K Reverse counterfactual: matched-geometry prior switch

In the reachable overlap interior, matched-geometry Gaussian and GMM interventions yield boundaries explained by each cell’s own variational boundary. A regression controlling for that full geometry finds no detectable direct prior-label effect, while the endpoint geometries remain unreachable across priors.

K Reverse counterfactual: matched-geometry prior switch

  • The reverse intervention uses GMM width floors and centre contraction because changing GMM KL weight alone leaves g at 3.1–6.8, outside the Gaussian-ELBO image on this pack.The two prior-internal interventions modify the prior distribution while leaving the decoder, score model, and reward untouched.Source: K Reverse counterfactual:… · p. 45

Results.

  • The measured boundary changes track changes in each pair’s own variational boundary, while controlling only for scalar g leaves an incomplete mediation because it discards absolute inter-centre scale.At similar g, width-arm boundaries and their variational predictions differ together as centre separation varies.Source: Results · p. 46
  • At matched geometry in the reachable interior, measured boundaries agree within the ±25% confidence interval, and prior switching neither removes nor adds the finite-pipeline shortfall.The GMM coefficient is 0.995× with 95% CI [0.79, 1.25] after controlling for the full own-geometry boundary.Source: Results · p. 46
  • The symmetry result is limited to reachable geometries: the tight Exp6/V GMM endpoint has no Gaussian counterpart, and the compressed Gaussian tail has no tested GMM counterpart.The matched comparison is therefore performed within the reachable interior rather than at either endpoint.Source: Results · pp. 46–47

L Machine-checked skeleton of the theory (Lean 4 / Mathlib)

The Lean 4 formalization machine-checks algebraic and logical steps in the theory skeleton under stated modeling assumptions. It covers prior geometry costs, mediation identities, error decomposition, and boundary certification, but not probabilistic estimator or generalization claims.

L Machine-checked skeleton of the theory (Lean 4 / Mathlib)

  • The formalized Gaussian-prior KL has a minimum at width 1, strictly taxes narrower widths, and diverges as width vanishes; a matching mixture component instead incurs a finite −log π assignment fee independent of width.The mixture bound is stated as KL(q∥m) ≤ KL(q∥p) −log π.Source: L Machine-checked skeleton of the… · p. 47
  • The mediation formalization proves additive total, direct, and indirect effects, and shows that full mediation makes the direct effect zero; it also constructs confounded designs that cannot identify the direct effect without intervention.The formalization reports compilation with zero errors under the specified Lean and Mathlib toolchain.Source: L Machine-checked skeleton of the… · p. 47
  • The error decomposition proves etot = enet + eode + eanchor and that, when eode = 0, the remaining network-plus-anchor budget bounds the achievable boundary.At least one of the three non-negative sources must be at least etot/3.Source: L Machine-checked skeleton of the… · p. 47
  • The boundary certificate shows that uniform score error ε brackets the true half-width within ±ε, and vanishing solver error eventually certifies the true 1/2-coverage boundary.The solver-error refinement tightens the certificate.Source: L Machine-checked skeleton of the… · p. 47
  • The machine-checked scope is limited to deterministic inequalities; sampling convergence of Monte Carlo estimators and score-model generalization are not certified.The formal proofs take the modeling assumptions as hypotheses.Source: L Machine-checked skeleton of the… · p. 47

Related papers

Loading 2610.28849v1…