PaperlayerView exact revision on arXiv

Source-linked AI summary

World Models with Predictable Long-Horizon Marginals

Yuhao Du, Shunian Chen

arXiv:2609.32657v1cs.AI

TL;DR

Accurate one-step predictions do not ensure that world-model rollouts retain the data distribution, making the long-horizon marginal a distinct target. The paper learns a decoder of a preserved Gaussian reference and constructs controlled transitions that maintain behaviour occupancy. Across pixel-model evaluations, the occupancy model retains all evaluated chains while the four nonpreserving comparison arms lose chains.

  • Problem

    Accurate one-step predictions do not ensure that a world model’s rollouts retain the data distribution, so its long-horizon marginal is a distinct prediction target.

    Source: §1 · p. 1

  • Method

    The model decodes a fixed Gaussian reference preserved by a behaviour-averaged transition, using a conditional action chart for controlled dynamics.

    Source: §1 · pp. 1–2

  • Results

    Across 36 task–seed cells, occupancy retains all 294,912 evaluated chains and has rollout-minus-reference gap −0.0002±0.0003 (SE), while all four comparison arms lose chains.

    Source: §5.4 · p. 7

  • Takeaways & Limitations

    The results distinguish the distribution a world model approaches, its rate of approach, and the conditional dynamics it learns.

    Source: §7 · p. 9

    The occupancy interpretation depends on stationarity and action-chart conditions; approximate charts and policy shifts need error control, and finite-start bounds rely on bank-specific moments and real arithmetic.

    Source: §7 · p. 9

Abstract

from arXiv · show

Accurate one-step predictions do not ensure that a world model's rollouts retain the data distribution. We make the model's decoded stationary law explicit by learning a decoder of a fixed Gaussian reference and constraining the behaviour-averaged transition to preserve that reference. For controlled systems, a joint transition uses a conditional action chart to preserve behaviour occupancy without requiring invariance at each fixed action. Joint state--action rotations and parallel Gaussian noise give an exactly preserving transition with a tractable conditional density. We derive an absolute convergence bound from finite initialization banks and control departure from the reference through conditional action-space divergence. Across $216$ fitted pixel checkpoints on twelve control tasks, the occupancy model with a reference mixture retains every evaluated chain at $10^5$ steps in all $36$ task--seed cells, with a rollout-minus-reference energy-statistic difference of $-0.0002\pm0.0003$ (training-seed standard error). Each of the four nonpreserving comparison arms loses chains, although the Gaussian arm is more accurate at ten steps. An offline DreamerV3 reference also achieves better short-horizon accuracy. These results distinguish three properties of a world model: the distribution it approaches, the rate of approach, and the conditional dynamics it learns.

1 INTRODUCTION

One-step accuracy does not ensure that rollouts retain the data distribution, so the long-horizon marginal is a distinct prediction target. The paper makes that target explicit through a preserved latent reference and develops controlled transitions and bounds around it.

1 INTRODUCTION

  • A rollout can leave the data-supported region despite accurate one-step predictions, while chaotic trajectories can diverge even under correct dynamics.Teacher-forced likelihood evaluates transitions under the data law, unlike rollouts that visit the model’s evolving distribution.Source: §1 · p. 1
  • The model’s stationary law is represented by a decoded Gaussian reference preserved by the behaviour-averaged transition; observation-law matching remains approximate.The transition still determines conditional dynamics and how quickly other initial laws approach the reference.Source: §1 · p. 1
  • Controlled transitions need only preserve the behaviour-averaged occupancy, avoiding the restriction that every action kernel preserve the state law.The construction combines occupancy preservation, an exact conditional density, and finite-initialization bounds in one parallel transition.Source: §1 · p. 2
  • The paper derives computable marginal guarantees from Gaussian entropy contraction and controls policy deviation using conditional action-space KL divergence.Its comparisons distinguish marginal predictability from conditional accuracy and planning return.Source: §1 · p. 2

2 STATIONARY DATA AS A PRESERVED REFERENCE

The stationary state law is preserved after averaging transitions over the fixed behaviour policy, not necessarily by each action-specific kernel. An invertible state chart maps this constraint to preservation of a standard Gaussian reference.

2 STATIONARY DATA AS A PRESERVED REFERENCE

  • Stationarity requires the behaviour-averaged controlled transition to preserve the state law, not every fixed-action transition.Independent uniform actions with next state equal to the action yield uniform stationary states even though each fixed action collapses the state distribution.Source: §2 · p. 2
  • An invertible chart mapping the stationary law to a standard Gaussian converts state-law preservation into Gaussian-reference preservation.With arbitrary measurable charts and kernels, the representation covers pair laws with equal atomless marginals; differentiable density representations require additional transport regularity, and finite networks cover only part of the constraint set.Source: §2 · p. 2
  • The conditional action chart Gaussianizes behaviour actions independently of latent state, allowing joint state–action transformations to preserve occupancy.With state-dependent charts, fixing a chart coordinate and fixing a physical action are different interventions, so the preservation classes need not be nested.Source: §2 · p. 2
  • For the recurrent pixel model, recorded pre-tanh actions provide the exact chart for a frozen tanh-Gaussian policy; learned approximate charts require residual-error control.During generation the policy reads the decoded observation, keeping its innovation independent of latent state.Source: §2 · p. 2

3 GAUSSIAN-PRESERVING TRANSITIONS

The transition architecture preserves a standard Gaussian at every parameter value through constrained transformations and parallel noise. It combines joint state–action mixing, tractable conditional density, and optional full-state reference refresh.

3 GAUSSIAN-PRESERVING TRANSITIONS

  • Norm-changing maps can be constructed by transforming through Gaussian quantiles, applying a volume-preserving map, and transforming back.The approximation result permits unrestricted smooth profiles and depth, while finite-network capacity remains a separate constraint.Source: §3 · pp. 2–3
  • Gaussian-preserving block-affine couplings require zero shift and an orthogonal linear map.Conditioned parallel Givens rotations implement the constraint without projection; changing coordinate-pair partitions across layers keeps depth independent of latent width.Source: §3 · p. 3
  • Joint state and action innovations are mixed by a Gaussian-preserving bijection before a parallel noise stage refreshes every recurrent coordinate.At the reference law, the mixed variables are independent Gaussians; conditional on the action-side variable, the noise stage and post-map retain the Gaussian reference.Source: §3 · p. 3
  • The implemented rotational post-map has no inverse-Jacobian contribution to the conditional density, and the input mixer transforms known conditioning variables.The block-scan parameterization in the fixed-latent benchmark is separate from the stochastic architecture.Source: §3 · p. 3
  • An optional reference mixture preserves the reference while refreshing the full state with probability r.Its added conditional loss is bounded by −log(1−r), equal to 0.001001 nats per transition at r = 10^-3; it also bounds extreme log-loss relative to the fitted marginal.Source: §3 · p. 3
  • Pixel training jointly learns the decoder, recursive Gaussian posterior, reward head, and transition using an ELBO for behaviour-generated sequences.The objective includes observation and reward reconstruction, an initial KL to the Gaussian reference, and transition KL terms using the exact prior density.Source: §3 · p. 4

4 MARGINAL CERTIFICATES

Reference preservation makes the decoded marginal exact when initialized from the reference, while contraction bounds how finite initializations approach it. Policy shifts and approximate charts require separate error control.

4 MARGINAL CERTIFICATES

  • From a Gaussian-reference initialization, the decoded observation law equals the model’s static reference law at every horizon.This holds for the behaviour action chart and the specified transition, with or without the full-state reference mixture.Source: §4 · p. 4
  • The guarantee concerns the generative prior, while recurrent inference learns an approximate posterior.Source: §4 · p. 4
  • A rollout discrepancy decomposes into static-reference discrepancy plus a signed transient, with energy-statistic error bounded by the finite-start total-variation distance.For observations of diameter Δ, the energy-statistic difference is bounded by 2Δ times that distance; unbounded state-space distances need additional moment control.Source: §4 · p. 4
  • Parallel noise supplies finite-start relaxation: under an exact behaviour chart, the state-kernel KL contracts by c⋆^2.The bound assumes Gaussian-preserving mixing and post-map with |c_i(b)| ≤ c⋆ < 1; the finite-bank moment is exactly computable, while a population certificate needs a population moment bound.Source: §4 · p. 4
  • At 10^5 steps, the reference-mixture population marginal-gap bound is below 1.5 × 10^-43, far below evaluation sampling error.For the specified initialization banks, the finite-start bound becomes informative after 3×10^3–10^4 steps; these guarantees are in real arithmetic.Source: §4 · p. 5
  • The policy-deviation bound concerns departure from the behaviour reference, not predictive error; fitted charts require divergence to their induced policy.For nonsingular tanh-Gaussians sharing a tanh transform, conditional KL equals the pre-tanh Gaussian KL; deterministic action selection has infinite KL against this behaviour law.Source: §4 · p. 5

5 EXPERIMENTS

Experiments compare static marginal prediction, finite-start guarantees, conditional fit, and planning across controlled examples and recurrent pixel models. The occupancy model preserved long-horizon behavior across the evaluated task–seed cells, while conditional accuracy and planning outcomes remained distinct.

5.1 STATIC PREDICTION OF THE LONG-HORIZON MARGINAL

  • Across seven constrained Lorenz-63 models, static energy statistics spanned 31× and each 105-step measurement was within one nominal combined SE of its prediction.The SE omits the unavailable prediction–rollout covariance; an unconstrained flow improved held-out pair likelihood by 0.83–0.99 nats per dimension, but only 14%–94% of chains survived.Source: §5.1 · p. 5, Figure 2 · p. 6

5.2 FINITE-START BOUNDS AND CONDITIONAL LIKELIHOOD

  • All 114 bounded-observable comparisons for finite starts and known policy shifts satisfied the population bounds with the simultaneous sampling allowance.The finite initialization bank has infinite KL to a Gaussian but a known second moment; prior-only sampling took 2.50–4.57 ms per batch of 32 as state width increased from 256 to 1024.Source: §5.2 · p. 6, Figure 3 · p. 7

5.3 CONTROLLED OCCUPANCY AND PLANNING

  • With τ = 0, fixed-action preservation incurred 0.99–1.00 times its predicted excess-loss floor, while the occupancy model’s excess loss was 0.0011 nats.On a controlled Duffing oscillator, occupancy-based CEM planning had mean excess cost 0.242 at horizon 500, versus 0.001 for the Gaussian head; the comparison also differs in fit and transition size.Source: §5.3 · p. 6

5.4 JOINTLY TRAINED RECURRENT PIXEL MODELS

  • All 294,912 occupancy-arm chains survived across 36 task–seed cells, with a rollout-minus-reference gap of −0.0002±0.0003; four alternative arms had survival fractions of 0.51, 0.24, 0.34, and 0.44.The alternatives were Gaussian, conditional RealNVP, latent overshooting, and invariant penalty, respectively.Source: §5.4 · p. 7, Figure 4 · p. 8
  • The Gaussian arm had the smallest measured mean ten-step MSE, with contrast −0.0563 [−0.1067, −0.0024] against occupancy, despite losing almost half its chains.Ten occupancy checkpoints had observation NLL above 1, reaching 17.37 on pendulum swingup, while retaining every chain with marginal gaps within ±0.007.Source: §5.4 · p. 7
  • Scaling the occupancy model’s prior output by 1.001 left 105-step survival unchanged but removed exact preservation; moment diagnostics distinguished the models despite identical survival.The worst-cell second-moment deviation grew from 0.15 to 0.54 to 6.9 for gains 1, 1.001, and 1.003.Source: §5.4 · p. 8
  • Of 348 planning evaluations, 172 failed because imagination produced too few finite candidates for the required CEM elites; return contrasts were conditioned on completion.The native reference completed all 12 evaluations and led in three of the four cells where both it and occupancy completed.Source: §5.4 · pp. 8–9
  • The offline DreamerV3 reference had lower observation NLL and ten-step MSE (0.0024 ± 0.00004 versus 0.1206±0.0244), and both models retained every chain at 103 steps.Exposure was matched at 25.6M frames, but capacity and context differed and wall time was not matched; the native model’s 105-step marginal was not measured.Source: §5.4 · p. 9

6 RELATED WORK

The related work covers structured dynamics and objectives for long-run statistics, as well as target-preserving transitions and flow-based transport. This paper distinguishes its approach by learning coordinates with an exactly preserved latent reference and bounding relaxation to it while also fitting conditional dynamics.

World models and rollout error

Invariant measures and structured dynamics

Flows, conditional transport, and mixing

  • The work identifies which block-affine couplings preserve a Gaussian reference exactly and derives finite-start and policy-deviation bounds by applying Gaussian entropy contraction to joint state–action innovation.The action chart uses conditional Rosenblatt transport.Source: §6 · p. 9

7 CONCLUSION AND LIMITATIONS

The conclusion makes the decoded reference explicit and distinguishes marginal predictability, conditional accuracy, and planning. Its limitations bound the occupancy interpretation and finite-start guarantees, while the accompanying results characterize when Gaussian-preserving representations are exact or structurally restricted.

7 CONCLUSION AND LIMITATIONS

  • An explicitly preserved latent reference makes long-horizon marginals statically sampleable; joint state–action mixing fits controlled dynamics, and parallel noise supplies finite-start bounds.The experiments distinguish marginal predictability, conditional accuracy, and planning.Source: §7 · p. 9
  • Stationarity and action-chart conditions delimit occupancy interpretation; approximate charts and policy shifts require error control, while finite-start bounds use bank-specific moments and real arithmetic.Architecture contrasts combine preservation with achieved fit; the matched intervention isolates output scaling.Source: §7 · p. 9

A.1 TRANSPORT OF STATIONARY PAIR LAWS

  • Smooth density representation uses conditional CDFs that are jointly C^k with positive derivatives and full-line support limits; smoothness of the joint density alone need not suffice.Locally uniform integrable bounds on density derivatives can justify differentiation of marginal integrals.Source: §A.1 · p. 13
  • With an exact chart and a joint density, unrestricted Gaussian-preserving transition kernels represent a stationary pair law with zero marginal and conditional excess loss.Finite parameterizations retain approximation restrictions, and restricting transports or transitions can introduce approximation or optimization error.Source: §A.1 · p. 14

A.2 RIGIDITY AND PRESERVING DIFFEOMORPHISMS

  • For an exactly volume-preserving diffeomorphism, a population second moment equal to the dimension implies Gaussian preservation, but a near-zero Monte Carlo estimate does not establish that equality.Cancellation and sampling can conceal nonzero pointwise residuals.Source: §A.2 · p. 14
  • A Gaussian-preserving diffeomorphism with a triangular Jacobian in a fixed coordinate order can only flip coordinate signs; compositions in that same order remain restricted to sign flips.Changing the order between layers removes the common-order hypothesis.Source: §A.2 · pp. 14–15
  • For a block map that leaves its first coordinates unchanged, Gaussian preservation requires each conditioned second-block map to preserve its Gaussian reference; scalar blocks reduce to a constant sign flip.Depth and conditioner expressiveness do not change the common-order rigidity result for triangular compositions.Source: §A.2 · p. 15

A.3 IDEAL APPROXIMATION AND AUGMENTED REPRESENTATION

This section develops ideal-approximation and augmented-representation results for controlled occupancy preservation, including its limits, transient behavior, marginal weighting, and conditions for path-law guarantees. It relates Gaussian-preserving transitions and convergence guarantees to scan structure, action charts, contraction, and an approximation argument accounting for state–action dependence away from the reference.

A.3 IDEAL APPROXIMATION AND AUGMENTED REPRESENTATION

  • Finite compositions of pairwise quantile swirls approximate any stated smooth Gaussian-preserving diffeomorphism in C1 on compact sets.The result assumes a compactly supported Gaussian-preserving isotopy and allows arbitrary profiles and unrestricted composition length; it gives no fixed-depth, capacity, or rate guarantee.Source: §A.3 · pp. 15–16
  • A Gaussian-preserving kernel with positive conditional densities admits a representation using d auxiliary Gaussian coordinates, modulo null sets, and a C1 representation under the stated conditional-CDF hypotheses.The controlled extension represents the full conditional K(z, α, ·), not only its action-averaged state kernel.Source: §A.3 · pp. 16–17

A.4 THE CONTROLLED CONSERVATION LAW

  • If the deployed policy’s action-chart image is Gaussian at every state, a Gaussian-preserving joint map keeps the state law Gaussian at every step, including at initialization.The chart condition holds under the behaviour policy and can also hold for observation-conditioned policies; preservation at fixed physical actions is not asserted for state-dependent charts.Source: §A.4 · pp. 18–19
  • With a state-independent action chart, per-action Gaussian-preserving kernels form a strict subclass of behaviour-averaged Gaussian-preserving kernels, including smooth positive-density transitions.The cited construction realizes the strict inclusion with orthogonal augmented maps.Source: §A.4 · p. 19
  • When the averaged kernel preserves the reference and contracts in total variation with coefficient ρ < 1, bounded conditional action deviation ηt implies lim sup dT ≤ η̄/(1 − ρ) if ηt ≤ η̄.For the observation-conditioned chart, the bound gives ηt ≤ min{2, √2κt}; invariance alone does not establish convergence.Source: §A.4 · p. 20

A.5 STATIC MARGINALS AND THE SIGNED TRANSIENT

  • A fixed-probability mixture of the Gaussian reference component preserves the state marginal, and reference preservation yields transient bounds that vanish only when total-variation convergence holds.The decoded statement follows through a fixed observation map or observation kernel; energy-statistic deviations have a bound for bounded observation spaces.Source: §A.5 · pp. 20–21
  • For population optima of Jw = wM + C, increasing w ≥ 1 weakly decreases M and weakly increases C and J1; adding unpaired observations motivates w = 1 + Nunpaired/Npaired.This ordering applies to population cross-entropy optima, not held-out energy distances or finite optimization results.Source: §A.5 · p. 21

A.6 ONE-STEP ACCURACY AND PATH-LAW DIVERGENCE

  • The ensuing path-law identities assume a shared behaviour rule and a state variable sufficient for the Markov property; a pixel observation alone may not suffice.Source: §A.6 · p. 21
  • A misspecified linear Gaussian transition can shift its stationary variance or produce divergent rollouts at |m| ≥ 1, despite a reverse one-step KL proportional to δ^2v_t.For |m| < 1, the stationary variance differs from 1 by 2λδ/(1 − λ^2) + O(δ^2).Source: §A.6 · p. 21
  • With invariant model marginals and constant reverse conditional KL, joint path divergence is (T − 1)ℓrev_0; forward sequence KL is affine in T without model invariance.The forward expression includes the initial KL and (T − 1)ℓfwd_0.Source: §A.6 · p. 22

A.7 SEQUENTIAL SCAN, COPULAS, AND OPERATOR CERTIFICATES

  • Sequential scans preserve the Gaussian reference when each angle depends only on permitted untouched coordinates; including an earlier input can invalidate the guarantee.The passage notes a scalar counterexample where individually preserving branches yield a nonpreserving mixture.Source: §A.7 · p. 22
  • Gaussian-preserving scalar kernels correspond bijectively to doubly stochastic copula densities, with Gaussian copulas forming a one-parameter subset; the multivariate scan is not one global one-parameter family.Conditional coefficients in the multivariate scan can vary with admissible untouched coordinates.Source: §A.7 · p. 23
  • For a Gaussian-preserving kernel, the mean-zero L2 norm σ2 gives geometric χ2 contraction and a population sufficient horizon for output-metric error; finite-basis estimates do not certify that horizon.The horizon bound is an upper bound, not an equality for a particular observable.Source: §A.7 · pp. 23–24

B AN EXPLICIT CONVERGENCE CERTIFICATE FOR PARALLEL CONTROLLED

  • The parallel controlled construction combines preserving joint maps with parallel Gaussian noise; its entropy bound contracts joint-input divergence by c⋆^2 and, under an exact behaviour action chart, bounds the averaged state-kernel divergence likewise.The state output preserves the reference at the joint reference, although its law at a fixed physical action need not; the convergence bound uses the specified initialization law, including finite empirical laws.Source: B AN EXPLICIT CONVERGENCE CERTIFICATE… · p. 26, §B.1 · p. 26

B.1 JOINT MIXING AND PARALLEL CONDITIONAL NOISE

  • The diagonal OU stage factors into a scalar-coefficient kernel and coordinatewise kernels that preserve the reference, including reflection when a coefficient is negative.Data processing through this factorization gives a c⋆^2 factor; applying it through S_b and convexity over ρ_b proves equation 29.Source: §B.1 · pp. 26–27
  • Away from the reference, the chain-rule term includes dependence between h and b.Source: §B.1 · p. 27
  • Under the behaviour chart, the joint input is μ ⊗ γ_m, yielding the state-kernel statement.Source: §B.1 · p. 27

B.2 A FINITE WARM START AND A DECODED PLATEAU BOUND

Together, these parts develop finite-start and policy-shift guarantees and report numerical and experimental evaluations, including output-gain sensitivity, planning contrasts, and an offline recurrent baseline. The scope includes recurrent pixel-model evaluation, optional refresh, and a traced initialization-bank guard crossing; comparisons are limited to measured horizons or completed cells and do not establish a suite-wide planning comparison.

B.2 A FINITE WARM START AND A DECODED PLATEAU BOUND

  • A finite-second-moment initialization, including an atomic finite bank, gives an absolute-convergence bound under the exact chart and pointwise norm-preserving map; bounded decoder energy-statistic error is controlled by total variation.A finite bank certifies only its specified initialization law, while an unknown population requires a valid moment upper bound. The evaluator’s distance must be bounded in its actual units.Source: §B.2 · p. 27

B.3 POLICY DEVIATION IN THE ORIGINAL ACTION SPACE

  • Under the exact behaviour chart, policy shift yields a recurrence based on conditional action-space divergence; deterministic CEM selection gives an infinite KL bound against a nonsingular tanh-Gaussian behaviour policy.A stochastic observation context can make the joint-divergence inequality strict, and the policy expectation is over the deployed model’s joint state-context law. For more general controlled kernels, the Pinsker perturbation bound has different architectural hypotheses.Source: §B.3 · p. 28

B.4 AN OPTIONAL REFERENCE COMPONENT

  • A whole-state independent refresh component gives a total-variation bound of 2(1−r)^T, while the parallel kernel’s KL contraction uses q = (1−r)c2⋆.For the parallel kernel, the warm-start bound becomes (1−r)B⋆, and its policy recurrence is Ht+1 ≤ q(Ht + κt). The refresh must reset recurrent memory too; its effects on short-horizon prediction and control require separate evaluation.Source: §B.4 · p. 28

C.1 DATA AND OCCUPANCY SCREENING

  • The twelve evaluated pixel tasks were selected from eighteen that passed finite-data occupancy screens, which do not establish stationarity or a latent Markov representation.Screening covered 22 tasks and used temporal, coverage, and matched leave-one-out energy-distance checks; swimmer was retained with a metastability flag, while concentrated occupancies led to exclusions. The pixel study used sequential-scan transitions under an external OU process, a narrower setup than the state-dependent occupancy construction.Source: §C.1 · p. 29

C.2 SEQUENTIAL-SCAN FITTING OBJECTIVES AND BUDGETS

  • Sequential-scan fitting uses distinct likelihood and regression objectives, and the stated conditional-density bounds assume the mathematical kernel, exact behaviour innovations, and a uniform coefficient cap.Flow-matching loss is not a comparable likelihood value; the Lorenz soft-stabilization table reports nats per pair rather than per-dimension NLL. Numerical diagnostics do not provide a global roundoff certificate.Source: §B.5 · p. 28, §C.2 · p. 30

C.3 DISTANCES, UNCERTAINTY, AND REFERENCE DEPENDENCE

  • The occupancy-gap mean remains −0.000182, while its reported within-task SE is 0.000325 and its grouped shared-seed SE is 0.000602.Both summaries are compatible with zero, but only three shared-seed groups limit uncertainty calibration; neither covers collection uncertainty or a population of tasks. Paired estimates also condition on the fitted model and dataset, and temporal dependence can affect reference terms.Source: §C.3 · pp. 30–31

C.4.1 MODEL AND SEQUENTIAL OBJECTIVE

  • The recurrent study jointly learns an encoder, stochastic posterior, decoder, reward head, and transition prior, then evaluates six prior arms with held-out forecasts and long rollouts.All arms share the CNN, posterior, decoder, and reward architecture; the occupancy prior’s certificate concerns the generative prior, not the aggregate variational posterior. Evaluation uses three paired training seeds, with a frozen initialization bank and distinct short-horizon, terminal-gap, and planning protocols.Source: §C.4.1 · pp. 31–32, §C.4.2 · p. 32

C.4.3 POSTERIOR INITIALIZATION AND GUARD CROSSINGS

  • 21 of 8,192 conditioner-only chains crossed the latent guard at the first step, while the Gaussian arm showed progressive leakage between 10^4 and 10^5 steps.The conditioner-only escape cell was swimmer, seed 1103; the γd-initialized control cloud survived all 8,192 chains to 105 steps. The bank’s second moment was 7.4 × 10^10, and the fitted posterior means traced the large initial states to one held-out episode segment.Source: §C.4.3 · p. 33

C.4.4 OUTPUT-GAIN CONTROLS

  • All output-gain cells retain their chains at measured horizons, while second-moment deviations increase with gain.Gain 1.001 was validation-selected and gain 1.003 exploratory; scaling also affects the reference-reset branch, changing the Gaussian reference law, with reset probability 10−3 retained.Source: §C.4.4 · p. 33

C.4.5 PLANNING PROTOCOL AND COMMON-CELL CONTRASTS

  • The primary CEM comparison uses horizon 20, 64 candidates, eight elites, three iterations, four particles, and ten paired 200-decision episodes.Each fitted model entails 30.72 million imagined transitions; horizons 5 and 50 form a separate prespecified sweep on four priority tasks with occupancy and Gaussian models.Source: §C.4.5 · p. 34
  • Across completed common plan cells, candidate-minus-occupancy normalized-return differences are descriptive and do not establish a suite-wide planning comparison.The equal-task mean differences (cross-task SE; common cells) were −0.0000 ± 0.0007 for conditioner-only (17), −0.0053 ± 0.0068 for Gaussian (13), +0.0089 ± 0.0080 for conditional RealNVP (11), −0.0128 ± 0.0135 for latent overshooting (12), and +0.0051 ± 0.0066 for invariant penalty (12). Against the native reference, its returns were higher on three of four completed priority-suite cells and lower on swimmer seed 1103: 0.1312 and 0.1246 on swimmer seeds 1101–1102 and 0.1098 on quadruped, versus 0.1109, 0.1188, and 0.1041; it returned 0.1152 versus 0.1211 on swimmer seed 1103.Source: §C.4.5 · p. 34

C.4.6 NATIVE RECURRENT BASELINE

  • The offline DreamerV3 baseline uses a larger recurrent model and has no Gaussian latent reference, static-limit identity, or convergence certificate.Its 8.85-million-parameter model trains for 25,000 updates on 25.6 million frames, matching primary fits’ frame exposure; native rollouts start from filtered data and extend through 103 steps. Native forecasts and CEM use the same physical actions and evaluation budgets, but the passage provides no measured comparison values.Source: §C.4.6 · p. 34

C.4.7 DETAILED RECURRENT AND NATIVE-MODEL RESULTS

The recurrent comparison contrasts survival, prediction, marginal agreement, and planning, while the Lorenz analyses test whether static reference scores predict rollout plateaus. Results are scoped to the collected data, eligible tasks, fixed behaviour policy, paired seeds, and reported budgets.

C.4.7 DETAILED RECURRENT AND NATIVE-MODEL RESULTS

  • All four nonpreserving arms lose chains at 105 steps, while occupancy retains every evaluated chain.Figure 5 reports paired candidate-minus-occupancy survival contrasts over twelve tasks and three seeds.Source: Figure 5 · p. 35
  • The primary comparison covers 216 checkpoints, but 168 have at least one unavailable estimate because of rollout censoring or planning failure.Of 648 primary evaluation stages, 545 complete and 103 have deterministic planning failures; the protocol does not estimate unconditional distances from censored rollouts or full-grid returns when a required planner fails.Source: §C.4.7 · p. 35
  • The native DreamerV3 reference has lower observation NLL and 10-step MSE, while survival is tied at 1.0 at the shared evaluation horizon.The comparison covers four priority tasks; the native 105-step marginal is unmeasured and its categorical state has no Gaussian-reference identity.Source: Figure 7 · p. 37

C.4.8 SCOPE OF THE RECURRENT COMPARISONS

  • Comparisons condition on collected data, eligible tasks, the fixed fitted behaviour policy, three paired seeds, and reported budgets.Planning contrasts use common completed cells, and the prior certificate applies to the specified initialization bank.Source: §C.4.8 · p. 39

D LORENZ CALIBRATION AND COMPETING STABILIZATION OBJECTIVES

  • The unconstrained Lorenz models have lower static reference scores but retain only 13.95%, 93.94%, and 46.02% of their chains at 105 steps.Their static decoded-Gaussian scores do not predict their limiting laws; these comparisons do not establish that an unconstrained kernel lacks an invariant distribution.Source: D LORENZ CALIBRATION AND COMPETING… · p. 40

D.1 REWEIGHTING THE MARGINAL FIT

  • For constrained w = 1 to 10, held-out pair NLL increases by 0.1869 nats per dimension while static distance decreases.Both unconstrained arms retain only about 6% of chains at 105 steps, and their negative distance estimate is not evidence of better full-distribution fidelity.Source: §D.1 · p. 41

D.2 SOFT STABILIZATION OBJECTIVES

The Lorenz experiments compare soft stabilization penalties under finite training budgets, alongside a time-varying negative control. The fixed-latent pixel setup uses separately trained, frozen autoencoders, so it tests transition families within a fixed representation.

D.2 SOFT STABILIZATION OBJECTIVES

  • Only two of nine penalized seed-zero configurations retain at least 99% of chains, and both retain every chain.Measure-penalty weight 0.1 has distance 0.1058 versus 0.2125 for the stationary model, while its static reference score still does not become a predicted limit.Source: §D.2 · p. 42
  • On the ramped Lorenz system, stationary-model pair NLL is 4.7793 versus 3.5166 for the unconstrained model, in nats per dimension.The unconstrained model also fails, and the reference two-half distance is 0.0052, showing that this finite-data diagnostic misses the imposed nonstationarity.Source: §D.2 · p. 42

E FIXED-LATENT PIXEL MODELS: MARGINAL PREDICTABILITY AND CONDITIONAL FAILURE

  • Fixed-latent pixel transition comparisons share each task’s frozen 256-dimensional autoencoder latent, decoder, smoothing scale, and data split.These are transition comparisons in a fixed representation, not end-to-end recurrent pixel-world-model comparisons.Source: E FIXED-LATENT PIXEL MODELS: MARGINAL… · p. 42

E.1 COMPLETE TWELVE-TASK OUTCOMES

Across twelve pixel tasks, stationary models retain every chain, but pixel fidelity is not uniformly best. The section also distinguishes typical conditional fit from extreme losses and reports alternate observation-space diagnostics with their evaluation constraints.

E.1 COMPLETE TWELVE-TASK OUTCOMES

  • Stationary models retain every chain on all twelve tasks, but their pixel fidelity is not uniformly best.Among ten tasks where flow matching also retains every chain, five favour flow matching and five favour the stationary model; the other two flow-matching scores condition on partial survival.Source: §E.1 · p. 44
  • At 10^5 steps, both Walker flow-matching conventions reach approximately 2.27, despite similar distances to the stationary model at 10^3 steps.The sampled flow-matching second moment is 1.06 versus 0.80 for the stationary model, although its distributional distance is worse.Source: §E.1 · p. 44

E.2 TYPICAL CONDITIONAL FIT AND EXTREME LOSSES

  • The Swimmer median of 1.43 × 10^7 and 97% of pairs above ten nats per dimension indicate broad conditional misfit.Reacher instead has a median near 0.85 with an extreme upper tail; large observed losses do not establish infinite population moments.Source: §E.2 · p. 45

E.3 ALTERNATE OBSERVATION-SPACE DIAGNOSTICS

  • The alternate metrics use a state probe whose R2 = 0.616 is a checkpoint-selection score, not an independent test score.Probe error, ambiguity, and failures on generated images limit the physical interpretation.Source: §E.3 · p. 46
  • The free-flow model has lower survivor-conditioned distances in every alternate geometry, while only 60.82% remain finite and 60.79% remain inside the box at the final horizon.The stationary arm keeps all chains; the four geometries share trajectories within this evaluation.Source: §E.3 · p. 46

F ARCHITECTURAL CONTRACTION, FIDELITY COST, AND RECURRENT

The cap experiments examine contraction guarantees alongside fitted fidelity, deterministic memory, rollout variability, and reference-mixture loss control. The results distinguish finite-horizon checks from evidence about convergence and show trade-offs across these settings.

F ARCHITECTURAL CONTRACTION, FIDELITY COST, AND RECURRENT

  • All twelve capped runs pass the flatness check between 10^4 and 10^5 steps, but this does not locate convergence or verify completion by the theoretical horizon.The horizons exceed 4,603 steps; Reacher’s static-versus-terminal gap is −0.0079±0.0015, while the other eleven paired gaps have |t| < 3.Source: F ARCHITECTURAL CONTRACTION, FIDELITY… · p. 47, Table 22 · p. 47
  • Tighter caps shorten the relative-contraction bound but do not consistently improve fitted fidelity: the best capped static discrepancy is worse on Pendulum and Reacher, and better on Quadruped.Moving from cap 0.999 to 0.9 improves conditional NLL on Pendulum and Quadruped but worsens Reacher by about 0.65 nats per dimension; the shortest bound and smallest static discrepancy occur at different caps.Source: F ARCHITECTURAL CONTRACTION, FIDELITY… · pp. 47–48, Table 23 · p. 48

F.1 DETERMINISTIC MEMORY AND PARTIAL NOISE

  • Rotation-only memory retains second moment 11.51 rather than eight, while both stochastic-memory models reach moments near eight and distances near the Gaussian control.A cap on noisy coordinates does not certify contraction of a state that retains deterministic memory; the absolute bound reaches 0.01 in total variation for T ≥474.Source: §F.1 · pp. 48–49, Table 24 · p. 49

G.1 INDEPENDENT STATIC RESAMPLING AT A FIXED REACHER CHECKPOINT

  • Independent static resampling places the resampled Reacher average below the recorded static draw, with rollout laws closer to that average; the contrasts remain unresolved.The independent rollout replication also gives a zero-compatible gap, consistent with sampling variation in the original static draw.Source: §G.1 · p. 49, Table 25 · p. 49

G.1.1 INDEPENDENT REACHER ROLLOUT REPLICATION

  • At T = 100000, the independent Reacher replication reports a data-initialized gap of 0.001711 ± 0.002227; its uncertainty is conditional on one checkpoint and fixed held-out data.The evaluation uses frozen-autoencoder coordinates and clipped OU actions, and measures rollout-sampling variation rather than replicating training.Source: §G.1.1 · pp. 49–50

G.2 REFERENCE MIXTURES BOUND LOSSES WITHOUT RESTORING DYNAMICS

  • For Swimmer, the reference component has posterior density responsibility above one half on 99.15% of default-checkpoint pairs and 98.26% of capped-checkpoint pairs, despite a 0.1% sampling probability.The reduction in scored log-loss is supplied mainly by the static reference component; for the reset mixture, the additional conditional loss is at most −log(1−r).Source: §G.2 · p. 50

G.3 UNTRAINED PARALLEL KERNELS AND ATOMIC STARTS

The appendix checks finite-start bounds and implementation behavior using untrained parallel kernels, analytic specialization, and CPU timing. These checks support numerical consistency within their tested settings, not proofs beyond them.

G.3 UNTRAINED PARALLEL KERNELS AND ATOMIC STARTS

  • All 114 reported bounded-observable comparisons are compatible with their finite-start bounds, testing implementation consistency rather than the proof itself.The parallel-kernel CPU experiment uses random, untrained weights; its fixed atomic bank begins with infinite KL to the Gaussian, so the bound starts after the first noisy transition.Source: §G.3 · pp. 50–51
  • The two-dimensional specialization’s implemented one-step density agrees with the analytic conditional Gaussian density, and its reported mixture KL and L1-TV integrals lie below their finite-start bounds.Quadrature and finite-sample checks do not replace the proof; marginal Gaussian action fit alone fails to supply the conditional chart in the negative control.Source: §G.3 · p. 51, Table 27 · p. 51
  • The prior-only CPU timing uses 0.30, 0.48, and 0.85 million parameters at latent widths 256, 512, and 1024, respectively.The benchmark excludes observation networks, optimization, and data loading, and does not predict GPU throughput or trained-model accuracy.Source: §G.3 · p. 51, Table 28 · p. 51

H.1 A SYSTEM WITH AN EXACT CONDITIONAL ACTION CHART

The controlled-system examples examine how conditional action charts and preservation constraints affect transition fit and planning, while numerical checks probe density identities and architecture scope. Their conclusions remain specific to the tested models and settings.

H.1 A SYSTEM WITH AN EXACT CONDITIONAL ACTION CHART

  • At ϱ = 0.8, the implemented scalar OU family has excess costs 0.5108, 0.3681, and 0.1408 for τ = 0, 0.6, and 0.9, but only the first is an unrestricted per-action-class floor.The approximate, uncertified class optima for τ = 0.6 and 0.9 are about 0.32 and 0.04; the action chart removes state–action dependence that a marginal action transform cannot.Source: §H.1 · pp. 51–53, Table 29 · p. 52

H.2 PLANNING ON THE STOCHASTIC DUFFING OSCILLATOR

  • On the stochastic Duffing oscillator, the Gaussian baseline is close to the oracle at reported horizons, while the occupancy model has mean excess 0.242 at horizon 500 and the per-action arm has positive excess in all 15 cells.The comparison does not isolate occupancy drift as the cause of the per-action deficit; the oracle also becomes less effective at long horizons.Source: §H.2 · pp. 53–54, Table 32 · p. 53

I.1 INDEPENDENT DENSITY AND PRESERVATION CHECKS

  • The additive control has Gaussian-identity residual 0.217, while the largest tested analytic-versus-autograd log-determinant discrepancy is 8.8 × 10^-14.The tests are finite numerical checks, not proofs for arbitrary weights or conditioning; coordinate-dependent branch selection can fail despite reference-preserving components.Source: §I.1 · pp. 54–55, Table 34 · p. 54

I.2 ARCHITECTURE SCOPE: PARALLEL PRIOR AND SEQUENTIAL SCAN

  • The parallel finite-start bound applies to its specific conditioning graph, whereas the sequential-scan L2 bound is proved under a different graph.The parallel update rotates joint state–innovation input and refreshes every persistent stochastic coordinate; sequential-scan conditioners exclude the coordinate being updated.Source: §I.2 · p. 55

I.3 A SCALAR ILLUSTRATION OF ERROR SENSITIVITY

A scalar autoregressive example illustrates that small conditional error can coexist with a larger stationary-law change.

I.3 A SCALAR ILLUSTRATION OF ERROR SENSITIVITY

  • A 0.01% coefficient perturbation at λ = 0.999 increases stationary variance by about 11% while costing 2.5 × 10^-6 nats of conditional KL.Tying the fitted coefficient and scale through m2 + s2 = 1 removes this particular variance error.Source: §I.3 · p. 55

J.1 FINITE TRANSITION CAPACITY

These experiments compare finite transition capacity and a non-Gaussian copula witness, then test what finite-basis operator estimates can establish about the full norm.

J.1 FINITE TRANSITION CAPACITY

  • Combining rotations and swirls improves on either primitive, but unconstrained models remain better at this training budget, so these are fitted-model comparisons, not converged limits.Broader stochastic dependence also helps.Source: §J.1 · pp. 55–56
  • At 60,000 updates, OU pair NLL is 2.1777 nats per dimension versus 2.3221, 2.1895, and 2.3104 for three broader copula variants despite 3–10% more parameters.The kernel-only copula improvement does not automatically transfer to joint chart–transition learning.Source: §J.1 · p. 56

J.2 A NON-GAUSSIAN COPULA EXAMPLE

  • 0.1123±0.0005 is the learned cosine basis’s three-seed score, while adaptive Gaussian scores 0.0636±0.0005; scalar Gaussian-copula correlation loses the example’s information.The basis contains the example’s truth at β2,1 = 0.45; the adaptive Gaussian arm has no guaranteed Gaussian output marginal.Source: §J.2 · p. 56

J.3 FINITE-BASIS OPERATOR ESTIMATES

  • Finite-basis population compressions cannot certify the full Markov operator norm; sampling can inflate estimates and truncation can omit slow modes.Constant-OU controls recover known norms within 0.0047, while orthogonal deterministic maps return norm one.Source: §J.3 · p. 57
  • A nonnormal Gaussian example has singular norm 0.900 but spectral radius 0.474; quadratic estimates 0.806 and 0.812 detect the true mode 0.810, unlike the linear estimate 0.004.These examples show why singular norm and spectral radius cannot be interchanged and why a linear-only estimate can miss a quadratic mode.Source: §J.3 · p. 57

Related papers

Loading 2609.32657v1…