Source-linked AI summary
Correcting a learned physical invariant improves world-model rollouts
Richard Bao
TL;DR
World models may predict video without reliably preserving the dynamics that generated it, so this paper tests whether a frozen DreamerV3 learns a scalar preserved by its own latent transition. A label-free latent search identifies an energy-like invariant in conservative pendulum models but not matched damped models, and correcting its rollout drift reduces prediction error in all three conservative models. The study therefore separates a dynamically used invariant from a merely decodable correlate while exposing a gap between encoding a physical constraint and respecting it during autonomous prediction.
Problem
Video prediction does not establish that a world model learned dynamics its transition reliably preserves, and latent probe correlations can appear before learning.
Method
A label-free search fits polynomial scalars in frozen DreamerV3 latent states, evaluates conservation and flow alignment, and projects autonomous states toward the recovered invariant level set.
Results
Correcting the recovered invariant lowers rollout error by 2.9%, 3.3%, and 3.5% across the three conservative models, while matched random constraints show a median change of +53.9%.
Takeaways & Limitations
A world model can learn a physical constraint from pixels yet violate it during autonomous imagination, while intervention provides evidence beyond mere decodability.
Takeaways & Limitations
The study covers one one-degree-of-freedom system and architecture, uses three trained seeds, and estimates an in-sample intervention effect with ranges rather than confidence intervals.
Abstract
from arXiv · showhide
World models can predict video without learning dynamics that they reliably preserve. We test whether a frozen DreamerV3 trained only on pendulum video learns a scalar that its own latent transition treats as approximately conserved. A label-free search recovers the same energy-like invariant across independently trained conservative models, while the same procedure finds no comparable invariant in matched damped models. During autonomous rollouts, this quantity drifts. Projecting the latent state back toward its initial level set reduces rollout error in all three conservative models, whereas matched random constraints usually increase it. These results distinguish a dynamically meaningful invariant from a merely decodable correlate and reveal a concrete failure mode: a world model can learn a physical constraint from pixels yet violate that constraint when it imagines forward.
1 Introduction
World models can predict future video without learning dynamics they reliably preserve, especially beyond training trajectories. This study asks whether a frozen video-trained DreamerV3 contains a transition-preserved scalar and whether violating it contributes to prediction error.
- Motivation: Video prediction alone does not establish that a world model has learned the observed system’s underlying dynamics.The distinction matters when predictions extend far beyond the trajectories used for training.
- Motivation: Probe accuracy reveals information retained in latent states, but not whether the model’s transition uses that information.Six randomly initialized DreamerV3 models produced polynomial energy correlations up to 0.908, showing that strong correlations can precede learning.
- Research question: The study tests whether a trained world model contains a scalar its own transition approximately preserves and whether failures to preserve it contribute to prediction error.The model is a frozen DreamerV3 trained only on 64 × 64 pendulum video, without physical labels, actor, or critic.
- Contribution: Independently trained conservative models recover nearly identical energy-like invariants, while matched damped models lack a comparably conserved scalar.The recovered invariant drifts during autonomous imagination; correcting that drift improves 50-step predictions in every conservative model, whereas matched random corrections usually worsen them.
2 Recovering a latent invariant
The method searches latent-state polynomials for scalars that remain constant along trajectories while differing across trajectories, then uses flow alignment to select among conserved candidates. It operates on frozen DreamerV3 latent states and the model’s own transition, with controls for damping, random initialization, and preliminary development experiments.
- Model and data: Pendulum data use zero actions, sampled simulator states, cropped and downsampled frames, and trajectories rejecting the simulator’s |˙θ| = 8 speed clip.Because a single frame does not reveal velocity, the model must infer ˙θ from frame sequences.
- Latent trajectories: The study separates observation-conditioned trajectories from autonomous imagination rollouts, using the model’s one-step transition T without new observations for autonomous updates.Latent flow is estimated by applying T once at states obtained from real video, while rollout tests repeatedly apply T from one encoded state.
- Latent coordinates: After freezing the model, extraction and perturbation operate in the top 12 principal directions of deterministic recurrent states.The projection removes unsupported directions and centers states by their mean.
- Candidate family: Candidates are degree-4 polynomials in 12 latent coordinates, comprising 1819 nonconstant monomials so energy can be expressed through an unknown nonlinear latent encoding.The basis is rich enough to approximate pendulum energy while remaining constrained enough for fitting.
- Invariance criterion: The invariance score is the within-trajectory variance of a candidate divided by its total variance, favoring scalars that are constant within trajectories but differ across them.The denominator prevents the trivial zero candidate from scoring perfectly.
- Candidate selection: A generalized eigenproblem ranks conserved candidates, and alternating least squares fits an antisymmetric flow operator among the top eight eigenvectors.The flow criterion helps distinguish candidates such as E and E^2, while reducing variation across random seeds; conservation drives most recovery and intervention effects.
- Validation and controls: The reference pendulum energy is compared after label-free extraction, while the search itself sees only latent states and the model’s one-step transition.Simulator integration produces about 12% relative oscillation in textbook energy without secular drift, treated as a noise floor.
3 Training makes invariant recovery reproducible
Training makes the recovered scalar reproducible across independently trained conservative models, beyond the energy-correlated readouts already present in random Dreamers.
- |ρ|E = 0.973, 0.967, and 0.975 across three trained conservative models, with within-trajectory variance only 1.1 × 10−4 of total variance.
- Randomly initialized Dreamers span |ρ|E = 0.170–0.908, showing that strong energy correlation can occur without training.
- The trained and untrained groups separate without overlap on energy correlation and pairing residual, but the small sample sizes are not statistically decisive.
- The trained models cluster between 0.967 and 0.975, whereas untrained models span 0.170–0.908, indicating greater recovery reproducibility after training.
4 The invariant disappears under matched damping
Matched damped models provide a control in which the underlying dynamics remove the corresponding conservation law, and the search finds no comparable approximately conserved scalar.
- The damping control tests whether the pipeline manufactures conserved-looking scalars when the underlying dynamics lack the corresponding conservation law.
- Damped trajectories converge to one fixed point, leaving no nontrivial continuous first integral throughout the basin.
- The damped system uses a preselected damping level that avoids trivial latent collapse caused by excessively strong damping.
- The recovered C varies about 2400× more within damped trajectories, while damped held-out invariance ratios are 0.983–0.994 versus 0.0027–0.0121 conservatively.
- The conservative and damped arms share architecture, budget, observations, and extraction; only training-data dynamics differ, without proving absence of every possible invariant.
5 Correcting invariant drift improves rollouts
The recovered invariant drifts during autonomous imagination, and projecting latent states toward its initial level set improves conservative-model rollouts more reliably than matched random constraints.
- The recovered scalar is nearly constant on observation-conditioned trajectories but drifts when the model repeatedly applies its own transition.
- The intervention projects each imagined latent state toward the level set determined by its initial C value, without retraining or modifying the decoder.
- The experiment compares each recovered conservative invariant with a norm-matched random degree-4 polynomial and with the damped model’s own recovered C.
- Enforcing the recovered constraint lowers 50-step rollout error by 2.9%, 3.3%, and 3.5% across the three conservative models.
- Across 60 random-constraint evaluations, the median error change is +53.9%, and only 7 of 60 constraints lower error.
- Specificity is not uniform: one conservative model has random constraints outperforming the recovered constraint in slope and strongest-projection error.
- Overall, recovered-invariant enforcement improves all conservative models, while matched random constraints usually worsen rollout error.
- The recovered correction preferentially acts along higher-sensitivity directions rather than simply the highest-variance latent directions.
6 Limitations
The study is narrow in system, architecture, training duration, and sample size; intervention effects are modest, nonuniform, and partly in-sample.
- The experiments cover one physical system, one degree of freedom, one architecture, and a much shorter training schedule than standard DreamerV3.
- Three trained seeds and six random draws show non-overlapping ranges but cannot provide precise effect sizes or confidence intervals.
- The intervention improves all conservative models, but specificity against matched random constraints holds for only two models and evaluation is in-sample for invariant fitting.
- The pairing residual is a diagnostic rather than a confidence score because it does not track recovery quality across extraction dimensions.
- The experiments do not test contacts, multiple interacting objects, nonzero actions, or distinguish weighted from unweighted invariant-error accumulation.
7 Related work
Prior work often probes representations or imposes physical structure by design, whereas this paper tests whether conservation emerges in a conventionally trained world model and matters for its rollouts.
- Probes reveal recoverable information but not whether latent dynamics use or preserve the corresponding physical variable.
- Equation-discovery methods infer governing equations, while Hamiltonian and Lagrangian networks impose conservation architecturally.
- The paper instead analyzes physical structure emerging in a conventionally trained world model using standard linear-algebraic tools.
- Prediction and control metrics do not reveal whether a latent transition preserves a learned physical constraint.
- Rollout intervention tests whether violating a recovered constraint contributes to error, while matched damping tests for conserved-looking artifacts.
8 Conclusion
The study finds that DreamerV3 learns an energy-like quantity from pendulum video, but its autonomous dynamics fail to preserve it. Damping and intervention tests provide the stronger evidence for this distinction.
- DreamerV3 learns an energy-like latent quantity approximately preserved on observation-conditioned pendulum trajectories.
- Randomly initialized models also admit strongly energy-correlated readouts, so correlation alone does not establish learned dynamical structure.
- The invariant disappears under dissipative training dynamics, while correcting its drift during autonomous imagination improves prediction in every conservative model.
- The result identifies a gap between encoding a physical constraint and respecting it during autonomous rollout.
A Experimental details and extraction ablations
The experiments document training and evaluation choices, including an in-sample correction analysis and slope-based scoring. Ablations examine selection criteria and extraction dimension.
- Training and data: The correction is evaluated on the same 52 trajectories used to fit the invariant, making its absolute effect in-sample with respect to invariant fitting.
- Scoring: Rollout-error slopes are scored across a fixed α grid rather than selecting the most favorable grid minimum.
- Extraction ablations: The joint criterion recovers energy at 0.973 versus 0.950 for conservation alone, while conservation alone has a slightly better mean intervention slope of −0.087 versus −0.077.
- Extraction ablations: Energy recovery rises from 0.721 to 0.967 as extraction dimension increases from 6 to 12, while pairing residual worsens from 0.738 to 0.876.
B Latent sensitivity analyses
Latent sensitivity analysis distinguishes variance from rollout damage and examines how corrections align with consequential directions. Basis-independent properties are more reproducible than coordinate-based selectors.
- Sensitivity measures: For each extracted direction, the analysis measures variance and rollout damage from a displacement along that direction.
- Sensitivity measures: The study uses ϵ = 0.25|z|, the smallest displacement whose damage exceeds 10% of baseline while remaining smoothly responsive.
- Sensitivity measures: After 40 steps, displacement retains 6.3×, 2.8×, and 1.9× its initial size at ϵ/|z| of 0.05, 0.25, and 0.5.
- Sensitivity measures: At a 40-step horizon, variance and damage rankings anticorrelate on all three models, with damage spanning 13.5× to 118.8× across directions.
- Sensitivity measures: Correction magnitudes align positively with higher-damage directions at +0.385, +0.692, and +0.287, but only partly concentrate there.
- Cross-model reproducibility: Basis-independent recovery, intervention slopes, and top-3 eigenvalue mass vary by 0.008, 0.016, and 0.046 across models, outperforming unstable coordinate selectors.
C Additional sweeps
Additional sweeps show that energy information can reside outside the highest-variance latent directions, while accumulated invariant error predicts rollout degradation better than a fitted power law.
- Energy outside top directions: 2.23× and 2.13×: directions 7–12 carry about twice the energy information of the top six in two models, while the third reverses the ratio to 0.91.Models are reported separately because a median would hide this sign change.
- Accumulated invariant error: R2 = 0.187 for an integral predictor versus 0.021 for a fitted power law in time at the horizon where both perform best.Shuffling destroys the relation.