Source-linked AI summary
When Does Dynamics Randomization Test Context Inference?
Oleg Smirnov, Sofiane Ennadir, John Pertoft, Bjartur Hjaltason, Sara Karimi
TL;DR
Randomized-dynamics benchmarks are used to test whether world models need explicit context inference, but randomization may leave little predictive information for it to add. The paper measures this remaining one-step information and separates query-coverage changes from context-weight changes. Its examples show low headroom for different reasons, while state resets increase headroom.
Problem
Randomized-dynamics benchmarks test whether world models benefit from explicit context inference, but parameter randomization may leave little predictive information for it to add.
Source: §1 · p. 1Method
Results
Wind query matching reduces the reference by 29% and visitation weights remove a further 88% of the matched reference; Resistance retains about 93%, while Attractor headroom is about 11% of its interior reference.
Source: §3 · p. 3Takeaways & Limitations
Dynamics randomization alone does not establish that a benchmark tests context inference; reporting headroom helps distinguish limited opportunity from weaknesses of the compared methods.
Source: §5 · p. 5The measurements concern one-step prediction on the collection distribution, so low on-distribution headroom can coexist with broad counterfactual coverage.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Environments with randomized dynamics are widely used to ask whether world models need explicit context inference. However, randomization can leave little predictive information for a context encoder to add, because distinct latent parameters may produce similar transitions at visited states, or those states may already reveal the active context. We formalize the remaining one-step information as predictive headroom and compare kernels under fixed and visited state--action distributions to distinguish changes in query coverage from changes in context weights. We analyze discrete and continuous environments and find low headroom both when kernels differ little and when visitation reduces an otherwise large discrepancy. At the same time, resetting state components increases headroom, showing that context evidence accumulates in state. We propose reporting this headroom alongside the randomized parameters, so that gains from explicit context inference can be interpreted relative to the information missing from the current state and action.
1 Introduction
Randomized-dynamics benchmarks test whether world models benefit from explicit context inference, but parameter randomization may leave little predictive information for context inference to add. This paper diagnoses how much transition information remains beyond the current state and action.
1 Introduction
- Randomized dynamics are used to compare context-aware models with context-blind predictors, but parameter variation alone may leave little information for context inference to add.Latent physical parameters can vary across episodes.Source: §1 · p. 1
- Low remaining information can reflect either small kernel differences at visited queries or states that already reveal the hidden parameter.A wall can block wind displacement, while persistent force can leave evidence in position and velocity.Source: §1 · p. 1
- The proposed diagnostic compares kernels under fixed and visited state–action distributions, then reweights contexts by visitation to estimate predictive headroom.The paper examines discrete and continuous settings and recommends reporting headroom when comparing world models.Source: §1 · p. 2
2 Predictive headroom
Predictive headroom measures the one-step transition information context adds beyond the current state and action. The paper separates kernel differences due to query coverage from changes in context weights associated with visitation.
2 Predictive headroom
- Predictive headroom is the minimum expected one-step log-loss difference between predicting from the context mixture and revealing the active context.It measures transition information beyond the current (s, a) pair and can be estimated using known kernels and fitted context mixtures.Source: §2 · p. 2
- Istruct averages prior-weighted kernel separation over a fixed query distribution, whereas Imatch averages it over visited queries while retaining prior context weights.The fixed reference supports comparisons at common queries; the matched measure evaluates differences where data were collected.Source: §2 · p. 2
- The structural-to-matched comparison changes query coverage, while the matched-to-visited comparison holds queries fixed and replaces prior weights with visitation-dependent context weights.The reweighting can reflect state-dependent evidence and marginal shifts from pooling episodes of unequal length.Source: §2 · p. 2
- A concentrated marginal context distribution can reduce headroom even without state-dependent evidence, so marginal context shifts must be considered when interpreting reweighting.Uniform random actions mean any context evidence in predictor inputs comes from state.Source: §2 · p. 2
3 Controlled evidence
Across Wind, Resistance, and Attractor, low visited headroom arises from different combinations of kernel variation and visitation-dependent context weights. State resets increase headroom, while the estimates are fitted-predictor estimates rather than exact mutual information.
3 Controlled evidence
- Wind visitation reduces the fixed reference by 29% through query matching and removes a further 88% of the matched reference through visitation-dependent context weights.Resistance retains about 93% of its matched reference, while Attractor’s visited headroom is about 11% of its large interior reference.Source: Table 1 · p. 3, §3 · p. 3
- The experiments use 5,000 episodes per setting with uniform random actions, evaluating transition prediction without training policies or comparing returns.MiniGrid episodes last up to 256 steps and Attractor episodes last 50 steps, avoiding duration-induced context reweighting in Attractor.Source: §3 · pp. 2–3
- MiniGrid visited headroom uses exact kernel KL divergences and fitted context mixtures, while Attractor scores logged next states against a fitted nearest-neighbor context posterior.The reported estimates retain all transitions; Attractor intervals omit posterior-refitting uncertainty, and the paper reports fitted-predictor estimates rather than exact mutual information.Source: §3 · p. 3
- Frequent Wind resets increase headroom, and resetting both Attractor position and velocity raises it more than resetting position alone.The transition family and episode context remain fixed, but resets change query coverage and the posterior over context; the comparisons do not isolate these effects.Source: §3 · p. 4, Table 2 · p. 4
4 Implications for evaluating world models
The section recommends evaluating context information alongside model error, because headroom depends on what the inputs retain and on the collection distribution. Reset protocols and deployment conditions can change which states and actions are represented.
4 Implications for evaluating world models
- Comparisons should measure both information available in the inputs and actual model errors under matched data and compute.Low headroom may explain a small gain from explicit context inference, but a context encoder may still improve learning or approximation.Source: §4 · p. 4
- Report kernel distinctness at overlapping observed states, collection rules, transition weighting, and marginal context frequencies relative to the design prior.For learned kernels, account for prediction error; a low matched reference suggests changing queries or perturbations, while a large matched-to-visited reduction calls for separating context imbalance from information retained in state.Source: §4 · p. 4
- Initial-state distributions, episode length, and observed physical coordinates influence how much context remains hidden; a reset that changes position but preserves velocity can retain context evidence.Report matched-query variation under each reset protocol’s collection distribution to separate prediction-support changes from context-weight changes.Source: §4 · p. 4
- The measurements concern one-step prediction on the collection distribution, whereas planning or new goals may require predictions at rarely represented states and actions.Broad counterfactual coverage can coexist with low on-distribution headroom; learned policies and physical robots require measurements under their own prediction and control conditions.Source: §4 · p. 4
5 Conclusion
Dynamics randomization alone does not establish that a benchmark tests context inference. The conclusion argues that headroom measurements help distinguish limited opportunity for context inference from weaknesses of the methods being compared.
5 Conclusion
- Dynamics randomization alone does not establish that a benchmark tests context inference.Source: §5 · p. 5
- Similar predictive headroom can accompany very different kernel variation, and observed state can retain evidence about hidden dynamics.Source: §5 · p. 5
- Reporting headroom would help distinguish limited opportunity for context inference from weaknesses of the compared methods.Source: §5 · p. 5
A Implementation and estimation details
The implementation details specify discrete MiniGrid and continuous attractor settings, along with procedures for estimating visited headroom and context posteriors. A reset-stratum check compares fitted and known-prior estimates but does not validate estimates across all visited states.
A Implementation and estimation details
- The MiniGrid setup uses an Empty-8x8 layout, logged episodes beginning at (1, 1, 0), goal termination or 256-step truncation, and 1008 fixed-reference state–action queries.Wind drift can be blocked by walls; Resistance sometimes leaves position and direction unchanged, and continuous contexts are integrated by quadrature without binning.Source: A Implementation and estimation details · p. 6
- Visited MiniGrid headroom groups known kernels by logged state–action cell and fits context mixtures using other episodes with a uniform pseudo-row.The score integrates KL over next-state outcomes; episode-level bootstrap resampling recomputes mixtures, while the point estimate weights pooled transitions equally.Source: A Implementation and estimation details · p. 6
- The attractor setup uses four uniformly sampled target contexts, independent uniform actions, noisy motion, and damped boundary reflections.The interior kernel is evaluated at fixed and matched references by coordinate-wise quadrature; visited scores use reflected-density preimages and their Jacobians.Source: A Implementation and estimation details · p. 6
- The attractor context posterior is fitted with episode-level folds and nearest-neighbor frequencies from standardized position and velocity, with one Laplace pseudo-count per context.Bootstrap intervals condition on fitted predictions and do not capture full estimation uncertainty; even known component kernels do not make the visitation-dependent posterior exact.Source: A Implementation and estimation details · p. 6
- After a zero-velocity position reset, the fitted mean log-density contrast is 1.0110 nats versus 0.9961 using the known prior at interval ten, and 1.0093 versus 1.0071 at interval two.The check applies to the reset stratum, not all visited states; Figure 2 contrasts four context-dependent force fields with the prior-averaged force.Source: A Implementation and estimation details · p. 6, Figure 2 · p. 6