PaperlayerView exact revision on arXiv

Source-linked AI summary

StarWM: Self-Supervised Trained Attention Routing for Robust World Models

Zeqiang Zhang, Fabian Wurzberger, Maximilian Otte, Daniel Schmid, Sebastian Gottwald, Arne Peter Raulf, Daniel Alexander Braun

arXiv:2609.30667v1cs.CVcs.LG

TL;DR

Visual world models must balance pixel-level grounding against filtering irrelevant content without losing useful information. STARWM routes self-supervised attention to decide where reconstruction applies, and its variants show robust returns across both video-distractor settings.

  • Problem

    Reconstruction-based world models can emphasize image area rather than dynamical relevance, while reconstruction-free objectives may lose relevant information and shared-latent combinations require balancing conflicting gradients.

    Source: §1 · pp. 1–2

  • Method

    STARWM uses self-supervised cross-attention to route relevant content into a dynamics stream and a separate decoder to reconstruct attended and residual content.

    Source: §1 · p. 2, §3 · p. 3, §3.1 · p. 3

  • Results

    Across six tasks, STARWM and STARWM-R are the only methods with high returns under both VH and VHR, and STARWM-R has the highest IQM and lowest optimality gap on the combined pool.

    Source: §4.2 · p. 8

  • Takeaways & Limitations

    On NoisyShape, STARWM achieves near-perfect retention of relevant attributes through imagination, while baselines degrade.

    Source: §4.1 · p. 6

    Experiments focus on single-agent continuous control from images; multiple agents, non-spatial inputs, and real-world domain shifts remain untested.

    Source: §6 · p. 9

Abstract

from arXiv · show

A robust world model must strike the balance between faithfully capturing environmental dynamics and abstracting away from irrelevant content. While reconstruction-based world models ensure faithful supervision, they misallocate representational capacity by pixel area rather than dynamics relevance for visual tasks, which can cause task-irrelevant content to dominate the learned representation. Alternatively, reconstruction-free methods avoid this bias but risk discarding possibly relevant information. We propose StarWM, which uses a cross-attention module trained on self-supervised dynamics to decide where reconstruction applies. A dual-stream decoder then restricts reconstruction to the attended regions, with stop-gradient barriers preventing interference between the two objectives. These components allows reconstruction to supervise the visual content of attended regions without contaminating the latent with non-predictive information. On DeepMind Control with dynamic video backgrounds, default (reward-free) StarWM achieves the strongest performance under random-frame distractors and substantially outperforms reconstruction-based baselines under sequential video. In addition, its reward-augmented variant matches or exceeds reconstruction-free methods on sequential video, achieving the highest overall return across all distractor regimes. Mechanistic probing confirms StarWM preserves state attributes with near-perfect fidelity through long-horizon imagination while systematically discarding distractors.

1 INTRODUCTION

World-model learning must capture environmental dynamics while filtering task-irrelevant visual content, but reconstruction and reconstruction-free objectives each have limitations. STARWM separates their roles through attention routing and evaluates that design with controlled diagnostics and distraction-robustness tests.

1 INTRODUCTION

  • Reconstruction-based models can favor large image regions over dynamical relevance, while reconstruction-free methods risk losing relevant information.Self-supervised signals can focus on action-conditional or predictable regions but do not provide pixel-level grounding.Source: §1 · p. 1
  • Combining reconstruction and auxiliary signals in a shared latent creates gradient tension: one preserves all content while the other selects specific content, requiring careful balancing.Source: §1 · pp. 1–2
  • STARWM addresses this conflict through architectural separation on the encoder’s spatial feature map, isolating supervision sources at the parameter level through attention routing.Source: §1 · p. 2
  • Its dual-stream architecture uses self-supervised attention routing to separate where to attend from what to extract.The cross-attention mask is supervised by auxiliary objectives, while a residual stream absorbs irrelevant content.Source: §1 · p. 2
  • NoisyShape provides controllable dynamics, randomized video distractors, and ground-truth labels for probing attribute encoding and long-horizon imagination.Source: §1 · p. 2
  • Evaluations report that STARWM outperforms baselines on average across distractor setups and retains relevant attributes while discarding irrelevant ones.Probe analysis reports near-perfect temporal coherence over long horizons.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries frame world models as compact latent simulators for model-based reinforcement learning and describe reconstruction-based latent dynamics and policy learning. DreamerV3 supplies the representative RSSM setup.

2 PRELIMINARIES

  • World models aim to simulate environmental dynamics in a compact latent space, enabling sample-efficient policy optimization.They should maintain long-horizon temporal coherence, resist task-irrelevant distractors, and align representations with the objective when available.Source: §2 · p. 2
  • Reconstruction-based world models train an encoder–decoder to reconstruct observations while predicting forward in latent space.DreamerV3 is presented as a representative RSSM-based model.Source: §2 · p. 2
  • The RSSM updates deterministic recurrent state h_t from prior state, latent, and action; it infers posterior z_t from h_t and observation o_t, and predicts prior ẑ_t from h_t.Source: §2 · p. 2
  • DreamerV3 learns its policy with an actor-critic agent on RSSM-imagined latent trajectories, with the actor maximizing estimated critic return over rollouts.Source: §2 · p. 2

3 METHOD: STARWM

STARWM modifies the encoder and decoder while retaining the existing latent dynamics model and policy-learning procedure. Self-supervised attention selects what enters the dynamics stream, while a separate reconstruction pathway handles visual content and irrelevant background.

3 METHOD: STARWM

  • STARWM changes only the encoder and decoder, splitting spatial features into a dynamics stream and a residual reconstruction stream while separating attention guidance from reconstruction.The latent dynamics model and policy-learning procedure remain unchanged; stop-gradient barriers keep the two objectives from interfering.Source: §3 · pp. 2–3, Figure 1 · p. 3

3.1 CROSS-ATTENTION SPATIAL ROUTING

  • Learnable queries attend over spatial features to form per-entity probability masks, routing attended features to the RSSM and residual features to an auxiliary VAE.The encoder adds positional encodings to keys and normalized coordinates to values.Source: §3.1 · p. 3

3.2 SELF-SUPERVISED LOCATION/CONTENT SEPARATION

  • Inverse dynamics guides attention toward regions whose visual changes are caused by the agent’s actions, while temporal contrastive learning guides it toward predictable content.The default signals are complementary; adding reward prediction yields STARWM-R, which better tracks passively moving task-relevant entities.Source: §3.2 · p. 4
  • Stop-gradient copies keep attention parameters trained purely by self-supervised signals, while the encoder receives gradients from self-supervision, RSSM losses, and reconstruction.The decoder uses the fixed attention map as its routing mask.Source: §3.2 · p. 4, §3.3 · p. 4

3.3 DUAL-STREAM DECODER

  • The dynamics decoder paints predicted dynamic features into attended positions, making the RSSM retain rich attribute information.The residual stream uses a stochastic bottleneck to absorb irrelevant background details for reconstruction; its latent is excluded from RSSM imagination.Source: §3.3 · p. 4
  • An aggregate coverage mask blends the two streams at the feature level before a shared transposed convolution reconstructs the observation.Feature-level composition prevents the soft mask from implicitly encoding shape through its spatial extent.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments test whether STARWM separates relevant from irrelevant content, transfers that separation to policy robustness under video distractors, and depends on self-supervised guidance and gradient isolation.

4.1 LATENT SEPARATION: PROBE ANALYSIS ON NOISYSHAPE

  • STARWM retains objective-relevant color, shape, and size near-perfectly through imagination, while every baseline degrades and each tested ablation loses at least one attribute.DenoisedMDP is not evaluated because its decomposition requires reward signals.Source: §4.1 · p. 6, Figure 3 · p. 6
  • STARWM discards randomly resampled position, but retains position through 15 imagined steps when the object follows a predictable trajectory.Ablations without the dual-stream decoder retain random position, implicating that component in filtering unpredictable position.Source: §4.1 · p. 6
  • Cross-sample swaps reveal an emergent factorization: the residual stream carries background, the dynamics stream entity content, and the attention map location, without explicit disentanglement.Figure 4 shows the corresponding isolated reconstructions and controlled component swaps.Source: §4.1 · p. 6, Figure 4 · p. 7

4.2 POLICY PERFORMANCE: DMC UNDER VIDEO DISTRACTORS

  • Across clean and distracting DMC scenes, four attention queries consistently localize the agent rather than drifting to visually dominant backgrounds.Routing also remains stable across matched textures, low-contrast agents, and photorealistic scenes.Source: §4.2 · p. 7, Figure 6 · p. 7
  • Across six tasks, STARWM and STARWM-R alone perform strongly under both video regimes, and STARWM-R leads pooled IQM and optimality gap; default STARWM leads 5 of 6 VHR tasks.Default STARWM remains competitive on high-DOF locomotion under sequential distractors, while STARWM-R improves tasks where default STARWM underperforms.Source: §4.2 · pp. 7–8, Figure 5 · p. 7

4.3 COMPONENT ANALYSIS

  • On DMC with video distractors, removing gradient isolation or self-supervised guidance collapses performance, although reconstruction gradients can guide attention on NoisyShape when shape aligns with action.The two design choices jointly restrict the attention signal to a dynamics-grounded source.Source: §4.3 · p. 8, Figure 7 · p. 8
  • Inverse dynamics alone gives near-optimal NoisyShape routing, whereas contrastive learning alone fails to localize the foreground; on DMC, contrastive guidance helps under VHR but is partly co-opted under VH.Inverse dynamics targets regions whose visual changes are explained by action; temporal similarity alone cannot distinguish the agent from other smooth regions.Source: §4.3 · p. 8

5 RELATED WORK

Prior approaches address distraction through reconstruction-free objectives, shared-latent decomposition, spatial localization, or object-centric attention. STARWM instead derives spatial routing from environment dynamics while preserving reconstruction for attended content.

World models and distraction-robust representation learning

  • Reconstruction-based methods provide dense grounding but favor dominant pixel areas, while reconstruction-free methods avoid that bias at the cost of pixel-level grounding; STARWM combines the two by routing reconstruction away from distractors.The reconstruction-based examples include the Dreamer family (Hafner et al., 2020; 2021; 2025), and reconstruction-free examples include TD-MPC (Hansen et al., 2022; 2024).Source: §5 · p. 8
  • Shared-latent decomposition methods TIA, DenoisedMDP, and Iso-Dream face gradient tension between reconstruction and auxiliary objectives.Other spatial methods use keypoints, simulator masks, or SAM-based models, while SMG (Zhang et al., 2024) requires task-specific augmentations and is model-free.Source: §5 · p. 9
  • STARWM derives spatial routing endogenously from environment dynamics, needing neither external priors nor task-specific augmentations while retaining dense reconstruction for attended content.Its attention pathway separates where to route from what to extract, with reconstruction gradients blocked from attention.Source: §5 · p. 9

Self-supervised methods for RL

  • Unlike prior whole-image self-supervised RL representations that drop spatial layout, STARWM uses dynamics signals to train spatial attention and keeps reconstruction for attended visual details.The routing mechanism can also use task-specific signals such as reward prediction.Source: §5 · p. 9

6 CONCLUSION

STARWM uses learned attention to guide reconstruction toward relevant regions, and its reward-free setup outperforms reconstruction-based baselines under visual distractions. The extended discussion distinguishes its spatial, modular routing from prior approaches that target different kinds of observation content.

6 CONCLUSION

  • STARWM’s learned attention guides reconstruction toward relevant regions; its reward-free setup outperforms reconstruction-based baselines under visual distractions.STARWM-R can add reward prediction to better track passively moving task-relevant entities.Source: §6 · p. 9

A.1 TAXONOMY OF OBSERVATION COMPONENTS

  • The observation taxonomy separates components by predictability, controllability, and task relevance.The discussion contrasts these axes with prior approaches targeting different subsets, including reward-based methods and latent-level decompositions.Source: §A.1 · p. 14

A EXTENDED RELATED WORK

  • Related approaches include HRSSM, SeeX, and ExoRL, which respectively use reward-aware global features, whole-image branches, or an action-independent state-factor framework.The text notes that these approaches differ from STARWM’s spatial routing and visual-control focus.Source: §A.1 · p. 14

A.1 TAXONOMY OF OBSERVATION COMPONENTS

  • STARWM routes at spatial resolution with cross-attention and can use differentiable live-path signals, including reward when available.Its default guidance combines inverse dynamics and temporal contrastive learning without reward.Source: §A.1 · p. 14

A.2 SELF-SUPERVISED SIGNALS AND THEIR LIMITATIONS

STARWM uses self-supervised signals to route attention while reconstruction separately supervises attended visual content. Its architecture intervenes before the RSSM, addressing upstream encoding and spatial-entanglement problems that latent noise handling alone cannot undo.

A.2 SELF-SUPERVISED SIGNALS AND THEIR LIMITATIONS

  • STARWM uses inverse dynamics to steer attention toward action-conditional regions and contrastive learning for complementary temporal guidance; reconstruction separately supervises attended content.Stop-gradient isolation prevents reconstruction gradients from corrupting attention parameters or vice versa.Source: §A.2 · p. 15

A.3 DREAMERV3’S BUILT-IN NOISE HANDLING

  • DreamerV3’s stochastic latent mechanisms act after image compression, leaving upstream encoding pressure, spatial entanglement, and contaminated imagination rollouts unresolved.The text describes background contamination in imagined trajectories as degrading policy quality.Source: §A.3 · p. 15
  • STARWM intervenes before the RSSM, routing attended entity features into dynamics and reconstructing background through a separate VAE pathway discarded during imagination.This leaves the RSSM’s noise handling focused on endogenous-entity dynamics.Source: §A.3 · pp. 15–16

A.4 COMPARISON WITH OBJECT-CENTRIC AND SLOT-BASED METHODS

  • Unlike reconstruction-supervised attention, STARWM trains attention with self-supervised signals and blocks reconstruction gradients from reaching attention parameters.Its authors contrast this with Slot Attention’s iterative refinement, which couples attention weights to the decoder.Source: §A.4 · p. 16

B.1 ALGORITHM

  • The architecture splits spatial features into a dynamics stream and a residual reconstruction stream, with attention routing trained by self-supervision and isolated from reconstruction gradients.The implementation intercepts an 8×8 feature map and applies cross-attention; Table 1 lists the model’s key hyperparameters.Source: §B.1 · pp. 16, 18, Figure 8 · p. 17, §B.2 · p. 19

B.3 IMPLEMENTATION OF BASELINE ALGORITHMS AND COMPUTE RESOURCES

The evaluation uses common visual-distractor conditions and baseline implementations, while the Shape environments provide a controlled setting for testing representation under noise and changing attributes.

B.3 IMPLEMENTATION OF BASELINE ALGORITHMS AND COMPUTE RESOURCES

  • Baselines use recommended hyperparameters and a unified Video Hard distractor wrapper; runs use three seeds, with 3M steps for VH and 1M for VHR.STARWM runs comparably to reconstruction-based world models but more slowly than R2-Dreamer, which omits the image decoder.Source: §B.3 · pp. 19–20, Table 2 · p. 20

C.1 SHAPE ENVIRONMENTS

  • Shape environments vary object attributes and background conditions to test how models encode relevant dynamics amid pixel-area imbalance.The object occupies 2%–20% of pixels, while the background covers 80%–98%; reward is uniform random noise, so it cannot guide representation learning.Source: §C.1 · pp. 20–22

C.2 PROBE ANALYSIS METHODOLOGY FOR NOISYSHAPE

The appendix specifies how latent probes and distractor evaluations are run, and points to full results tables for NoisyShape and DeepMind Control. The supplied passages describe evaluation scope and table contents but do not provide result values.

C.2 PROBE ANALYSIS METHODOLOGY FOR NOISYSHAPE

  • The NoisyShape probe study uses cached random-policy trajectories, frozen RSSM features, three-layer MLP probes, and open-loop imagination for 15 steps.Evaluation reports classification accuracy or position-regression MSE.Source: §C.2 · p. 22

C.3 DEEPMIND CONTROL SUITE ENVIRONMENTS

  • The DeepMind Control evaluation covers six continuous-control tasks under Clean, sequential-video VH, and random-frame VHR settings.VH uses 3M environment steps, while Clean and VHR use 1M steps.Source: §C.3 · p. 23

D FULL RESULTS & AGGREGATE METRICS

  • The appendix presents full numerical results, including standard deviations across seeds, as support for the main figures and tables.The supplied passage describes this scope but gives no aggregate values.Source: D FULL RESULTS & AGGREGATE METRICS · p. 23

D.1 FULL RESULTS IN NOISYSHAPE ENVIRONMENT

  • Table 4 reports NoisyShape latent-probe accuracy at encoding and after 15 imagination steps, with standard errors across three seeds.Its caption says each ablation isolates one STARWM component.Source: §D.1 · p. 23, Table 4 · p. 24

D.2.1 PER-TASK RESULTS

  • Table 5 reports DMC returns by task and setting, with standard errors across three seeds and the best result bolded for each task-setting pair.VH uses 3M steps and VHR uses 1M steps; the accompanying passage distinguishes two DreamerV3 Clean references.Source: Table 5 · p. 24, §D.2.1 · p. 24

D.2.2 AGGREGATE STATISTICAL ANALYSIS

Across both video-distractor conditions, STARWM variants are strongest under random-frame distractors and STARWM-R leads pooled performance, while R2-Dreamer leads under sequential video. Reward prediction improves results on tasks with passive, reward-critical objects, and STARWM-R localizes Cartpole’s cart and pole where default STARWM’s attention drifts.

E.1 REWARD AS ADDITIONAL ATTENTION ROUTING SIGNAL

  • On Ball-in-cup Catch VHR, STARWM-R improves from 98 to 303 versus R2-Dreamer’s 239; on Cartpole Swingup VH, it improves from 308 to 756 versus 652.The passages attribute these improvements to reward guidance for passive, reward-critical objects that inverse dynamics supervises weakly.Source: §E.1 · p. 25

D.2.2 AGGREGATE STATISTICAL ANALYSIS

  • On the VH+VHR pool, STARWM-R leads all four aggregate point estimates and has the lowest optimality gap, while STARWM ranks second on IQM, Mean, and Optimality Gap.STARWM-R also stochastically dominates every baseline in the performance profile.Source: §D.2.2 · p. 25, Figure 10 · p. 26
  • Under sequential video (VH), R2-Dreamer has the highest aggregate scores across all four point estimates; STARWM-R ranks second and default STARWM third.P(STARWM-R > R2-Dreamer) = 0.46, while P(STARWM > R2-Dreamer) = 0.22.Source: §D.2.2 · p. 25, Figure 10 · p. 26
  • Under random video (VHR), STARWM and STARWM-R lead every aggregate metric, and both stochastically dominate the other methods across the full score range.P(STARWM-R > B) = 1 for every non-STARWM baseline, and P(STARWM > B) ≥0.85 for every baseline except STARWM-R.Source: §D.2.2 · p. 25, Figure 10 · p. 26

E.1.1 CASE ANALYSIS: ATTENTION DRIFT ON CA R T P O L E SW I N G U P VH

  • On Cartpole Swingup VH, default STARWM attends near track endpoints with little query specialization, whereas STARWM-R localizes the cart and pole in distinct query regions.The paper identifies this as the task where STARWM most strongly underperforms STARWM-R.Source: Figure 11 · p. 27, §E.1.1 · p. 27

E.2 GRADIENT ISOLATION ANALYSIS

The gradient-isolation analysis finds that self-supervised guidance and blocking reconstruction gradients are jointly needed to route attention under video distractors, though their alignment makes isolation less consequential in NoisyShape. The accompanying ablations show that inverse dynamics and contrastive learning contribute differently by setting, while four queries provide a balanced default.

E.2 GRADIENT ISOLATION ANALYSIS

  • On NoisyShape, probe accuracy remains near optimal across gradient-isolation configurations because reconstruction and self-supervised signals favor the same foreground target.The passages contrast this alignment with video backgrounds, where those signals can disagree.Source: §E.2 · p. 28, Figure 12 · p. 28
  • Under VHR distractors, removing detach B lets reconstruction gradients pull attention toward the background, while removing self-supervised guidance leaves RSSM gradients unable to distinguish foreground from coherent background.The two failure modes support the joint need for detach B and self-supervised guidance.Source: §E.2 · pp. 28–29
  • Without detaching, simultaneous self-supervised, RSSM, and reconstruction gradients produce nearly uniform attention; under video distractors, RSSM-only guidance lets just one of four queries reliably localize the agent.Source: §E.2 · p. 29

E.3 HYPERPARAMETER SENSITIVITY

  • Across the reported hyperparameter variations, Walker Walk VHR performance remains between 787 and 925; removing βbg annealing yields 918.Higher λinv and βfinal_bg slow convergence within the 1M-step budget, but the configurations remain functional.Source: §E.3 · p. 31, Table 9 · p. 31

E.3.1 DECOMPOSITION OF SELF-SUPERVISED OBJECTIVES (INV VS. CTR)

  • On NoisyShape, inverse-dynamics-only matches full STARWM on attribute probes, whereas contrastive-only guidance degrades shape and size retention through imagination.The passages characterize inverse dynamics as direct action-linked supervision and contrastive learning as indirect temporal-consistency supervision.Source: §E.3.1 · p. 31, Figure 15 · p. 31
  • On DMC, contrastive learning substantially benefits VHR, where the agent is the only reliably coherent feature, but is neutral under Clean and VH, where sequential video backgrounds can share its temporal cue.Under VH, inverse-only is slightly better; the passage attributes this to contrastive gradients splitting between agent and background.Source: §E.3.1 · pp. 31–32, Figure 16 · p. 32
  • Self-supervised observation dynamics alone cannot distinguish a predictable task-relevant landmark from a predictable distractor, so contrastive weight encodes a designer-held prior.The passage says resolving that ambiguity requires information beyond observation dynamics, such as reward.Source: §E.3.1 · p. 32

E.3.2 QUERY COUNT N.

  • On DMC VH at 3M steps (episode return, mean ± SE, 3 seeds), N=1 underperforms across tasks; N=2 and N=4 achieve strong returns, while N=8 improves Cartpole Swingup but adds Walker Walk variance; N=4 is the balanced default.Source: §E.3.2 · p. 33, Table 10 · p. 33

E.4 SELECTIVE SPATIAL ENCODING ANALYSIS

Predictable motion changes what STARWM retains: it encodes position through long-horizon imagination when position is predictable, while its background-identity probe remains at chance. Together, the two NoisyShape variants show selective encoding rather than indiscriminate spatial-information loss.

E.4 SELECTIVE SPATIAL ENCODING ANALYSIS

  • With predictable motion, STARWM’s position MSE falls to ∼0.0008 at encoding and remains well below chance after 15 imagination steps.The passage reports a two-orders-of-magnitude drop relative to random position; the slight H15 gap to DreamerV3 is attributed to the N = 4 attention bottleneck.Source: §E.4 · p. 33, Figure 18 · p. 34, Table 11 · p. 34
  • DreamerV3 classifies background identity at 82.4% over 20 classes, while STARWM is at 5.0%, exactly chance; this holds regardless of position predictability.The two position settings together show that STARWM discards position when irrelevant and encodes it when relevant, while filtering background identity.Source: §E.4 · pp. 33–34, Figure 18 · p. 34, Table 11 · p. 34

E.5 PROGRESSIVE DISTRACTION ANALYSIS ACROSS NOISYSHAPE VARIANTS

Across NoisyShape variants, STARWM retains relevant attributes as distraction complexity grows; extensions test small objects, multiple objects, and reduced visual contrast. Its performance and probe results remain strong in several of these settings, while spatial granularity and task conditions bound what it retains.

E.5 PROGRESSIVE DISTRACTION ANALYSIS ACROSS NOISYSHAPE VARIANTS

  • DreamerV3’s shape H15 falls to 29.0% in NoisyShape, while STARWM retains ≥99.6% across distraction levels.DreamerV3 declines progressively as irrelevant complexity increases; STARWM also retains size at ≥99.99% and color at 100.0% through H15.Source: §E.5 · p. 34, Figure 19 · p. 35, Table 12 · p. 35

F.1 SPATIAL GRANULARITY & SMALL OBJECTS

  • With agents at one-quarter pixel area, all methods degrade under distractors, but STARWM still leads both tasks on VHR; 8×8 routing can miss very small objects.On small-object Walker Walk, STARWM’s VHR return falls from approximately 850 to 350; the proposed 16×16 tokens trade greater compute for finer routing, and reward-augmented recovery remains unverified.Source: §F.1 · p. 36, Figure 20 · p. 36

F.2 MULTI-OBJECT EXTENSION: NOISYTWOSHAPE

  • Four unsupervised attention queries cover both NoisyTwoShape objects, and STARWM retains both objects’ attributes through 15-step imagination better than DreamerV3.STARWM preserves color and shape at ≥99% at H0; the small-object shape probe declines from approximately 99% to approximately 85% at H15, while DreamerV3’s small-shape probe is at chance.Source: Figure 22 · p. 37, §F.2 · p. 37, Figure 21 · p. 37
  • STARWM’s position MSE matches chance for both objects, whereas DreamerV3 encodes both positions precisely.The objects’ chance MSE values differ because their position-sampling margins depend on object size.Source: §F.2 · p. 37, Figure 21 · p. 37

F.3 TEXTURE, LOW CONTRAST, AND PHOTOREALISTIC SCENES

  • Across the texture ladder, STARWM retains shape and texture better than DreamerV3 at H15 while preserving object size and filtering irrelevant position.At L3, STARWM’s texture accuracy is 0.929 versus DreamerV3’s 0.252; STARWM’s size accuracy is 0.698 versus 0.395, while position MSE is near chance.Source: Figure 23 · p. 38, Table 13 · p. 38, §F.3 · pp. 38–39
  • On desaturated Walker Walk, STARWM scores 890 under VHR versus DreamerV3 394 and R2-Dreamer 118; VH returns are similar.VH returns are STARWM 893, DreamerV3 799, and R2-Dreamer 895.Source: Figure 25 · p. 39, §F.3 · p. 39, Table 14 · p. 39
  • On photorealistic CALVIN, attention concentrates on the gripper and active block above shuffled nulls, but not static blocks, consistent with controllability-based routing.Gripper and active-block concentration are 8.69× and 6.09× versus nulls of 2.83× and 2.56×; static blocks are 1.52× versus a 2.03× null.Source: Figure 26 · p. 40, §F.3 · p. 40, Table 15 · p. 40

F.4 PASSIVE AND NON-ACTION-DRIVEN TASK-RELEVANT OBJECTS

Default self-supervised routing can retain passive task-relevant features under some background conditions, while coherent video can reduce position retention. For delayed action effects, matching inverse-dynamics supervision to the actual transition restores representation metrics.

F.4 PASSIVE AND NON-ACTION-DRIVEN TASK-RELEVANT OBJECTS

  • Passive object B identity remains fully preserved through 15 rollout steps in both backgrounds, but position R2 falls from ≈0.8 on random frames to ≤0.13 at sequential-video H0.Position recovery drops below zero after imagination under sequential video.Source: Figure 28 · p. 41, §F.4 · p. 41, Table 16 · p. 41
  • Temporal contrastive learning retains B’s position when randomized backgrounds make it a cross-step episode identifier; coherent video can supply that identifier instead.InfoNCE negatives typically come from different episodes, making B’s episode-varying attributes useful unless the background already distinguishes episodes.Source: §F.4 · p. 42
  • On Reacher, STARWM-R raises VH return from STARWM’s 391 to 789; under VHR, STARWM and STARWM-R score 690 and 657, while R2-Dreamer scores 66.The static target provides neither inverse-dynamics gradients nor distinct contrastive signals for standard STARWM.Source: Figure 29 · p. 42, §F.4 · p. 42, Table 17 · p. 42
  • Default routing retains episode-static, cross-episode-distinct features unless coherent video already identifies the episode; reward-conditioned routing adds missing task signals when self-supervision does not isolate passive elements.Source: §F.4 · p. 42, Table 18 · p. 43

F.5 DELAYED ACTION EFFECTS

  • The test compares adjacent (t, t+1) and shifted (t+k, t+k+1) supervision windows for actions whose effects appear after delays k=1, 2, or 4.The shifted window captures the frame pair directly modified by the action.Source: §F.5 · pp. 42–43
  • Under delayed action effects, mismatched supervision disperses attention (object mass 0.35–0.44 vs. 0.85 without delay); aligning the window to the actual action outcome restores mass to 0.73–0.82 and shape and size accuracy above 0.99 across all delays.Restored attention concentration is 11–13× uniform and matches the undelayed baseline.Source: §F.5 · p. 43, Table 19 · p. 43

Related papers

Loading 2609.30667v1…