Beyond a single latent space: a dual-latent world model for long-horizon planning

1 INTRODUCTION

• Long-horizon planning needs both predictive consistency and goal distances that preserve meaningful comparisons beyond local neighborhoods.Locally accurate transitions alone do not ensure consistent rollouts or informative goal distances.Source: §1 · pp. 1–2
• A shared state geometry must serve local control's sensitivity to small changes and long-range planning's need to compare progress toward distant goals.Local prediction objectives do not directly enforce long-range discrimination.Source: §1 · p. 2
• Dual-WM separates primitive-action dynamics and representations from distinct high-level dynamics and representations for learned macro-actions.At inference, it plans high-level latent subgoals, refines them into primitive actions, then switches to direct low-level planning for precise convergence.Source: §1 · pp. 2–3
• LORE supervises self-generated rollouts at both temporal scales with separately parameterized horizon weights.The weights extend predictive supervision beyond one-step transitions.Source: §1 · p. 3
• The evaluation covers five goal-conditioned visual planning tasks spanning continuous control and discrete-action, game-like planning.Comparisons include flat and hierarchical world models, with diagnostics of learned representations.Source: §1 · p. 3

Latent prediction and control

• Latent dynamics support planning from pixels and imagined trajectories, while LoRe retains recursive dynamics and supervises self-generated rollouts at both temporal scales.Other approaches change how extended futures are predicted (Du et al., 2026; Gao & Xu, 2026; Huo et al., 2026).Source: §2 · p. 3

Geometry and temporal abstraction

• Planning-oriented objectives reshape representations through action alignment, physical-state distances, reachability, or temporal ordering (Wang et al., 2026a; Hu et al., 2026; Li et al., 2026; Bai & Xiong, 2026).SD-JEPA separates progression and content within an embedding (Thil et al., 2026).Source: §2 · p. 3
• Unlike HWM and Hi-LeWM, Dual-WM uses separate state spaces and predictors across scales, with a learned interface for macro-action subgoals and primitive-action control (Zhang et al., 2026a; Caselli et al., 2026).LoRe uses scale-specific exponential decay motivated by recursive error propagation, unlike HWM's uniform rollout weights (Zhang et al., 2026a).Source: §2 · p. 3

3 METHOD

• Dual-WM separates local execution and long-range planning into different state spaces, coupled through a learned projection used during subgoal refinement.Figures 2 and 3 depict training and three-stage planning.Source: §3 · p. 3

3.1 PROBLEM SETTING AND NOTATION

• The model learns from offline observation–action trajectories and plans from current and goal observations without reward or ground-truth state-distance targets for its predictive objectives.Each low-level input covers f environment steps; a high-level transition spans k low-level steps, or kf environment steps.Source: §3.1 · pp. 3–4

3.2 DUAL-LATENT DYNAMICS AND MACRO-ACTIONS

• Separate low- and high-level encoders and predictors model fine-scale transitions and temporally extended transitions, while an action encoder represents macro-actions.MAPS regularizes the macro-action posterior toward a standard Gaussian prior to support sampling and optimization during planning.Source: Figure 2 · p. 4, §3.2 · p. 4

3.3 HORIZON-WEIGHTED SELF-GENERATED ROLLOUTS

• LoRe trains each dynamics model on recursive predictions using separately parameterized horizon weights for the two temporal scales.Finite exponential decay tempers amplified distant errors while retaining supervision at included horizons; uniform weighting is recovered at zero decay.Source: §3.3 · p. 5

3.4 COARSE-TO-FINE PLANNING ACROSS THE TWO SPACES

• Planning first optimizes macro-actions for high-level latent subgoals, reducing route-search depth before low-level refinement.A route covering k high-level steps' worth of low-level model steps needs only the high-level transition count during route search.Source: §3.4 · p. 5
• During refinement, low-level dynamics produce latent windows compared with high-level subgoals through EH; the planner later switches to direct low-level goal convergence.High-level matching can miss completion-relevant details, for which direct low-level goal matching retains sensitivity.Source: §3.4 · p. 6

4.2 Q1: DOES DUAL-WM IMPROVE LONG-HORIZON PLANNING?

• 84.4% versus 75.9% at offset 50 and 69.5% versus 61.4% at offset 100; Dual-WM also beats LeWM on all five tasks at 100.Mean success improvement over LeWM at offset 100 is 30.8 percentage points.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 Q2: DO LATENT GOAL DISTANCES DISTINGUISH LONG-RANGE PROGRESS?

• LeWM’s normalized distance approaches the isotropic-Gaussian reference after short physical ranges, while learned spaces preserve more long-range ordering; Reacher high-level correlation is 0.562 versus 0.318 low-level and 0.190 LeWM.On PushT, goal discrimination improves most clearly, while orientation correlation is statistically indistinguishable from LeWM.Source: §4.3 · p. 8

4.4 Q3: DOES ROLLOUT TRAINING PRESERVE PHYSICAL-STATE INFORMATION?

• Rollout-trained checkpoints have lower point estimates at every tested horizon for all four probed physical quantities, clearest on Reacher configuration and fingertip position and PushT block position.PushT orientation estimates have wider uncertainty.Source: §4.4 · p. 8

4.5 Q4: WHICH DESIGN CHOICES IMPROVE PLANNING SUCCESS?

• At offset 100, Dual-WM achieves 93% success versus 41% for Window-Concat using the same input window and low-level checkpoint.In the farthest distance bin, Dual-WM-H reaches 84.9% goal-ordering accuracy versus 51.0% for Window-Concat.Source: §4.5 · p. 8, Figure 4 · p. 9
• At offset 100, dual-scale LoRe achieves 92% success versus 48% with one-step supervision, exceeding the stronger single-scale setting by 13 percentage points.Both model levels and hierarchical planning remain active; only which levels receive LoRe changes.Source: §4.5 · p. 8, Figure 5 · p. 9
• Under low-level-only planning, offset-100 success rises from 32% at N = 1 to 69% at N = 5, then falls to 37% at N = 10; at N = 5, exponential weighting reaches 69%.Linear and uniform weighting reach 62% and 54%, respectively, at N = 5.Source: §4.5 · p. 9
• At offset 100, stochastic macro-action encoding reaches 93.5% success with MAPS (β = 10−2), versus 81% without MAPS, despite higher latent prediction error.Increasing β to 5 × 10−2 reduces success to 91.5%.Source: §4.5 · p. 9

• Dual-WM separates local execution and long-range planning into distinct state spaces and dynamics, with LoRe training recursive rollouts at both scales and MAPS shaping macro-actions for search.Source: §5 · p. 9
• Across five visual-control tasks with continuous and discrete actions, the design improves extended-offset goal reaching; diagnostics and ablations support complementary roles for goal discrimination, recursive consistency, and precise low-level convergence.Source: §5 · p. 9
• Evaluation is offline with fixed temporal abstraction, and gains depend on task and action-proposal mechanism.Source: §5 · p. 9

A LATENT DISTANCE CONCENTRATION: ANALYSIS

• At d = 192, the independent-Gaussian reference distance has mean approximately 19.5704 and standard deviation about 0.9993.The analysis uses this reference to evaluate distance contrast in learned representations.Source: A LATENT DISTANCE CONCENTRATION:… · p. 13

A.1 CANDIDATE RANKING UNDER PREDICTION ERROR

• A candidate-cost margin greater than the sum of both endpoint prediction-error bounds is sufficient to preserve their ranking.As the margin shrinks, tighter prediction-error bounds are needed to certify the same ordering; smaller or cancelling cost errors can also preserve it.Source: §A.1 · p. 14

B FINITE-HORIZON ROLLOUT WEIGHTING

• The finite-horizon weighting rationale analyzes fixed predictors over rollout horizons and the trade-off in supervision across those horizons.Source: B FINITE-HORIZON ROLLOUT WEIGHTING · p. 14

B.1 PROPAGATION ALONG A FIXED TRAJECTORY

• Along a fixed trajectory, a Lipschitz update and bounded residual yield a recursive error bound, conditional on the rollouts considered.The constants are not measured global model properties, and assuming no residual cancellation can make the bound conservative.Source: §B.1 · p. 14

B.2 A FINITE-HORIZON REFERENCE FOR WEIGHTING

• Normalized finite-horizon weights equalize squared error-envelope contributions, motivating the exponential weighting family used in the paper.The reference has an exponential adjacent-horizon decay only in the expanding regime; finite horizons need not reach that regime.Source: §B.2 · p. 15

B.3 DECAY AND THE BALANCE OF HORIZON SUPERVISION

• Increasing decay can reduce the envelope objective while suppressing multi-step supervision; as terminal weight shrinks, the limiting objective is one-step prediction.The two temporal scales use separate decay parameters because their spans and error-amplification rates can differ.Source: §B.3 · p. 15

C.1 LATENT WORLD MODELS FOR CONTROL

• Dual-WM reorganizes reconstruction-free, goal-conditioned latent planning so primitive transitions and distant subgoal evaluation need not use the same representation geometry.Source: §C.1 · pp. 15–16

C.2 PREDICTION BEYOND ONE-STEP TRAINING

• LORE trains recursive trajectories with geometrically decaying horizon weights at both primitive-action and macro-action scales, unlike cited uniform or increasing-weight objectives.Related approaches also change prediction interfaces to reduce dependence on long chains of local transitions.Source: §C.2 · p. 16

C.3 REPRESENTATION GEOMETRY FOR GOAL EVALUATION

• Prior geometry methods reshape representations or costs with alignment, reachability, or temporal objectives, whereas Dual-WM uses a dedicated high-level latent space without a distance target or reachability head.Long-range subgoals are evaluated in the high-level space, while final goal convergence uses the low-level space.Source: §C.3 · p. 16

C.4 TEMPORAL ABSTRACTION AND HIERARCHICAL PLANNING

• Unlike hierarchical precedents that retain a shared latent interface, Dual-WM maps predicted low-level windows into a distinct high-level state space and uses low-level distance for final approach.This lets coarse subgoal matching and fine control use different geometries with a common visual input.Source: §C.4 · p. 16

C.5 AMORTIZING TEST-TIME SEARCH

• Search-amortization methods alter candidate proposal or refinement, while Dual-WM changes the latent spaces and rollout supervision used to evaluate candidates.Source: §C.5 · p. 17

C.6 RELIABILITY AND DIAGNOSIS

• The paper frames distance-saturation and per-horizon rollout-error analyses as diagnostics of two planning failure modes, not only as architectural motivation.Source: §C.6 · p. 17

C.7 THEORY

• Appendix A provides a reference regime for distance-contrast loss, complementing the rollout-error analysis that motivates horizon weighting.Source: §C.7 · p. 17

D TRAINING AND PLANNING CONFIGURATIONS

• Each dataset is split 9:1 into training and test episodes, with prediction-loss weights λL = λH = 1 and MAPS coefficient β = 10−2 on all tasks.The high-level state-regularization weight is 0.30; low-level weights are task-specific.Source: D TRAINING AND PLANNING CONFIGURATIONS · pp. 17–18
• Table 3 reports task-specific training configurations, with paired entries for low and high levels and approximate environment-step totals.N and M count model transitions at their respective scales.Source: Table 3 · p. 18
• The from-scratch architecture uses a 12-layer ViT-Tiny low-level encoder, six-layer causal low-level predictor, and separate high-level encoder and predictor over low-level windows.The macro-action encoder parameterizes a diagonal-Gaussian posterior with du = 16.Source: D TRAINING AND PLANNING CONFIGURATIONS · p. 18
• The VFM-adaptor variant freezes DINOv2-S/14, trains an adaptor and low-level predictor with LoRe plus spatial-structure and semantic-consistency losses, then trains the high-level branch.The two supplementary losses are weighted 0.1 each.Source: D TRAINING AND PLANNING CONFIGURATIONS · p. 18
• Planning uses CEM with 300 low-level candidates and 100 high-level candidates per iteration; Sokoban-Long uses categorical low-level search, while other tasks use Gaussian search.Maximum execution budgets are 50, 75, 100, and 125 environment steps at offsets 25, 50, 75, and 100, respectively, wherever evaluated; planning horizons are task-specific.Source: D TRAINING AND PLANNING CONFIGURATIONS · p. 18, Table 4 · p. 18

E FULL PLANNING RESULTS

• The full planning tables report goal-reaching success across the five evaluation environments and goal offsets; reproduced evaluations use three planning seeds and matched task sets across methods.The figure identifies the environments as TwoRoom, Reacher, Sokoban-Long, PushT, and Cube-Single.Source: E FULL PLANNING RESULTS · p. 18, Figure 6 · p. 19, Table 5 · p. 19
• 98.50% at offset 100 makes the VFM-adaptor strongest on Reacher, but it underperforms the from-scratch model on PushT.The paper notes that pretrained-feature benefits are task-dependent; the main cross-task comparison uses the from-scratch variant.Source: E FULL PLANNING RESULTS · p. 19

E.1 ACTOR-GUIDED COMPARISON ON CUBE-SINGLE

• At offset 50, actor guidance improves INTACT but lowers Dual-WM success from 62.67% to 58.50%, so its benefit depends on the evaluation setting.The comparison separates action proposals from the world-model comparison; Dual-WM retains its dual-latent dynamics and hierarchical planner with or without actor guidance.Source: §E.1 · p. 19

E.2 SUCCESS–TIME TRADEOFF ON TWOROOM

• 90–98% success across the measured planning-budget range; at nearly equal planning times, Dual-WM reaches 90% at 92.4 ms versus 68% for low-level-only at 93.3 ms.At its largest tested budget (247.7 ms), low-level-only reaches 84%; LeWM remains at 40%, while HWM rises from 7% to 33%.Source: §E.2 · p. 21

F QUALITATIVE EXECUTION ACROSS FIVE TASKS

• In TwoRoom, Dual-WM passes through the doorway toward the goal in the other room, while LeWM remains on the starting side of the wall.The selected executions use matched initial states and goals with a common 125-environment-step budget; offset 100 separates the dataset frames by 100 steps.Source: F QUALITATIVE EXECUTION ACROSS FIVE… · p. 21, Figure 8 · p. 22
• On PushT and Cube-Single, Dual-WM changes the block or object configuration toward the goal, while LeWM leaves the block near its initial state or fails to move the object as required.In Cube-Single, LeWM changes the gripper pose without completing the required object movement.Source: Figure 9 · p. 22, Figure 11 · p. 22
• On Reacher, Dual-WM brings the arm toward the goal configuration, while LeWM moves it but retains a different configuration.Source: Figure 10 · p. 22
• In Sokoban-Long, Dual-WM navigates to a box and performs aligned pushes toward the goal, whereas LeWM does not complete the required box movement.The example is a supplemental held-out source pair outside the standard 200-task sample.Source: Figure 12 · p. 23
• The examples connect long-range goal structure to useful intermediate choices, while low-level representations support precise action refinement.Completing the sequences also requires action-conditioned predictions to remain informative across repeated planning updates.Source: F QUALITATIVE EXECUTION ACROSS FIVE… · p. 23

G ADDITIONAL REPRESENTATION DIAGNOSTICS

• Figure 13 plots dimension-normalized latent distance against A* path distance in TwoRoom and wrapped joint-angle distance in Reacher, with episode-bootstrap intervals and an isotropic-Gaussian reference.Only bins with at least 100 valid pairs are displayed; the task-relevant ranges are 0–150 px and 0–3 rad.Source: Figure 13 · p. 23
• Figure 14 compares A* reference fields with LeWM and Dual-WM-L/H latent-distance fields for selected TwoRoom queries, using shared scales and wall-safe, cross-validated smoothing.The nine-anchor set is geometry-only, and bandwidth selection does not use A* similarity or visual appearance.Source: Figure 14 · p. 24
• Figure 15 compares wrapped joint-angle distance with LeWM and Dual-WM latent fields; reported correlations use only 1,118 observed cells with at least ten samples.Sparse cells are interpolated and lightly smoothed for visualization.Source: Figure 15 · p. 24
• Figure 16 measures PushT goal geometry through physical-distance correlations and same-episode goal ordering on 128 held-out episodes, excluding exact physical ties.Dots show paired differences for Dual-WM-L and Dual-WM-H relative to LeWM, with episode-cluster bootstrap confidence intervals.Source: Figure 16 · p. 25
• Physical-state probes fit ridge regressors on predicted latents pooled across five horizons, selecting regularization on validation episodes and evaluating on disjoint test episodes.Probe fitting and evaluation use the same predicted-latent regime.Source: G ADDITIONAL REPRESENTATION DIAGNOSTICS · p. 25

H COMPONENT-STUDY PROTOCOLS AND SENSITIVITY RESULTS

• At offsets 50 and 100, Window-Concat achieves 69% and 41% success, compared with 100% and 93% for Dual-WM.Both use high-level planning followed by low-level refinement; Window-Concat uses the same latent window but direct concatenation instead of learned encoding.Source: H COMPONENT-STUDY PROTOCOLS AND… · p. 25
• For recursive low-level physical-state probes, the rollout-trained checkpoint has lower point estimates at all horizons, while PushT orientation has the widest uncertainty.A separate horizon-pooled ridge probe is fitted for each checkpoint and physical target, using disjoint validation and test episodes.Source: Figure 17 · p. 26
• In the farthest goal-distance bin, Dual-WM-H reaches 84.9% ordering accuracy, versus 51.0% for Window-Concat and 55.6% for Dual-WM-L.The diagnostic uses matched held-out state pairs and goals; Dual-WM-H and Window-Concat receive identical observation windows.Source: H COMPONENT-STUDY PROTOCOLS AND… · p. 26, Figure 18 · p. 27
• The LoRe scale comparison retains the hierarchical planner; the both-level setting applies recursive supervision at both scales while the other configurations retain one-step supervision at the inactive scale.These comparisons are distinct from the low-level-only planning variant; component-study means are based on three planning seeds and shown without uncertainty bars.Source: H COMPONENT-STUDY PROTOCOLS AND… · pp. 25–26
• At evaluation horizon 10, position-probe error is 5.7 px for N = 5, versus 12.0 px for N = 1 and 8.9 px for N = 10.The probe measures accessible position information after recursive prediction, not raw latent-vector discrepancy; the study uses low-level-only planning.Source: H COMPONENT-STUDY PROTOCOLS AND… · p. 27, Figure 19 · p. 27
• At N = 5, the linear rollout-weight profile is (5, 4, 3, 2, 1)/15; the weighting study uses low-level-only planning.The figure reports that profile differences grow as goal offset increases, without specifying a winning profile.Source: H COMPONENT-STUDY PROTOCOLS AND… · p. 27, Figure 20 · p. 28

I SOKOBAN-LONG: DISCRETE-ACTION IMPLEMENTATION

• Sokoban-Long encodes discrete actions as five-dimensional one-hot vectors, and its macro-action encoder maps k-action windows to continuous macro-actions.The low-level action planner uses categorical CEM with Gumbel-max sampling and refits its distribution to elite-sequence action frequencies.Source: I SOKOBAN-LONG: DISCRETE-ACTION… · pp. 28–29
• Sokoban-Long uses Gaussian CEM over continuous macro-actions at the high level and categorical search over discrete actions at the low level.High-level search evaluates macro-actions, while the low-level planner searches discrete actions to realize latent subgoals.Source: I SOKOBAN-LONG: DISCRETE-ACTION… · p. 29

J BASELINE DESCRIPTIONS AND EVALUATION PROTOCOLS

• Each planning seed uses 200 matched goal-reaching tasks per task and offset, with the same episode IDs, start states, and goals across methods.Evaluations also share environment-step budgets, success criteria, and early-termination rules; results quoted from prior publications retain their original protocols.Source: J BASELINE DESCRIPTIONS AND EVALUATION… · p. 29
• Baseline entries marked † are reproduced evaluations; unmarked entries retain the original publication's reporting convention, apart from Dual-WM results.DINO-WM and HWM use the same datasets as Dual-WM, while several other reproduced baselines use released checkpoints on matched test tasks.Source: J BASELINE DESCRIPTIONS AND EVALUATION… · p. 29
• The task-wise strongest eligible non-Dual-WM aggregates are 75.9% at offset 50 and 61.4% at offset 100.The selection excludes actor-guided proposals, retains the external Gemini controller, and uses reported results available for each task and offset.Source: J BASELINE DESCRIPTIONS AND EVALUATION… · p. 30
• Allowing actor-guided competitors raises the task-wise aggregates to 76.9% at offset 50 and 65.9% at offset 100.This broader proposal-eligible comparison is distinct from the grouping used in the main table.Source: J BASELINE DESCRIPTIONS AND EVALUATION… · p. 30

K ADDITIONAL TWOROOM DISTANCE FIELDS

• Figure 22 presents six additional TwoRoom query states, each with an A* reference followed by LeWM, Dual-WM-L, and Dual-WM-H distance fields.The A* and model columns use the same respective shared scales as Figure 14.Source: Figure 22 · p. 31, K ADDITIONAL TWOROOM DISTANCE FIELDS · p. 31