PaperlayerView exact revision on arXiv

Source-linked AI summary

What Must a World Model Distinguish for Planning?

Rongzhe Wei, Hans Hao-Hsun Hsu, Peizhi Niu, Yifan Li, Pan Li

arXiv:2609.33030v1cs.LGcs.RO

TL;DR

World models predict candidate-action consequences, but planning may not require preserving every physical distinction needed for accurate prediction. The paper formalizes query-dependent information needs and evaluates where query information should enter planning. Its modular design uses query-guided proposals with reusable action-conditioned predictions, while empirical regret comparisons vary between seen and unseen objectives.

  • Problem

    It is unclear which physical distinctions world models must preserve for planning, since different physical outcomes can favor the same action while small physical changes can reverse the preferred action.

    Source: §1 · p. 1

  • Method

    The paper defines mechanism, response, and decision sufficiency and compares query-conditioned joint action-and-outcome modeling with query-guided proposals paired with action-conditioned prediction.

    Source: §1 · pp. 1–2

  • Results

    Modular reduces held-out regret by 0.011 relative to ACWM and seen-objective regret by 0.014 relative to Joint, while WAM's seen-objective advantage largely disappears on unseen objective combinations.

    Source: §1 · p. 2, §6.2 · p. 9

  • Takeaways & Limitations

    Query-guided candidate generation with action-conditioned physical prediction allows predictions to be reused across objectives while search adapts to the current objective.

    Source: §7 · p. 10, H SCOPE, LIMITATIONS, AND FUTURE… · p. 32

    Empirical rates depend on the observation process, model class, optimization, coding scheme, and readout, so they characterize the studied systems rather than universal information-theoretic limits.

    Source: H SCOPE, LIMITATIONS, AND FUTURE… · p. 32

Abstract

from arXiv · show

World models simulate the consequences of action candidates, but good planning need not preserve every physical distinction required for accurate prediction. We formalize this gap through a hierarchy of mechanism, response, and decision sufficiency. Given a candidate set, the planning query determines which physical variations matter and how precisely they must be preserved: coarse decisions can discard much of the information needed for prediction, whereas fine decisions may require nearly the same resolution. In practice, planners often adaptively search to construct candidates, and information unnecessary for final selection may still be needed to discover good candidates. What a world model must preserve therefore depends on the query, the candidate set, and the planner. We study these effects in a collision system, nonlinear dynamics, and robotic planning. These varying requirements raise a design question: where should query information enter the planning system? A model that jointly generates actions and outcomes conditioned on the query achieves lower regret than an action-conditioned world model on seen objectives, but this advantage largely disappears when generalizing to unseen objectives. Motivated by this, we propose a modular design in which the query determines where to look and an action-conditioned model predicts what will happen, allowing the same predictions to be reused across objectives.

1 INTRODUCTION

The paper asks which physical distinctions world models must preserve for planning, arguing that predictive fidelity alone is not the right criterion. It develops query-, candidate-set-, and planner-dependent requirements, then compares ways to place query information in the planning system.

1 INTRODUCTION

  • Two worlds can produce different trajectories yet favor the same action, while a small physical change can reverse the preferred action.Thus the distinctions required for action choice need not match those required for accurate prediction.Source: §1 · p. 1
  • The mechanism–response–decision hierarchy distinguishes preserving physical dynamics, preserving query-specific candidate costs, and retaining enough information to choose a low-regret action.The resolution needed between response and decision depends on the query: coarse decisions can discard substantial cost detail, while fine decisions may need nearly the full cost resolution.Source: §1 · p. 1
  • Adaptive search may need distinctions that final action selection can discard, especially early in the search.Candidate actions are often constructed during planning, and predictions can guide search-distribution updates as well as the final choice.Source: §1 · pp. 1–2
  • Query-guided proposals combined with action-conditioned physical prediction form a modular design that reuses predictions across objectives.Jointly conditioning action generation and outcome prediction on the query has lower regret on seen objectives, but that advantage largely disappears on unseen objective combinations.Source: §1 · p. 2

2 RELATED WORK

Prior work studies world models for planning, task-relevant abstractions, and decision-aware learning. This paper distinguishes information needed to predict candidate costs from information needed to choose a low-regret action, and examines how query placement affects generalization.

World models for planning

Task-relevant abstractions for planning

Decision-aware representation learning

3 PHYSICAL DISTINGUISHABILITY FOR WORLD-MODEL PLANNING

The section formalizes planning around candidate actions and query-specific costs, then defines a hierarchy of increasingly permissive world distinctions. The hierarchy ranges from preserving physical dynamics to preserving only what is needed for the selected action.

3.1 WORLD MODELS AND PLANNING QUERIES

  • A world model maps observation-action history to a predictive state and predicts outcomes for candidate action sequences, while a query assigns each outcome-action pair a scalar cost.The true candidate cost is computed from the physical outcome, and the planning choice minimizes that cost over the candidate set.Source: §3.1 · pp. 2–3

3.2 A HIERARCHY OF WORLD-MODEL SUFFICIENCY

  • Mechanism sufficiency preserves differences in physical dynamics, whereas response sufficiency can merge physically different worlds with the same query-specific candidate-cost profile.Decision sufficiency is weaker still: distinct cost profiles may be grouped when they select the same action, yielding the ordering from mechanism through response to decision sufficiency.Source: §3.2 · p. 3
  • These sufficiency requirements are information requirements, not guarantees that a representation can attain them when history cannot distinguish worlds.The framework also extends to approximate response prediction and low-regret decisions through tolerance-aware covers.Source: §3.2 · p. 3

4 QUERIES DETERMINE WHAT A WORLD MODEL MUST PRESERVE

The section formalizes query-dependent information needs from physical mechanisms through candidate costs to low-regret decisions, then examines how decision resolution and planning horizon affect the information required.

4.1 QUERY-DEPENDENT RELEVANCE AND RESOLUTION

  • Mechanism sufficiency preserves physical distinctions, response sufficiency preserves candidate costs, and decision sufficiency retains enough information for a low-regret choice; the query sets relevance and precision.Source: §4.1 · p. 3

4.2 CONTROLLED PHYSICS: EXACT QUERY-DEPENDENT STRUCTURE

  • At matched ∆=2ϵ, response–decision gaps grow without bound for threshold queries but remain bounded for tracking; decision supervision saves bits for thresholds, not systematically for tracking.Source: §4.2 · pp. 4–5, Figure 2 · p. 5

4.3 LEARNED TEMPORAL DYNAMICS: RESOLUTION SHAPES INFORMATION NEEDS

  • 0.016, 0.080, and 0.782 bits per context are required at K = 2, 8, and 64; at K = 8, T = 1 and 4 s require 0.039 and 0.080.At K = 8, T = 2 s requires 0.046 bits per context; these rates are at normalized regret δ = 0.02.Source: §4.3 · p. 5, Figure 3 · p. 6

5 PLANNING CHANGES WHAT MUST BE DISTINGUISHED

Planning information needs vary with the candidate set and with the stage at which predictions enter an adaptive planner.

5.2 DIFFERENT PLANNING STAGES REQUIRE DIFFERENT DISTINCTIONS

  • 0.691 mean selected-action cost follows one-bit search, versus 0.180 with full-prediction search and 0.179 with one-bit messages only at final selection.With equal budgets of 60 nominal bit-iterations per candidate, early four-bit Cost then one-bit Decision messages yielded mean executed cost 0.189, versus 0.490 for the reverse schedule and 0.275 for two-bit Cost throughout; four-bit Cost throughout reached 0.173 at twice the budget.Source: Figure 4 · p. 6, §5.2 · p. 7

5.1 CANDIDATE SETS CHANGE RELEVANT DISTINCTIONS

  • Under the coarse query, the regret gap between one-bit messages and full predictions widens from 0.019 on the narrowest candidate sets to 0.115 on the broadest.Thus, information sufficient for one candidate set may not suffice when the alternatives change.Source: Figure 4 · p. 6, §5.1 · p. 6

6 FROM PLANNING REQUIREMENTS TO MODEL DESIGN

The section tests how to preserve planning-relevant information under limited representation capacity and where query information should enter when objectives change. Its results favor planner-oriented supervision for compact representations and query-guided proposals paired with reusable, action-conditioned physical prediction.

6.1 SUFFICIENCY DOES NOT DETERMINE SUPERVISION

  • With one bit per candidate, mean planning regret is 0.272 for Hybrid, 0.353 for Planner, and 0.448 for Value, versus 0.069 for Full.The compact-representation ordering also appears in closed-loop final cost; the advantage weakens as capacity increases.Source: §6.1 · p. 7, Figure 5 · p. 7

6.2 WHERE SHOULD QUERY INFORMATION ENTER?

  • Query-aware proposals reach 30.83% success with N = 8 candidates, versus 28.19% for query-blind proposals at N = 128.The comparison spans 512 new scenes and four target directions, using the same query-independent, action-conditioned predictor.Source: §6.2 · p. 8, Figure 6 · p. 8
  • Joint reduces regret by 0.027 relative to ACWM on training objectives, but by only 0.004 on held-out compositions.The comparison concerns complete interfaces, which also differ in predictive action exposure and evaluation of arbitrary supplied actions.Source: §6.2 · p. 9, Figure 8 · p. 9
  • Modular reduces held-out regret by 0.011 relative to ACWM and seen-objective regret by 0.014 relative to Joint.On held-out objectives, candidate-set regret falls by 0.024 while selection regret rises by 0.013, for a net improvement of 0.011.Source: §6.2 · p. 9, Figure 8 · p. 9
  • Modular uses query-aware proposals with an action-conditioned predictor, so predicted outcomes can be rescored across objectives.The study combines target reaching with optional keep-out regions, a peak-speed constraint, and an effort penalty; evaluation includes held-out objective compositions.Source: §6.2 · pp. 8–9

7 CONCLUSION

The conclusion argues that planning requires preserving distinctions useful for both search and action choice, not predictive fidelity alone. It recommends matching information, resolution, and model design to the planner’s needs.

7 CONCLUSION

  • World models must preserve distinctions required for search and action choice, rather than predictive fidelity alone.The mechanism–response–decision hierarchy captures how query requirements determine which physical variations matter and how precisely to resolve them.Source: §7 · p. 9
  • Adaptive search may need information that final action selection no longer requires.When representation capacity is limited, planner-oriented supervision helps preserve planning-useful distinctions; query-guided proposals can adapt search while physical prediction remains reusable across objectives.Source: §7 · pp. 9–10
  • An effective world model should preserve the right distinctions at the resolution and stage where the planner needs them.The Modular interface improves planning while retaining reusable action-conditioned predictions across objectives.Source: §7 · p. 10

SUPPLEMENTARY MATERIAL

The supplement establishes formal links among mechanism, response, and decision sufficiency, and shows that adaptive planning may need information beyond what final action selection requires. Its collision analysis illustrates query-dependent distinctions and cover complexity, while the learned-study details describe how representations are compressed and evaluated.

A.1 EXACT REFINEMENT AND THE ROLE OF TIES

  • With fixed query, candidate set, initial conditions, and tie rule, equality of dynamics implies equal cost profiles and selected actions; selected-minimizer equality avoids the non-transitivity of overlapping minimizer sets.Redundant parameterizations may represent identical dynamics, and approximate parameter recovery needs an additional continuity bound to imply response accuracy.Source: A FORMAL PROPERTIES AND PLANNER… · p. 14, §A.1 · p. 14

A.2 RESPONSE ACCURACY CERTIFIES LOW REGRET

  • Response accuracy within ε certifies regret at most 2ε, but the converse fails: a one-action decision cover can coexist with an arbitrarily large response cover.The argument assumes a decoded profile attains its minimum; this holds for finite candidate sets and compact continuous intervals.Source: §A.2 · p. 14

A.3 WHAT AN ADAPTIVE PLANNER MUST PRESERVE

  • Adaptive search may require richer summaries than final selection: CEM uses elite candidates, MPPI uses normalized relative-cost weights, and intermediate updates affect later candidates.The complete candidate-cost profile supports all three operations but can contain more information than the planner uses.Source: §A.3 · p. 14, Table 1 · p. 15

B.5 COMPRESSION AND LOSSLESS CODING

  • Compression trades representation rate against task-loss degradation, and reported rates measure complete lossless coded streams rather than theoretical cover complexity.Dfull is pre-compression loss without a rate constraint, D0 is the zero-information baseline, and the tolerance parameter controls permitted degradation.Source: B COLLISION: ANALYTIC STRUCTURE AND… · p. 15, §B.5 · pp. 17–18

B.1 QUERY-RELEVANT PHYSICAL QUANTITIES

  • In the collision family, fixed-velocity and fixed-impulse queries depend on κ and λ, respectively, and neither quantity determines the other across mechanisms.Across the stated parameter ranges, κ spans [0.2, 1.6] and λ spans [0.25, 2].Source: §B.1 · p. 15

B.2 THRESHOLD AND TRACKING COVERS

  • For threshold queries q1 and q2, two actions suffice for zero-regret decisions, while response-cover complexity keeps growing as precision tightens, making the matched-tolerance log-cover gap diverge.For q1, each response-cover element spans at most 2ε/v0 in κ; the same threshold argument applies to q2 using λ.Source: §B.2 · p. 15
  • For continuous tracking under the stated endpoint condition, response and decision covers both scale as Θ(1/∆), so their log-cover difference remains bounded at matched tolerance.These asymptotics concern a continuous candidate interval; the learned experiment instead uses 64 actions and has a finite exact decision cover.Source: §B.2 · p. 16

B.3 WORLDS, OBSERVATIONS, AND QUERIES; B.4 REPRESENTATION, SUPERVISION, AND COMMON SCORER

  • The collision learner observes two reset-collision outcomes and the query, encodes observations as variable-length discrete messages, and evaluates candidates with a matched scorer.The simulator supplies exact-cost supervision, while the encoder does not receive mechanism labels; the matched scorer isolates differences in information retained by each representation.Source: §B.3 · p. 16, §B.4 · pp. 16–17

B.6 FRONTIER CONSTRUCTION AND SUPPORTED COMPARISONS

The Double Pendulum experiments show that required information rises with finer candidate choices and longer horizons, while the reported rate comparisons are limited by frontier support and representation capacity. The section also validates the learned dynamics model and describes the Push-T query-cost evaluation.

B.6 FRONTIER CONSTRUCTION AND SUPPORTED COMPARISONS

  • Both threshold queries show positive mean paired savings at reported benchmark tolerances, but savings can reverse elsewhere along the frontier.At q3 and δ ≈0.001, some frontiers do not reach the target, leaving required rates undefined.Source: §B.6 · p. 18, Table 4 · p. 18

C DOUBLE PENDULUM: LEARNED DYNAMICS AND DECISION REQUIREMENTS

C.1 PHYSICAL SYSTEM, OBSERVATIONS, AND PLANNING QUERY

  • The Double Pendulum planner selects among fixed nested candidate sets of 2, 8, or 64 velocities, with response supervision predicting costs for all 64 actions.Source: §C.1 · p. 19

C.2 LEARNING AND VALIDATING THE PHYSICAL REPRESENTATION

  • Across rollout horizons and planning resolutions, the learned dynamics model substantially outperforms the mean-mechanism baseline.At K = 8, median normalized regret is 0.0095 for the learned model and 0.0226 for the baseline; all five seeds met the fidelity criterion.Source: §C.2 · p. 20, Table 5 · p. 20

C.3 COMPRESSION, COMMON SCORING, AND RATE ESTIMATION

  • Rate comparisons use matched supported seeds, within-seed changes, and a minimum of three seeds; large frontier gaps are excluded from interpolation.Source: §C.3 · p. 21

C.4 REQUIRED RATES AND INPUT CONTROLS

  • At δ = 0.05 and 0.02, the learned representation requires more information as candidate resolution or planning horizon increases.Paired within-seed changes have the same positive direction for both adjacent resolution and horizon comparisons.Source: §C.4 · p. 21, Table 6 · p. 21

C.5 AUTOREGRESSIVE ROLLOUT FIDELITY

D.1 VISUAL PREDICTION AND QUERY COSTS

  • Push-T planning evaluates query costs from predicted terminal poses, using separate coarse and fine queries and simulator costs for retrospective regret.Source: §D.1 · pp. 22–23

D.2 MESSAGES, DATA, AND TRAINING

The experiments show that selection difficulty depends on candidate-set breadth and that compressed messages used during search can damage the candidate set even when the final selector is unchanged. Query-aware candidate proposals also achieve higher success with substantially fewer evaluations than query-blind proposals.

D.2 MESSAGES, DATA, AND TRAINING

  • Decision and Cost messages are discrete candidate representations, followed by a common scorer trained on the decoded message and query information.The matched readout isolates information retained by each message from differences between task-specific heads.Source: §D.2 · p. 23, Table 7 · p. 23

D.3 CEM AND RETROSPECTIVE EVALUATION

  • CEM uses 30 iterations of 300 candidates and ten elites, and retrospective simulator costs do not guide search or selection.Source: §D.3 · pp. 23–24

D.4 DEPENDENCE ON THE CANDIDATE SET

  • The regret gap between Decision and Full rises from 0.019 on the narrowest candidate sets to 0.115 on the broadest.The representations are evaluated on the same saved candidate populations.Source: §D.4 · p. 24

D.5 SEARCH VERSUS SELECTION AND INFORMATION TIMING

  • Using the one-bit Decision message during search substantially degrades the candidate set, while using it only for final selection has almost no effect.Under the fine query, mean selected-action costs for Full/Full, Full/Decision, Decision/Full, and Decision/Decision are 0.180, 0.179, 0.691, and 0.732, respectively.Source: §D.5 · p. 24

D.6 COST-MESSAGE CAPACITY ACROSS CANDIDATE SETS

  • Selection regret decreases as Cost messages retain more information, while broader candidate sets remain more demanding across message capacities.Source: §D.6 · p. 24, Figure 11 · p. 24

E.1 ENVIRONMENT, OBSERVATIONS, AND PHYSICAL DATA

  • The PushCube experiment reuses query-independent physical trajectories across objectives, with a diverse action bank covering varied cube motions.Source: §E.1 · p. 25

E.2 SHARED PREDICTOR AND MATCHED PROPOSALS

  • PushCube uses a shared action-conditioned physical predictor, while matched proposal networks differ in whether they receive the target query.Source: §E.2 · p. 25

E.3 CANDIDATE BUDGETS AND EXECUTION

  • With N = 8, the query-aware proposal achieves lower mean terminal cost and higher success than the query-blind proposal with N = 128, using 16× fewer candidate evaluations.Source: §E.3 · p. 26

E.4 CANDIDATE QUALITY AND SELECTION ERROR

The candidate-quality study separates proposing good actions from selecting them, while the Push-T supervision comparison finds that planner-aligned signals help most under a one-bit bottleneck. At eight bits, Value instead outperforms Hybrid on the reported measures.

E.4 CANDIDATE QUALITY AND SELECTION ERROR

  • For query-aware proposals, the best simulator-scored candidate improves as the budget grows, but the learned selector does not always improve at the same rate.A single N = 128 example shows a low-cost candidate available but missed by the selector, distinguishing proposal from selection error.Source: §E.4 · p. 26, Figure 12 · p. 26

F LEARNING PLANNER-RELEVANT REPRESENTATIONS

  • The Push-T comparison tests Value, Planner, and Hybrid supervision under a fixed discrete bottleneck with shared inputs, capacity, and downstream planner.Source: F LEARNING PLANNER-RELEVANT… · p. 26

F.1 DATA AND DISCRETE BOTTLENECK

  • The Push-T training bank contains saved CEM populations, and each candidate receives a discrete code that cannot be bypassed by continuous features or actions.Source: §F.1 · pp. 26–27, Table 9 · p. 27

F.2 VALUE, PLANNER, AND HYBRID SUPERVISION

  • Value preserves absolute predicted costs, whereas Planner emphasizes candidate rank and separation around the CEM elite boundary; Hybrid combines both objectives.Source: §F.2 · p. 27

F.3 LOW-CAPACITY CONFIRMATION AND CAPACITY CONTROL

  • At one bit, Planner improves over Value on both elite agreement and planning regret, while Hybrid performs best; at eight bits, Value has lower regret and higher elite agreement than Hybrid.Source: §F.3 · pp. 27–28, Table 10 · p. 27

F.4 CLOSED-LOOP EVALUATION AND ROBUSTNESS CONTROLS

Closed-loop results show a one-bit Hybrid advantage over Value, but the ordering reverses at eight bits. The following PushCube experiment compares query placement across familiar and recombined objectives while reusing physical predictions where possible.

F.4 CLOSED-LOOP EVALUATION AND ROBUSTNESS CONTROLS

  • At one bit, Hybrid lowers mean closed-loop cost relative to Value by 0.113, while the eight-bit Hybrid–Value cost difference is +0.062.Hybrid also beats Value in all nine tested CEM configurations; additional data does not remove the low-capacity difference.Source: §F.4 · p. 28, Figure 13 · p. 28

G QUERY PLACEMENT ACROSS OBJECTIVES

G.1 OBJECTIVE FAMILIES AND PHYSICAL DATA

  • Held-out evaluation recombines familiar objective terms into unseen compositions, using shared physical data across objectives and ten candidate plans per planner.Source: §G.1 · p. 29

G.2 QUERY CONSTRUCTION, COSTS, AND SUCCESS

  • A plan succeeds only if it reaches the target tolerance and satisfies every active constraint, as well as table and workspace conditions.Source: §G.2 · pp. 29–30

G.3 ACWM, JOINT, AND MODULAR INTERFACES

  • ACWM and Modular can reuse a predicted physical outcome across objectives, whereas Joint specializes its prediction pathway to the planning query.Source: §G.3 · p. 30, Table 11 · p. 30

G.4 TRAINING AND ACTION EXPOSURE

  • Predictive training exposure differs across interfaces: ACWM uses query-independent broad action samples, Modular reuses its predictor, and Joint receives query-weighted examples.Source: §G.4 · p. 30

G.5 REGRET DECOMPOSITION AND OBJECTIVE SHIFT

Regret is decomposed into candidate generation and final selection, clarifying where query placement changes planning performance. The comparisons show the trade-off between query-aware search and reusable, query-independent physical predictions.

G.5 REGRET DECOMPOSITION AND OBJECTIVE SHIFT

  • Regret separates whether the proposal generates good alternatives from whether the predictor selects well among them.The comparison uses a shared simulator-evaluated reference set, pooling proposal candidates with additional goal-directed actions.Source: §G.5 · p. 30
  • 0.011 lower regret than ACWM on held-out objectives is the net improvement for Modular.Modular reduces candidate-set regret by 0.024, while its selection regret is 0.013 higher.Source: §G.5 · pp. 30–31
  • Joint improves over ACWM on seen objectives, but its advantage becomes smaller on held-out objective compositions.The same pattern holds with raw costs, so it does not arise from objective-specific normalization.Source: §G.5 · p. 31

G.6 FIXED-HISTORY REUSE AND HIGHER-CAPACITY DIAGNOSTIC

  • For a fixed observation and action sequence, ACWM and Modular predictions can be cached and rescored under a new query.Joint+action lowers regret further, but its added capacity and predictive training exposure make it a diagnostic rather than a matched comparison.Source: §G.6 · p. 31

G.7 QUALITATIVE OBJECTIVE COMPOSITIONS

  • Figure 15 visualizes trajectories from separately reset rollouts under different objective compositions, including one held out during training.The examples show behavior as constraints are added; they are qualitative illustrations, not an additional quantitative comparison.Source: §G.7 · p. 31, Figure 15 · p. 32

H SCOPE, LIMITATIONS, AND FUTURE DIRECTIONS

The empirical conclusions characterize the studied systems, not universal information-theoretic limits. The paper identifies practical directions for reusing predictions across objectives while noting settings and planners that remain to be examined.

Scope and limitations

  • Empirical rates depend on the observation process, model class, optimization, coding scheme, and readout, so they are not universal information-theoretic limits.The robotic studies use simulated Push-T and PushCube environments to vary planning components while controlling the underlying physics.Source: H SCOPE, LIMITATIONS, AND FUTURE… · p. 32
  • Adaptive-search results are demonstrated primarily with CEM, and other planners may require different intermediate distinctions.The continuous-action analysis isolates an idealized resolution effect approximated by finite candidate sets in learned experiments.Source: H SCOPE, LIMITATIONS, AND FUTURE… · p. 32

Future directions

  • When objectives change frequently, query-guided candidate generation and action-conditioned prediction can support prediction reuse across tasks.The proposed directions include caching physical predictions, updating candidate proposals, and allocating predictive detail to the current search stage.Source: H SCOPE, LIMITATIONS, AND FUTURE… · p. 32
  • Future work could make selected parts of predictive representations query-dependent while retaining reusable physical structure.The paper also identifies longer-horizon manipulation, richer embodied tasks, and large learned world models as settings for further study.Source: H SCOPE, LIMITATIONS, AND FUTURE… · p. 32

Related papers

Loading 2609.33030v1…