PaperlayerView exact revision on arXiv

Source-linked AI summary

When More Planning Hurts: Learned Planners for World Models

Josef Chen

arXiv:2610.37891v1cs.ROcs.AIcs.LG

TL;DR

The paper tests whether a fixed action updater can improve planning outcomes on PushT and whether extra updates help. It compares the updater with conventional and learned controllers and finds that it does not meet the task–cost criterion; additional updates also do not establish a general mechanism for score changes and failures.

  • Problem

    The study asks whether one fixed updater works across related predictors and horizons, improves task success and cost over other controllers, and benefits from additional updates.

    Source: §1 · p. 1

  • Method

    A matched intervention study revisits planning decisions from the same observed history while varying effort, using a supervised action updater.

    Source: §1 · p. 1, §4.1 · p. 5

  • Results

    LePlanner solves 16/16 reserved scenes versus 8/16 for the updater, which does not meet the prespecified task–cost criterion; extra updates lower selected scores in one scene without establishing why.

    Source: §2 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    Compatibility across predictors and horizons is not evidence of task success, and the tested updater should not be taken to generalize to other model families.

    Source: §6 · p. 8

    The authors make no claim of practical utility or generalization to other model families, and only a subset of experiments and replays were verified for execution.

    Source: §6 · p. 8

Abstract

from arXiv · show

On PushT, our fixed action updater (1,505 parameters) solves 8 of 16 scenes reserved from our configuration selection. The comparison planners perform as follows: Selected Adam (7 tasks solved, total planning and encoding time 218.440s), adapted iCEM (2 tasks solved, total planning and encoding time 49.309s), released LePlanner (16 tasks solved, total planning and encoding time 5.652s). The total planning and encoding time for our action updater is 59.079s. Note that our action updater does not satisfy the task-cost criterion we fixed before evaluation. We also conduct another matched experiment where we only replay the first ten commands in each context and then allow the next solve 4, 8 or 12 updates (subsequent solves all get 8 updates). Among all 16 contexts (8 scene clusters), the 3 budgets get 9, 8 and 7 successes, respectively. Interestingly, the extra updates did not rescue any failing task in this experiment. All the changes in success/failure happen in one scene cluster, where we observe that the extra updates decrease the costs of the selected latent world-model trajectories for our updater, while the tasks that used to be successes become failures. The code, weights, evaluators, execution traces and cost accounting accompany the paper.

1 Introduction

The paper uses matched interventions on PushT to test a fixed action updater’s compatibility, task performance, and response to additional planning effort. The results caution that better predicted scores do not guarantee execution success, and the tested updater misses the task–cost criterion.

1 Introduction

  • The matched intervention study revisits the same planning point from the same observed history while varying effort to test compatibility, task success and cost, and the effect of extra updates.Compatibility is tested separately from questions requiring execution of the chosen actions.Source: §1 · p. 1
  • On the sixteen reserved scenes, the updater does not meet the prespecified task–cost criterion or outperform released LePlanner.The introduction notes that additional updates cannot recover tasks and make success worse in one scene.Source: §1 · p. 1
  • The reserved comparison scenes were not used for configuration selection, though the authors do not know whether public-model pretraining had seen them.Initial development experiments on the original 24 scenes are reported separately in Appendix A.Source: §1 · p. 1
  • In scene 18651, four updates succeed at command 25, while twelve updates select a lower score and fail at command 50.Success also requires matching the pusher position, not only block position and orientation.Source: Figure 1 · p. 2

2 The updater fails the task–cost criterion

On sixteen reserved scenes, released LePlanner solves all tasks and has the best observed task-cost performance, while the updater solves eight and fails the prespecified criterion. The comparison includes all costs, including failures and cold starts.

2 The updater fails the task–cost criterion

  • 16/16 scenes are solved by released LePlanner, versus 8/16 by the updater, 7/16 by selected Adam, and 2/16 by selected iCEM.LePlanner has the best observed task-cost performance; its summed planning plus encoding time is 5.652 seconds versus 59.079 seconds for the updater.Source: §2 · p. 2, Figure 2 · p. 3
  • The updater fails the prespecified criterion: versus Adam its net gain is +1, below +3; versus iCEM its net gain is +6 but its cost ratio is 1.198, above 1.Against iCEM, the updater takes 9.770 seconds more in summed planning plus encoding time.Source: §2 · p. 2
  • The confirmation study covers 64 trials across four fixed controllers and sixteen reserved scenes, summing planning and encoding time and including failures and cold starts.Plan trajectories and backward calls measure different aspects of work and are not directly comparable across controllers.Source: Table 2 · p. 3

3 Controller and evaluation protocol

The controller minimizes a terminal latent-distance objective with a shared learned action updater, fitted to imitate teacher improvements. The evaluation uses reserved scenes and fixed comparison criteria, with explicit limits on what the comparisons and intervals establish.

3.1 Objective, update and carried plan

  • The updater minimizes a terminal latent-distance objective and applies a shared coordinate-wise head to update the complete action plan.The comparison between five-knot and direct-action approaches does not isolate architecture because their implementations and optimized variables differ.Source: §3.1 · p. 3

3.2 Fitting and exposure

  • The updater imitates normalized teacher improvements rather than using task-reward reinforcement learning, with pooled fitting on 144 correlated snapshots and 5,760 coordinate examples.The fit uses 100 epochs (2,300 Adam updates); the updater is trained on official and FT17 H3/H5 teachers.Source: §3.2 · p. 4

3.3 Controls, execution and statistical unit

  • The protocol pauses simulation during planning and varies the first continuation solve’s update count while fixing later continuation solves at eight updates.Episodes end at success or 50 executed commands; PushT success is judged by joint pusher/block-position error and symmetry-aware block-angle error.Source: §3.3 · p. 4

3.4 Reserved-scene comparison and task–cost criterion

  • The reserved-scene comparison evaluates four full controllers under shared predictor, cadence, action, reset/goal, and command-budget conventions, with a success-first task–cost criterion set before outcomes.The criterion requires learned8 to beat each selected conventional comparator by at least three net successes, use no more total planning and encoding time, and complete all assigned trials; LePlanner is excluded from that criterion.Source: §3.4 · p. 5
  • The reported 95% intervals are descriptive and conditional on fixed fits, related predictors, and the exposed set, and do not account for refit or model-family uncertainty.Scene resampling preserves within-scene cells; the intervals are not evidence of equivalence or noninferiority even when they include zero.Source: §3.4 · p. 5

4 Matched continuation experiment

The matched continuation experiment changes the update budget at one planning decision while holding prior history and later policy fixed. Extra updates did not rescue failures, and in the affected scene lower selected scores accompanied lost successes.

4.1 One encountered solve, fixed subsequent policy

  • The intervention replays ten commands from eight scenes under official and FT17 H5 predictors, then varies the next solve across K ∈ {4, 8, 12} while later solves use eight updates.All arms reconstruct the same reset, command/render order, history, incoming plan, and random state; continuation runs without fitting.Source: §4.1 · p. 5
  • Across 16 contexts, none of the tested update budgets beat always using four updates in success or call count, and extra effort rescued no task.K8 loses one official success relative to K4, and K12 loses one FT17 success relative to K8; both losses occur in scene 18651.Source: §4.1 · p. 6

4.2 Lower selected scores precede failure

  • Official scene 18651 succeeds with K4 at selected cost 19.6764 but fails with K8/K12 at 10.2942; in both affected contexts, more updates lower the score and lose the task.With FT17 in the same scene, K4/K8 succeed at cost 21.6850, while K12 selects 7.5485 and fails.Source: §4.2 · p. 6, Figure 3 · p. 6

4.3 Whole-policy controls and accounting

  • For whole-policy controls, CEM solves 7/16 and Adam solves 5/16, with planning and encoding times per task of 0.669s and 6.573s, respectively.The comparison uses different controller histories; CEM is more computationally efficient, while the similar success rates do not establish which algorithm is better.Source: §4.3 · p. 6

5 Related work

The related work situates the paper among latent-world-model planners, learned plan-update rules, and methods that allocate planning effort. It distinguishes several forms of objective mismatch and notes that the cause of lower scores preceding task failure is unresolved.

Latent planning methods

  • Latent planners differ in their representations and objectives: PlaNet uses predicted rewards, DINO-WM plans toward pretrained goal embeddings, and LeWM minimizes terminal latent distance.LeWM is trained from offline pixel/action trajectories with prediction and SIGReg losses; the paper focuses on its goal-distance objective across related predictors, updaters, and horizons.Source: §5 · pp. 6–7

Learned update rules and planning controls

  • Learned planners update plans through distinct mechanisms, while this updater imitates teacher improvements of a terminal latent-distance objective.The paper tests bounded iCEM adaptation but leaves MPPI untested; it does not reproduce RP-1’s reported performance.Source: §5 · p. 7

Compute allocation

  • Compute-allocation methods choose horizons, replanning times, or internal prediction steps, whereas this intervention varies outer-loop updates at one encountered planning call without fitting an allocator.SufficientPlan reports budget non-monotonicity, including success at a reduced budget when full planning fails.Source: §5 · p. 7

Objective mismatch

  • Objective mismatch can mean a gap between model-training likelihood and controller reward, or that improving a fixed optimization metric does not ensure eventual task success.When lower controller scores precede failure, the paper leaves inaccurate dynamics, unsupported actions, and objective-to-task mismatch as possible explanations.Source: §5 · p. 7

6 Discussion and limitations

The PushT updater falls short of the predefined task-cost criterion, and additional updates did not rescue failures. The discussion limits interpretation to this setting and cautions against claims of practical utility or generalization.

6 Discussion and limitations

  • The updater succeeds on 8 of 16 reserved scenes but misses the task-cost criterion against conventional controls; LePlanner solves all 16.Comparisons do not isolate architecture because controller capacity, fitting data, objectives, normalization, and initialization also differ.Source: §6 · p. 8
  • Additional updates did not recover failing tasks and reduced successes in one scene, but the authors say a single example cannot establish a general mechanism.They report that four updates matched the outcome oracle, with no lower-call budget maintaining its successes.Source: §6 · p. 8
  • The study covers one task, related predictors, and fixed control cadence; public pretraining exposure, seed robustness, independent model-family transfer, and MPPI remain unexamined.The cost-only control used four corrective-learning rounds, unlike the 20-round DAgger regime, so the full approach is not ruled out.Source: §6 · p. 8
  • The authors claim neither practical utility for the learned allocator nor a new optimizer family, and do not claim generalization to other model families.They provide failures, weights, and configurations, but only a subset of experiments and replays were execution-verified.Source: §6 · p. 8

Research assistance

The manuscript received writing suggestions from StealthGPT, and generative AI tools assisted with coding, literature searches, analysis, and drafting.

Research assistance

  • StealthGPT suggested writing improvements, while generative AI tools assisted with coding, literature searches, analysis, and manuscript drafting.Pangram was used for text classification; suggested manuscript changes were checked for consistency with the methods and results.Source: Research assistance · p. 8

A Compatible reuse and development comparisons

Across direct-action predictor and horizon cells, the shared updater performs comparably to Adam, while development probes and untested alternatives leave the interpretation and transfer scope bounded.

A Compatible reuse and development comparisons

  • 140/216 direct-action tasks are solved by the shared updater using either shared or padded features, versus 132/216 for Adam and 119/216 for CEM.The shared-minus-Adam success difference is +3.70pp [−6.48, +13.89]; mean planning plus encoding time is 5.832 seconds for shared and 6.289 seconds for Adam.Source: Figure 4 · p. 10, A Compatible reuse and development… · p. 10, Table 4 · p. 10
  • The four fixed specialists solve 110, 135, 141 and 127 tasks versus 140 for the pooled updater; removing gradient magnitude and scale yields 110/216, and forgetting the retained plan yields 63/216.The best specialist leads by one goal, but teacher exposure and fitting work are not matched; the authors say these probes do not establish new warm-start methods.Source: A Compatible reuse and development… · p. 10
  • 9/16 primary goals are solved by both fixed8 and the earlier episode-start threshold, but the threshold misses the screen despite 7.2569% lower planning time.It needed two more primary goals without increased planning cost, or at least a 25% planning-time reduction without losing solved goals; on 20 challenge contexts, both solve eight and the threshold costs slightly more.Source: A Compatible reuse and development… · p. 11

A.1 Released-controller and World/CEM solver development probes

  • The released LePlanner alone solves all four development-probe scenes, versus two for learned8 and Adam and one for bounded one-step CEM.LePlanner uses 6.072 s summed planning/encoding time versus 16.713 s for learned8, but the controllers differ in capacity, training, objectives, normalization and plan state.Source: §A.1 · p. 11
  • 2/2 versus 0/2 scenes are solved by World/CEM under the two shared-scene protocols, but history length and cadence vary together, so the cause cannot be isolated.The controls are unbounded, and the reported elite-average score is not directly comparable to the paper’s best-scored-plan selection rule.Source: §A.1 · pp. 11–12

A.2 Matched conventional calibration and separate validation

  • On four separate validation scenes, learned8 solves 2/4, selected Adam 1/4, and selected iCEM 0/4; their summed planning plus encoding times are 16.417, 64.410 and 21.011 seconds, respectively.The family configurations were selected by success count on bounded calibration, without a timing tiebreak.Source: §A.2 · p. 12, Table 6 · p. 12

D Missing comparisons and tests that could overturn the findings

  • The authors flag unresolved alternatives: cheap contextual policies, independent model/task shifts, MPPI, the official LeWM planner, and sufficient matched training could change interpretation.They also state that current evidence does not establish broad transfer beyond similar weights and a single task.Source: D Missing comparisons and tests that… · p. 14

C Experimental records and accounting

I Original comparison and compiled conventional controls

Across the original fixed-five-knot and compiled-control comparisons, the learned control did not show a robust advantage over Adam. Direct-action results and fixed-specialist transfer were descriptive and conditional on their particular study setups.

I Original comparison and compiled conventional controls

  • 56/216 cost-only cases were solved by learned control A115, versus 114/216 by the network-free best65 selector (paired difference −26.85 pp).The descriptive scene interval was [−41.20, −12.50].Source: I Original comparison and compiled… · p. 15
  • 128/216 compiled tasks were solved by shared SFT, 122/216 by Adam, and 106/216 by CEM; the shared-minus-Adam difference was +2.78pp [−7.41, +12.50].Mean planning-plus-encoding time was 3.321 s for shared SFT and 1.640 s for Adam; the intervals do not establish a robust joint advantage.Source: I Original comparison and compiled… · pp. 16–17, Table 8 · p. 17

J Direct-action parameterization and fixed specialist transfer

J.1 Four fixed specialists, one historical reference

  • FT17-H3/H5 specialists solved 141/127 of 216 problems, Official-H3/H5 specialists 110/135, and the shared reference 140; the one-goal FT17-H3 difference was +0.46pp [−8.33, +8.80].Transfer outcomes depend on the particular fixed training cell and do not represent a learned routing policy.Source: §J.1 · p. 17, Table 10 · p. 17
  • Specialists and pooled models had unequal data exposure and fitting work, so their reported computation times do not establish comparable total-cost savings or general specialization benefits.Specialists used 1,080/1,800 coordinates and 500/800 updates, versus 5,760/2,300 for the pooled version.Source: §J.1 · p. 17

K FT17 adaptation, fitting cost and official retention

Warm FT17 adaptation produced descriptive target-goal and retention results close to the fixed-role baselines, without a declared benefit threshold. The extra fitting did not appear to pay off in this setting.

K FT17 adaptation, fitting cost and official retention

  • 35/48 target-goal episodes succeeded for warm adaptation, pooled, and fresh roles, compared with 34/48 for the unchanged official baseline.Warm adaptation's target-goal mean difference was +2.08 percentage points, with a scene interval from −12.50 to +16.67pp; no adaptation-benefit threshold was set.Source: K FT17 adaptation, fitting cost and… · p. 18, Table 11 · p. 18
  • 33/48 official-goal retention episodes succeeded for warm adaptation, versus 34/48 for pooled and fresh roles and 32/48 for the unchanged official role.These are descriptive directions, not noninferiority conclusions.Source: K FT17 adaptation, fitting cost and… · p. 18
  • For direct-action 384, the controller took 1,935.978 seconds and workers took 1,846.263 seconds; in this setting, the extra fitting has not paid off for adaptation.Source: K FT17 adaptation, fitting cost and… · p. 19

L Fit seeds, retrained feature masks and plan reset

Across refits and feature-mask controls, outcomes varied by seed and some masks reduced solved tasks; resetting the previous solution sharply reduced performance. In held-out effort allocation, the threshold tied fixed8 on ID successes but did not meet the prespecified screen.

L Fit seeds, retrained feature masks and plan reset

  • 140, 131, and 141 of 216 tasks were solved by fit seeds 13, 23, and 37; masking gradient-magnitude and scale channels while retaining gradient sign yielded 110/216.Masking cost channels yielded 120/216; masking budget, previous-update, and coordinate/length channels yielded 135, 135, and 130, with intervals including zero in those three cases.Source: Figure 9 · p. 19, Table 12 · p. 19, L Fit seeds, retrained feature masks… · p. 20
  • 63/216 tasks were solved after resetting the previous solution at each decision, versus 140/216 without reset (−35.65pp [−45.37, −25.93]).The setting still used the real observed/action history and all 12 within-solve updates; it relies on the receding-horizon solution as the initial solution.Source: L Fit seeds, retrained feature masks… · p. 20

M Actual effort allocation and runnable-flow validation

  • 9/16 ID goals were solved by the primary threshold and fixed8, but threshold planning took 46.518 seconds versus 50.158 seconds; the held-out test did not confirm the TRAIN allocation result.On OOD contexts they both solved 8/20, while threshold planning took 95.835 seconds versus 94.635 for fixed8; neither policy met the prespecified overall criterion.Source: M Actual effort allocation and… · p. 20, Table 13 · p. 20

M.1 What the released flow demonstrates

  • The released-flow smoke test had CEM solve the task after 41 commands, while Adam and shared failed after 50 commands.It was a quick sanity check, not a rerun of the full 864-case matrix.Source: §M.1 · pp. 20–21

N Cost boundaries, uncertainty and limits

The paper separates timing quantities and cautions that instrumented planning measurements are not physical-robot response-time estimates. Its uncertainty analyses are conditional, and several factors limit broader interpretation of the results.

N Cost boundaries, uncertainty and limits

  • 0.814 seconds was the shared updater's mean decision time, compared with 0.863 seconds for Adam; the shared updater's 95th percentile was 1.177 seconds versus Adam's 1.143 seconds.These decision-time summaries do not imply that deadlines are met on a real robot.Source: N Cost boundaries, uncertainty and… · p. 21, Table 14 · p. 21
  • Two expensive tasks account for 62.96% of the compiled SFT planning-plus-encoder time, and the full-task compiled times retain unsuccessful tasks and logged cold work.The timing distribution is not an isolated measure of compiler-generated kernels.Source: N Cost boundaries, uncertainty and… · pp. 21–22, Table 15 · p. 22
  • Instrumented timing measures action-sequence generation, not per-decision latency, and simulator episode timing includes planning and encoding.The reported timings do not measure uninstrumented neural-network latency or physical-robot response time.Source: N Cost boundaries, uncertainty and… · pp. 21–22
  • Confidence intervals are conditional on fixed heads and predictors, while refit/model-family uncertainty and multiple-comparison correction are not accounted for.The analysis also notes that exposure to public fitting data may limit generalization, and uses a single hardware family.Source: N Cost boundaries, uncertainty and… · p. 22
  • The first objective decreased without progress toward the center in 85/216 shared episodes, and its first-five-command progress proxy may not correlate with Full-H scores.The correlations ranged from 0.244 to 0.641 for the shared updater; the proxy and Full-H scores use different horizons.Source: N Cost boundaries, uncertainty and… · p. 22

O Reproducibility and recorded demonstrations

A pinned-environment rerun reproduced the original confirmation results, while a second-host rerun matched success labels but not numerical values. The section also documents limits on available traces and on what the reproduction establishes.

O Reproducibility and recorded demonstrations

  • The released materials include reproduction instructions, task IDs, full results, evaluation code, and versioned weights, but per-call traces omit compiled and original updater experiments.Available plan and neural-network traces allow inspection of individual decisions; demonstrations replay simulator-recorded results.Source: O Reproducibility and recorded… · p. 22
  • A fresh Python 3.12 environment with all 65 packages pinned reproduced 64 results, 2,192 commands, numerical traces, and saved frames identically.Timing measurements from this rerun are reported separately in the reproduction guide.Source: O Reproducibility and recorded… · p. 22
  • The HTTP predictor-and-horizon change did not pass in the fork run, and the check does not establish equivalence with direct-action execution or recovery and superior transfer.The baseline passed in 28 commands; the fork used 50 commands and 29/51 actual frames.Source: O Reproducibility and recorded… · p. 22

O.1 Separate-machine reproduction

  • All 64 success labels matched the reference results on a second host, but numerical equivalence was not achieved and this was not an independent test sample.The rerun used the same cases, model weights, inputs, and evaluator.Source: Table 16 · p. 23, §O.1 · p. 23
  • Across hosts, timing and numerical differences remained unexplained; deterministic algorithms were off, and environments differed in Python, C library, kernel/container, and GPU.Frames differed in 3/64 streams, and 38,383 of 4,865,835 numeric values exceeded diagnostic tolerance.Source: §O.1 · p. 23
  • The instrumented reproduction took 320.643 s for the supervised campaign and 511.621 s from launch through cleanup; these are not production latencies.The totals include cold or failed tasks and should not be added to P+E.Source: §O.1 · p. 23
Loading 2610.37891v1…