PaperlayerView exact revision on arXiv

Source-linked AI summary

Not All Errors Matter: Decision-Relevant Prediction Error Predicts Planning Quality

Linhao Wang, Yiyan Fan, Dongjin Huang

arXiv:2609.32322v2cs.LG

TL;DR

Prediction error is commonly used to judge world models, although planning depends on only some state information. The paper introduces DRPE and an iso-error protocol; its experiments find that decision-relevant error tracks planning quality more closely than total error, with rankings also changing across tasks.

  • Problem

    World models are often evaluated by prediction accuracy, but the state dimensions relevant to decisions may differ from those that dominate total error.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces DRPE for multi-step error on decision-relevant dimensions and an iso-error protocol that varies error allocation at fixed total error.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    Across models, DRPE closely tracks planning success while total error is weakly associated with it, and model rankings can reverse across tasks at the same total error.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    World models should be assessed by errors on the state information that decisions depend on, rather than by total prediction error alone.

    Source: §6 · p. 10

    The study uses a 12-dimensional gridworld, leaving transfer to pixel-scale models untested; DRPE also requires prior knowledge of relevance.

    Source: §6 · p. 10

Abstract

from arXiv · show

World models are typically trained and evaluated by prediction error, assuming that more accurate predictions lead to better decisions. We show that this assumption can fail because models with similar total error can differ substantially in planning performance when their errors occur on different state dimensions. We introduce Decision-Relevant Prediction Error (DRPE), which measures prediction error on the state dimensions that affect decisions. We also develop an iso-error evaluation protocol that varies error allocation while keeping total error fixed. In a factored gridworld with known state relevance and a standardized planner, we evaluate 55 controlled and learned models across different error levels and allocations. Total prediction error is weakly related to planning success (Spearman $\rho=-0.25$), whereas DRPE is strongly predictive ($\rho=-0.84$; $-0.98$ within the controlled family). Models with only a 1\% difference in total error can differ by 60 percentage points in planning success (97\% vs 37\%). The relevant error also depends on the task, with model rankings reversing across tasks at the same total error. Deeper imagination further amplifies decision-relevant errors, while learned models exhibit systematic bias on rare but decision-critical events. We formalize sufficient conditions under which DRPE correctly ranks models and total prediction error cannot.

1 INTRODUCTION

Prediction error can miss whether a world model preserves the information planning uses. The paper isolates error allocation, proposes DRPE, and finds that decision-relevant error tracks planning quality more closely than total error.

1 INTRODUCTION

  • Prediction accuracy is widely treated as evidence of a better world model, although planning agents use only some state dimensions to choose actions.Prior work points to a gap between prediction fidelity and decision value but does not isolate how error allocation affects it.Source: §1 · pp. 1–2
  • DRPE measures multi-step prediction error on state dimensions that affect decisions, linking model error to the information consumed by planning.Known relevance in the controlled environment avoids reliance on a learned relevance estimator.Source: §1 · p. 2
  • Across 55 controlled and learned models, total prediction error is weakly associated with planning success, while DRPE closely follows it; model rankings can also change across tasks at fixed total error.Deeper imagination strengthens the effect as decision-relevant errors accumulate through simulated trajectories.Source: §1 · p. 2
  • Learned models can have similar average prediction error yet show systematic bias on rare events critical to a decision, which may determine planning outcomes.Such events contribute little to total error.Source: §1 · p. 2
  • The paper formalizes sufficient conditions for DRPE to rank models correctly and shows when total prediction error cannot provide a correct ranking.The authors conclude that world models should be evaluated against the state information decisions depend on, not total error alone.Source: §1 · p. 2
  • The iso-error protocol fixes total prediction error while moving error across relevant and irrelevant state dimensions, isolating allocation's effect on planning.The controlled factored gridworld has known relevance by construction.Source: §1 · p. 2

2 RELATED WORK

Related work studies decision-aware model learning and world-model evaluation, but the cited approaches do not isolate error allocation at matched total error. The figure illustrates how errors on relevant versus irrelevant dimensions can produce different behavior.

2 RELATED WORK

  • Figure 1 contrasts oracle-like behavior under irrelevant noise with failure to pick up the key under comparable relevant-dimension noise; model (b) succeeds 97% of the time.The relevant-noise model times out at a total error close to model (b).Source: Figure 1 · p. 3
  • Decision-aware learning methods shape models or representations around value, state abstraction, pessimism, or uncertainty, whereas this work measures error allocation in arbitrary world models.The paper positions its approach as a measurement layer complementary to methods for training decision-aware models.Source: §2 · pp. 2–3
  • World-model evaluation commonly uses fidelity metrics such as FVD and PSNR, while observational benchmark analyses report prediction and decision quality separately.The paper distinguishes its controlled error-allocation evidence from these evaluation approaches.Source: §2 · p. 3

3 PRELIMINARIES: A CONTROLLED LABORATORY FOR WORLD MODEL ERROR

The controlled laboratory uses a factored gridworld with known decision relevance, allowing error effects to be isolated. A common planner evaluates models using only the relevant projection of predicted states.

3 PRELIMINARIES: A CONTROLLED LABORATORY FOR WORLD MODEL ERROR

  • The environment is deterministic except where noted, making model error the only source of quality differences.Source: §3 · p. 3
  • The 12-dimensional state has six decision-relevant variables and six irrelevant variables that never affect reward or relevant dynamics.The irrelevant variables are observable and partly predictable, so models may spend capacity learning them.Source: §3 · p. 3
  • Value sensitivity supplies graded relevance weights: the expected absolute change in optimal value from perturbing one state dimension.The normalized weights are positive for all six relevant dimensions and zero for the six irrelevant ones.Source: §3 · p. 4
  • A standardized planner searches over model rollouts, applies the true sparse reward, and scores leaves using the exact optimal-value table on rounded relevant coordinates.Its heuristic consumes only the relevant projection of each predicted state.Source: §3 · p. 4

4 METHOD

DRPE measures rollout error on decision-relevant dimensions, while the controlled model family varies relevant and irrelevant noise to compare models at similar total error. The theory explains when relevant error determines planning behavior and identifies limits to DRPE’s ranking scope.

4.1 DECISION-RELEVANT PREDICTION ERROR

  • DRPE weights rollout error on relevant dimensions over a state-action distribution, while uniform weights across all dimensions yield total error.The paper evaluates graded ground-truth, binary-mask, and estimated weights, and recommends reporting the per-dimension error profile.Source: §4.1 · p. 4
  • Along iso-total-error contours, planning success varies by up to 60 points as relevant and irrelevant noise allocations change.In Task A both static bits are irrelevant; in Task B c1 selects the active goal and the other goal is a terminal hazard.Source: Figure 2 · p. 5, Figure 3 · p. 5

4.2 CONTROLLED ISO-ERROR MODEL FAMILIES

  • The controlled family injects independent Gaussian noise at separate scales on relevant and irrelevant dimensions, with errors compounding across imagined steps.Binary flags are thresholded and position noise can cross cell boundaries; iso-error matching uses measured total error.Source: §4.2 · p. 5

4.3 THEORETICAL ANALYSIS

  • Near-identical total-error models can have return gaps exceeding any C below C†, while matching relevant predictions across planning support preserves behavior regardless of irrelevant error.The theory therefore makes total error insufficient and irrelevant error provably free under its stated conditions.Source: §4.3 · pp. 5–6
  • Predictions that round to the same relevant grid cells as the true state induce the same greedy action, so error below decision resolution is absorbed.This predicts a flat-then-collapse pattern and explains why shallow planners tolerate relevant error that deeper imagination amplifies.Source: §4.3 · p. 6
  • DRPE ranks models correctly when error structure is fixed, but is not sufficient across bias and noise structures or at event-level granularity.The paper identifies the distribution of errors over decision-relevant events as a target for future metrics.Source: §4.3 · p. 6

5 EXPERIMENTS

Across six experiments, the paper tests how error allocation, task relevance, planning depth, and learned-model failures relate to planning quality. The results show that total error can miss decision-critical failures, motivating evaluations that account for task relevance and critical events.

5 EXPERIMENTS

  • The study reports six experiment groups and evaluates success as reaching the goal within the horizon, with means and uncertainty estimates across seeds and episodes.The groups cover controlled error allocation, model rankings, learned models, relevance weights, task shifts, and depth, horizon, and learned-model failure cases.Source: §5 · p. 6

5.1 MATCHED TOTAL ERROR WITH DIFFERENT DECISION QUALITY

  • 60 percentage points separate the success rates of two models whose total multi-step errors differ by only 1%.Success is constant along fixed-σR rows despite Etot varying by a factor of 14 to 91; below one-cell decision resolution, coordinate rounding absorbs noise.Source: §5.1 · p. 6, Table 2 · p. 7

5.2 DRPE PREDICTS PLANNING QUALITY WHILE TOTAL ERROR DOES NOT

  • ρ = 0.97 across 77 iso-error pairs: changes in DRPE closely track changes in planning success, while total error is a weak predictor across the pooled models.In the controlled relevance sweep, graded and binary relevance each correlate with success at ρ = −0.99; among seven learned variants, both metrics correlate at only +0.43.Source: §5.2 · pp. 6–7, Table 1 · p. 7

5.3 RELEVANCE IS RELATIONAL: SAME MODELS, DIFFERENT TASK

  • Task B reverses the models’ relative performance: c1 noise lowers success to 0.82 ± 0.02, while matched c2 noise leaves it at 0.97 ± 0.01.c1 selects the active goal in Task B but is irrelevant in Task A; task-weighted DRPE identifies the failing model.Source: §5.3 · p. 7, Figure 5 · p. 8

5.4 WHAT AVERAGES MISS: DEPTH, BIAS, AND EVENTS

  • Deeper imagination makes relevant noise more harmful, while irrelevant noise remains harmless across planner depths.A depth-D search evaluates 4^D leaves; accumulated relevant error can make the planner evaluate its heuristic at the wrong cells.Source: §5.4 · p. 7, Figure 6 · p. 8
  • A model trained on 12k uniform-random transitions misses key pickup, predicts carrying as 0.19 below the 0.5 door threshold, and succeeds 0% of the time despite Etot matching models with 97% success.With balanced data, event-weighted training cuts pickup and door-transition error threefold and yields 0.79 success versus 0.75 for uniform training; dimension-level relevance weighting does not help.Source: §5.4 · pp. 8–9

5.5 AN EVALUATION PROTOCOL

  • The evaluation protocol recommends reporting task-relevant error profiles, task-conditional DRPE, matched-total-error comparisons, and errors on critical events.These practices address cases where scalar averages obscure task-specific performance or event-level failures.Source: §5.5 · p. 9

6 CONCLUSION

The paper proposes DRPE and an iso-error protocol to evaluate whether error allocation matters when total error is fixed. It concludes that models should be judged by errors reaching decisions, while noting limits in setting, planner, and relevance knowledge.

6 CONCLUSION

  • DRPE scores prediction error on state dimensions a decision depends on, alongside an iso-error protocol that varies allocation at fixed total error.Source: §6 · p. 9
  • A 1% total-error difference accompanied a 60-point success gap, and model rankings could reverse across tasks at the same total error.The paper proposes varying error allocation while holding total error fixed.Source: §6 · pp. 9–10
  • The 12-dimensional gridworld leaves transfer to pixel-scale models untested; the exact V* heuristic may also lead other planners to weight errors differently.DRPE additionally requires prior relevance knowledge, and exploratory data-based estimation failed.Source: §6 · p. 10

A SUPPLEMENTARY

The supplementary material clarifies the conditions, evaluation protocol, and limitations behind the paper’s claims. It reports sensitivity to relevance masks, a failed relevance-estimation approach, comparisons with value-prediction error, and extensions to correlated errors and pixel-scale settings.

Scope of the propositions

  • The propositions apply when a planner reads only relevant prediction components; if it also reads irrelevant components, the stated guarantee no longer directly applies.For practical planners, relevance should include every feature they consume, including features read by value or reward heads.Source: A SUPPLEMENTARY · pp. 12–13

Experiments and protocol

  • The evaluation uses held-out H=10 teacher-forced rollouts, averages squared error over dimensions and steps, and reports behavior-policy rather than planner-action errors.The shared planner and environment are used across six experiments; controlled and learned model families differ in error structure.Source: A SUPPLEMENTARY · p. 13

What the results do and do not establish

  • Total prediction error is not monotone in decision quality in practice, while a binary relevance mask matches graded ground-truth relevance; event-level error structure remains a blind spot for DRPE.The supplementary material says profiles can reveal failures at the event level.Source: A SUPPLEMENTARY · pp. 13–14

Learned-model results

  • Within the seven learned variants, neither total error nor DRPE reliably ranks models; both correlations of +0.43 are driven by data-budget confounds.The pooled decoupling result is dominated by the controlled family, whose error allocation is known by construction.Source: A SUPPLEMENTARY · p. 14

Relevance estimation

  • Exploratory relevance weights placed 39%–42% of their mass on irrelevant dimensions and yielded DRPE–success ρ = −0.24, matching total error.The authors conclude that DRPE is practical when relevance is structurally known or cheaply estimated; reliable automatic estimation remains open.Source: A SUPPLEMENTARY · p. 14

Approximate relevance masks

  • False positives in a relevance mask are more damaging than false negatives: adding irrelevant c1 changed DRPE–success ρ from −0.98 to −0.45.Dropping relevant dimensions left the correlation at −0.98 in the controlled setting because relevant-dimension errors remained proportional.Source: A SUPPLEMENTARY · p. 14, Table 4 · p. 14

Value-prediction error

  • On the 48-model grid, VPE matched DRPE at ρ = −0.98, but requires the task’s exact V* and shares the blindness of averages.The authors describe VPE as a more demanding oracle than a relevance mask.Source: A SUPPLEMENTARY · p. 14

Correlated errors and extensions

  • With AR(1) noise, irrelevant-dimension noise remained harmless and relevant-dimension noise harmful, though the relevant-noise dose response was not monotone.The authors attribute partial absorption of persistent offsets to replanning from the true state at every step; pixel-scale evaluation would require relevance over learned representations and planner-consumed features.Source: A SUPPLEMENTARY · pp. 13, 15

B PROOFS

The proofs establish when matching relevant predictions preserves decisions and show that near-matched total error can coexist with a large return gap. The supplementary sections specify relevance weights and experimental procedures, and illustrate an event-level failure in a learned model.

B PROOFS

  • Identical predictions on relevant coordinates over the planning support yield identical closed-loop policies and returns when reward and dynamics depend only on those coordinates.The proof shows the planner’s action scores depend on predictions only through the relevant projection.Source: B PROOFS · p. 15
  • Total errors of 0.151 and 0.152 accompanied success falling from 0.97 to 0.37 when noise shifted onto relevant dimensions.Proposition 2 establishes that arbitrarily close total errors can accompany a return gap greater than any C below C† for a suitable task and models.Source: B PROOFS · p. 15
  • When predictions round to the same relevant grid cells as true states along planned paths, the greedy actions match the zero-error model’s actions.Errors smaller than the distance to a cell boundary leave the rounded input unchanged; deeper rollouts provide more chances for errors to cross a boundary.Source: B PROOFS · pp. 15–16

C GROUND TRUTH RELEVANCE WEIGHTS

  • The six relevant dimensions receive normalized graded weights from the exact value function, while the six irrelevant dimensions receive zero weight.The graded profile is reported for completeness; a binary mask over the same dimensions performs identically.Source: C GROUND TRUTH RELEVANCE WEIGHTS · p. 16, Table 5 · p. 16

D EXPERIMENTAL DETAILS

  • The 8×8 gridworld uses an 8×8 map with a wall at x=5, a single door, and a 60-step horizon; Task B makes c1 select a goal or terminal hazard.Exact value iteration and the full-width expectimax planner use the same standardized evaluation setup.Source: D EXPERIMENTAL DETAILS · p. 16
  • The controlled grid uses depth 4 with 10 seeds and 30 episodes, while the relevance sweep uses depth 4 with 4 seeds and 30 episodes.The compute setup runs on CPU and all experiments take under two hours in total.Source: D EXPERIMENTAL DETAILS · p. 16

E ADDITIONAL QUALITATIVE RESULTS

  • A uniform-random-trained model oscillated for 60 steps without reaching the key or goal, despite small errors on every dimension.It failed to predict carrying above the 0.5 threshold at pickup; the controlled model succeeded 97% of the time while the learned model never reached the goal.Source: E ADDITIONAL QUALITATIVE RESULTS · p. 17, Figure 8 · p. 17

F FULL ISO-ERROR GRID

The full iso-error grid makes the paper’s matched-total-error comparison directly checkable. A companion figure shows that models with matched total error can occupy both ends of the planning-performance range.

F FULL ISO-ERROR GRID

  • Models whose total errors agree within 1% differ in success by 60 points in the example rows (0, 0.3) and (0.35, 0.1).The full sweep covers 48 model types, with 10 seeds and 30 episodes each.Source: F FULL ISO-ERROR GRID · p. 17
  • Reading down each column block, success changes only when σR changes.Source: F FULL ISO-ERROR GRID · p. 17
  • At matched total error, the grid contains both the best and worst planners, while coloring by σR resolves the cloud.Source: F FULL ISO-ERROR GRID · p. 17

G PER-DIMENSION ERROR PROFILES

Per-dimension error profiles expose differences that total error conceals: models with matched total error can have very different planning success, and average error can miss failures at rare events.

G PER-DIMENSION ERROR PROFILES

  • At matched total prediction error, both the best and worst planners appear; error allocation separates them.Figure 9 colors models by irrelevant-dimension noise, showing that total error alone does not distinguish planning outcomes.Source: Figure 9 · p. 18
  • The irrelevant-noise model is exact on all six relevant dimensions, while the relevant-noise model is exact on irrelevant dimensions.The two controlled models differ in where their error sits, not simply in its presence.Source: G PER-DIMENSION ERROR PROFILES · pp. 17–18
  • The random-data model’s carrying MSE (0.070) is below the balanced model’s (0.080), but it almost never reaches the pickup event whose transition it failed to learn; closed-loop results locate failure there, not in the average.Source: G PER-DIMENSION ERROR PROFILES · p. 18

H TASK B TRAJECTORIES ACROSS SEEDS

Task B failures arise when corrupted c1 beliefs misdirect planning or prevent commitment; across model families, deeper imagination magnifies divergence from the true path for relevant-dimension noise.

H TASK B TRAJECTORIES ACROSS SEEDS

  • 0.82 was the c1-noise model’s Task B success rate: 35 of 240 episodes ended at the hazard and 8 timed out.In three illustrated failures it chose the inactive goal and walked into the hazard; in one it circled until the horizon expired.Source: H TASK B TRAJECTORIES ACROSS SEEDS · p. 18, Figure 10 · p. 19

I IMAGINATION FANS ACROSS MODEL FAMILIES

J STRONGEST ISO-ERROR PAIRS

Among the largest success gaps at matched total error, every pair contrasts irrelevant-only noise with relevant-dimension noise, and the DRPE gap tracks the success gap’s direction.

J STRONGEST ISO-ERROR PAIRS

Related papers

Loading 2609.32322v2…