Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by Trust

• World models support planning by feeding predicted states back into the model, but extrapolation beyond experience can compound prediction errors along imagined trajectories.Source: §1 · p. 2
• Predictive entropy, variance, and ensemble disagreement can remain insensitive to unfamiliar inputs or become more confident as reliability deteriorates.Source: §1 · p. 2
• Doubt concerns experiential support for a state-action pair, while trajectory trust must also account for uncertainty inherited from preceding imagined transitions.Source: §1 · p. 2
• LucidWM uses transition-head evidence to derive doubt, compounds its complement into trajectory trust, and uses trust to reweight imagined returns and guide action selection.It requires no additional learnable parameters or forward passes for uncertainty estimation.Source: §1 · p. 2
• Across four world-model backbones and seventeen uncertainty readouts, LucidWM detects environmental changes and rollout drift; it alarms before all 20 evaluated falls and reaches the goal in 190 steps versus 362 for the base model.Source: §1 · p. 3

• A world model encodes observations as latent states, predicts next states from state-action pairs, and trains its transition head to match a posterior informed by the arriving observation.Imagination feeds sampled predictions back as inputs for future steps and uses λ-returns to train actor and critic.Source: §2 · p. 3
• Subjective Logic represents beliefs over outcomes with a Dirichlet whose concentration combines prior weight and observed evidence, separating uncertainty from the predicted mean.The uncertainty mass is the prior’s share of total concentration and falls as evidence accumulates.Source: §2 · pp. 3–4
• Existing world-model uncertainty readouts use predictive distributions, frozen-feature scores, or disagreement among models, whereas this paper reads evidence behind each imagined step and carries doubt along the rollout.Source: §2 · p. 4

3 Method

• The method replaces a standard world model’s fixed prediction mix and fixed λ-return share with transition doubt and trust derived from the transition head’s logits.Source: §3 · p. 4

3.1 Learning doubt from experience

• Standard heads encode constant doubt ε, equivalent to asserting the same evidence total for every transition, while LucidWM lets evidence strength and doubt vary by transition.For ε = 0.01 and W = 2, the standard head’s implied evidence total is S = 198.Source: §3.1 · p. 4
• An evidence discipline lets unsupported evidence recede toward a vacuous opinion, while fused observation evidence sharpens the posterior and constrains the prediction’s doubt.The dynamics loss alone leaves total evidence unconstrained; cumulative fusion adds observation evidence without double-counting prediction evidence.Source: §3.1 · pp. 4–5

3.2 Carrying trust through imagination

• LucidWM converts transition doubt into step trust and uses λ multiplied by trust as the share of imagined future retained in the return.A doubtful step passes less imagined future to the return and more to the critic.Source: §3.2 · p. 5
• Trust compounds multiplicatively along a rollout, reducing the weight of returns beyond doubtful steps and making the return fall back on the critic as model experience weakens.Source: §3.2 · p. 5

3.3 Deciding by trust

• Trust-weighted returns preserve critic convergence to the imagined-model value, while weighting each depth’s model error by the share of return reaching that depth.The error bound tightens wherever doubt precedes error.Source: §3.3 · p. 6
• For action selection, LucidWM scores candidate futures by trust-weighted rewards, so a course whose trust collapses can be vetoed rather than winning on the critic’s estimate.Source: §3.3 · p. 6

4.2 Main results

• Across every base, LucidWM recognizes the opened room after observation and its doubt rises under corrupted imagined actions, while the unmodified-base readout often moves little or the wrong way.The tests report AUROC for opened-room versus familiar frames and lift under corrupted versus true actions, respectively.Source: §4.2 · pp. 7–8, Table 1 · p. 7

4.3 Doubt rises beyond experience

• Within ten rollout steps, LucidWM’s median doubt crosses the experienced-state 95th percentile; by the end, nine in ten rollouts exceed it, while base doubt stays flat.Both models’ random-action rollouts drift equally far beyond the experienced-state cloud.Source: §4.3 · p. 8
• In both altered rooms, LucidWM raises an alarm while the base readout raises none, including when the opened room uses only familiar wall textures.Doubt rises when the agent faces the opening and falls when it turns away.Source: §4.3 · p. 8, Figure 5 · p. 8

4.4 Doubt falls as experience grows

• At one imagined step, doubt halves as pixel error falls nearly 25-fold across checkpoints; at chaotic step 24, doubt remains highest at every checkpoint.The pixel error and transition-evidence doubt are measured independently.Source: §4.4 · p. 9, Figure 6 · p. 9

4.5 Ablating trust

• Trust removes 11.0% of error within each depth across walker, cheetah, finger, and cartpole, more than any other tested readout.The reported average spans ten runs.Source: Figure 7 · p. 9

4.6 Acting on trust

• In the repainted dead end, trust’s veto redirects the agent to the goal at step 190, versus step 362 for the same policy ignoring doubt.The veto fires after doubt remains above its alarm line for three frames; 143 of 145 vetoes on both altered maps occur inside changed rooms.Source: §4.6 · p. 9, Figure 8 · p. 10

5 Conclusion

• LucidWM makes a standard world model a special case by replacing fixed doubt and return constants with quantities derived from transition-head logits, without adding parameters or forward passes.The conclusion says doubt compounds along imagination into trust that weights learning and decisions.Source: §5 · p. 10, B What LucidWM changes · p. 14, Table 3 · p. 15

C Why the doubt reads experience

• The appendix’s argument proceeds from notation and model changes to evidence-based doubt, joint experience counts, and the prediction-and-fit identifiability limit.The listed results are organized as successive steps in the appendix.Source: A Notation · p. 14, C Why the doubt reads experience · p. 15, Table 4 · p. 15

C.1 The doubt measures the evidence total

• Doubt is the prior’s share of the prediction and depends on evidence through its total; it equals one with no evidence and falls strictly as evidence grows.Observed counts add to the evidence through posterior fusion.Source: §C.1 · p. 15

C.2 In a table, the doubt counts experience

• For a tabular model, doubt depends on the joint scene–action count: a pair never experienced has doubt one, even if its scene and action were seen separately.No count from another pair contributes to that pair’s evidence.Source: §C.2 · p. 16

C.3 The prediction and the fit cannot see the evidence total

• For a fixed predictive mean, multiple evidence totals yield the same readouts and fit, so prediction-based quantities cannot identify how much evidence supports a transition.The passage characterizes this as non-identifiability when the loss sees only the predictive mean.Source: §C.3 · pp. 16–17, Figure 9 · p. 16

C.4 The discipline makes the doubt count experience

• The discipline grows with evidence total while the fit depends on the predictive mean, so a minimizer takes the smallest total allowed by its mean.The resulting opinion has its least likely class holding no evidence.Source: §C.4 · p. 17
• With full-support targets and positive βu, every loss minimizer chooses the most doubtful opinion compatible with its mean; without experience doubt is one, and with experience it is at least K min_i q_i.For non-uniform q, the excess over K min_i q_i is Θ(βu/N) as βu/N approaches zero.Source: §C.4 · p. 17
• Compared with regularized evidential learners whose uncertainty is set by regularization weight, this discipline uses the weight only for an excess of order βu/N above a target-fixed level.For a never-experienced pair, the example returns doubt one.Source: §C.4 · p. 18
• For shared network parameters, the discipline lowers evidence where no transition supports it, while the fit rewards directions that generalize across pairs.The lucid head requires no density model for this network setting.Source: §C.4 · pp. 17–18

C.5 The doubt sets the expected error

• For a fixed predicted mean, expected error rises with doubt; the same prediction has eighteen times lower expected error at S = 198 than at S = 8.The example reports expected errors of 0.0024 at S = 198 and 0.044 at S = 8.Source: §C.5 · p. 18

D Why trust is carried in the return

• Carrying doubt as trust in the return preserves uncertainty from earlier imagined steps, whereas carrying it through the imagined state can lose that lineage.The paper presents return-level trust as the reason for carrying doubt in the return.Source: D Why trust is carried in the return · pp. 18–19

D.1 Observation never raises the doubt

• Cumulative fusion never raises doubt and moves the predicted mean toward the observation in proportion to the doubt it removes.A vacuous observation leaves the prediction unchanged; in the example, fusion lowers doubt from 0.2 and 0.33 to 0.14 and shifts the mean 29% toward the observed evidence.Source: §D.1 · p. 19

D.2 In the state, the lineage is lost

• When |σ| is small, state-based deduction forgets an earlier step’s doubt after later imagined steps.With |σk| ≤ 0.01 in the eight-step example, states 5–8 return to doubt 0.01 despite doubtful transitions at steps 3 and 4.Source: §D.2 · pp. 19–20

D.3 In the return, the doubt compounds

• Return-level trust compounds as Cn = the product of step trusts, carrying a doubtful transition’s effect into every later imagined step.In the example, carried trust is 0.61 at step 3 and 0.37 from step 4 onward, while state deduction loses that lineage.Source: §D.3 · p. 20
• Figure 10 contrasts trust deduced through states with trust carried in the return, and compares their one- to eight-step return weights.The caption identifies σ = 0.009 for the state-trust example.Source: Figure 10 · p. 20

D.4 The lucid return keeps the fixed point and discounts model error

• The lucid return mixes n-step returns with nonnegative weights that sum to one, and the total weight on returns longer than n is Mn.With τ ≡ 1, the weights reduce to the standard geometric λ-return weights.Source: §D.4 · pp. 20–21
• With λ = 0.95, the eight-step return’s weight falls from 0.70 under the λ-return to 0.26 under the lucid return.The three- and four-step returns gain 0.34 and 0.18, respectively.Source: §D.4 · p. 21
• The lucid-return critic update is a γ-contraction to the value of the policy in the imagined model.The theorem also states a bias bound relative to the true value, with depth-dependent model error terms.Source: §D.4 · p. 21

D.5 When trust tightens the bound

• Lowering a step’s trust tightens the bias bound when the γjMj-weighted mean of downstream model errors exceeds (1 − γ)∆v.With the critic at its true value, the bound discounts each depth’s error by trust lost before that depth; a doubtful step discounts later errors, not earlier ones.Source: §D.5 · p. 22

D.6 Doubt ranks actions before acting

• Before acting, doubt ranks candidate actions inversely by their evidential strength: greater doubt corresponds exactly to lower evidence above the floor ε.The doubt is computed from the state-action pair alone, before the outcome is observed.Source: §D.6 · p. 22

E.1 Protocols

• Maze tests compare a sealed training map with opened and repainted maps, while all trained agents still reach the goal in 100 test episodes.The repainted dead end changes paint in its deep half while keeping every wall fixed.Source: §E.1 · p. 22
• For corrupted-action tests, imagined actions are randomly replaced at a set probability, and readout lift compares fully corrupted with true-action values.Maze walks are scripted from agent position alone, so every readout sees the same frames.Source: §E.1 · pp. 22–23

E.2 Four bases

• The four bases use ε = 0.01 and λ = 0.95, with Table 6 reporting categorical-state sizes, prior weights, and imagination horizons.Source: Table 6 · p. 23, §E.2 · p. 23

E.3 Every experiment at a glance; E.4 Seventeen readouts and their cost

• The comparison covers seventeen readouts, with each evaluated on the same frames and imagined futures and oriented so higher values mean less familiarity.The readout set includes single-pass signals, fitted heads, and methods requiring multiple forwards.Source: Table 7 · p. 23, §E.4 · p. 24, Table 8 · p. 24

F.1 The alarm fires in the new room; F.4 The alarm fires on every base

• LucidWM alarms in the new room on 25 of 29 walks and on eight of ten further walks, while the base misses the first set and several alternatives miss the latter.The doubt crosses its alarm line on all four tested bases.Source: Figure 11 · p. 24, §F.1 · pp. 24–25, Figure 12 · p. 25

F.2 The alarm fires before the fall; F.3 The alarm fires at nightfall

• For twenty consecutive cheetah falls, the doubt alarms a median of four steps ahead while same-prediction entropy alarms on none; at Crafter nightfall, doubt crosses its line within 25 steps.The Crafter base readout never reaches half of its own alarm line.Source: Figure 13 · p. 25, §F.2 · p. 26, Figure 14 · p. 26

F.5 The worse the actions, the higher the doubt

• As action corruption increases, doubt rises over fivefold on OC-STORM walker and over twofold on Crafter, and over threefold on EMERALD walker and cheetah from one pass.On EMERALD, MC dropout, Laplace, and snapshot ensemble instead fall despite up to twenty passes.Source: §F.5 · p. 26, Figure 16 · p. 27

F.6 The alarm fires at any line, any checkpoint

• In the repainted room, doubt alarms under all 28 tested rules, while the base alarms under four; across 38 checkpoints, doubt ranks opened-room frames above familiar ones at every checkpoint.The base ranks the opened-room frames below chance at every checkpoint.Source: Figure 17 · p. 27, §F.6 · p. 27

F.7 The doubt drains as the model learns

• As the model learns, doubt declines at one step and across 24 imagined steps, while same-prediction entropy averaged over those steps rises from 0.79 to 1.01.From 48k to 180k environment steps, one-step doubt falls from 0.194 to 0.093 and 24-step average doubt from 0.181 to 0.096.Source: §F.7 · p. 27, Figure 18 · p. 28

F.8 Trust fades faster where the dream drifts

• When imagined futures drift from reality, walker pixel error grows almost fivefold, while Crafter rollout trust reaches 0.16 against 0.42 for calm starts at step 6.Source: Figure 19 · p. 28, §F.8 · p. 28

F.9 Trust keeps its edge across ten runs

• Across ten runs and four tasks, trust removes 11.0% of model error within depth, more than the compared readouts, and does so on each run.The comparison averages walker, cheetah, finger, and cartpole results.Source: §F.9 · pp. 28–29, Figure 20 · p. 29

F.10 Hyperparameter analysis

• On cheetah from pixels, error removed rises from 18.7% at trust window w = 1 to 31.5% across all 15 steps; walker return stays between 924 and 949 as βu varies.The walker range holds for βu from 3 × 10^-4 to 10^-2.Source: §F.10 · p. 29

F.11 Choosing futures by trust cuts deaths

• Across 50 seeds, trust-guided selection reduces deaths from 4.08 to 3.80 per 1,000 steps and raises median life from 236 to 253 steps.Trust changes the selected future at 43% of steps; both scoring methods evaluate the same imagined futures.Source: Figure 21 · p. 29, §F.11 · p. 30

F.12 LucidWM learns Crafter faster than its base

• The trust-guided Crafter agent reaches evaluation return 8 after 0.35M steps versus 0.96M for its base, and scores 15.6% versus 9.0%.It also unlocks 11.0 achievements per episode against 8.9 for the base over the last 100 evaluation episodes.Source: §F.12 · p. 30

F.13 Vetoes fire where the world changed

• Of 145 vetoes on two altered maps, 143 fall inside changed rooms: all 127 in the opened room and 16 of 18 in the repainted dead end.Source: Figure 22 · p. 30, §F.13 · p. 30