Hallucination in World Models is Predictable and Preventable

Nicklas Hansen; Xiaolong Wang

Source-linked AI summary

Hallucination in World Models is Predictable and Preventable

Nicklas Hansen, Xiaolong Wang

arXiv:2606.27326v1Jun 25, 2026cs.LGcs.CVcs.RO

TL;DR

World models can produce visually plausible rollouts that drift from ground-truth dynamics, but where and why hallucinations occur remains poorly understood. This paper introduces MMBench2, characterizes three hallucination modes, develops runtime predictors and coverage-based interventions, and finds that predictors track rollout error with ρ ≈0.80 while enabling adaptation to unseen environments with 50 real trajectories.

Problem
Where and why world-model rollouts hallucinate remains poorly understood, limiting understanding of failures that drift from ground-truth dynamics despite visual plausibility.
Method
The paper introduces MMBench2 and characterizes three hallucination modes, then uses runtime predictors for coverage-aware training and targeted online data collection.
Results
The predictors track rollout error with ρ ≈0.80, while coverage-aware training reduces all three failure modes and targeted collection adapts the model to unseen environments with 50 trajectories.
Takeaways & Limitations
The findings support data coverage as a central lens for predicting and mitigating hallucination in generative world models.
Takeaways & Limitations
Whether the findings transfer to billion-parameter models or real robot data with sensor noise and partial observability remains an open empirical question.

Abstract

from arXiv · show

Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates in low-coverage regions of the state-action space, where lightweight data-centric signals can both detect it and guide mitigation. To test this, we introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators, and train a 350M-parameter world model on it. We identify three distinct hallucination modes: perceptual, action-marginalized, and scene-diverging -- each anchored to a different stage of the pipeline, and develop three signals that accurately predict where the model will fail. To close coverage gaps at training time, we develop a coverage-aware sampling technique; to close them online, our hallucination predictors serve as curiosity rewards for targeted data collection, yielding a data-efficient finetuning recipe that adapts the pretrained world model to entirely unseen environments with as few as 50 real environment trajectories. Overall, our findings reveal that hallucination in world models is inherently a data coverage issue, and that the same signals used to detect it can also be used for mitigation. An interactive web version of our paper is available at https://www.nicklashansen.com/mmbench2

1 Introduction

The paper argues that world-model hallucination is primarily a data-coverage problem concentrated in low-coverage state-action regions, rather than merely an architectural limitation. Using MMBench2, it characterizes three failure modes, develops label-free predictors, and uses coverage-aware training and targeted online collection to mitigate them.

Hallucinated rollouts remain visually fluent and superficially plausible while drifting from ground-truth dynamics.
Hallucination is argued to concentrate in low-coverage regions of the state-action space, motivating data-centric detection and mitigation.
The proposed approach combines three label-free hallucination predictors with coverage-aware pretraining and targeted online data collection to reduce hallucination and close coverage gaps.
The study identifies three failure modes: perceptual hallucination in the encoder-decoder, action marginalization in the dynamics model, and scene divergence during multi-step rollouts.
MMBench2 provides 427 hours of visual world-modeling data with ground-truth actions, rewards, live environments, and behaviorally diverse data.

2 MMBench2: A Dataset for Visual World Modeling

MMBench2 is introduced as a large-scale, multitask dataset for visual world modeling that enables controlled study of hallucination through access to training data and processes. It combines 65,600 trajectories across 10 task domains, while its 200-task pretraining corpus remains highly non-uniform in episode length and visual diversity.

Dataset scale and scope: MMBench2 contains 65,600 trajectories totaling 427 hours and 23M frames of 224×224 video recorded at 15 fps.The dataset is designed to provide diverse behaviors for visual world modeling while preserving control over training data and processes.
Dataset scale and scope: The dataset spans 10 task domains covering locomotion, dexterous and tabletop manipulation, goal-conditioned navigation, and arcade behaviors.The domains include DMControl, DMControl Extended, Meta-World, ManiSkill3, MuJoCo, MiniArcade, Box2D, RoboDesk, OGBench, and Continuous Atari.
Coverage characteristics: The 200-task pretraining corpus contains 260 episodes per task, but episode lengths range from 25 to 1,000, producing a highly non-uniform corpus.The shortest episodes occur in certain ManiSkill3 manipulation tasks, while the longest occur in Atari games.
Coverage characteristics: Visual diversity also varies substantially across tasks, from relatively uniform Inverted Pendulum observations to more diverse Dungeon Explorer and Bipedal Walker environments.This variation is identified as an additional source of non-uniformity across the pretraining corpus.

3 Training a Large Visual World Model

The paper trains a 350M-parameter action-conditioned visual world model using a two-stage tokenizer–dynamics architecture inspired by Dreamer 4. It pretrains on MMBench2 and leverages ground-truth actions and rewards available for every trajectory before targeted finetuning experiments.

Architecture and training overview: The model is a 350M-parameter action-conditioned generative world model trained with a two-stage recipe: video-tokenizer pretraining followed by dynamics-model training.Its overall architecture and training recipe largely follow Dreamer 4.
Training corpus: MMBench2 contains 20M frames across 210 tasks with a heavy-tailed task distribution.The top 20 tasks account for 26% of frames, while the bottom 20 contribute 0.7%; the per-task median is 65k frames.
Tokenizer: The video tokenizer uses a symmetric encoder-decoder Transformer that maps 224×224 RGB frames into 64-dimensional bounded latent codes.Frames are patchified at stride 14 into 256 patch tokens, with 64 learnable latent queries; the latent stream is projected through a tanh-bounded bottleneck.
Dynamics model: The dynamics model is a 250M-parameter block-causal Transformer operating over packed tokenizer latents and conditioning on actions.Each timestep includes an action token, shortcut-conditioning token, 32 packed spatial latent tokens, 4 register tokens, and optional agent tokens for reward and behavior-cloning heads.
Training recipe: Training first pretrains the tokenizer on the full corpus and then pretrains the action-conditioned dynamics model before targeted finetuning experiments.Unlike the setting motivating Dreamer 4, MMBench2 supplies ground-truth actions and rewards for every trajectory, which the training procedure uses.

4 Hallucination in World Models

Hallucination arises when one of the world model’s sequential learned stages extrapolates beyond its finite state-action coverage, producing fluent outputs decoupled from reality. The paper distinguishes three stage-specific failure modes, develops label-free predictors for them, and frames coverage as the common target for mitigation.

Failure taxonomy: The encoder, action-conditioned dynamics head, and decoder can each fail independently outside the finite state-action regions represented in training.Because these stages compose sequentially, early hallucinations propagate and amplify through later stages.
Failure taxonomy: Three hallucination modes target disjoint pipeline components: perceptual, action-marginalized, and scene-diverging failures.They probe the tokenizer, action-conditioning of dynamics, and multi-step dynamics-error accumulation, respectively.
Failure taxonomy: Perceptual hallucination distorts observations before rollout, action-marginalized hallucination produces action-insensitive plausible futures, and scene-diverging hallucination predicts physically implausible events.Scene-diverging hallucination is most frequent in poorly covered states; action sensitivity can be tested by shuffling actions and measuring flow MSE.
Hallucination predictors: Three mechanistically distinct, label-free predictors strongly track hallucination events: tokenizer round-trip residual, flow instability, and inter-seed variance.They measure tokenizer-manifold failure, unstable dynamics conditioning, and disagreement across denoising seeds, respectively.
Hallucination predictors: Dynamism-normalized variants are the primary predictors because scene activity inflates raw tokenizer residuals, flow instability, and seed-to-seed dispersion.Normalization divides each signal by the per-step RMS latent change, using a per-task average or an online running estimate.
Coverage-based mitigation: The three failure modes are unified as coverage gaps in reconstruction, action-conditional transitions, or trajectory regions, motivating coverage-aware resampling and targeted curiosity-driven collection.The predictors can score candidate model rollouts so the highest-ranked trajectory is executed in the live environment to obtain data covering the gap.

5 Experiments

Experiments evaluate a 350M-parameter RGB world model through hallucination detection, coverage-aware training, and targeted finetuning on unseen tasks. The results show that hallucination predictors track rollout error, coverage-aware sampling improves the model, and curiosity-guided data collection enables efficient adaptation.

Experimental setup: The study trains a 350M-parameter action-conditioned world model on approximately 20M RGB frames from 200 continuous-control tasks, reserving 3M frames for testing.Experiments strictly use RGB observations despite available low-dimensional state information.
Hallucination detection: The three hallucination predictors strongly track open-loop rollout error, achieving Spearman correlation ρ ≈0.80 with rollout ∆PSNR and outperforming unnormalized and latent-motion or frame-count baselines.The predictors consistently outperform their unnormalized counterparts and baselines based on latent scene motion or per-task frame count.
Coverage-aware training: Coverage-aware sampling benefits both the tokenizer and dynamics model, with applying it to both achieving the best overall performance across held-out trajectories from 200 tasks.The comparison extends pretraining by 30k steps for each component and contrasts uniform frame sampling with uniform task sampling.
Targeted data collection: Zero-shot transfer to unseen tasks reaches 0.276, or 2.3× the 0.118 random policy baseline, while 50 curiosity-guided trajectories per task raise performance to 0.325.Curiosity-guided finetuning reaches within ∼90% of the 0.362 expert/human oracle without privileged behavior.
Conclusions and limitations: The experiments support data coverage as the primary source of hallucination, identifying perceptual, action-marginalization, and scene-divergence failures mitigated by coverage-aware sampling and curiosity rewards.The study operates at the 350M-parameter scale across 210 simulated control tasks, leaving transfer to billion-parameter models and real robot data open.

A Related Work

Related work spans world models for control, hallucination and uncertainty estimation in generative models, and coverage-aware training and data collection. The paper situates its contribution within these areas while emphasizing comparatively limited study of rollout failure in autoregressive world models.

World models for control: World models for control range from abstract latent dynamics models to high-capacity generators that render full pixel observations, with recent work scaling to heterogeneous video corpora.This literature spans model-based reinforcement learning and increasingly large video-based generative models.
Hallucination and uncertainty in generative models: Hallucination is well studied in language, image, and video generation, but comparatively little work examines where autoregressive world models fail along rollouts.Related work also includes uncertainty estimation in deep networks through deep ensembles.
Coverage-aware training and data collection: Coverage-aware training and data collection research treats data scale and composition as first-order drivers of generative-model performance and coverage as a constraint on offline-RL policy improvement.This motivates curated offline-RL datasets and connects to curiosity-driven exploration.

B Comparison to Existing Datasets

MMBench2 is positioned against datasets spanning offline reinforcement learning, robot imitation learning, and large-scale generative video and world modeling. It combines mixed-quality data across more tasks and domains with ground-truth actions, rewards, and live environments.

Comparison scope: Table 4 compares MMBench2 with representative datasets for offline reinforcement learning, robot imitation learning, and generative video and world modeling.The offline RL comparison includes RL Unplugged, V-D4RL, ExoRL, the TD-MPC2 multi-task dataset, and Atari DQN Replay.
Dataset characteristics: MMBench2 spans more tasks and domains than the existing datasets summarized in Table 4.The comparison describes its data as mixed-quality and broad in task and domain coverage.
Dataset characteristics: MMBench2 uniquely combines ground-truth action and reward labels with live environments in the comparison.The table notes that some existing datasets provide only binary trajectory success flags, pseudo-label actions, or estimated scale.

C Task Domains

MMBench2 spans 210 tasks across 10 domains, covering diverse continuous-control settings with varied task complexity, horizons, observation and action dimensions, and reward formulations.

Task coverage: 210 tasks span 10 domains, including robot manipulation, locomotion, navigation, arcade games, and classic control problems.The domains comprise diverse continuous-control tasks.
Task coverage: The task set varies in complexity, time horizon, observation and action space dimensionality, and reward formulation.Table 5 summarizes this breadth across task types and task specifications.
Finetuning task sets: Finetuning experiments use 10 seen tasks from pretraining and 10 held-out unseen tasks developed specifically for MMBench2.The two task sets are listed in Table 6.

D Data Collection

MMBench2 expands expert-only collection into a large, diverse dataset for visual world modeling by combining augmented expert behavior with non-expert and human data. Collection spans 210 live environments and uses behavior switching to explore varied state-action regions while retaining ground-truth labels.

Data Collection: MMBench2 builds on MMBench’s 4k expert demonstrations but adds behavioral diversity to support visual world modeling and hallucination research.MMBench provides live environments with ground-truth action and reward labels, while expert-only demonstrations lack behavioral diversity.
Data Collection: The collection combines random, no-op, expert, transformed-expert, hallucination-seeking, and human-play policies with distinct diversity and task-performance tradeoffs.Random actions are diverse but perform poorly; no-op actions model dynamics without agent interference; expert actions perform well but provide low diversity; human play is diverse but not task-driven.
Data Collection: Behaviors are frequently switched within episodes, using structured noise to visit infrequent state-action regions before returning to expert behavior for recovery.Human play data is the exception to this switching strategy.
Data Collection: 1,400 trajectories are collected through a keyboard-based web interface for human interaction data.The saved interaction data is later used for world model training.

E Additional Results

Additional experiments show that the proposed predictors reliably identify hallucination events, while coverage-aware collection is evaluated across more tasks and finetuning settings. Reward finetuning does not produce a significant difference in results.

Hallucination detection: The three proposed predictors reliably detect action-ignored and scene-diverging hallucinations on held-out data from all 200 pretraining tasks.Table 7 evaluates per-task AUROC, with higher values indicating better prediction.
Data coverage: State-density comparisons examine no-op, random, expert, curiosity, and human data collection across three additional tasks.These results extend the coverage analysis beyond the tasks shown in Figure 6.
Targeted finetuning: Targeted data collection is evaluated for finetuning on 10 unseen tasks using 50 trajectories per task, with separate expert-test and human-play evaluations.The experiments vary data source and finetuning strategy on 10 seen plus 10 unseen tasks.
Targeted finetuning: Targeted data collection is also evaluated for finetuning on 10 seen tasks using 50 trajectories per task and closed-loop MPC performance.Task performance is normalized to [0, 1] across three episodes per task.
Reward finetuning: Reward finetuning produces no significant difference in results compared with reward-free control when both variants add 30k dynamics steps.The comparison averages results over all 200 pretraining tasks on a held-out test set.

F Implementation Details

The implementation reproduces Dreamer 4 with frozen CLIP task embeddings, a block-causal Transformer, modality-aware token processing, spatial packing, and shortcut flow matching. Separate reward and behavior-cloning heads operate on agent tokens, with defined inference and planning schedules.

Overview: The world model reproduces the original Dreamer 4 method while documenting implementation and design choices.The paper identifies this section as an overview of its reproduction and design decisions.
Language embeddings: Task conditioning uses frozen 512-dimensional CLIP embeddings, conditioning only the reward prediction and BC policy heads.The embeddings come from openai/clip-vit-base-patch32 and represent per-task language instructions.
Transformer backbone: Each Transformer combines space self-attention, causal time self-attention, and a SiLU-gated MLP with ratio 4.Attention uses RoPE, QK-normalization, and RMSNorm pre-normalization without normalization-layer biases.
Spatial packing: Spatial packing transforms per-frame latents from (nL, db) = (64, 64) to (nspatial, dspatial) = (32, 128), halving spatial attention cost.The packing factor is k=2, doubling channel dimension, and inverse unpacking is applied during decoding.
Reward and BC heads: The reward and BC heads pool agent-token representations, predicting L=8 reward distributions and a deterministic Gaussian policy over the ground-truth 16-dimensional padded action.The reward head uses 255 bins over [−10, 10], while the policy is trained with MSE.
Sampling and planning: Inference uses a shortcut Euler integrator with step size d=0.125, while CEM planning uses horizon H=32 and replans every K=16.The CEM planner runs 3 iterations with population size 32 and 2 rollouts per candidate.