PaperlayerView exact revision on arXiv

Source-linked AI summary

MEND: Label-Free Detection, Localisation, and Correction of Latent Hallucination in World Models

Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam, James Bailey, Naveed Akhtar

arXiv:2609.39182v1cs.CVcs.AI

TL;DR

Latent world-model predictions can decode to scenes that never occur while remaining numerically plausible, making hallucinations silent and compounding in autoregressive rollouts. MEND uses one conditional score field for label-free detection, localisation, and inference-time correction; on two navigation environments, it detects hallucinations without actions and localises errors, while correction remains bounded by manifold-tangent error.

  • Problem

    Latent world-model predictions can decode to scenes that never occur while remaining numerically plausible, making hallucinations silent and compounding in autoregressive rollouts.

    Source: §I · p. 1

  • Method

    MEND uses one conditional score field to detect and localise hallucinations and supply an inference-time correction direction, without error labels.

    Source: §I · p. 1, 2) Conditional score network · p. 4, 3) One field, three roles · p. 5

  • Results

    Up to 0.80 detection AUROC without actions exceeded a single-Gaussian density baseline, while per-token AUPRC reached up to 0.87.

    Source: §I · p. 2

  • Takeaways & Limitations

    Detection and localisation are solid across two environments, while correction reliably reduces latent error but is bounded by error along the data manifold.

    Source: §VIII · p. 7

    The approach assumes hallucinations lie off-manifold, and whether representation-level correction gains transfer to a downstream controller remains unknown.

    Source: §VIII · p. 7

Abstract

from arXiv · show

World Models are appearing as the next major frontier in computer vision. However, their robustness is currently largely unexplored. We identify the phenomenon of hallucination in latent World Models: given a state and an action, the predicted next latent can decode to a scene that never occurs. Because the prediction is statistically ordinary and is fed back autoregressively by the model, the error is both silent and compounding. We study whether such latent hallucination can be detected, localised, and corrected at inference time, on a frozen self-supervised world model in the absence of ground-truth error labels. We introduce Masked Empirical-Bayes Neural Denoising (MEND), a single conditional score network trained by denoising score matching on real transitions, whose score field serves three roles: its magnitude detects hallucination, its per-token field localises it to specific image patches, and it defines an inference-time correction direction. On two navigation environments MEND detects hallucination with an AUROC of up to 0.80 without using actions, exceeding a single-Gaussian density baseline while also localising the error (per-token AUPRC up to 0.87) and correcting it, all from one score field. Our correction reliably reduces single-step latent error and improves predictions. We identify that a part of the error is tangent to the data manifold, hence, we focus on detection and localisation while highlighting promises of the correction.

I. INTRODUCTION

Latent world models can make plausible predictions that decode to scenes that never occur, and autoregressive rollouts can compound these silent errors. This paper introduces MEND, a label-free detect–localise–correct approach, and reports detection and localisation results across two environments.

I. INTRODUCTION

  • Because a hallucinated latent is fed back as autoregressive context, its effect on rollout accuracy grows with the imagination horizon.The latent can remain statistically plausible while decoding to a scene that never happens.Source: §I · p. 1
  • Evaluation has focused mainly on downstream task performance, while hallucination detection and correction inside world models remain largely unexplored.Source: §I · p. 1
  • MEND uses one conditional score field to detect hallucination, localise errors to image patches, and provide an inference-time correction direction.The paper presents correction as a natural extension alongside its stronger detection and localisation findings.Source: §I · p. 1
  • The paper characterises latent hallucination as spatially sparse and proposes a label-free detect, localise, and correct loop instantiated by MEND.Source: §I · p. 1
  • Up to 0.80 detection AUROC without actions and per-token AUPRC up to 0.87 accompany performance above a single-Gaussian density baseline.The reported detection and localisation results cover two environments and are robust to the labelling threshold.Source: §I · p. 2

II. RELATED WORK

Related work places MEND among latent world models and score-based approaches to prediction confidence and restoration. The paper defines hallucination by reachability under the true dynamics, extends that definition to rollouts, and reports that errors concentrate in a small subset of tokens.

II. RELATED WORK

  • MEND targets latent world models without accuracy labels, computing confidence directly from predicted latents, unlike the supervised per-patch confidence probe for video generation C3 [14].The related work frames this as a first application of hallucination detection to latent world models.Source: §II · p. 2

III. DEFINITION OF LATENT HALLUCINATION

  • A predicted next latent is hallucinated when it lies outside the states reachable under the true transition, regardless of prediction error.The environments studied are deterministic, so the reachable successor under a given action reduces to a single point.Source: §III · p. 2
  • A rollout is hallucination-free only when every predicted state is reachable at its corresponding step; a single hallucinated step reduces the entire rollout’s probability and errors accumulate with depth.Source: §III · p. 2
  • The most suspicious fifth of tokens accounts for roughly a third of single-step error and over 80 percent during deep free-running rollouts.This concentration makes per-token localisation and masked correction well suited to the studied WALL and POINTMAZE environments.Source: §III · p. 3

IV. METHOD

The detect–localise–correct loop offers inference-time mitigation when retraining is unavailable or prohibitively expensive. A differentiable validity score can unify detection, token selection, and masked correction while leaving the rest of the latent unchanged.

A. Detect, localise, and correct loop

  • Inference-time mitigation leaves the world model unchanged when retraining or fine-tuning is unavailable or prohibitively expensive.The alternative is to collect detected failures and use them to fine-tune the model, which requires a data-collection pipeline and computational budget.Source: A. Detect, localise, and correct loop · p. 3
  • The loop instantiates a general detect–localise–correct framework whose detector may combine models or heuristic indicators, provided the score is differentiable for localisation and correction.Source: A. Detect, localise, and correct loop · pp. 3–4
  • A differentiable validity score can detect hallucination, use its gradient to identify suspicious tokens, and provide a correction direction for those tokens.The approach avoids a separate attribution mechanism by using the score gradient for localisation.Source: A. Detect, localise, and correct loop · p. 3
  • Each iteration scores the prediction, selects suspicious tokens, applies a masked correction, and re-evaluates until validity or the iteration budget is reached.The mask trades greater error coverage against unnecessary changes to regions that are already correct.Source: A. Detect, localise, and correct loop · p. 3

B. Masked Empirical-Bayes Neural Denoising

MEND instantiates the mitigation loop with the score of a learned conditional density over next states. The method uses this single differentiable object as its unified detector, localiser, and corrector.

B. Masked Empirical-Bayes Neural Denoising

1) Mathematical basis:

The Bayes factorisation separates next-state reachability from action consistency, yielding two complementary detector terms. In the deterministic simulators, the framework uses reachability alone after finding little added benefit from the action term.

Mathematical basis

  • The Bayes factorisation makes the next-state score proportional to reachability and action consistency, because the action likelihood given the current state is constant across candidate predictions.The terms are D1, p(ẑt+1 | zt), and D2, p(at | zt, ẑt+1).Source: 1) Mathematical basis · p. 4
  • The factors describe environment dynamics and can be estimated from logged transitions without hallucination or accuracy labels, making the detectors self-supervised and world-model-agnostic.They assign higher scores to predictions likely under true environment dynamics and lower scores to likely hallucinations.Source: 1) Mathematical basis · p. 4
  • D1 tests whether a predicted state is reachable from the current state, while D2 tests whether the observed action explains the transition.D1 is sensitive to type S hallucinations and D2 to type A.Source: 1) Mathematical basis · p. 4
  • In the deterministic simulators, D1 alone provides strong detection and localisation, so the framework uses it while evaluating D2 as a principled ablation.The relative usefulness of the terms depends on the world model’s inductive biases.Source: 1) Mathematical basis · p. 4

2) Conditional score network:

MEND estimates the score of the conditional density of reachable next states, using a network trained on real transitions with denoising score matching. The learned score applies to predicted states without assuming their errors are Gaussian corruptions.

Conditional score network

  • MEND treats a hallucinated prediction as a valid successor plus an error and learns a conditional score pointing toward greater validity.The network is conditioned on the current latent and a Gaussian smoothing noise scale.Source: 2) Conditional score network · p. 4
  • Denoising score matching trains on real next states corrupted by Gaussian noise, with the network predicting the score of each corrupted sample.The regression target is −ε/σ, and σ is sampled from a geometric noise schedule.Source: 2) Conditional score network · p. 4
  • The training corruption ε is distinct from hallucination error e, and the population optimum is the score of the smoothed conditional density of valid next states.This learns the score directly without requiring the density’s normalising constant.Source: 2) Conditional score network · p. 4
  • Because the network sees only real transitions, it models environment dynamics rather than a particular predictor, and Gaussian noise acts as smoothing rather than an error assumption.The score identity holds at any input, including a predicted state not formed by adding Gaussian noise.Source: 2) Conditional score network · pp. 4–5

3) One field, three roles:

One score field supplies detection, token-level localisation, and a correction direction. Correction iteratively updates a fixed support of suspicious tokens while anchoring edits to the original prediction.

One field, three roles

  • A single score evaluation yields both a scalar detection score and a per-token localisation map, while correction reuses the same network.The three roles therefore come from one score field rather than separate components.Source: 3) One field, three roles · p. 5
  • The score magnitude measures distance from reachable states, its per-token magnitude estimates how far each token must move, and Tweedie’s formula makes it a correction direction.The per-token estimate requires no attribution method or backward pass.Source: 3) One field, three roles · p. 5
  • For points away from the reachable set, the small-noise score points toward the nearest valid neighbour with magnitude r/σ^2.Detection reads vector length, localisation its per-token parts, and correction follows the vector.Source: 3) One field, three roles · p. 5
  • Correction fixes the top-p% most-displaced tokens as support, moves them along the score, and uses a proximal term to keep edits near the original prediction.Larger correction noise scales are destructive, and correction must be reapplied at each rollout step because a single correction may not persist.Source: 3) One field, three roles · p. 5

V. EXPERIMENTAL SETUP

The experiments evaluate frozen DINO-WM checkpoints in WALL and POINTMAZE, using held-out trajectories and an error-based proxy for correct versus incorrect predictions. Detection, localisation, and correction are assessed with separate metrics and comparison baselines.

Experimental setup

  • Experiments use public frozen DINO-WM checkpoints with a DINOv2 ViT-S/14 encoder and a small predictor in WALL and POINTMAZE.The encoder has 196 tokens and dimension 384; WALL uses one history frame and POINTMAZE uses three.Source: §V · p. 5
  • Held-out trajectory predictions are split into balanced correct and incorrect sets at the median per-token error, defining the organic-error calibration set.In these deterministic environments, per-token prediction error against the realised next latent serves as a proxy for reachability; stochastic dynamics would require reachability-based labels to avoid counting valid alternatives as errors.Source: §V · p. 5
  • The inference loop scores each prediction, ranks token displacements, freezes the top-p% support on its first iteration, and returns the lowest-score iterate.It stops when the score falls below τ or the update magnitude falls below δ.Source: §V · p. 5
  • Detection uses AUROC, localisation uses per-token AUPRC against the ground-truth error map with a 0.50 random baseline, and correction uses latent-error reduction and fraction improved.Detection is compared with a single diagonal-Gaussian density baseline, inverse-model detection, and action-conditioned score variants.Source: §V · p. 5
  • The four-layer token Transformer has 10.3M parameters and is trained by denoising score matching on roughly 500 held-out trajectories.Detection uses σ = 0.39; correction uses σ = 0.05, η = 0.3, ρ = 0.1, and K = 10 Tweedie steps, with scales above 0.08 destructive.Source: §V · p. 5

VI. DETECTION AND LOCALISATION RESULTS

MEND’s detection and localisation are related but distinct: detection varies with environment, while token-level localisation remains strong, including in deep rollouts.

A. Across environments

  • MEND substantially outperforms the single-Gaussian density floor on WALL and improves on it in POINTMAZE; POINTMAZE localisation reaches per-token AUPRC 0.87.Table I compares MEND with a diagonal-Gaussian density reference.Source: Table I · p. 6, A. Across environments · p. 6
  • Detection is strongest with sharp, nearly deterministic WALL dynamics, while POINTMAZE uncertainty lowers detection AUROC even as concentrated errors support accurate localisation.Detection and localisation need not improve together.Source: A. Across environments · pp. 5–6
  • MEND’s differentiable score field provides a global detection score, dense per-token localisation map, and inference-time correction direction; the single-Gaussian baseline provides detection only.Detection remains stable across labelling thresholds and saturates after a few hundred training trajectories.Source: A. Across environments · p. 6

B. Localisation examples

  • In deep free-running POINTMAZE rollouts, the score field concentrates on the erroneous tokens even after the imagined state has drifted far off the manifold.The figure compares decoded ground truth and prediction with ground-truth token error and the D1 score field.Source: B. Localisation examples · p. 6, Figure 3 · p. 6
  • Localisation is robust to rollout depth: as total error grows, per-token AUPRC barely changes, and the same localisation signal is used by correction.Equivalent deep-rollout examples on WALL are reported in the supplementary material.Source: B. Localisation examples · p. 6

VII. ABLATION STUDY AND DISCUSSION

In these deterministic environments, action-consistency and direct action conditioning add little to D1 detection; the authors relate this to how the signals are constructed.

Ablation study

  • D2 performs at approximately chance level, while direct action conditioning matches but does not exceed D1 detection.The detector variants are ablated on WALL in Table II.Source: §VII · p. 6, Table II · p. 7

Discussion

  • Because the predictor is already action-conditioned, hallucinated predictions can still reconstruct the observed action accurately, leaving the inverse-dynamics residual weakly related to validity.This explains why D2 can be a weak detection signal despite accurate action reconstruction.Source: §VII · p. 6
  • The weak D2 result is specific to deterministic environments; action-consistency is expected to help in stochastic or multimodal settings where reachability alone cannot identify the correct transition.D2 distinguishes reachable futures that differ by the action that produced them.Source: §VII · pp. 6–7
  • Directly conditioning the score network on action offers no benefit and can let it use action as a shortcut; classifier-free guidance recovers the action-free model’s performance but does not improve it.The correction table reports single-step relative latent-error change, with negative values indicating improvement.Source: §VII · p. 7, Table III · p. 7

A. Correction results

MEND’s score-field correction reduces latent error, but recovery is limited by error tangent to the data manifold. The conclusion presents detection and localisation as stronger findings and bounds correction’s demonstrated scope.

A. Correction results

  • The Tweedie update reuses the per-token score field, moving suspicious tokens more and confident tokens less, so an explicit mask is unnecessary.The correction edits the full latent.Source: A. Correction results · p. 7
  • Only error normal to the reachable set is recoverable; tangent error is an aleatoric floor, making reductions modest and near-manifold errors the easiest to correct.The authors frame correction as a feasibility result rather than a solved problem.Source: A. Correction results · p. 7
  • Per-step correction sustains a roughly constant reduction in latent error with rollout depth, whereas a first-step correction washes out.Table IV compares mean per-token latent error across rollout depths; lower is better, and parentheses show reduction relative to no correction.Source: Table IV · p. 7, A. Correction results · p. 7

VIII. CONCLUSION

  • Across two environments, detection and localisation are label-free and robust to labelling threshold and data budget, while correction reliably reduces latent error.The same signal may guide targeted data collection and direct model repair.Source: §VIII · p. 7
  • The study assumes hallucinations lie off-manifold, an assumption reported for the navigation environments, and leaves stochastic dynamics and downstream-controller transfer open.Whether representation-level correction gains transfer to a controller remains future work.Source: §VIII · p. 7
Loading 2609.39182v1…