PaperlayerView exact revision on arXiv

Source-linked AI summary

Where World Models Break: Natural-Input Failure Discovery

Zhanpeng Shi, Zi Liang, Rong Feng, Shiqin Tang, Xuyang Chen, Hongzong Li

arXiv:2608.22421v1cs.AI

TL;DR

World-model evaluations can miss severe prediction failures on rare but environment-valid conditions and action prefixes. The paper formalizes finite-budget natural-input failure discovery and proposes BASINLENS, which combines uncertainty-guided global search with typed local replacements. Across multiple world-model families and interfaces, BASINLENS finds failures with held-out and neighborhood evidence, while typed structure is most useful in heterogeneous interacting spaces and standard search remains competitive in simpler ones.

  • Problem

    Existing aggregate evaluations provide limited evidence about severe world-model prediction failures under rare or unobserved valid condition-action combinations.

  • Method

    BASINLENS combines uncertainty-guided global proposals with typed local replacements over environment-valid inputs under a finite query budget.

  • Results

    54.7% of BASINLENS runs found a fixed-threshold failure at budget 64 on a 12,016-candidate PushT grid, versus 40.6% for GP-UCB and 10.9% for random search.

  • Takeaways & Limitations

    The results support using independent held-out and neighborhood evidence to turn hidden world-model weaknesses into inspectable failure records, with typed search most useful for heterogeneous interacting inputs.

  • Takeaways & Limitations

    Cross-model, task, and interface factors vary together, so the observed settings where typed structure helps are scope observations rather than controlled causal comparisons.

Abstract

from arXiv · show

World models predict action-conditioned futures and serve as critical internal simulators for downstream planning and control. However, catastrophic prediction failures of world models could dangerously propagate through the control pipeline, as subsequent agent or model training and decision-making depend heavily on the continuous environment evolution forecasted by these world models. Existing evaluations overlook this systemic risk: by aggregating average errors over benign generations from general queries, they fail to stress-test the model against catastrophic collapses under rare or unobserved condition-action combinations. To bridge this gap, we formalize the natural-input failure discovery problem: under a finite query budget, finding environment-valid conditions and action prefixes that induce severe prediction risk, verifying whether these failures reproduce on fresh seeds, and testing their persistence under nearby valid edits. Discovering such critical failures is computationally challenging, as valid condition-action combinations explode exponentially, rendering exhaustive search or standard sampling infeasible given the high cost of noisy rollouts. To tackle this, we propose BasinLens, which exploits the underlying structure of valid inputs, where each coordinate possesses environment-defined semantic types and admissible domains, by pairing uncertainty-guided global search with typed local replacements. Across diverse benchmarks and world-model families, BasinLens exposes reproducible and locally persistent failure modes that conventional evaluations fail to reveal, showing that average-case benchmarks can mask important vulnerabilities in world-model-driven control.

1 Introduction

The paper formalizes natural-input failure discovery to find environment-valid conditions that induce severe world-model prediction risk under finite budgets. BASINLENS combines uncertainty-guided global proposals with typed local replacements, exposing reproducible failures across diverse models and settings.

  • Motivation: Aggregate benchmarks can miss localized forecast errors that change action rankings despite strong average prediction error or task return.The paper motivates evaluating which valid resets or action sequences make forecasts unreliable.
  • Problem: Natural-input failure discovery searches a combinatorial valid-input space under costly, noisy rollouts and requires independent tests for reproducibility and local persistence.The target is high-risk resets or action prefixes within a finite query budget.
  • Results: At budget 64 on a 12,016-candidate PushT grid, discovery occurs in 54.7% of BASINLENS runs, versus 40.6% for GP-UCB and 10.9% for random search.The GP-UCB comparison at this threshold is descriptive.
  • Approach: BASINLENS combines uncertainty-guided global proposals with valid typed replacements to return executable, inspectable test cases.Typed structure uses environment-defined coordinate semantics and admissible domains.
  • Results: Across five world-model families and four prediction interfaces, the study exposes valid failures and identifies settings where typed structure helps or standard search remains competitive.Selected PushT cases are independently verified.

2 Problem Formulation

The formulation treats failure discovery as finite-budget search over executable, environment-valid resets or action prefixes, with prediction risk aggregated across evaluation seeds. It separately defines held-out point validation and fresh valid-neighborhood evaluation rather than estimating deployment prevalence.

  • Inputs: A valid natural input specifies an environment-valid condition, such as reset variables, a goal, or an action prefix, with edits that preserve validity.Action prefixes execute before the fixed rollout policy continues.
  • Risk: Prediction risk compares world-model forecasts with environment-derived future targets using a nonnegative discrepancy and a prespecified aggregation operator.Aggregation may use the mean, maximum, or upper-tail CVaR.
  • Evidence: Held-out evaluation changes seeds while preserving the input, whereas neighborhood evaluation applies valid local edits on fresh seeds without updating search.These protocols distinguish pointwise reproducibility from local persistence.
  • Budgeted discovery: An adaptive search evaluates at most B valid inputs and returns a shortlist for inspection rather than estimating average risk under a deployment distribution.Only shortlisted candidates receive held-out validation.
  • Scope: Validity guarantees executability and prespecified checks, not prevalence; finite libraries define reproducible search domains rather than deployment failure frequencies.Risk values are specific to prediction interfaces, seeds, and validity rules.

3 A Structured Implementation: BASINLENS

BASINLENS structures budgeted search by combining distant global-surrogate proposals with typed local replacement frontiers. It freezes the search trace before independent held-out and neighborhood evaluation, producing inspectable failure records.

  • Global exploration: Global surrogate proposals can reach candidates far from current anchors, complementing frontier expansion that may remain locally confined.The surrogate uses normalized typed coordinates and GP-UCB scores.
  • Typed frontier: The typed frontier replaces one coordinate of a high-scoring anchor with another admissible value, preserving executable candidates.Exploration uses broader replacements, while persistence uses adjacent-grid edits.
  • Acquisition: BASINLENS merges global and typed-local candidate pools, then ranks them with acquisition terms for local evidence, proximity, and novelty.Fixed acquisition settings are used within the reported protocols.
  • Design principle: Normalization provides numerical coordinates for global modeling, while typed edits define meaningful environment-valid changes without requiring gradients or a differentiable simulator.Each edit maps to a named field and admissible value.
  • Evidence records: Each discovery record stores typed values, seeds, per-seed losses, activity measurements, ranking scores, and the ordered query trace for reproducible inspection.Validation is attached only after the query trace and shortlist are frozen.
  • Analysis variants: Component-isolation variants test whether coordinate interactions or multi-anchor expansion explain gains under different input structures.BASINLENS-INTERACTION adds pairwise features, while BASINLENS-BEAM expands multiple anchors without a global GP-UCB refresh.

4 Experimental Setup

The experiments evaluate BASINLENS across multiple world-model families, tasks, prediction interfaces, candidate libraries, and protocol-specific comparisons. Risk measures and validation results are interpreted within their respective protocols because prediction spaces differ.

  • Models and tasks: The study uses LeWM on PushT and TwoRooms, with additional checks using DINO-WM, JEPA-WM, DIAMOND, and IRIS across other tasks.These settings span PushT, PointMaze, and Atari.
  • Prediction measures: LeWM PushT risk is upper-tail CVaR of final-step state-probe MSE across evaluation seeds, with task-activity penalties applied during online acquisition.The prediction setup observes three frames and predicts three frames from a 40-step rollout.
  • Readout analysis: In the displayed TwoRooms case, future-position MAE rises from 13.2 to 89.0 pixels while probe MAE changes from 1.13 to 1.66 pixels.The evaluation uses a latent-to-state probe rather than the optional RGB decoder.
  • Protocols: The experiments separate online discovery, held-out point validation, fresh-neighborhood evaluation, component isolation, and cross-model scope studies.Cached libraries include 8,177 agent-start candidates, 3,928 valid inputs, and 431 valid combinatorial combinations.
  • Budgets and libraries: The main online grid contains 12,016 valid candidates and uses 64 paired restarts, eight search seeds per candidate, and budgets of 16, 32, and 64.A separate validation library uses 16 disjoint validation seeds.
  • Comparisons: Baselines include random search, GP-UCB without a typed frontier, and cached CEM comparisons matched on budgets, libraries, and restart streams.Protocol-specific held-out references and local controls support the comparisons.

5 Results

BASINLENS finds high-risk, environment-valid inputs more effectively than random search, with selected failures reproducing on held-out seeds and persisting in nearby valid neighborhoods. Results across world-model families show that typed structure helps most in combinatorial, semantically heterogeneous spaces, while standard search can remain competitive elsewhere.

  • Discovery under a Finite Budget: 54.7% of budget-64 BASINLENS runs crossed Reval ≥1.60, versus 40.6% for GP-UCB and 10.9% for random search.The comparison with GP-UCB was descriptive at this fixed threshold, while the comparison against random search was statistically significant.
  • Discovery under a Finite Budget: At budget 64, BASINLENS achieved mean best search score 1.572, compared with 1.547 for GP-UCB and 1.512 for random search.Threshold-averaged discovery rates were 0.572, 0.474, and 0.255, respectively, across thresholds from 1.50 to 1.65.
  • Reproducibility and Local Persistence: Held-out prediction CVaR was 1.311 for 24 complete BASINLENS selections versus 1.186 for the default reset on the same disjoint seeds.This protocol establishes reproducibility but does not compare shortlist quality across methods.
  • Reproducibility and Local Persistence: Fresh radius-1 neighborhoods around top-score anchors reached mean prediction CVaR 1.252, versus 0.608 around mid-score controls and 0.546 around random controls.The independent tests treated anchors as independent units and both had p ≤6.99×10−4.
  • What the Discovered Failures Reveal: Changing only the valid agent start raised PushT’s predicted final block–goal distance ratio from 0.941 to 1.110 across 64 paired seeds, higher in 98.4% of pairs.Qualitative cases included temporal collapse in TwoRooms, prediction stalling in PushT, and a missed wall-opening transition in DINO-WM.
  • When Does Typed Structure Help?: Typed search variants led on the cached combinatorial grid, while GP-UCB led on the single-factor grid; across models, typed expansion was most useful with semantic coordinate distinctions and interactions.BASINLENS-BEAM and BASINLENS-INTERACTION reached top-5% hit rates of 0.963 and 0.961, whereas GP-UCB reached 0.875 on the single-factor grid.

6 Related Work

The paper situates natural-input failure discovery among testing, falsification, adversarial, and black-box search methods, while distinguishing its target: fixed-model prediction failures under valid inputs.

  • World-model planning spans latent-space, discrete-token, pixel, and pretrained visual-feature interfaces, motivating interface-aware evaluation.
  • Existing testing and falsification methods search coverage, behavioral transformations, scenarios, requirement violations, or agent failures rather than fixed-model prediction failures.
  • Table 2 positions the paper relative to adjacent evaluation settings, noting that individual methods may combine multiple elements.
  • Adversarial methods target classifiers, policies, sensed objects, physical-conditioning channels, or imagined-trajectory rankings, whereas this work searches valid inputs for prediction failures.

7 Conclusion

The paper concludes that natural-input failure discovery complements aggregate benchmarks by locating valid, reproducible, locally persistent prediction failures through structured search and independent evaluation.

  • Natural-input failure discovery formalizes finite-budget search for environment-valid resets or action prefixes causing reproducible, locally persistent prediction failures.
  • BASINLENS combines global proposals with typed local replacements and returns a ranked discovery record for held-out and neighborhood inspection.
  • The experiments use separate protocols for discovery, validation, neighborhood risk, component isolation, cross-model scope, and controller consequences.
  • Legal reset variables define executable tests, while the environment supplies the matched future used to evaluate the fixed world model.
  • The workflow does not observe an oracle risk landscape; it identifies finite discovery records rather than enumerating all high-risk regions.

C BASINLENS Implementation

BASINLENS implements budgeted search over a fixed valid library by combining global surrogate proposals with typed local replacements, then freezing the trace before follow-up evaluation.

  • Typed locality: Each coordinate is normalized over the valid library, while the search frontier uses valid one-coordinate replacements and radius-1 validation uses adjacent grid levels.
  • Model-based search: The implementation combines a Gaussian-process surrogate with fixed kernel and noise settings for LeWM and DINO-WM online protocols.
  • Complete procedure: The protocol constructs and verifies the typed library, creates paired scouts, evaluates unseen candidates, builds pools, ranks their union, and returns a shortlist.
  • Workflow: Figure B.1 depicts legal-input execution, alternating global and typed-local search, and held-out or nearby-valid evaluation; its contours are schematic.
  • Evaluation separation: Follow-up evaluation uses separate seeds and does not update acquisition or rerank the completed search trace.

D Protocol Details

The protocol details define model-specific prediction risks, typed valid libraries, disjoint seed partitions, and independent point and neighborhood evaluations across several world-model interfaces.

  • Models and risks: The study evaluates LeWM on PushT and extends the formulation to DINO-WM, JEPA-WM, DIAMOND, and IRIS using each interface’s prediction target.
  • Validity and locality: Validity is environment-level, and radius-1 locality is stricter than arbitrary valid typed replacement; categorical action templates use typed replacement without geometric radius.
  • Seed design: Search, point validation, and neighborhood evaluation use disjoint environment-seed sets, while restart seeds control scout order and acquisition ties.
  • Candidate libraries: The PushT libraries retain only tuples satisfying agent–block and block–goal distance constraints, including 12,016 candidates from 15,625 Cartesian combinations.
  • Risk and activity: Reported task-active best scores equal unpenalized CVaR, with task activity determined by zero normalized violations for progress, contacts, and path length.
  • Validation groups: Top-score anchors, mid-score controls, and random controls are formed from deduplicated task-active records and evaluated with candidate-level CVaR.
  • Scope: The LeWM probe diagnostic does not establish uniform accuracy on every searched latent, so results concern the complete frozen model-to-state-readout pipeline.

F Consolidated Quantitative Ledger

The consolidated ledger separates protocol-specific online search endpoints rather than combining incompatible measurements. It distinguishes mean-best scores from threshold crossing, with held-out and neighborhood risks reported separately.

  • The detailed tables above provide the consolidated endpoint ledger for existing run-level records.
  • The ledger preserves separate budgets, thresholds, and paired comparisons for the principal online PushT searches.Its purpose is to provide a single endpoint view without averaging across incompatible protocols.
  • Mean-best search score ranks methods within one candidate library, while threshold crossing makes rare-event discovery explicit.Held-out point risk and fresh-neighborhood risk are follow-up measurements, not additional online-search endpoints.

G Qualitative Failure Cases

The qualitative cases show that valid starts and configurations can produce sharply divergent predicted futures despite similar or active environment rollouts. These examples illustrate temporal and topological failures while quantitative validation remains protocol-specific.

  • TwoRooms: The selected TwoRooms start reaches 89.0-pixel future-position MAE versus 13.2 pixels for the reference start.Its final target-distance gap is 149.3 pixels versus 27.2 pixels, while probe MAE remains 1.66 versus 1.13 pixels.
  • PushT examples: The DINO-WM searched PushT case has mean horizon MSE 3.074 versus 1.105 for the displayed lower-risk case.Both matched rollouts are active, with 73 and 72 contact steps respectively.
  • Probe diagnostic: Figure D.1 is an illustrative LeWM PushT probe diagnostic and is not counted as an additional statistical sample.
  • Validation protocol: The 576-candidate reference is the simulator reset without a geometry override, reevaluated on the same held-out seeds as the selected candidate.Incomplete validation availability is reported rather than filled by reusing search seeds.
  • Wall: Ninety-two of 900 prioritized DINO-WM Wall inputs satisfy the visible no-cross criterion.The result is a scoped qualitative failure bank, not a distribution-wide frequency estimate or dominance claim.
  • PushT examples: The selected PushT start has final-step MSE 2.086 versus 0.935, 0.937, and 1.065 for three comparison starts.The panel is qualitative; held-out and neighborhood protocols provide independent quantitative evidence.

G.1 What the Discovered Cases Reveal

The discovered cases reveal temporal and topological prediction failures that average benchmark scores do not localize. The accompanying protocols separate search efficiency from reproducibility, neighborhood persistence, and downstream controller consequences.

  • Failure patterns: Temporal failures map requested later futures to early or weakly changing predicted states while the environment continues evolving.
  • Failure patterns: Topological failures keep predicted futures on the wrong side of a traversable opening.
  • Search endpoints: At budget 64, threshold discovery reaches 54.7% for BASINLENS, 40.6% for GP-UCB, and 10.9% for random search.The paired GP-UCB comparison at this threshold is descriptive.
  • Statistical comparisons: BASINLENS and comparator searches are evaluated as paired runs, with exact sign tests removing ties from directional comparisons.
  • Controller consequence: MPC compares model-ranked and environment-oracle-ranked controllers using identical proposal seeds and a shared state-conditioned proposal generator.
  • Statistical comparisons: Threshold crossing uses discordant paired restarts, while threshold-averaged discovery averages prespecified hit indicators within each run.
  • Neighborhood persistence: Fresh-neighborhood comparisons use eight anchor-level neighborhood means per group, retaining candidate overlap because anchors are the inferential units.
  • Qualitative comparison: Figure G.1 contrasts a selected high-error start whose prediction collapses after the early horizon with a reference start that tracks the requested future.

M Interpretation of Cross-Model Results

Cross-model results show that natural-input failure discovery transfers across world models when executable inputs and comparable prediction targets are available. However, typed search is not uniformly strongest, and the main validation evidence remains concentrated in LeWM PushT.

  • Scope: Natural-input failure discovery applies whenever a world model exposes an executable input specification and a prediction target comparable with the environment.
  • Method behavior: DINO-WM and JEPA-WM provide positive cases for typed expansion, whereas Atari studies show random search or UCB can be stronger.The paper treats the discovered failure and its input condition as the central object, not a universal optimizer leaderboard.
  • Evidence boundaries: The main validation claim is tied to LeWM PushT, where selected points and neighborhoods use disjoint seeds.DINO-WM supplies method-level replication but not the same held-out neighborhood protocol; TwoRooms and Wall provide interpretable temporal and topological views.
  • Reproducibility: The anonymous supplement provides entry points and record schemas for reproducing discovery and validation results, while upstream checkpoints remain externally maintained.
  • Reporting: A complete discovery result records the input condition, failed prediction quantity, and evidence against invalid-input or reused-noise artifacts.The search algorithm is only one part of that record.
  • Interpretation: A high search score alone does not establish reproducibility or local persistence, and a comparator win is not required to return a valid high-risk case.
  • Reporting: The reporting fields separate discovery efficiency from evidence about the returned condition.
Loading 2608.22421v1…