PaperlayerView exact revision on arXiv

Source-linked AI summary

Attacking the Trusted Imagination: Oracle-Level Integrity Attacks on Imagine-then-Act World Models

Linghan Chen, Kaiyan Ji, Minyu Guo

arXiv:2606.22966v1cs.LGcs.AIcs.CR

TL;DR

The paper examines whether downstream oracles that trust WAM-imagined futures remain robust when the imagination, rather than the reactive action pathway, is attacked. The attacker applies true PGD through a differentiable observation-to-imagination map, targeting either off-manifold corruption or a chosen imagination, alongside parameter-free denoiser detection. At ϵ=0.01, adversarial perturbations collapse success to 0.05 versus random noise at 0.70, with adversary-specific failure also observed in native imagination-driven MPC.

  • Problem

    The paper examines whether downstream oracles that trust WAM-imagined futures remain robust when the imagination, rather than the reactive action pathway, is attacked.

    Source: §1 · pp. 1–2

  • Method

    The attacker applies true PGD through a differentiable observation-to-imagination map, targeting either off-manifold corruption or a chosen imagination, alongside parameter-free denoiser detection.

    Source: §1 · p. 2, §4 · p. 4, §5.6 · p. 8

  • Results

    At ϵ=0.01, adversarial perturbations collapse success to 0.05 versus random noise at 0.70, with adversary-specific failure also observed in native imagination-driven MPC.

    Source: §1 · p. 2, §5.9 · p. 9

  • Takeaways & Limitations

    Reactive-policy robustness does not establish robustness for planners, verifiers, or simulators that treat WAM imagination as a trusted future.

    Source: §1 · p. 2, §7 · p. 11, §9 · p. 12

    The evaluation is simulation-only, limited in closed-loop scale, and insufficient for universality claims without a larger multi-task study.

    Source: §8 · p. 11

Abstract

from arXiv · show

Many recent vision-language-action (VLA) policies adopt an imagine-then-act design. A world-action model (WAM) first imagines a short future as a latent trajectory z~, on which the action is then conditioned. We identify this trusted imagination, rather than the reactive policy, as the exposed attack surface. A downstream oracle, such as a safety gate, a visual model-predictive-control (MPC) planner, or an imagine-then-check verifier, consumes z~ as a prediction of the future. The robustness of the policy therefore does not entail the robustness of systems that rely on the WAM. The underlying phenomenon is an asymmetry. Corrupting the imagination is easy, since it requires only displacing z~ from its natural-future manifold. Steering it precisely is hard, since it must reach a specified on-manifold target. We adopt a capability-based threat model with an L-infinity-bounded observation perturbation. The attacker applies projected gradient descent through the fully differentiable observation-to-imagination map. The same off-manifold property motivates a parameter-free denoiser detector. We evaluate three targets: RynnVLA-002, LingBot-VA, and LaDi-WM. Untargeted corruption is roughly 60x stronger than random and is detected at AUC 1.0. Targeted control remains bounded. An adaptive attacker evades detection only by forgoing corruption. The reactive policy remains robust to corrupted imagination. A native imagination-driven MPC, however, exhibits the first adversary-specific task failure (at epsilon=0.01, success 0.70 versus 0.05; Fisher p < 10^-4).

1 Introduction

Imagine-then-act systems expose a trusted-imagination attack surface distinct from the reactive action pathway. The paper argues that imagination is easy to corrupt, hard to steer precisely, detectable off-manifold, and consequential mainly for downstream oracles rather than reactive policies.

  • Downstream oracles trust imagined futures without real-world corrective feedback, so corrupting the imagination directly corrupts their decisions.Source: §1 · pp. 1–2
  • A continuous, differentiable observation→imagination map enables the first white-box integrity attack against trusted WAM imagination using true PGD.Source: §1 · p. 2
  • Untargeted corruption is easy, whereas targeted steering is hard, with the asymmetry consistent across targets and supported by decoded-frame evidence.Source: §1 · p. 2
  • A parameter-free denoiser self-consistency detector reaches AUC 1.0, and adaptive attackers evade detection only by surrendering corruption.Source: §1 · p. 2
  • Reactive policy robustness does not imply oracle robustness: the policy remains robust to corrupted imagination, while the native imagination-driven MPC exhibits an adversary-specific task failure.Source: §1 · p. 2, Figure 1 · p. 2

2 Related Work

Prior work studies imagine-then-act policies, action-pathway attacks, and world-model robustness, but not white-box integrity attacks on trusted imagination consumed by downstream oracles. This paper evaluates that distinct threat on LIBERO while distinguishing related simulation benchmarks as out of scope.

  • Imagine-then-act policies condition actions on imagined futures, with RynnVLA-002, LingBot-VA, and LaDi-WM coupling world models to action generation in different architectures.Source: §2 · p. 3
  • Prior robot-policy attacks target observation-to-action pathways and measure action error or task-success drops, whereas this work attacks oracle-consumed imagination.Source: §2 · p. 3
  • JailWAM is a black-box semantic jailbreak, while this work uses a white-box ℓ∞-bounded single-observation attack through a differentiable observation-to-imagination map.Source: §2 · p. 3
  • Robustness studies of world-action models measure natural visual and language perturbations rather than worst-case adversaries, leaving trusted-imagination integrity unaddressed.Source: §2 · p. 3
  • Evaluation uses LIBERO [11], while SimplerEnv [10] is complementary and out of scope.Source: §2 · p. 3

3 Threat Model

The threat model assumes a white-box attacker that perturbs only the current camera observation within an ℓ∞ budget while a downstream oracle trusts the WAM imagination. Harm is assessed at the oracle decision level, with end-to-end closed-loop success instantiated only for visual MPC.

  • The attacker perturbs only the current camera observation within an ℓ∞ budget, knows the WAM and encoder, and cannot modify weights, the oracle, or the true future.Source: §3 · p. 3
  • The threat model measures verifier flips, MPC mis-selection, and trusted-imagination divergence rather than restricting harm to executed-action error.Source: §3 · p. 3
  • The three oracle instantiations differ in scope: only visual MPC is evaluated end-to-end with closed-loop success, while the verifier and safety gate are decision-level probes.Source: §3 · p. 4

4 Method

The method attacks a differentiable observation-to-imagination pipeline with true PGD under two objectives: untargeted off-manifold corruption and targeted steering toward a chosen imagination. It also provides a parameter-free detector based on denoiser self-consistency.

  • The attacker optimizes a bounded observation perturbation through the encoder and world model to change the imagined future consumed by an oracle.Source: §4 · p. 4
  • True PGD requires frozen parameters and differentiable forward passes; in-place KV-cache writes can sever gradients, motivating SPSA fallback when gradients are unavailable.Source: §4 · pp. 4–5
  • Untargeted optimization maximizes divergence from the clean imagination, while targeted optimization drives it toward an attacker-chosen target imagination.Source: §4 · p. 5
  • The off-manifold detector uses mean future-frame velocity-prediction norm at small noise levels and requires no clean reference.Source: §4 · p. 5

5 Experiments

Experiments across three imagine-then-act targets show strong untargeted corruption but bounded targeted steering, with effects depending on how imagination couples to action. The strongest attack corrupts imagined futures visibly and flips a fidelity verifier, while independent decision-level harm remains more limited.

Setup and scope

  • The three-target benchmark spans different imagination-to-action coupling strengths, while the reported rollout failure is limited to a single LaDi-WM MPC task with N=20 per ϵ.Non-rollout conclusions about corruption, detection, and adaptive defense are identified as the most robust.Source: §5.1 · pp. 5–6

RynnVLA-002

  • RynnVLA-002 exhibits strong same-model gradient coupling but zero cross-model task transfer, with transferred perturbations yielding 100% success equal to random and clean.Directly attacking the victim’s action instead yields 0% task success, distinguishing imagination transfer from action-path attacks.Source: §5.2 · p. 6

Targeted manipulation

  • Targeted steering remains bounded for cross-scene targets: gap_closed=0.207, verifier flips 0/8, and decoded frames remain the original scene.The adversarial imagination shows only mild artifacts rather than the attacker’s different target scene.Source: §5.3 · p. 6, Figure 3 · p. 7, Table 3 · p. 7

Untargeted corruption

  • 60× stronger-than-random untargeted corruption flips the fidelity verifier at 100% from ϵ=0.1, with 0% false positives and visible decoding damage.The attack was evaluated over N=5 tasks.Source: §5.4 · pp. 6–7, Table 4 · p. 7

Verifier scope

  • The fidelity verifier’s 100% flip is partly by construction, whereas an independent task classifier misclassifies only 12% of corrupted imaginations and the denoiser safety gate reduces effective task success by 0.18 to 0.39.The paper therefore reports fidelity, decision-level, and availability measures separately.Source: §5.4 · p. 7

5.5 Negative control: corruption is imagination-channel-specific

The attack selectively amplifies divergence in imagined futures while leaving the current-observation read-out comparatively unaffected, confirming an imagination-channel-specific exploit. A parameter-free denoiser detects this off-manifold corruption, and adaptive evasion requires giving up corruption.

Negative control: corruption is imagination-channel-specific

  • Untargeted corruption produces visible smearing, ghosting, and tearing in decoded imagined frames, unlike the targeted case.The damage is directly observable in both decoded camera views.Source: Figure 4 · p. 8
  • 24.5× amplification at ϵ=0.1 shows future divergence exceeds observation-read-out divergence, while adversarial future divergence is much larger than random.The amplification stays above 1 throughout the tested range, indicating that the attack primarily targets the imagination channel.Source: §5.5 · pp. 7–8, Table 5 · p. 8
  • A parameter-free denoiser separates clean from adversarial imagination almost perfectly, whereas latent RMS and max-abs surface statistics remain at chance.The detector uses mean future-frame velocity-prediction norms at small noise levels and needs no clean reference.Source: §5.6 · p. 8, Table 6 · p. 8
  • As adaptive detection pressure increases, corruption falls from 0.279 to 0.064 while detector AUC drops below 0.5 toward the clean baseline.The results support an intrinsic trade-off: evading detection requires surrendering imagination corruption.Source: §5.7 · p. 9, Table 7 · p. 9

5.8 Universal transfer

A scene-agnostic perturbation transfers from training tasks to held-out tasks and causes adversary-specific failure in an imagination-driven MPC. The failure is localized to corrupted trusted imagination rather than the outer planning and ranking layer, although the observation attack cannot separate shared execution pathways.

5.8 Universal transfer

  • A single frozen perturbation transfers to held-out tasks, producing test corruption 0.199 versus random 0.039, approximately 5× stronger.The perturbation is optimized on tasks [0, 3] and evaluated on tasks [7].Source: §5.8 · p. 9

5.9 LaDi-WM sampling MPC: adversary-specific real task failure

  • The MPC samples K=4 candidate actions, imagines and scores each, and selects the arg max every five steps under an ℓ∞-PGD observation attack.The evaluation uses task 4 with N=20 rollouts.Source: §5.9 · p. 9
  • At ϵ=0.01, adversarial success is 0.05 versus random 0.70, with non-overlapping intervals and Fisher p < 10^-4.Across the adversary-specific window ϵ∈[0.01, 0.03], adversarial success remains 0.05 while random success is at least 0.65; at ϵ=0.06 both collapse.Source: §5.9 · p. 9, Table 8 · p. 9, Figure 5 · p. 10
  • Feeding the adversarial observation only to MPC scoring restores success to 0.60, implicating the shared corrupted image in execution and imagination-refinement paths rather than the outer ranking layer.The final_only result of 0 does not establish a pure direct-vision failure because both paths share the encoder and corrupted image.Source: §5.9 · pp. 9–10, Table 9 · p. 10

6 Mechanism: off-manifold is intrinsic to corrupting imagination

The paper explains its results through a natural-future manifold: untargeted corruption can move imagination anywhere off-manifold, whereas targeted steering must reach a specific alternative point on the manifold. The same constraint makes corruption detectable and adaptive evasion costly.

6 Mechanism: off-manifold is intrinsic to corrupting imagination

  • Untargeted corruption is easy because any wrong direction can leave the natural-future manifold, yielding 60× random corruption and AUC 1.0 detection.The denoiser prior flags the resulting off-manifold imagination.Source: §6 · p. 10
  • Targeted steering is bounded because it must both leave the original manifold point and re-land on a specified on-manifold target.The cross-scene gap is 0.2, decoding remains unchanged, and the verifier flips 0/8.Source: §6 · p. 10
  • Adaptive evasion and corruption cannot both succeed: staying on the learned manifold avoids detection but removes the off-manifold displacement required for corruption.This single mechanism accounts for the asymmetry between easy corruption, hard steering, and reliable detection.Source: §6 · pp. 10–11

7 The task-level null, and why it motivates the oracle threat

Corrupting imagination does not materially reduce reactive closed-loop task success, so policy robustness is not evidence that downstream imagination-trusting oracles are robust. The null motivates evaluating native oracle consumers, where adversary-specific failure appears.

7 The task-level null, and why it motivates the oracle threat

  • A changed imagination cosine or verifier flip is not equivalent to real task failure; the reactive closed-loop experiment supplies the paper’s explicit negative control.This constrains interpretation of latent and verifier attacks outside trusted oracle settings.Source: §7 · p. 11
  • Reactive success remains near clean performance under corrupted imagination: 0.978 clean, 0.966 under oracle attack, and 0.943 under direct observation attack.At mid-budget, random noise drives success to 0 while a universal adversarial perturbation preserves 0.83 to 1.0.Source: §7 · p. 11, Table 10 · p. 11
  • The reactive null motivates the distinction policy robustness ≠ downstream-oracle robustness, because planners or verifiers may trust the corrupted imagination without real-world correction.LaDi-WM’s native imagination-driven MPC is the oracle consumer where adversary-specific real failure appears.Source: §7 · p. 11

8 Limitations

The evaluation has attribution, control, scale, and verification limits. In particular, full-scale availability loss is milder than the pilot suggested, and closed-loop evidence remains simulation-only and narrow.

Attribution

  • Observation-space attacks cannot separate direct-vision and imagination-refinement paths because they share an encoder.Imagination-specific attribution is possible only under the oracle threat model with direct z-space manipulation.Source: §8 · p. 11

Targeted control

  • Targeted steering remains bounded: arbitrary-scene control is not achieved, with gap 0.2, unchanged decode, and verifier success 0/8.The supported threat is corruption rather than arbitrary control.Source: §8 · p. 11

Availability

  • At full scale (N=179), availability loss is mild but statistically significant, with effective-SR drops of −0.18 to −0.39 and gate AUC 0.771.This replaces the early N=3 pilot’s catastrophic effective-SR collapse and AUC 1.00 as the reported availability result.Source: §8 · p. 11

Evaluation scope

  • The evaluation is simulation-only and limited in closed-loop scale, using task 4 MPC rollouts with N=20 across an ϵ sweep.A multi-task study with 20 to 50 episodes is needed before claiming universality.Source: §8 · p. 11

Verification

  • The semantic CLIP verifier is unreliable at decoded resolution, so the denoiser fidelity verifier is the primary evidence.Source: §8 · p. 11

9 Conclusion

WAM imagination is easy to corrupt but hard to steer precisely, making trusted downstream oracles vulnerable even when reactive policies remain robust. The paper’s central result is a native imagination-driven MPC failure, while adaptive detection remains effective.

Conclusion

  • Untargeted corruption flips a verifier at ϵ0.1, is perceptually visible, and reaches AUC 1.0 detection, whereas targeted steering stays bounded.The asymmetry is consistent with corruption being off-manifold and precise control requiring an on-manifold target.Source: §9 · pp. 11–12
  • A native imagination-driven MPC shows the first adversary-specific real task failure, while reactive policy performance remains robust to corrupted imagination.This establishes that policy robustness does not imply oracle robustness.Source: §9 · p. 12
  • Because corruption is off-manifold, the parameter-free detector remains effective against adaptive attackers unless they forgo corruption.Source: §9 · p. 12

Reproducibility

  • All reported numbers were measured on a single-A100 cluster, with code, attack and defense scripts, decoded frames, and rollout JSONs released by the authors.Source: §9 · p. 12

Related papers

Loading 2606.22966v1…