PaperlayerView exact revision on arXiv

Source-linked AI summary

BadWAM: When World-Action Models Dream Right but Act Wrong

Qi Li, Xingyi Yang, Xinchao Wang

arXiv:2607.15207v1cs.LGcs.RO

TL;DR

World-action models may execute harmful actions even when their imagined futures remain plausible, leaving action-imagination alignment insufficiently tested. BadWAM formalizes attacks exploiting this gap and substantially reduces closed-loop success across WAM variants, with reported drops of around 30%-50%.

  • Problem

    It remains unclear whether adversaries can substantially change WAM actions while keeping the model’s imagined future apparently plausible.

  • Method

    BadWAM models and evaluates action-only and imagination-preserving world-action drift attacks using small observation perturbations.

  • Results

    Around 30%-50% drops in closed-loop success occur across WAM variants, while imagination-preserving attacks keep predicted futures closer to clean rollouts with substantial attack strength.

  • Takeaways & Limitations

    WAM’s coupling of action generation and future prediction can itself become an attack surface through action-imagination decoupling.

  • Takeaways & Limitations

    The evaluated defenses are incomplete because the attacks are non-adaptive, preprocessing may harm clean performance, and consistency detection is insufficient at practical false-positive rates.

Abstract

from arXiv · show

World-action models (WAMs) are emerging as a promising foundation for embodied control: rather than predicting actions alone, they learn representations that couple action generation with future world prediction. This coupling is often viewed as a source of robustness, interpretability, and safety, as a robot's action can in principle be checked against its imagined future. In this paper, we show that this assumption is fragile. We introduce BadWAM, a unified framework for modeling and evaluating World-Action Drift Attacks: a new class of WAM-specific adversarial attacks that use small visual perturbations to break the alignment between what a WAM imagines and what it executes. BadWAM characterizes this attack surface along two natural criteria: attack strength and stealthiness. When the adversary prioritizes disruption, BadWAM instantiates an action-only adversarial attack, which directly drives the model toward task-failing actions. When the adversary additionally prioritizes stealth, BadWAM instantiates an imagination-preserving adversarial attack, which seeks to induce harmful action shifts while keeping the model's predicted future close to its clean imagination. Together, these two attacks capture a spectrum of WAM-specific failures: from overt action hijacking to stealthier cases where the model appears to imagine a plausible future but executes a desynchronized action. We evaluate BadWAM across different variants of WAMs. Results show that our attacks substantially reduce task success rates under closed-loop execution. For example, our action-only attack reduces the model performance from 96.5% to 43.1% success. The results of our imagination-preserving attack further exposes a WAM-specific vulnerability: moderate future-preserving regularization can maintain strong attack performance while reducing future imagination drift.

1 Introduction

BadWAM shows that small visual perturbations can desynchronize a WAM’s executed action from its plausible imagined future, undermining future prediction as a safety signal. Its two attack variants expose vulnerabilities ranging from overt action hijacking to stealthy imagination-preserving failures.

  • Background: WAMs couple action generation with future world prediction, a design intended to support richer representations, planning, verification, and safety monitoring [22] [27].Recent variants differ in whether they explicitly generate future observations or latent futures, use video modeling auxiliarily, or target specialized manipulation and contact-rich control [5] [18] [25].
  • Method: BadWAM models the attack surface along attack strength and stealthiness under black-box access to deployed WAMs.The action-only variant prioritizes task disruption, whereas the imagination-preserving variant causes failures while preserving plausible predicted futures.
  • Empirical findings: Action-only attacks reduce closed-loop task success from 96.5% to 43.1%, while imagination-preserving attacks maintain strong attack performance and reduce imagination drift.Experiments across WAM variants on LIBERO [16] and RoboTwin [6] show vulnerability under bounded visual perturbations.
  • Problem and vulnerability: BadWAM identifies World-Action Drift Attack, where adversarial perturbations hijack actions even while predicted futures remain visually plausible.This asynchronous failure creates a dangerous gap between what the WAM imagines and what the robot executes.

2 Related Work

World-action models couple action generation with learned future-world prediction across several architectural forms, motivating future-based safety checks. BadWAM identifies the unaddressed security gap: actions can be hijacked while imagined futures remain plausible, complementing prior attacks on world models and imagination integrity.

  • World-Action Models: WAMs differ from reactive policies by coupling action prediction with latent states, decoded frames, or jointly modeled future representations.Some use future prediction at inference, while others use it primarily as a training signal; the coupling can still shape vulnerable action representations.
  • World-Action Models: Recent variants extend WAMs across object-centric manipulation, unified mobility-and-manipulation control, and visual-tactile contact-rich manipulation [18] [5] [25].These systems illustrate the expansion of WAMs beyond visual manipulation into richer embodied settings.
  • WAM Security Gap: Prior WAM work treats future prediction as a representation, planning interface, or safety signal [3] [17], but monitors may inspect imagination without verifying action alignment.This leaves open whether an observation perturbation can substantially change execution while preserving an apparently plausible imagined future.
  • Adversarial Attacks: Existing adversarial-attack research spans image classifiers, black-box settings, physical transformations, and embodied perception or policy failures [9] [19] [24] [4] [11] [26] [8] [15].These attacks generally target predictions or reactive decisions rather than the alignment between executed actions and imagined futures.

3 Threat Model

BadWAM models inference-time attacks on closed-loop WAM policies using small bounded visual perturbations, with action-only and imagination-visible access levels. It also evaluates stealthy attacks that disrupt execution while preserving the model’s imagined future, without modifying instructions, dynamics, parameters, training data, or the robot directly.

  • Inference-Time Setting: The threat model targets deployed WAM policies in closed-loop execution, where each perturbed observation produces an action chunk and optionally an imagined latent or decoded future.The robot executes part of the chunk, observes the environment again, and replans.
  • Adversarial Capability: The adversary applies a bounded perturbation to the full visual input, including concatenated multicamera observations, while leaving the goal, state, dynamics, parameters, and training data unchanged.The attacked observation remains visually close to the clean observation.
  • Stealthiness: Stealthy attacks seek to degrade execution while keeping the imagined future close to its clean prediction, targeting the gap between imagined outcomes and executed actions.This gap matters because monitors may accept actions when the predicted future appears plausible [3].
  • Non-Goals: The scope excludes training-time poisoning, model extraction, reward hacking, prompt injection, physical environment modification, direct robot commands, and post-action controller alteration.The focus is limited to small bounded observation perturbations exploiting WAM world-action interfaces at inference time.

4 BadWAM

BadWAM frames WAM security as an alignment problem between action outputs and imagined futures, and attacks this interface with bounded visual perturbations optimized online. It spans high-strength action disruption and stealthier imagination-preserving attacks, including a failure mode where plausible imagined futures coexist with desynchronized execution.

  • BadWAM framework: BadWAM models world-action drift with D_act for action deviation and D_img for imagination drift, using bounded visual perturbations to target their alignment.The framework treats WAMs as queryable predictive-action systems and optimizes perturbations at each replanning step.
  • Imagination-preserving attack: The imagination-preserving attack maximizes action disruption while constraining future drift through a Lagrangian trade-off controlled by λ.λ = 0 yields pure action disruption; larger λ favors stealthier attacks that alter imagined futures less, potentially reducing damage.
  • Imagination-preserving attack: The attack targets a WAM-specific failure in which the robot appears to imagine a plausible future but executes a desynchronized action.Qualitative results indicate that the preservation term keeps adversarial futures closer to clean predictions while action-space failure remains possible.
  • Optimization and evaluation: BadWAM is a black-box, query-based online attack that uses zeroth-order finite differences and projects perturbations into an ℓ∞ ball without gradients, weights, or training data.The same optimizer supports both attacks by changing only the scalar objective, and recomputes perturbations at each replanning step.

5 Evaluation

BadWAM substantially degrades closed-loop task success across WAM variants, including imagination-based models, with failures that are structured, progressive, and persistent across repeated trials. Evaluation further shows that attack strength, future preservation, and runtime effectiveness are governed by distinct tunable tradeoffs.

  • RQ1: Closed-loop results: In LIBERO, BadWAM lowers action-only WAM success from 96.5% to 43.1%, a 53.4% drop, while also reducing joint-WAM success to 61.5% or 63.0%.Clean joint and IDM WAMs reach 96.7% and 100.0% success, respectively.
  • RQ1: Closed-loop results: Random perturbations are weaker than BadWAM: joint-WAM success remains 71.0% under random noise versus 61.5%/63.0% under BadWAM, while white-box attacks reach 49.2%.For IDM WAM, random noise leaves success at 75.2%, compared with 66.1%/68.1% under BadWAM and 52.8% under white-box attacks.
  • Task-level failure structure: BadWAM is most damaging on spatial and long-horizon tasks, where small action errors compound, while object-centric tasks retain comparatively high success.For the action-only WAM, Spatial success falls from 96.5% to 16.0%, Goal success from 97.0% to 40.0%, and Long-horizon success from 92.5% to 23.5%; Object success remains 93.0%.
  • Repeated-trial reliability: Under repeated trials, attacked performance remains separated from clean performance: the clean action-only WAM stays near 1.0 pass@k, whereas the attacked model remains around 0.40-0.43 through 20 trials.This persistence indicates systematic reliability loss rather than a few unlucky seeds.
  • RQ3: Mechanism analysis: Attacks produce progressive, locally plausible action drift: action distance increases during optimization while future-video distance changes more mildly under the preservation objective.The imagination-preserving attack lowers average predicted-future distance from 14.01 to 13.04 under matched resources and is lower on 39 of 40 LIBERO tasks.
  • RQ5: Sensitivity analysis: The attack is tunable: increasing λ reduces future distance, larger perturbation budgets increase attack strength, and query budgets improve success reduction only up to a nonmonotonic strength-runtime frontier.At ε = 0.01, joint and IDM success are 95.8% and 98.3%; at ε = 0.20, both reach 0.0%, while budget 32 costs more without further reducing success.

6 Conclusion

The conclusion argues that coupling perception, prediction, and control in world-action models can itself become an attack surface. BadWAM is introduced as a black-box framework that induces task failures through small observation perturbations and exposes action-imagination decoupling beyond action-only attacks.

  • 6 Conclusion: BadWAM shows that coupling action generation with imagined futures can create an attack surface in world-action models.The framework targets the connection between perception, prediction, and control.
  • 6 Conclusion: BadWAM is a black-box framework for world-action drift attacks that induce task failures through small observation perturbations.
  • 6 Conclusion: Beyond action-only attacks, BadWAM exposes the WAM-specific failure mode of action-imagination decoupling.

Appendix · A Details of Experiment Setup · A.1 Full and Subset Evaluation Protocols

The evaluation uses a full LIBERO benchmark sweep for standard results and a balanced subset for expensive diagnostic studies. The subset preserves diversity across all four manipulation suites while enabling matched comparisons across ablations, transfer experiments, and defenses.

  • A.1 Full and Subset Evaluation Protocols: The full LIBERO sweep covers four suites, with 10 tasks per suite and 20 trials per task, totaling 800 closed-loop episodes per model–attack pair.The suites are Spatial, Object, Goal, and Long-horizon.
  • A.1 Full and Subset Evaluation Protocols: Expensive diagnostic studies use a balanced subset containing three tasks per suite—task IDs 0, 4, and 9—with 10 trials per task, totaling 120 episodes.The subset reduces evaluation cost while retaining representatives from every suite.
  • A.1 Full and Subset Evaluation Protocols: Table 4 identifies which reported results use the full benchmark sweep and which use the balanced LIBERO subset.This distinction separates standard benchmark coverage from expensive diagnostic evaluation.
  • A.1 Full and Subset Evaluation Protocols: The subset preserves suite diversity by including spatial reasoning, object manipulation, goal-conditioned manipulation, and long-horizon manipulation.Each subset experiment contains representatives from all four categories.
  • A.1 Full and Subset Evaluation Protocols: The same subset is used across future-preserving-weight, perturbation-budget, and query-budget ablations.Using one fixed subset makes comparisons across these ablations matched by task.
  • A.1 Full and Subset Evaluation Protocols: The fixed subset also supports matched comparisons for transfer experiments and defense baselines.These studies use the same task selection as the ablations.

A.2 Training Details

The experiments use official or FastWAM-based training configurations for action-only, joint, and IDM WAMs, with RoboTwin variants trained for 50,000 steps on 8 H100 GPUs. LIBERO and RoboTwin use benchmark-specific multi-camera inputs and 33-frame training windows, while joint and IDM variants retain the FastWAM architecture and schedulers.

  • Training configurations: RoboTwin joint and IDM WAMs are trained for 50,000 optimization steps on 8 H100 GPUs using the FastWAM recipe.The action-only model uses official FastWAM checkpoints, while LIBERO joint and IDM variants use the default FastWAM configuration.
  • Model configuration: The joint and IDM variants retain the FastWAM architecture, backbone, tokenizer, and default action and video schedulers.The configuration uses Wan-AI/Wan2.2-TI2V-5B as the video backbone and Wan-AI/Wan2.1-T2V-1.3B as the tokenizer, with 1000 training timesteps and shift value 5.0.

A.3 Attack Hyperparameters and Query Accounting

BadWAM perturbs the full preprocessed visual input online during closed-loop replanning, using a default ℓ∞ budget of ε = 0.06 and 17 WAM forward queries per attacked replan. The attack objectives differ by future-preserving regularization: action-only sets λ = 0, whereas imagination-preserving uses a default λ = 0.015.

  • Attack Hyperparameters: ε = 0.06 bounds online perturbations to the full preprocessed visual input at each closed-loop replanning step.The perturbation is applied at every attacked replan.
  • Query Accounting: 17 WAM forward queries per attacked replan comprise 16 perturbation queries plus one clean reference query, organized as eight paired finite-difference updates.This is the default zeroth-order finite-difference search configuration.
  • Attack Hyperparameters: The action-only objective sets the future-preserving coefficient to λ = 0, while the imagination-preserving objective penalizes predicted-future drift with default λ = 0.015.Figure 12 uses a larger coefficient for matched-strength comparisons between the two objectives.

A.4 White-Box Gradient-Access Reference · A.5 Random Perturbation Baseline · A.6 Evaluation Metrics

The evaluation distinguishes BadWAM’s query-based threat model from stronger white-box diagnostics and model-agnostic random perturbations, while measuring attack effects through closed-loop success, behavioral distances, perturbation statistics, and repeated-trial reliability. The white-box reference serves as an upper-bound diagnostic rather than part of the primary threat model.

  • A.4 White-Box Gradient-Access Reference: White-box gradient access is used only as a diagnostic reference to contextualize how much stronger attacks become when backpropagation through action outputs is available.BadWAM itself assumes query access and does not require gradients or model parameters.
  • A.4 White-Box Gradient-Access Reference: The white-box reference optimizes one bounded episode-start perturbation for 16 projected-gradient steps, maximizing clean-versus-perturbed action-chunk deviation under ||δ||∞≤0.06.It uses step size 0.01, random initialization within the ℓ∞ ball, and standard closed-loop inference after optimization.
  • A.4 White-Box Gradient-Access Reference: Gradient access can amplify vulnerability beyond query-based BadWAM attacks, while random perturbations under the same budget are substantially weaker than targeted attacks.Accordingly, the white-box result is treated as a diagnostic upper bound rather than the primary threat model.
  • A.5 Random Perturbation Baseline: The random baseline samples independent uniform perturbations within BadWAM’s ℓ∞ budget, clips inputs to the valid range, and removes optimization objectives and model-specific signals.It uses no action outputs, future predictions, gradients, or task feedback, testing whether failures arise from arbitrary visual corruption alone.
  • A.6 Evaluation Metrics: Closed-loop task success is the primary metric, with lower attacked success indicating a stronger attack.The evaluation also reports action distance, predicted-future distance, and input perturbation statistics; the first two are mean ℓ2 deviations from clean behavior.
  • A.6 Evaluation Metrics: Pass@k measures the fraction of successful executions among the first k trials for each task, averaged across tasks to assess reliability across repeated evaluations.This distinguishes isolated unlucky failures from attacks that consistently reduce success as more trials are considered.
Loading 2607.15207v1…