Source-linked AI summary
State Backdoor: Towards Stealthy Real-world Poisoning Attack on Vision-Language-Action Model in State Space
Ji Guo, Wenbo Jiang, Yansong Lin, Yijing Liu, Ruichen Zhang, Guomin Lu, Aiguo Chen, Xinshuo Han, Hongwei Li
TL;DR
VLA backdoor attacks must induce targeted behavior while preserving clean performance, yet visible triggers are vulnerable to real-world variability. State Backdoor uses the robot arm’s initial state as a trigger and PGA to optimize it, achieving 90% ASR across five models and five real-world tasks without degrading normal functionality.
Problem
Visible VLA backdoor triggers are vulnerable to environmental variability, while backdoor behavior can cause unsafe physical actions despite normal clean-input performance.
Method
State Backdoor poisons training data with robot-arm initial states as triggers and uses PGA to optimize stealthy, effective state-space shifts.
Results
90% ASR was achieved across five representative VLA models and five real-world tasks while preserving normal functionality.
Takeaways & Limitations
State Backdoor exposes an underexplored vulnerability in embodied AI and remains effective against fine-pruning and image-compression defenses.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) models are widely deployed in safety-critical embodied AI applications such as robotics. However, their complex multimodal interactions also expose new security vulnerabilities. In this paper, we investigate a backdoor threat in VLA models, where malicious inputs cause targeted misbehavior while preserving performance on clean data. Existing backdoor methods predominantly rely on inserting visible triggers into visual modality, which suffer from poor robustness and low insusceptibility in real-world settings due to environmental variability. To overcome these limitations, we introduce the State Backdoor, a novel and practical backdoor attack that leverages the robot arm's initial state as the trigger. To optimize trigger for insusceptibility and effectiveness, we design a Preference-guided Genetic Algorithm (PGA) that efficiently searches the state space for minimal yet potent triggers. Extensive experiments on five representative VLA models and five real-world tasks show that our method achieves over 90% attack success rate without affecting benign task performance, revealing an underexplored vulnerability in embodied AI systems.
I. INTRODUCTION
VLA backdoors can preserve clean-input behavior while inducing attacker-specified actions, but visible triggers are unreliable under real-world variability. State Backdoor instead uses the robot arm’s initial state and PGA to seek stealthy, effective triggers, achieving 90% ASR across five models and five real-world tasks while preserving normal functionality.
- VLA models integrate visual perception, language understanding, and action generation for robots following natural-language instructions.
- Backdoor models behave normally on clean inputs but execute attacker-specified actions on triggered inputs, creating physical risks in robotic-arm deployments.
- Visible object triggers are sensitive to lighting, viewpoint, background, and placement, making simulated evaluations poorly representative of real-world conditions.
- State Backdoor uses the robotic arm’s initial state as a trigger because state information remains more stable across environments.
- PGA searches black-box state space for triggers that balance realistic state shifts against sufficient attack effectiveness.
- 90% ASR was achieved across five representative VLA models and five real-world tasks while preserving normal functionality.
II. BACKGROUND AND RELATED WORKS
VLA models unify perception, language understanding, and action generation, while prior VLA backdoor methods implant hidden behaviors through poisoned data or architectural components. Related attack categories also include adversarial and jailbreak attacks.
- A. Vision-Language-Action Model: VLA systems combine visual perception, language understanding, and action generation to execute low-level control actions from high-level commands.
- A. Vision-Language-Action Model: For a 6-DoF robotic arm, the model receives a task instruction, initial joint state, and visual observations, then produces control actions.
- Backdoor attacks: Existing VLA backdoors include poisoned-data training in simulation and dedicated architectural components that activate malicious behavior under specific conditions.
- Related attacks: Adversarial attacks perturb inputs to induce incorrect actions, whereas jailbreak attacks optimize instructions to bypass safety constraints.
III. FORMULATION OF VISION-LANGUAGE-ACTION MODELS
The VLA formulation represents embodied control as a multimodal policy that maps visual observations, language instructions, and robot state to actions. Training uses demonstration data and behavior cloning, while deployment repeatedly updates state through closed-loop interaction.
- A multimodal policy πθ maps visual observation It, instruction T, and current robot state st to the next action at.
- The policy fuses visual, language, and state representations before producing an action through a task-specific decoder.
- VLA training uses demonstration tuples containing observations, instructions, states, and actions, commonly optimizing behavior-cloning discrepancy.
- During inference, live camera input, language commands, and robot states are encoded into an action embedding before motor commands are executed.
- The closed-loop system updates the robot state through environment or robot dynamics after each action.
IV. THREAT MODEL
The threat model assumes black-box poisoning during real-world fine-tuning, with inference-time manipulation limited to the robot arm’s initial state. The attack seeks clean functionality preservation, reliable task failure, and stealthy triggers.
- The adversary poisons real-world data during fine-tuning of a VLA model initialized from pretrained weights.
- The attacker accesses only the training dataset, uses a surrogate model, and cannot inspect the victim model or alter its training pipeline or architecture.
- At inference, the attacker perturbs only the victim robot arm’s initial state, an assumption motivated by maintenance access and established robotics research settings.
- The attack aims to preserve clean performance, reliably induce untargeted task failure, and keep triggered samples inconspicuous during inference.
- State Backdoor uses a specific robotic-arm initial state as the trigger, and PGA searches for an effective state change that is less perceptible.
- The three-stage attack generates poisoned data, fine-tunes the VLA model, and activates the backdoor by setting the arm to the optimized triggered state.
B. Objective Function of PGA
PGA optimizes a composite objective that balances attack effectiveness, clean functionality preservation, and trigger stealthiness. A lightweight surrogate model supplies the scoring signal for attack effectiveness.
- The PGA objective jointly balances attack success, clean functionality preservation, and trigger subtlety.
- Attack effectiveness measures how well a trigger fools the model into producing the desired target action.
- The attack-effectiveness term uses a lightweight surrogate model trained briefly for scoring.
- Functionality preservation encourages normal behavior under clean initial states.
- Stealthiness is optimized by minimizing the magnitude of the trigger perturbation.
- The final objective combines the attack-effectiveness, functionality-preservation, and stealthiness criteria.
C. Search Process of PGA
PGA searches the robot arm’s state space for an effective, stealthy trigger by evaluating candidate perturbations and iteratively refining the best candidates. The pipeline then uses the selected state to create poisoned data and train the backdoor model.
- Search Process: Each candidate trigger is a perturbation vector applied to the clean initial state, and PGA minimizes an objective combining effectiveness and stealth constraints.
- Search Process: PGA initializes N candidate triggers sampled from a uniform distribution.
- Training Backdoor VLA Model: The selected triggered state is used to synthesize poisoned samples, mix them with original training data, and train or fine-tune a backdoor VLA model.
- Search Process: The method evaluates candidates, selects the top-K with the lowest objectives, and generates new candidates through crossover and Gaussian mutation.
- Search Process: The search repeats for T generations and returns the candidate with the lowest final objective as t∗.
D. Target Action Trajectory Generation
The section constructs failure-inducing target trajectories and defines the evaluation setup for testing attack effectiveness and preserved task functionality.
- Random poisoned trajectories degrade normal functionality and perform poorly because their high action variance makes them outliers.
- Opposite Action Trajectories negate every action component of each normal trajectory to create structured poisoned counterparts.
- The structured trajectories produce similar action-space distributions, helping the model associate the trigger state with task failure.
- The evaluation uses five real-world tasks, five VLA models, and two existing VLA backdoor methods as baselines.Each dataset contains 100 real-world samples, and models are trained with a 10% poisoning rate.
- SR measures clean-task completion, whereas ASR measures triggered episodes in which the attack successfully causes task failure.
B. Effectiveness Evaluation
Across real-world tasks, models, and supplementary evaluations, State Backdoor provides strong and stable attack performance while preserving clean behavior and improving stealth over visual triggers.
- State Backdoor consistently achieves the highest ASR across five tasks, outperforming vision-based triggers in real-world evaluation.The authors attribute this to state triggers being less vulnerable to visual distribution shifts.
- Clean samples produce successful normal actions, whereas triggered samples activate backdoor behavior that causes task failure.The visualized examples cover Pick-and-Place and Tennis Pushing.
- State Backdoor achieves high ASR while largely preserving normal functionality in the LIBERO simulation environment.
- PGA finds triggers with lower time overhead than competing optimization methods and consistently achieves the highest ASR among their discovered triggers.
- State-based triggers modify only the robot’s initial state without visible objects or artifacts, making them substantially harder to detect than visual triggers.
D. Ablation study
Ablation and robustness studies examine trajectory design, stealth regularization, surrogate-model transfer, and defenses against State Backdoor.
- Random trajectories reduce both normal functionality and attack effectiveness compared with structured opposite action trajectories.
- The stealthiness constraint regularizes state perturbations, preventing excessive and unnatural triggered states.
- State Backdoor remains largely surrogate-independent, with ASR and SR varying by less than 5% across different surrogate models.
- Fine-pruning leaves ASR above 90% across pruning levels from 0% to 10%, while increasing pruning degrades normal functionality.
- At 50% image quality, ASR remains 86% while SR drops significantly, because the backdoor does not rely on visual-space features.
- Semantic Shield has almost no impact on attack performance because it cannot affect the state space used by State Backdoor.
VII. STATE BACKDOOR FOR DATESET WATERMARKING
State Backdoor can watermark VLA datasets through keyed robot states, enabling reliable ownership verification while preserving normal performance, though extensive fine-tuning weakens the watermark.
- Dataset watermarking embeds keyed joint or pose configurations into training episodes and verifies them by querying candidate states for predefined behaviors.
- Watermark verification uses validation accuracy, Top-k keys accuracy, and log10 p to measure utility, key identification, and statistical significance.
- Top-1 and Top-10 keys accuracy reach 97.3% and 99.7% on average, while validation accuracy remains 86.9% on average.
- The watermark supports effective and stealthy dataset ownership verification while maintaining model utility and enabling traceability of training-data usage.
- Light fine-tuning causes only minor verification declines, but extensive fine-tuning gradually weakens and can partially overwrite the watermark associations.
VIII. DISCUSSION
State Backdoor remains effective in from-scratch imitation learning while preserving normal functionality, and the broader method is reported to resist common defenses. The approach also raises dual-use concerns and motivates detection and certification research.
- Attack for training from scratch: State Backdoor achieves high ASR while preserving normal functionality when DP and ACT policies are trained from scratch on three manipulation tasks.The tasks are Pick-and-Place, Button Pressing, and Tennis Pushing.
- Societal impact: The state-space trigger is presented as both highly effective and extremely stealthy, creating ethical and societal concerns in safety-critical applications.The discussion also identifies dataset ownership verification and intellectual-property protection as possible positive uses.
- Future work: Future work targets broader embodied-intelligence domains and more systematic detection and certification of malicious or watermark-like state behaviors.
- Overall evaluation: The full evaluation covers five representative VLA models and five classic tasks, with high ASR and no degradation of normal functionality.
- Defense robustness: Fine-pruning and image compression fail to mitigate State Backdoor in the reported robustness assessment.