PaperlayerView exact revision on arXiv

Source-linked AI summary

TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes

Jun-Hui Liu, Kun-Yu Lin, Yi-Lin Wei, Xu-Han Chen, Yinghao Li, Zhuohao Li, Yuan-Ming Li, Qing Zhang, Xiaoyi Fan, Dongmei Jiang, Yan Li, Wei-Shi Zheng

arXiv:2608.26578v1cs.ROcs.CV

TL;DR

Prior VLA backdoor attacks typically detect only whether a task fails, leaving fine-grained control over how failures occur insufficiently studied. This paper introduces Configured Failure Trapping and TrapVLA, which reliably induces four configured failure modes while largely preserving clean-task performance across simulations and real-world settings.

  • Problem

    Prior VLA backdoor attacks usually treat failure as binary, leaving adversarial control over how robots fail insufficiently studied despite its stealth and detection relevance.

  • Method

    The paper introduces Configured Failure Trapping, TrapEngine, TrapEval, and TrapVLA with Target Residual Steering to learn trigger-induced action residuals for multiple configured failures.

  • Results

    TrapVLA largely preserves clean-task performance while reliably inducing four configured failure modes across Trap-LIBERO, Trap-RoboTwin, simulation, and real-world settings, reaching AVEs of 98.7–94.1.

  • Takeaways & Limitations

    The results reveal a practical backdoor threat in which textual triggers can configure behaviorally plausible robotic failures rather than merely causing unconstrained task failure.

Abstract

from arXiv · show

This work introduces Configured Failure Trapping, a novel backdoor attack task against Vision-Language-Action (VLA) models, which aims to activate attacks through stealthy textual triggers and induce configured failure modes. Unlike prior backdoor attacks that treat any task failure as a successful attack, Configured Failure Trapping requires the attacker to control how the robot fails (e.g., causing the robot to grasp with a specified positional offset), making it substantially more challenging and hard to detect. To support the new task, we propose an effective data engine for synthesizing high-quality target trajectories and an automated suite for measuring configured-failure fidelity. Then, based on this foundation, we construct two new benchmarks, namely Trap-LIBERO and Trap-RoboTwin, that instantiate Configured Failure Trapping across four representative failure modes. To address this task, we identify sparse action deviation as a critical challenge and accordingly propose a novel method named TrapVLA, which explicitly learns trigger-induced action residuals to steer the policy toward the configured failure behavior. Extensive experiments across simulation benchmarks and real-world robotic settings show that TrapVLA effectively injects configured failure modes into VLA models while largely preserving performance on clean data. Project page: https://john-liua.github.io/TrapVLA/

Introduction

The paper introduces Configured Failure Trapping, a stealthy VLA backdoor task that uses textual triggers to induce attacker-specified, behaviorally plausible failures. It supports this task with data synthesis, evaluation tools, benchmarks, and TrapVLA, which learns trigger-induced action residuals.

  • Motivation: Existing VLA attacks mainly target generic task failure, overlooking adversarial control over how the robot fails.Prior work commonly treats failure as binary rather than measuring the specific failure behavior.
  • Configured Failure Trapping: Configured Failure Trapping activates attacker-specified failure modes through stealthy textual triggers instead of inducing unconstrained trajectory deviations.The resulting failures can remain behaviorally plausible and resemble errors arising during normal task execution, making detection difficult.
  • Supporting Tools: TrapEngine automatically synthesizes high-quality backdoor data by configuring and replaying target trajectories with specified failure behaviors.It extracts task-relevant geometric metadata from successful expert trajectories and converts configurations into target trajectories.
  • Method and Results: TrapVLA learns trigger-induced action residuals from paired benign and target trajectories to steer policies toward configured failures despite sparse action deviation.Experiments cover π0.5 and OpenVLA-OFT in simulation and real-world settings, showing largely preserved clean-task performance and reliable induction of four configured failures.
  • Benchmarks and Evaluation: TrapEval measures configured attack success and failure fidelity, while Trap-LIBERO and Trap-RoboTwin instantiate four representative failure modes across LIBERO and RoboTwin.The evaluation suite introduces C-ASR for configured-failure execution and AVE for attack effectiveness and clean-task preservation.

Related Work

Vision-Language-Action models map visual observations and natural-language instructions to executable robot actions, with existing approaches mainly following two action-generation paradigms. Unlike prior attacks that disrupt execution without controlling failure manner, this work uses stealthy textual triggers to activate prespecified failure modes without modifying observations or intervening during execution.

  • Vision-Language-Action Models: VLA models map visual observations and natural-language instructions to executable robot actions.
  • Vision-Language-Action Models: Existing VLA approaches mainly follow two action-generation paradigms.
  • Prior attacks disrupt task execution without explicitly controlling how the robot fails.
  • This work activates a prespecified failure mode through a stealthy textual trigger without modifying visual observations or intervening during execution.

Configured Failure Trapping

Configured Failure Trapping injects trigger-conditioned, behaviorally plausible failure modes into VLA models while preserving normal behavior without the trigger. The task defines four offset-parameterized manipulation failures and evaluates whether triggered executions conform to the configured behavior using C-ASR.

  • Task Definition: Configured Failure Trapping activates target actions when a predefined textual trigger appears, while requiring normal task behavior in its absence.The induced actions steer the robot along a target trajectory into the configured failure mode, measured by Configured Attack Success Rate (C-ASR).
  • Representative Failure Modes: Four representative failure modes span temporal gripper-transition errors and spatial interaction displacements, each parameterized by an offset vector ∆∈R3.They include Early Close, Grasp Deviation, Early Open, and Release Deviation.
  • Representative Failure Modes: Early Close and Early Open create premature gripper transitions, whereas Grasp Deviation and Release Deviation shift interaction locations from intended positions.The four modes respectively cover premature closing, offset grasping, premature opening, and offset releasing.
  • Data Construction: TrapEngine constructs configured-failure backdoor data by filtering expert demonstrations, modifying relevant trajectory segments, verifying feasibility, and pairing trajectories with stealthy textual triggers.Its stages are Meta Data Collection, Target Trajectory Generation, and Backdoor Dataset Assembly.
  • Evaluation: C-ASR measures the proportion of triggered episodes whose first relevant gripper-transition displacement satisfies the configured failure constraint within tolerance γ.Closing transitions are evaluated for Early Close and Grasp Deviation, while opening transitions are evaluated for Early Open and Release Deviation.

Methodology

TrapVLA addresses sparse action deviation, where trajectory-wide textual triggers coincide with localized failure-inducing action changes, by learning trigger-induced action residuals. Its Target Residual Steering objective contrasts paired clean and backdoor outputs, reinforces deviations in target intervals, and suppresses unnecessary changes elsewhere.

  • Sparse Action Deviation: Configured Failure Trapping is difficult because most backdoor data retain task-consistent actions, overwhelming the sparse deviations that encode the configured failure.The textual trigger spans the trajectory, but salient action modifications occur only briefly, potentially causing the model to ignore the trigger.
  • Target Residual Steering: TrapVLA learns trigger-induced action residuals to amplify supervision in localized intervals where salient deviations activate the configured failure.Target Residual Steering strengthens the trigger association with localized action changes rather than indiscriminately associating it with all target-trajectory data.
  • Paired Data Construction: The method forms temporally aligned clean–backdoor pairs from the same task and nominal timestep, allowing one clean sample to pair with multiple configured failure modes.This paired structure supports direct comparison between clean and triggered policy outputs.
  • Target Residual Steering: TrapVLA partitions paired data into target-deviation subset S and task-consistent subset Sc, reflecting whether the injected failure requires salient action modification.For Early Open, approach, grasp, transport, and similar benign stages belong to Sc and comprise most of the execution trajectory.
  • Overall Objective: TRS aligns predicted and target residuals on S while suppressing unnecessary residuals on Sc within a fixed tolerance, complementing vanilla absolute-action imitation.The final objective combines vanilla backdoor training with TRS, whose contribution is controlled by λtrs.

Experiments

Experiments evaluate TrapVLA on Trap-LIBERO and Trap-RoboTwin using OpenVLA-OFT and π0.5, including four configured failure modes across simulation and real-world robotic settings. TrapVLA achieves strong configured-failure fidelity while preserving clean-task performance.

  • Benchmarks and victim models: Experiments use OpenVLA-OFT and π0.5 on Trap-LIBERO and Trap-RoboTwin, covering four configured failure modes: Early Close, Grasp Deviation, Early Open, and Release Deviation.Trap-LIBERO includes Object, Spatial, Goal, and Long suites, while Trap-RoboTwin includes bimanual Shoes and Fan tasks.
  • Trap-LIBERO with OpenVLA-OFT: TrapVLA reaches AVE scores of 98.7, 98.9, 95.8, and 94.1 on the Object, Spatial, Goal, and Long suites, respectively, with OpenVLA-OFT.These are the highest AVE scores across all four Trap-LIBERO suites.
  • Results on other VLAs and benchmarks: On π0.5, Vanilla-T performs considerably better than on OpenVLA-OFT, narrowing its gap with TrapVLA, possibly because π0.5 uses a dedicated action expert for continuous action flows.The comparison indicates that victim-model architecture affects vanilla textual-trigger performance.
  • Trap-RoboTwin: TrapVLA achieves high C-ASR on the challenging bimanual Shoes and Fan tasks while maintaining success rate on normal tasks.The results support effectiveness across different VLA architectures, simulation environments, and manipulation settings.
  • Real-world evaluation: 97.0 and 99.2 are TrapVLA’s AVE scores on Eggplant and Cup in real-world experiments, exceeding DropVLA by 44.7 and 43.3 percentage points, respectively.The experiments deploy π0.5 on a ROKAE 6-DoF robot with third-person-view and wrist-mounted cameras.

Conclusion

The paper introduces Configured Failure Trapping for VLA models, supported by tools and benchmarks for synthesizing and evaluating fine-grained failures. TrapVLA uses Target Residual Steering to activate diverse configured failures while largely preserving clean-task performance.

  • Configured Failure Trapping uses stealthy textual triggers to activate fine-grained configured failure modes in Vision-Language-Action models.
  • TrapEngine generates target trajectories, while TrapEval evaluates clean-task preservation and the specificity of induced failures.
  • Trap-LIBERO and Trap-RoboTwin instantiate Configured Failure Trapping across four representative failure modes.
  • TrapVLA addresses sparse action deviation through Target Residual Steering, directly supervising trigger-induced action residuals.
  • Extensive experiments show reliable activation of diverse configured failure modes across VLA architectures while largely preserving clean-task performance.

A Textual Trigger Selection · Stealthiness against ONION Detection

TrapEngine selects fluent textual prefixes conditioned on complete task instructions, ranks them by language-model perplexity, and retains the most natural candidates. On Trap-LIBERO, these triggers remain close to clean instructions in linguistic naturalness and largely evade ONION detection unless aggressive thresholds cause extensive collateral corruption.

  • A Textual Trigger Selection: TrapEngine uses GPT-5.4 to generate candidate prefixes conditioned on complete instructions, promoting fluent integration without grammatical inconsistencies, semantic shifts, or irrelevant content.The prefixes cover the clean condition and four configured failure modes.
  • A Textual Trigger Selection: The selection procedure evaluates complete triggered instructions using average token-level NLL and its corresponding perplexity as naturalness measures.Both metrics are computed from token-level conditional probabilities.
  • A Textual Trigger Selection: 0.24 is the largest NLL increase for triggered instructions relative to corresponding clean instructions across Trap-LIBERO suites and configured failure modes.The Close trigger yields slightly lower NLL and PPL than clean instructions on the Goal and Long suites.
  • Stealthiness against ONION Detection: ONION evaluation uses 37 clean and 148 triggered LIBERO prompts, with 37 prompts for each trigger type, across thresholds from −5 to 5.The default threshold is t = 0, and increasingly aggressive negative thresholds are also tested.
  • Stealthiness against ONION Detection: 2.7% of triggered prompts are detected at the default threshold t = 0, and none of the triggers are fully removed.Detection is counted when at least one trigger token is removed; full removal requires deleting every trigger token.
  • Stealthiness against ONION Detection: 56.8% detection at t = −5 coincides with 94.6% of clean prompts modified and 98.6% of triggered prompts losing non-trigger tokens.No tested threshold fully removes any trigger, while thresholds t ≥2 yield no detections.
  • Stealthiness against ONION Detection: The Close trigger is never detected, whereas Open is detected only at t = −5 and t = −4, with Grasp and Release detections concentrated at aggressive negative thresholds.No tested threshold reliably detects triggers while limiting collateral corruption.

Trigger Detection with Codex

Codex failed to recognize the paper’s fluent textual triggers as malicious under a blinded detection protocol. Across 185 prompts, it labeled every input CLEAN, producing zero triggered-prompt detection and 20.0% overall accuracy.

  • The evaluation set contains 185 prompts from 37 Trap-LIBERO tasks, with one clean prompt and four triggered prompts per task.
  • Each prompt was judged in an independent ephemeral Codex session without ground-truth labels, trigger information, demonstrations, or other evaluation prompts.The judge operated in an empty temporary directory with user configuration and project rules disabled.
  • Codex labeled all 185 inputs CLEAN, yielding TN = 37, FP = 0, FN = 148, and TP = 0.
  • The false-positive rate and triggered-prompt detection rate were both 0.0%, while overall accuracy was 20.0% and balanced accuracy was 50.0%.The 0.0% detection rate held for every trigger type.

Robustness to Synonym Substitution

The evaluation tests whether the language-triggered attack depends on exact trigger wording by replacing one content word with a semantics-preserving synonym. On Trap-LIBERO with π0.5, substituted triggers retain high C-ASR, with aggregate decreases of only 0.8–1.6 percentage points.

  • The study evaluates lexical robustness by testing whether the attack depends on the exact form of its textual triggers.
  • On Trap-LIBERO with the π0.5 model, one content word in each trigger is replaced while the original task instruction remains unchanged.Synonym candidates come from bert-base-uncased and are filtered with a conservative semantic whitelist.
  • 0.8–1.6 percentage points: aggregated C-ASR decreases by only this amount across the four trigger types after substitution.Several trigger–suite combinations even improve after substitution.
  • Substituted triggers retain high C-ASR across all four instantiated failure modes, showing effectiveness under semantics-preserving lexical perturbations.The results indicate that the attack does not depend solely on an exact trigger string.

B Training Details

TrapVLA trains on aligned clean and trigger-conditioned samples while combining the victim model’s native action objective with Target Residual Steering to supervise configured failure residuals. A data-ratio study shows that vanilla backdoor injection remains unreliable despite stable clean performance, indicating that effectiveness depends on more than backdoor-data quantity.

  • Training Pipeline: Each paired training group aligns one clean sample with multiple trigger-conditioned backdoor samples from corresponding target trajectories at the same nominal timestep.Clean samples retain the original instruction, while backdoor samples prepend failure-specific textual triggers to that instruction.
  • Training Objective: TrapVLA combines the victim model’s native action prediction objective with Target Residual Steering to supervise trigger-induced relative action changes.The native objective supervises absolute actions under both conditions, while TRS emphasizes residuals that instantiate configured failure modes and suppresses unnecessary deviations.
  • Backdoor-Data Sensitivity: 93.8–96.4: clean SR remains relatively stable across backdoor-data ratios from 0.1 to 0.9 on Trap-LIBERO Object using OpenVLA-OFT.The study evaluates clean-task SR and Early Close C-ASR while keeping other training settings fixed.
  • Backdoor-Data Sensitivity: 68.6 to 82.4, then 10.2, 86.8, and 93.2: Early Close C-ASR changes substantially and non-monotonically as the backdoor-data ratio increases.The sharp drop occurs at a ratio of 0.5, followed by recovery at higher ratios.
  • Backdoor-Data Sensitivity: Simply increasing backdoor-data quantity does not consistently resolve vanilla backdoor injection’s learning difficulty.The observed variation indicates that effectiveness depends not only on data quantity but also on how training data are triaged.

C Real-World Robot Demonstrations

Real-world evaluations use a dual-view ROKAE robot platform to test whether textual triggers activate configured failure modes during tabletop pick-and-place. The demonstrations show clean task completion and trigger-specific failures without changing visual observations or intervening during execution.

  • Platform: The evaluation platform combines a ROKAE 6-DoF robot with external Intel RealSense D435 and wrist-mounted Intel RealSense D405 cameras.The external camera provides third-person workspace views, while the wrist camera captures close-range gripper, object, and placement observations.
  • Evaluation Protocol: Two tabletop pick-and-place tasks involve an eggplant and a cup.
  • Evaluation Protocol: The study collects 50 demonstrations for training and conducts 30 independent trials under each of five evaluation conditions.The conditions include clean execution and triggered Early Close, Grasp Deviation, Early Open, and Release Deviation.
  • Results: Clean instructions produce benign trajectories that complete the specified task, whereas failure-specific textual triggers produce trajectories realizing the corresponding configured failure mode.
  • Results: Textual triggers activate distinct configured failure modes on a physical robot without modifying visual observations or requiring intervention during execution.

D Additional Related Work

Configured Failure Trapping differs from prior targeted VLA attacks in its threat model, behavior objective, optimization challenge, and evaluation criteria. TrapVLA activates stealthy, structured, and controllable failure behaviors through natural textual triggers while preserving task-related behavior.

  • Comparison with FreezeVLA: FreezeVLA uses visual perturbations to induce instruction-agnostic persistent inaction, whereas TrapVLA uses a training-time backdoor activated by a natural textual prefix.TrapVLA does not modify visual observations during inference.
  • Optimization challenge: TrapVLA addresses sparse action deviation because only a short interval of a triggered target trajectory contains actions instantiating the configured failure.Target Residual Steering supervises localized trigger-induced residuals while suppressing unnecessary deviations during task-consistent behavior.
  • Evaluation objective: TrapEval evaluates failure-mode-specific criteria, including gripper-transition events and configured spatial or temporal offsets, rather than exact low-level trajectories.This differs from BackdoorVLA, which evaluates success against an attacker-specified target action sequence.
  • Attack objective: Configured Failure Trapping targets structured, behaviorally plausible failures, extending beyond predefined primitives or trajectory redirection; its four temporal and spatial failures are representative, not restrictive.DropVLA activates a predefined low-level action primitive, while BackdoorVLA redirects the policy toward a predefined trajectory.
  • Stealthiness and controllability: TrapVLA’s stealthiness comes from natural textual activation without visual modification or execution-time intervention, while induced behavior remains task-related and physically plausible.The method also supports multiple failure-specific triggers within a single model, with TrapEval checking the corresponding configured-failure criteria.
Loading 2608.26578v1…