Detect and Suppress: A Mechanistic Defense against Adversarial Patches in VLA Models

1 Introduction

• Adversarial patches can disrupt VLA control by altering visual observations, while existing defenses preprocess inputs or fine-tune model parameters.Adversarial fine-tuning adds the computational cost of policy training.Source: §1 · p. 1
• Using SAE features for adversarial defense remains underexplored, and indiscriminate suppression may disrupt nominal policy behavior.The introduction frames feature choice and intervention timing as joint requirements.Source: §1 · p. 2
• The proposed approach detects attacks with a linear probe and suppresses an associated SAE feature only then, keeping VLA parameters fixed.On LIBERO-10 under UADA, conditional intervention improves success under intermittent attacks, while continuous suppression substantially degrades performance.Source: §1 · p. 2

2 Motivation: Attack-Related Features

• Several SAE feature activations track patch presence over time, a pattern observed across multiple episodes.Figure 2 presents SAE features associated with adversarial patch presence.Source: §2 · p. 2, Figure 2 · p. 2
• The analysis switches a UADA-generated patch on and off during a rollout to examine feature responses to its presence.The SAE is trained on internal representations from clean LIBERO trajectories.Source: §2 · p. 2
• The observed feature patterns motivate investigating suppression only when an attack is detected.Source: §2 · p. 2

3 Methodology: Conditional Feature Intervention

• The setup considers UADA, which optimizes a patch to make predicted actions deviate from ground-truth actions while leaving the language instruction unchanged.A VLA policy generates actions from visual observations and language instructions.Source: §3 · p. 2
• The method uses a TopK SAE to analyze patch effects, a linear probe to detect patches, and conditional suppression of a selected attack-related feature.Source: §3 · p. 2
• The selected feature is suppressed only when an attack is detected, with the stated aim of limiting performance degradation under benign conditions.Source: §3 · p. 2

1 Attack-Related Feature Identification. Let h(i)

• The intervention target is the layer-feature pair whose activation is most consistently correlated with patch presence across rollouts.Its decoder direction is used for intervention.Source: §1 · p. 3
• A linear probe detects patch presence from a token-mean-pooled VLA representation at layer k, with k no later than the intervention layer.The layer ordering makes the detection result available before intervention.Source: §1 · p. 3
• At inference, the method suppresses the selected feature contribution in each prefix-token representation only when the probe detects an attack.λ attenuates the contribution when between 0 and 1 and removes it at 1.Source: §1 · p. 3

4.1 Experimental Setup

• The evaluation tests π0.5 and SmolVLA on LIBERO-10 under intermittent UADA attacks targeting degree of freedom 1, comparing baseline, continuous, conditional, and oracle intervention.Each setting uses 50 initial states per task across 10 tasks; the selected feature, probe, and intervention strength are fixed before evaluation.Source: §4.1 · p. 3

4.2 Main Results

• The probe achieves recall above 81% and precision above 92% in both settings, with FPRs of 0.39% for n = 3 and 0.71% for n = 10.Both FPRs are below the 2% threshold-selection target.Source: Table 1 · p. 3, §4.2 · p. 4
• 54.6% vs. 54.4% SR on π0.5, while SmolVLA falls from 41.4% to 36.6% under conditional intervention on benign observations.Continuous intervention lowers benign SR to 35.2% and 19.0%, respectively.Source: §4.2 · p. 4, Table 3 · p. 4
• Under attack, SR peaks around λ = 0.3 for n = 3 and between λ = 0.2 and 0.25 for n = 10, then declines with stronger intervention.Under benign conditions, continuous intervention sharply reduces SR as λ increases.Source: §4.2 · p. 4

5 Conclusion

• On LIBERO-10, conditional suppression improves task success under intermittent UADA attacks on π0.5 and SmolVLA, with modest SmolVLA gains and remaining benign-performance degradation.The authors call for evaluation on unseen attacks and physical robots.Source: §5 · p. 4, Figure 3 · p. 4

A.1 Adversarial Patch Attacks and Defenses.

• Prior VLA defenses preprocess inputs [3] or fine-tune models [2], while this method intervenes in internal representations at inference without changing inputs or policy parameters.Adversarial fine-tuning incurs additional policy-training cost.Source: §A.1 · p. 8

A.2 Mechanistic Interpretability and Representation Intervention.

• Prior VLA SAE studies focus primarily on nominal behavior [5] [6], while this work conditionally suppresses a patch-associated feature based on a linear probe.The evaluation measures task success under both adversarial and benign conditions.Source: §A.2 · p. 8

Models

• The study evaluates π0.5 and SmolVLA; π0.5 is fine-tuned on LIBERO with the wrist camera replaced by black images, while SmolVLA uses its released checkpoint without further fine-tuning.The policies’ weights remain fixed in defense experiments.Source: B Experimental Details · pp. 8–9

Patches

• For each victim model, one universal UADA patch targets action DoF 1 across LIBERO-10 tasks and evaluation rollouts; intermittent attacks show the patch for n ∈ {3, 10} intervals of 10 frames.An additional unoptimized gripper-image patch is used in the appendix evaluation.Source: B Experimental Details · p. 9, Figure 4 · p. 9

SAE Training

• A separate TopK SAE is trained for each candidate layer on mean-pooled residual-stream representations from clean LIBERO rollouts, using K = 12 for π0.5 and K = 6 for SmolVLA.Each SAE uses a 1× expansion ratio and is trained for 6,677 optimization steps.Source: B Experimental Details · p. 9

Attack-Related Feature Selection

• The intervention target is the layer-feature pair whose activation correlation with patch presence exceeds 0.6 in the largest number of development rollouts, with mean correlation breaking ties.Selection uses 10 separate development rollouts, one per LIBERO-10 task.Source: B Experimental Details · p. 9

Linear Probe Training

• Probe thresholds are set on clean development data for FPR at most 2%; probes use token-mean-pooled representations, with model-specific classifiers and layers.The probe training data comprise 100 clean-and-attacked rollout pairs split into training, development, and test sets.Source: B Experimental Details · p. 9

Intervention Strength

• Intervention strength λ is selected on development rollouts and fixed before evaluation; the main evaluation uses model- and attack-condition-specific strengths.π0.5 uses λ = 0.3 and 0.2 for n = 3 and n = 10; SmolVLA uses λ = 0.2 and 0.25, respectively.Source: B Experimental Details · pp. 9–10

• The additional experiments cover gripper-patch attacks, detector specificity across patch types, attack-related feature selection, and persistent attacks.Source: C Additional Experiments · p. 10

C.1 Evaluation under Gripper Patch Attacks

• Under gripper-patch attacks, conditional suppression raises π0.5 SR from 30.4% to 46.4% at n = 3 and from 4.4% to 33.6% at n = 10.The corresponding baseline SRs are 30.4% and 4.4%.Source: Table 4 · p. 10, §C.1 · p. 10
• Conditional suppression reaches SRs comparable to oracle detection: 46.4% versus 50.0% at n = 3, and 33.6% versus 31.8% at n = 10.The comparisons are for π0.5 under gripper-patch attacks.Source: Table 4 · p. 10, §C.1 · p. 10
• Without detection, intervention reduces π0.5 SR to 0% at both n = 3 and n = 10.The baseline SRs are 30.4% at n = 3 and 4.4% at n = 10.Source: §C.1 · p. 10

C.2 Detector Specificity Across Patch Types

• The UADA-trained probe recalls 5.74% of white patches and 2.17% of black patches, but 100% of random-noise patches.It therefore detects random noise as well as UADA patches.Source: Table 5 · p. 10, §C.2 · p. 10
• The gripper-trained probe recalls 16.32–23.11% of white, black, and noise patches, with a 9.45% FPR.The reported differences suggest detector specificity depends on the training patch type.Source: Table 5 · p. 10, §C.2 · p. 10

C.3 Effect of Attack-Related Feature Selection

• At π0.5 and n = 10, the selected feature has the highest patch-presence correlation and SR among tested features.Some random features improve SR by smaller margins, while others reduce it.Source: Figure 5 · p. 10, §C.3 · p. 10

C.4 Performance under Persistent Attacks

• Under persistent attacks, no intervention method improves SR; even oracle detection does not recover success.The patch remains visible throughout the rollout, making oracle detection equivalent to continuous intervention.Source: Table 6 · p. 11, §C.4 · p. 11
• For π0.5, SR falls from 12.4% at baseline to 9.0% with the proposed method and 7.4% with either comparison intervention.The comparison interventions are w/o Det. and w/ Oracle Det.Source: Table 6 · p. 11, §C.4 · p. 11
• For SmolVLA, all three intervention methods yield 1.6% SR, compared with 1.8% at baseline.The methods are the proposed method, w/o Det., and w/ Oracle Det.Source: Table 6 · p. 11, §C.4 · p. 11