Source-linked AI summary
VLA-Forget: Vision-Language-Action Unlearning for Embodied Foundation Models
Ravi Ranjan, Agoritsa Polyzou
TL;DR
VLA policies can retain unsafe, spurious, or privacy-sensitive behaviors because undesirable knowledge is distributed across perception, grounding, and action generation. VLA-Forget uses component-aware selective editing with staged updates across these modules while jointly optimizing forgetting, perceptual preservation, and reasoning retention. Across the reported embodied evaluations, it improves the forget-retain trade-off and remains comparatively robust under quantization, but it does not provide certified erasure.
Problem
VLA unlearning must remove targeted unsafe, spurious, or privacy-sensitive behaviors without degrading perception, grounding, and action control.
Source: §1 · pp. 1–2Method
Results
VLA-FORGET achieves a stronger forget-retain success trade-off than prior baselines across Open X-Embodiment and PushT and remains comparatively robust under quantization.
Source: §5 · p. 9, §4.5 · p. 9Takeaways & Limitations
Effective embodied unlearning should be multimodal, execution-aware, and deployment-oriented rather than assessed through forget-side metrics alone.
Source: §4.5 · p. 9, §5 · p. 9VLA-Forget is approximate, depends on forget, retain, and boundary-set quality, and is evaluated only in benchmark-style manipulation settings.
Source: §4.5 · p. 9
Abstract
from arXiv · showhide
Vision-language-action (VLA) models are emerging as embodied foundation models for robotic manipulation, but their deployment introduces a new unlearning challenge: removing unsafe, spurious, or privacy-sensitive behaviors without degrading perception, language grounding, and action control. In OpenVLA-style policies, behavior is produced through a fused visual encoder, a cross-modal projector, and a language backbone that predicts tokenized robot actions, so undesirable knowledge can be distributed across perception, alignment, and reasoning/action layers rather than confined to a single module. Consequently, partial unlearning applied only to the vision stack or only to the language backbone is often insufficient, while conventional unlearning baselines designed for standalone vision or language models may leave residual forgetting or incur unnecessary utility loss in embodied settings. We propose VLA-Forget, a hybrid unlearning framework that combines ratio-aware selective editing for perception and cross-modal specificity with layer-selective reasoning/action unlearning for utility-preserving forgetting. VLA-Forget jointly optimizes three objectives: targeted forgetting, perceptual preservation, and reasoning retention, through staged updates over the visual encoder, projector, and upper action-generating transformer blocks. Across forget-set behavior probes and retain-task evaluations, VLA-Forget improves forgetting efficacy by 10%, preserves perceptual specificity by 22%, retains reasoning and task success by 9%, and reduces post-quantization recovery by 55% relative to strong unlearning baselines.
1 Introduction
VLA policies can produce physically plausible but instruction-inconsistent actions because undesirable behavior is distributed across perception, grounding, and action-generation components. VLA-Forget addresses this with component-aware, staged selective unlearning while preserving normal perception and task utility.
1 Introduction
- VLA deployment can retain unsafe, privacy-sensitive, or spurious behaviors, including visually plausible actions that contradict instructions.These failures are especially consequential because robotic errors become physical actions.Source: §1 · pp. 1–2
- Undesired behavior may be encoded jointly in visual features, cross-modal bindings, and instruction-conditioned action priors, making single-module edits often insufficient.Editing only vision can leave downstream action priors intact, while language-only edits can preserve harmful perceptual shortcuts.Source: §1 · p. 2
- VLA-Forget combines ratio-aware editing of the visual encoder and projector with significance-based editing of action-relevant transformer blocks.The framework uses staged adapter-first updates compatible with rollback and canary deployment.Source: §1 · p. 2
- The framework evaluates targeted forgetting together with perceptual preservation, reasoning retention, embodied performance, safety risk, and post-quantization robustness.Its stated goals are efficacy, specificity, and utility under an action-token interface and multimodal component entanglement.Source: §1 · p. 2
2 Related Work
Prior VLA work emphasized scaling and adaptation rather than post hoc behavior removal, while unlearning research exposed trade-offs between forgetting, retention, efficiency, and robustness. VLA-FORGET adapts selective, structure-aware unlearning to multimodal embodied policies evaluated through execution.
2 Related Work
- VLA research has largely focused on scaling, transfer, and adaptation rather than removing unsafe behaviors, undesirable concepts, or sensitive instruction-action associations.This leaves post hoc correction comparatively underdeveloped for embodied policies.Source: §2 · p. 2
- Unlearning methods balance erasure quality, efficiency, retained utility, and privacy, but broad edits can damage retention and apparent forgetting can fail after quantization.These limitations motivate selective, structure-aware updates instead of indiscriminate full-model changes.Source: §2 · p. 3
- VLA unlearning differs from unimodal settings because unwanted behavior can span visual features, cross-modal bindings, and action-token priors and must be judged through embodied execution.VLA-FORGET therefore combines ratio-aware perception/projector edits with significance-based reasoning/action updates.Source: §2 · p. 3
3 Methodology
VLA-Forget treats unlearning as a component-aware problem spanning visual encoding, cross-modal projection, and language/action generation. It selectively updates these components with complementary objectives, staged adapters, and conflict-mitigated optimization.
3 Methodology
- A VLA policy maps an observation image and instruction through a visual encoder, projector, and language/action backbone into continuous robot actions from tokenized predictions.The method forms forget, retain, and boundary sets for a target concept or behavior slice.Source: §3 · p. 3
- Undesired behavior can arise from perceptual memorization, erroneous visual-to-language binding, or action priors, so VLA-Forget combines multimodal and layer-selective editing.Vision-only or language-only updates may leave another harmful component unchanged.Source: §3 · p. 3
3.1 Hybrid Selective Localization
- VLA-Forget selects visual and projector layers using forget-retain ratios, then initializes a minimal action-layer update set with significance scores and expands it only when forgetting is insufficient.This limits unnecessary interference and global drift.Source: §3.1 · p. 4
3.2 Unlearning Objectives
- The objectives suppress target behavior, preserve retained behavior and visual grounding, and discourage recovery toward the original forgotten response.They combine forget loss, retain regularization, mismatch divergence, and perceptual feature preservation.Source: §3.2 · p. 4, Figure 2 · p. 4
3.3 Training and Unlearning Procedure
- Staged LoRA updates first weaken visual triggers, then correct cross-modal bindings, and finally suppress residual reasoning/action priors while preserving the native VLA interface.PCGrad resolves conflicts among objectives, and training stops early when forgetting and retention criteria are met.Source: §3.3 · pp. 4–5, Figure 3 · p. 5
4 Experiments
Experiments show that VLA-Forget balances targeted behavior removal with retained utility across OpenVLA and pi0fast settings. Ablations and qualitative analyses indicate that distributed multimodal editing is needed, while limitations remain around formal guarantees, benchmark scope, and repeated unlearning.
4.2 Experiment Results
- VLA-Forget achieves the best overall balance between forgetting and retention across OpenVLA benchmarks, preserving the highest RC, lowest RAD, and best TSR while GA has stronger raw forgetting but harms utility.The same trend holds on Open X-Embodiment and PushT, with lower SVR than SSD, SalUn, and NPO.Source: §4.2 · pp. 6–7, Table 1 · p. 6
- VLA-Forget preserves the strongest retain-task performance and high triggered-action suppression on pi0fast-base, maintaining the best overall trade-off despite GA's stronger raw forgetting.Canonical instructions probe retained behavior, while Triggered instructions probe target suppression; higher scores are better in both cases.Source: §4.2 · p. 7, Table 2 · p. 7
- Quantization degrades forgetting for every method, but VLA-Forget remains comparatively stable under both 8-bit and 4-bit inference while broad updates such as GA and NPO degrade more.Table 3 reports forget-side metrics after post-training quantization on OpenVLA-7B.Source: §4.2 · p. 7, Table 3 · p. 7
4.3 Ablation Study
- Single-component ablations produce weaker forgetting, whereas the full multimodal configuration most strongly suppresses forget actions because unwanted behavior spans perception, alignment, and language-action reasoning.Removing retain preservation increases raw forgetting but sharply harms RC, RAD, and TSR, showing that forgetting strength alone is insufficient.Source: §4.3 · p. 7, Figure 6 · p. 8, Table 4 · p. 8
4.4 Qualitative Analysis
- Vision-only editing reduces visual triggers but leaves target-directed behavior, projector editing causes ambiguous responses, and language editing leaves residual targeting; joint editing is cleanest.The full configuration also better preserves scene understanding than the single-component variants.Source: §4.4 · p. 8
- VLA-Forget suppresses sensitive human-directed behavior while preserving general scene understanding and safe non-target interaction.This qualitative result reflects targeted forgetting with retained perceptual grounding and action utility.Source: Figure 8 · p. 8, §4.4 · p. 9
4.5 Discussion
- The method is intended for post-deployment correction while retaining the native policy interface, and evaluation should include task success, safety violations, and quantization robustness rather than forgetting metrics alone.These criteria follow the paper's component-aware deployment perspective.Source: §4.5 · p. 9
- VLA-Forget provides approximate rather than formal erasure, depends on forget, retain, and boundary-set quality, and is evaluated only on benchmark-style manipulation settings.Repeated selective editing may also accumulate updates that degrade grounding or action robustness.Source: §4.5 · p. 9
5 Conclusion
VLA-FORGET is a staged, component-aware framework for unlearning unsafe, spurious, or privacy-sensitive behaviors in VLA policies while preserving the native interface and utility. Across OpenVLA-7B benchmarks, it improves the forget–retain trade-off over prior baselines and remains comparatively robust under quantization.
5 Conclusion
- VLA-FORGET selectively edits the vision encoder, projector, and language/action backbone with retain-aware adapter updates.This staged design removes targeted behaviors while preserving the native OpenVLA interface and overall utility.Source: §5 · p. 9
- VLA-FORGET achieves a stronger forget-retain success trade-off than prior baselines across OpenVLA-7B on Open X-Embodiment and PushT.The full hybrid design remains comparatively robust under quantization.Source: §5 · p. 9
- Ablations show that single-module edits are insufficient, supporting multimodal, execution-aware, and deployment-oriented unlearning.The conclusion frames effective embodied unlearning as requiring coordinated treatment across the policy components.Source: §5 · p. 9
Ethical Statement
The paper frames VLA-Forget as an approximate, audit-driven method that requires careful oversight and rollback safeguards. Its pipeline stages selective updates across vision, projection, and language/action components, while boundary-set construction preserves nearby valid behaviors.
Ethical Statement
- VLA-Forget should be deployed with human oversight, retain and safety-critical evaluations, and rollback safeguards because selective unlearning has dual-use risks.The method could in principle suppress desirable safety constraints or other important behaviors.Source: Ethical Statement · p. 9
A Pseudo Code
- The pipeline stages selective updates across the visual encoder, cross-modal projector, and LLM backbone while balancing forgetting, perceptual preservation, and reasoning retention.Ratio-based module selection, significance-based layer scoring, and PCGrad support the multi-objective procedure.Source: A Pseudo Code · p. 12
A.1 Boundary set construction and mismatch pairing.
- The boundary set uses near-neighbor retain samples and mismatch pairing to diverge on the forgotten mapping while preserving nearby valid behaviors.When exact instruction matches are unavailable, semantically related retain instruction-scene pairs provide the local specificity constraint.Source: §A.1 · p. 12
B Detailed Experimental Setting
The appendix records the hyper-parameter and reproducibility settings used for VLA-Forget. These settings cover data splits, optimization, unlearning weights, adapter configuration, and reporting.
B.1 Hyper Parameters
- Table 5 summarizes the data splits, optimization, unlearning weights, adapter settings, and reporting protocol used for reproducibility.Source: §B.1 · p. 12
B.2 Mathematical Definition of VLA-Forget Metrics
VLA-Forget evaluates forgetting, retention, embodied success, and safety through token-level, accuracy-drop, rollout, and violation metrics. Its staged procedure updates vision, projector, and action-relevant layers, then audits deployment robustness.
B.2 Mathematical Definition of VLA-Forget Metrics
- VLA-Forget represents each example as an observation image, instruction, and target action-token sequence for offline token-level evaluation on forget and retain sets.Exact-match sequence accuracy can also instantiate the lightweight evaluation, with corresponding forget/retain deltas.Source: §B.2 · p. 12
25 Continue in Algorithm 1.
- Retain utility is defined from the unlearned model’s retain-set loss, with higher scores indicating better preservation or improvement relative to the original policy.RC is reported as a scaled retain utility score for readability.Source: §B.2 · p. 12, §25 · p. 13
7 Stage 1: Vision unlearning
- The algorithm constructs forget, retain, and boundary batches, applies staged vision and projector unlearning, and can expand edits to action-token embeddings or language-model-head rows when forgetting is insufficient.Each stage may stop early when forget and retain criteria are satisfied, followed by global stopping and deployment auditing.Source: §B.2 · p. 13, §7 · p. 13, §16 · p. 13
15 Evaluate quantization robustness under bf16, int8, and int4
- The procedure includes adapter merging, significance-ranked block selection, and robustness audits across bf16, int8, and int4 deployment settings.The supplied algorithm fragments also specify safety metrics including task success, action jerk, gripper toggles, and bin saturation.Source: §15 · p. 14, §18 · pp. 14–15
18 Merge adapters if desired and return
- Embodied evaluation measures closed-loop task success and safety violation rate, including forbidden actions, wrong-target motion, and instruction-inconsistent trajectories.Contradiction probes expose grounding failures that token metrics or success rates may miss.Source: §18 · p. 14
B.2 Mathematical Definition of VLA-Forget Metrics
- FC and FAD measure forgetting strength, while RC and RAD measure retained utility; TSR and SVR extend evaluation to embodied success and safety.Higher FC, RC, FAD, and TSR are better; lower RAD and SVR are better.Source: §B.2 · p. 12, §25 · p. 13, §18 · p. 14
C.1 Ablation Study
The ablation selects a balanced configuration because it offers the best overall forgetting–retention trade-off, whereas more aggressive settings slightly improve forgetting but harm utility.
C.1 Ablation Study
- The balanced configuration provides the best overall forget-retain trade-off, maintaining strong forgetting, retain accuracy, low action drift, and stable task success.It uses moderate objective weights and η = 5 × 10−5.Source: §C.1 · p. 15
- More aggressive settings improve forgetting slightly but noticeably reduce retain-side utility.Source: §C.1 · p. 15
C.2 Safety Case Study
The safety case study shows targeted suppression of an unsafe affordance while preserving ordinary non-target manipulation. Its score panel jointly examines unsafe execution, safety violations, and retained-task success.
C.2 Safety Case Study
- VLA-Forget suppresses the unsafe human-directed action toward a restricted region while preserving ordinary object-directed manipulation under a safe instruction.Source: §C.2 · p. 15, Figure 9 · p. 16
- The case study combines unsafe target execution, safety violation rate, and retained-task success to assess safety correction alongside non-target utility.The reported comparison is qualitative rather than a table of numerical cells.Source: §C.2 · p. 16, Table 5 · p. 17, Table 6 · p. 18