Sparse Feature Policy Unlearning Mitigates State Hallucination in Vision-Language-Action Models

• VLAs still exhibit unreliable behaviors, including premature actions, long-horizon drift, and failures to recover, which matter in physical execution.The introduction also notes that erroneous actions directly affect the robot and its environment.Source: §I · p. 1
• State hallucination is a recurring pattern in which a VLA continues acting as if an unrealized robot-object state had been achieved.A failed grasp can lead to later transport and placement motions as though the object were held, allowing the mismatch to persist across stages.Source: §I · pp. 1–2
• Hallucinated rollouts show weaker attention to task-relevant regions and distinct sparse-feature activation patterns compared with successful behavior.The paper uses sparse autoencoders to analyze these feature patterns.Source: §I · pp. 1–2
• SOUL uses features associated with hallucinated failures and successful behavior as forgetting and retention targets, respectively.It suppresses hallucination-associated features while enhancing features supporting successful manipulation.Source: §I · p. 2
• Across OpenVLA and π0.5 experiments, SOUL achieves an average 21.7%p reduction in hallucination rate and 22.1%p improvement in task success while preserving 89% of previously successful behaviors.The evaluation spans LIBERO-Plus, RoboCasa, and a real-world Franka robot.Source: §I · p. 2

A. Vision-Language-Action Models

• Prior VLA work improves generalization through pretrained representations, robotic data, and action-generation architectures, while reliability work uses reasoning, recovery, sampling, or verification.These approaches primarily improve policy outputs during deployment rather than directly modifying knowledge associated with recurring failures.Source: A. Vision-Language-Action Models · p. 2
• The paper formulates recurring undesirable behavior as a policy-unlearning problem, encoding correction in pretrained parameters for persistence without repeated inference-time intervention.The approach removes knowledge associated with hallucinated behavior while preserving representations that support successful execution.Source: A. Vision-Language-Action Models · p. 2

B. Machine Unlearning

• Machine unlearning has been studied in classification and language tasks, with a central challenge of removing targeted knowledge while preserving useful capabilities.For robot policies, modifying one behavior can inadvertently disrupt tightly coupled skills, motivating explicit feature-level forgetting and retention targets.Source: B. Machine Unlearning · p. 2

C. Sparse Autoencoders

• Sparse autoencoders decompose dense activations into sparse latent features that can be associated with more localized, interpretable behavior patterns.The Top-k variant retains only the k largest latent activations; unlike repeated test-time steering, this work uses sparse features as targets for policy unlearning.Source: C. Sparse Autoencoders · p. 2

A. What Is State Hallucination?

• State hallucination means continuing to act based on a robot-object state that has not actually been achieved, including grasp, transport, and place events.The stage-specific examples distinguish closing without object contact, moving an ungrasped object toward a destination, and opening without holding it.Source: A. What Is State Hallucination? · p. 2
• In representative same-task rollouts, hallucination failures show weaker attention to the gripper and target object and increased attention to background regions.The comparison is between clean success and hallucination failure.Source: Figure 2 · p. 3
• Rollouts are labeled using ground-truth robot-object states in simulation and observable interaction states in the real world, then categorized as clean success, hallucination failure, or non-hallucination failure.Hallucination failure denotes a failed execution with at least one state-hallucination event.Source: A. What Is State Hallucination? · p. 3

• Hallucination failures attend less to the target object and gripper and more to background regions than clean successes in representative rollouts.The analysis aggregates action-token attention to visual tokens at each rollout step, projects scores onto the input image, and visualizes steps immediately before events such as grasping.Source: B. Attention Analysis of Hallucinated… · p. 3

C. Sparse Feature Characterization of State Hallucination

• Sparse autoencoders decompose policy activations into sparse features whose activations can be compared across clean successes, hallucination failures, and non-hallucination failures.The analysis uses visual-token activations from policy representations.Source: C. Sparse Feature Characterization of… · p. 3
• Region-conditioned feature activations are summarized by their maximum over rollout time and averaged within each execution outcome.Semantic regions come from simulation masks or manual annotation of real-world observations.Source: C. Sparse Feature Characterization of… · p. 3
• Feature association scores contrast execution outcomes and categorize features as more associated with hallucination failures or clean successes.These categories are called hallucination features and success features, respectively.Source: C. Sparse Feature Characterization of… · pp. 3–4
• Hallucination features activate around hallucination events in HF rollouts but remain weak in CS, with similar patterns across tasks and visual conditions.Rollout-level maximum activations are also higher for HF than CS.Source: C. Sparse Feature Characterization of… · p. 4, Figure 3 · p. 4
• Success features activate during successful subtasks or task completion in CS rollouts, while remaining weaker in HF.Rollout-level maximum activations are higher for CS than HF.Source: C. Sparse Feature Characterization of… · p. 4, Figure 3 · p. 4

IV. SPARSE FEATURE POLICY UNLEARNING (SOUL)

• SOUL targets sparse features associated with hallucination failures for forgetting and features associated with clean success for retention.The framework analyzes region-specific features across clean success, hallucination failure, and non-hallucination failure before selecting targets.Source: §IV · p. 4, Figure 4 · p. 5
• Forget features are selected for strong association with hallucination failures, separation from non-hallucination failures, and weak association with clean success.Retain features are selected for high association with clean success.Source: §IV · pp. 4–5
• With the SAE encoder frozen, SOUL optimizes policy parameters to penalize selected forget-feature activations on hallucination-failure samples and encourage retain-feature activations on clean-success samples.The forgetting and retention objectives are jointly balanced by Lunlearn = Lforget + λrLretain.Source: §IV · p. 5

Evaluation protocol

• Experiments use OpenVLA and π0.5 on LIBERO-Plus, RoboCasa, and a 7-DoF Franka Research 3, with gradient-ascent unlearning as a comparison baseline.The simulated evaluation focuses on structured pick-and-place tasks; the real-world setup uses the Franka arm.Source: A. Setup · p. 5

SAE-based feature analysis and unlearning

• The SAE analysis uses a 4096-dimensional latent space with k = 100 active features and selects five region-feature pairs each for forgetting and retention.The policy is initialized from publicly released pretrained weights; LoRA adapters are optimized while the original weights remain frozen.Source: A. Setup · p. 5

Metrics

• Task success and hallucination are measured at rollout level, including clean success, hallucination failure, and grasp-, transport-, and place-specific hallucination rates.TABLE I compares hallucination and task-success performance in percentages; lower and higher values are better, respectively.Source: A. Setup · p. 6, Table I · p. 6

Quantitative results

• SOUL reduces average hallucination from 55.0% to 33.5% and raises clean success from 27.5% to 52.2%, with hallucination reductions across grasp, transport, and place.Gradient-ascent unlearning does not consistently mitigate hallucination and substantially degrades task success.Source: B. Main Results · p. 6

Qualitative results

• After unlearning, benchmark rollouts complete tasks without hallucination as forget-feature activation falls and retain-feature activation rises during corresponding stages.This pattern is reported for OpenVLA on LIBERO-Plus and π0.5 on RoboCasa.Source: B. Main Results · pp. 6–7

Behavior preservation and correction

• SOUL’s preservation and correction measures distinguish retaining already successful, non-hallucinated behavior from correcting previously failed or hallucinated behavior.The figure defines these measures as fractions of executions or events that remain successful and clean, or become successful and non-hallucinated.Source: Figure 7 · p. 7

Real-world evaluation

• In 50 real-world rollouts across OpenVLA and π0.5, SOUL increases clean success by an average of 20%p and reduces hallucination failure by 22%p.Representative rollouts show suppressed forget-feature and increased retain-feature activations after SOUL.Source: B. Main Results · p. 7, Figure 6 · p. 7

Behavior preservation and correction

• SOUL preserves 89% of originally successful tasks and non-hallucinated events, converts 40% of originally failed tasks to success, and corrects 78% of hallucinated events.The correction measure covers hallucinations across grasp, transport, and place stages.Source: C. Analyses · p. 7

Effect of region guidance

• Region-guided feature selection achieves the lowest hallucination rates and highest success rates, outperforming tokenized and global selection.The comparison evaluates global, tokenized, and region-guided selection across OpenVLA on LIBERO-Plus and π0.5 on RoboCasa.Source: C. Analyses · pp. 7–8, Table II · p. 7

Contributions of forget score and retain objective

• Removing the retain objective maintains hallucination reduction but lowers clean success, while removing forget-score aggregation degrades both hallucination reduction and task success.The ablation indicates that aggregation and retention work together to balance selective forgetting with successful behavior.Source: C. Analyses · p. 8, Table III · p. 8

VI. CONCLUSION

• State hallucination coincides with weakened attention to task-relevant regions and distinct region-conditioned sparse feature patterns.The paper uses these interpretable features as targets for selectively unlearning hallucination-associated knowledge while preserving success-associated features.Source: §VI · p. 8
• Across VLA architectures and simulated and real-world settings, SOUL reduces state hallucination and improves task success while largely preserving manipulation capabilities.The conclusion frames interpretable sparse-feature analysis as a basis for modifying recurring undesirable behaviors in robot policies.Source: §VI · p. 8
• The study positions interpretable sparse-feature analysis as a way to move from diagnosing VLA failures to identifying actionable targets for policy unlearning.The proposed targets support selective modification of undesirable behaviors in embodied robot policies.Source: §VI · p. 8