Blackout vs. Freeze: Analyzing Physical Failure Modes of VLAs under Camera Faults

Motivation

• Camera blackouts and frozen observations can produce unintended contact, faulty gripper commands, and risks to people or objects during physical manipulation.The robot and surrounding objects keep moving, so visual input may be blocked or outdated during deployment.Source: §1 · p. 2

Research gap and approach

• Prior evaluations leave unclear how camera faults create distinct physical failure modes and whether other views or proprioception compensate for missing visual information.The study compares blackout and freezing across scene and wrist cameras, fault onsets, π0.5 and GR00T policies, and policies trained with or without proprioception in simulation and on a real robot.Source: §1 · p. 2

Main findings

• Blackout and freezing conceal different physical vulnerabilities despite similarly low success: freezing is associated with more extreme joint behavior, while post-closure wrist blackout leads to more drops.Mitigations can recover task success in selected conditions while increasing unintended contact or object disturbance in others.Source: §1 · p. 2

2 BEHAVIOR AND PHYSICAL CONSEQUENCES UNDER CAMERA FAULTS

• The evaluation compares π0.5 and GR00T policies with and without proprioception across LIBERO suites, using blackout or a repeated fault-onset frame on scene or wrist cameras.Faults begin either at episode start or after gripper closure; outcomes include task success and physical indicators.Source: §2 · p. 2, §2.3 · p. 5

2.3 SIMILAR TASK SUCCESS DOES NOT INDICATE SIMILAR PHYSICAL OUTCOMES

• For π0.5 under wrist faults at episode start, success was 21.3% with blackout and 0.0% with freezing, while freezing raised OD from 5.3% to 42.7% and NTC from 1.4% to 19.5% of steps.F5 also rose from 0.0% to 36.0% of episodes under freezing; Table 2 reports physical indicators across models, cameras, fault types, and timings.Source: Table 2 · p. 4, §2.3 · p. 5
• For GR00T under wrist-camera faults at episode start, freezing raised JVE from 3.3% under blackout to 82.0%, and JLP from 20.7% to 81.3%.The authors relate the joint-behavior differences to frozen-input commands diverging from clean commands over time, especially under wrist freezing.Source: §2.2 · p. 4, Table 2 · p. 4, §2.3 · p. 5
• After gripper closure, wrist blackout produced more drops than freezing: 4.0% versus 2.0% for π0.5 and 6.7% versus 4.0% for GR00T; scene-view ordering reversed.For scene faults, freezing produced higher Drop rates than blackout: 5.3% versus 2.0% for π0.5, and 4.7% versus 2.7% for GR00T.Source: Table 2 · p. 4, §2.3 · p. 5

2.4 REAL-WORLD VALIDATION

• In real WidowX trials, π0.5 succeeded in 9/10 scene-blackout trials and 5/10 scene-freezing trials, but failed all wrist-fault trials; non-target contact or drops still occurred in several faulted trials.Two of the nine successful scene-blackout trials involved non-target contact; real-world joint-limit margins did not consistently mirror the simulation freezing effect.Source: §2.4 · pp. 5–6, Figure 3 · p. 5

2.5 BENEFITS OF PROPRIOCEPTION

• Proprioception improved task success mainly under blackout, but provided little recovery under freezing; under wrist blackout, success fell to zero without proprioception for both models.For π0.5 and GR00T, removing proprioception reduced scene-blackout success by 10.6 and 12.0 points, respectively; wrist-freezing success remained near zero regardless.Source: §2.5 · p. 6
• Proprioception reduced joint-motion risks and drops under wrist blackout, but did not uniformly lower physical risks across fault conditions.For π0.5 under wrist freezing at episode start, proprioception coincided with higher NTC (19.5% vs. 12.6%) and OD (42.7% vs. 37.3%).Source: §2.5 · p. 6

3 WHICH VISUAL INFORMATION CONTRIBUTES TO FAULT RESPONSES?

• The study disables rendering of selected robot components or objects in one camera while leaving the other view unchanged and preserving their physics and collisions.Interventions begin at episode start unless otherwise noted and are compared with unmodified rendering.Source: §3.1 · p. 6, §3 · p. 6

3.1 DIFFERENT VIEWS RELY ON DIFFERENT VISUAL CONTENT

• Scene-view palm visibility preserves 75.3% task success for π0.5, while removing wrist-view fingers lowers success to 25.3% for π0.5 and 12.7% for GR00T.Scene-view finger removal leaves success largely unchanged.Source: §3.1 · p. 6
• Removing wrist-view objects reduces task success to 0–2.7%; the remaining scene view does not compensate, and non-target contact increases in both models.The removal also affects physical outcomes beyond task failure.Source: §3.1 · pp. 6–7
• Proprioception raises π0.5 success from 18.7% to 63.3% when scene arm links and the palm are removed, and lowers JLP from 42.7% to 8.0% of episodes.Under wrist-object removal, it does not improve task success, but reduces π0.5 NTC from 12.6% to 9.5% of steps.Source: §3.1 · p. 7

4.1 TRAINING-BASED MITIGATION: CAMERA BLACKOUT TRAINING

• Blackout training transfers partly to freezing: scene-freezing success rises from 2.7% to 53.0%, and wrist-freezing success from 0.0% to 30.0%.Wrist-freezing performance remains low after training.Source: §4.1 · p. 8
• Matching selected attention ratios raises scene-blackout success only from 22.0% to 26.7%, versus 94.7% for the trained reference.The attention ratios were matched on fixed probes; whether more targeted interventions can yield reliable recovery remains open.Source: §4.1 · p. 8
• Blackout training reduces some physical failures but worsens others: without proprioception, wrist-freezing JVE falls from 2.93% to 0.39% of steps while NTC rises from 5.91% to 9.98%.With proprioception, scene-blackout training reduces OD under scene freezing from 25.3% to 6.0% of episodes, compared with 1.3% clean.Source: §4.1 · p. 8
• Blackout training restores task success substantially under blackout and partly under freezing, but does not eliminate or uniformly reduce physical failures; wrist freezing remains difficult, especially without proprioception.This training-based mitigation requires retraining the policy.Source: §4.1 · p. 8

4.2 TRAIN-FREE MITIGATION: MEAN EMBEDDING REPLACEMENT

• The training-free method replaces each faulty camera’s embeddings with camera-specific per-token means computed from uncorrupted training images.For freezing this removes scene-specific content; for blackout it changes how the absent visual information is represented.Source: §4.2 · pp. 8–9
• Mean-embedding replacement improves scene-fault success for π0.5, but GR00T’s response depends on fault type; wrist-fault recovery is more limited, while physical effects also vary by fault.For π0.5, replacement can raise scene-blackout success with proprioception while increasing object drops.Source: §4.2 · p. 9
• For freezing, mean replacement reduces π0.5 NTC, OD, and JVE across both cameras, with and without proprioception; without proprioception, scene-freezing JVE falls from 3.98% to 0.03% of steps.Under blackout, OD increases in both cameras and proprioception conditions.Source: §4.2 · p. 9
• Mean replacement changes how a faulty camera is represented without restoring its current visual information; its inconsistent gains motivate investigating how replacements should be selected.Conditioning replacement on the remaining camera view and proprioception is proposed for future work.Source: §4.2 · p. 9

Visual robustness and safety

• LIBERO-Plus evaluates visual perturbations and wrist-view support for task completion, while SafeVLA-Bench and ForesightSafety-VLA assess physical risks beyond task success [Lyu et al., 2026].This work contrasts missing and outdated information through blackout and freezing and compares physical outcomes even when success rates are similarly low.Source: §5 · p. 9

Visual content and proprioception

• VLA-Trace masks visual elements to identify dependencies [Shi et al., 2026a]; GAP and ReViP address proprioceptive dominance or false completion.This work examines camera-specific robot and object depictions, proprioceptive compensation, and physical indicators.Source: §5 · p. 9

Mitigating observation failures

• RoboPanoptes and RobustVLA train against observation perturbations [Xu et al., 2025], while ActFovea addresses visual delay without retraining.This work tests blackout training and inference-time mean-embedding replacement, including transfer to freezing and physical outcomes.Source: §5 · p. 9

6 CONCLUSION

• Blackout and freezing produce distinct physical failure modes that depend on the affected camera and fault timing.Proprioception partly compensates for removed robot depictions but does not restore task success when wrist-view object information is removed, even with the scene view available.Source: §6 · p. 9
• Neither other sensory inputs nor recovered task performance alone establishes a safe fault response; policies should account for remaining robot and object information and the physical risks of continuing or changing execution.Source: §6 · p. 10

• AI tools supported theoretical modeling, hypothesis refinement, experiment design, implementation, translation, qualitative analysis, related-work searches, and interpretation.Source: §7 · p. 10
• The authors state that AI tools did not generate synthetic datasets, formulate mathematical claims, prove them, or clean and reformat datasets.Source: §7 · p. 10
• One author manually reviewed AI-assisted ideas, designs, code, analyses, interpretations, and manuscript text, revising as needed and accepting responsibility for the final content.Source: §7 · p. 10

• The appendix details policy interfaces, evaluation cohorts, physical metrics, measurement-window sensitivity analyses, and additional experiments; reproduction code is available.Source: §8 · p. 10

A.1 TRAINING DATA AND POLICY CHECKPOINTS

• The simulation training set combines four LIBERO suites into 1,693 episodes and 273,465 frames, with separate policies trained with and without proprioception.π0.5 and GR00T use model-appropriate data formats and training configurations.Source: §A.1 · p. 13, Table 3 · p. 13

A.2 CLOSED-LOOP SIMULATION EVALUATION

• Simulation evaluation tests Spatial, Object, and Goal tasks across clean and camera-fault conditions, selectively removing image content while preserving the other inputs.Success rates pool 150 episodes per suite; the evaluation includes episode-start and post-closure faults.Source: §A.2 · p. 14, Table 4 · p. 14, Table 5 · p. 14

A.3 REAL-ROBOT TRAINING AND EVALUATION

• Real-robot evaluation uses a WidowX to place a block in a bowl, with ten trials per input condition and operator-recorded contact and drop outcomes.Policies train on 100 clean demonstrations; physical-event labels and joint-limit margins are reported at the trial level.Source: §A.3 · pp. 14–15

B PHYSICAL MEASUREMENTS AND WINDOW SENSITIVITY

• Physical measures include non-target contact, object displacement, joint-limit and velocity events, force, action magnitude, and grasp-conditioned target drops.Primary traces end at task completion or episode end; recorded force is sampled after each executed step.Source: B PHYSICAL MEASUREMENTS AND WINDOW… · p. 15
• For GR00T without proprioception, the blackout-minus-freezing Drop difference falls from 36.0% in the primary window to 16.0% over 105 steps and 17.3% post-onset.Both short-window comparisons retain positive differences for all four policies; π0.5 JVE differences are zero in the first 50 post-onset steps.Source: B PHYSICAL MEASUREMENTS AND WINDOW… · p. 16

C.1 FULL-ACTION ROLLOUT CONTEXT

• Full-action plots show native seven-coordinate commands across clean and blackout rollouts, while a separate comparison summarizes early and later command directions against clean execution.The rollout plots use the same 30 selected episodes per policy and mark post-termination time in gray.Source: Figure 7 · p. 18, §C.1 · p. 18, Figure 8 · p. 19

C.3 IMMEDIATE GRIPPER RESPONSES TO VISUAL REMOVAL

• With both cameras blacked out at fixed observations, π0.5 gripper commands shift substantially without proprioception but remain closer to clean-input commands with it.Source: §C.3 · p. 19, Figure 9 · p. 20

D.1 EXACT SUCCESS RATES

• Selective content-removal tables report task success and physical outcomes after removing robot or object depictions from one view, with the other camera left unmodified.Physical outcomes include non-target contact and sustained non-target-object displacement; no condition-level results are supplied here.Source: Table 10 · p. 20, §D.1 · p. 20, Table 11 · p. 21

E.2 TEMPORAL REFERENCE FOLLOWING

• In paired-view probes, π0.5 follows the scene reference more often during transport, whereas GR00T follows the wrist reference more often in all four phases.Near-end samples mostly come from opening, so they underrepresent completions without opening.Source: §E.2 · pp. 21, 23, §E.1 · p. 21, Figure 10 · p. 22

E.3 SPATIAL TRANSLATION-ACTION PROJECTION

• Wrist-view target shifts produce larger median translation-action projections in every reported distance bin; scene-view projections increase with distance, especially for π0.5.These target-shift responses may differ from camera influence under blackout or freezing.Source: §E.3 · p. 24, Table 13 · p. 24

F.1 CAMERA-BLACKOUT TRAINING RECIPES AND SUITE RESULTS

• Blackout-training recipes selectively black out one camera in a fraction of training samples, and task success is evaluated across LIBERO suites for π0.5 and GR00T.The supplied tables specify the training recipes and evaluation scope but provide no condition-level outcome values here.Source: §F.1 · pp. 24–25, Table 14 · p. 25, Table 15 · p. 26

F.2 COMPONENT TRANSPLANTS AFTER BLACKOUT TRAINING

• 90.0% success after VLM transplant, versus 38.0% for the vision encoder and projector and 26.0% for the action expert, from a 22.7% baseline.The fully trained donor reached 94.7% in a separate evaluation; the transplant does not identify which changes within the VLM account for the result.Source: Table 16 · p. 26, §F.2 · p. 26

F.3 PHYSICAL OUTCOMES AFTER BLACKOUT TRAINING

• Blackout-training evaluations report whole-episode success and physical outcomes over up to 105 steps; Object and Goal extend the Spatial measurements.NTC and JVE are step percentages, while OD is an episode percentage; Goal non-target metrics use only rollouts with a defined target.Source: §F.3 · pp. 26–27, Table 17 · p. 27, Figure 11 · p. 27

F.4 MEAN-EMBEDDING REPLACEMENT: CALIBRATION AND FAULT-ONSET RESULTS

• Mean-embedding replacement averages clean-image representations by camera and token position, then uses detector-triggered replacement while retaining the other camera’s tokens.Calibration differs by policy: π0.5 uses 2,550 Spatial training frames, while GR00T uses clean rollouts from six held-out initial states per task.Source: §F.4 · p. 28

F.5 ATTENTION CHANGES AFTER BLACKOUT TRAINING

• Attention-share probes compare control and trained policies on a cohort separate from success rollouts, so they cannot establish that attention changes cause recovery.The probes cover scene vision, wrist vision, and state across clean and camera-fault conditions.Source: §F.5 · p. 31, Figure 14 · p. 31

G.1 INFERENCE-TIME ATTENTION REWEIGHTING

• Increasing state-token bias under scene blackout lowers success from 24.7% at zero bias to 0% at gstate = 4; matching trained-policy attention ratios recovers little of the blackout-training gain.Under scene blackout, full ratio adjustment recovers 6.4% of the gain, with a 95% interval of [0.0, 12.9]%; no improvement survives Holm correction across eight settings.Source: §G.1 · pp. 31–32, Table 20 · p. 32, Table 21 · p. 32

G.2 ORACLE CAMERA-TOKEN MASKING

• Prefix-and-expert masking raises scene-freezing success from 1.3% to 47.3% for the released checkpoint, with no comparable gain in the fine-tuned pair.Expert-only masking leaves wrist-freezing success at zero in all three policies; recovery from faults arising during manipulation remains untested.Source: §G.2 · p. 33, Table 22 · p. 33

G.3 TRAINING WITH OLDER OBSERVATIONS

• Training uses delayed images for one randomly chosen camera in 10% of samples; the short-budget comparison adds binary freshness indicators through a new state-projection pathway.Continued training from the released policy compares landmark freezing and delayed images, with mixtures sampling 8–40-frame or 2–8-frame delays; filtering yields different replacement rates of 5.7% and 5.0%.Source: §G.3 · pp. 33–34
• Augmentation raises scene-freezing success from 4% to 22% but leaves wrist-freezing success at 2%; freshness indicators raise wrist-freezing success to 10%, a nonsignificant difference.At 12,000 updates, freshness indicators change success by +6% under wrist freezing and −8% under scene freezing relative to augmentation alone; neither comparison is significant.Source: §G.3 · p. 33, Table 23 · p. 34
• 98.7% clean success with the 8–40-frame mixture accompanies 7.3% scene-freezing success, while the 2–8-frame mixture reaches 1.3%; neither recipe recovers wrist-fault success.Adding consistency lowers clean success to 73.3% and yields 4.7% scene-freezing success, with no wrist-freezing recovery.Source: Table 24 · p. 34, §G.3 · p. 34