PaperlayerView exact revision on arXiv

Source-linked AI summary

FailPatch: Failure Residual Patching for Vision-Language-Action Models

Peng Yu, Jiacheng Wang, Ziheng Zhang, Xuchong Zhang, Baoting Li, Zhuoyuan Yu, Yuxiang Chen, Tiancai Wang, Hongbin Sun

arXiv:2609.34175v1cs.RO

TL;DR

Successful demonstrations provide VLA action supervision but rarely cover failure-prone states, and deployment failures lack corrective actions. FailPatch uses demonstrations to ground residual patches and deployment trajectories to guide reliability-aware intervention in a frozen policy. It improves success across simulation and real-world tasks with only 0.52% trainable parameters.

  • Problem

    Successful demonstrations provide direct action supervision but rarely cover states reached after execution begins to deviate, while deployment failures lack corrective actions.

    Source: §1 · p. 1

  • Method

    FailPatch adds a Null-gated Residual Expert Bank to a frozen VLA’s action hidden space, using demonstration actions to ground patches and deployment trajectories to guide preservation or redirection.

    Source: §1 · p. 2, §3.1 · p. 2

  • Results

    11.0 percentage points over baseline on four RoboTwin tasks under clean evaluation, 9.5 points under clean-to-random generalization, and 16.7 points across three real-world tasks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Simulation and real-robot experiments demonstrate gains with high parameter efficiency and substantially reduced training time.

    Source: §5 · p. 7

    FailPatch relies on offline VLM failure-onset localization from RGB observations, which may be ambiguous when causal errors are occluded or depend on unobserved force and contact information.

    Source: S7 Limitations · p. 19

Abstract

from arXiv · show

Vision-Language-Action (VLA) policies are typically adapted using successful demonstrations, which provide direct action supervision but rarely cover failure-prone states. Deployment failures expose these states, yet lack the corrective actions needed for conventional supervised learning. We propose FailPatch, a failure-driven residual patching framework that decouples action supervision from execution-reliability supervision. Successful demonstrations ground how the policy should act, while deployment trajectories indicate when its behavior becomes unreliable. We further observe that action hidden representations exhibit clear linear separability between reliable and failure-associated states while directly conditioning action generation. Building on these insights, FailPatch introduces a Null-gated Residual Expert Bank into the action hidden space of a frozen VLA policy. A unified Preserve--Redirect--Trust objective retains the original policy in reliable states, selects residual experts in failure-associated states and redirects representations from failure regions toward success-associated regions under bounded intervention. With only 0.52% trainable parameters, FailPatch improves success rates by 11.0 percentage points on four long-horizon RoboTwin tasks under clean evaluation, 9.5 percentage points under clean-to-random generalization, and 16.7 percentage points over the baseline across three real-world tasks. Project and code: https://github.com/yupeng-2003/FailPatch.

1 Introduction

FailPatch addresses the gap between action-labeled demonstrations and deployment failures that reveal unreliable states but lack corrective actions. It uses the two sources of supervision to patch a frozen VLA policy, improving results in simulation and on real robots.

1 Introduction

  • Successful demonstrations provide direct action supervision but rarely cover states reached after execution deviates, while deployment failures lack corrective actions.In long-horizon tasks, small interaction errors can propagate across action chunks into poorly represented states.Source: §1 · p. 1

2 Related Work

  • Success-only fine-tuning uses demonstrations, while rollout-based reinforcement learning typically requires reward or value estimation and repeated interaction; supervised use of outcome-only failures remains underexplored.FailPatch uses demonstrations for action-grounded patches and deployment trajectories to supervise when the policy should be preserved or patched, without corrective-action labels.Source: §1 · p. 1, §2 · p. 2

1 Introduction

  • 11.0 percentage points over the 50-Task Base under clean evaluation and 9.5 points under clean-to-random generalization; real-world tasks improve by 16.7 points over baseline.The evaluation covers four long-horizon RoboTwin tasks and three physical manipulation tasks.Source: §1 · p. 2

3.1 Problem Formulation and Overview

  • The problem is to learn lightweight residual parameters while keeping the pretrained policy frozen, using demonstrations for action supervision and deployment rollouts for reliability supervision.Failed rollouts provide terminal outcomes but no temporally localized failure labels or corrective actions.Source: §3.1 · p. 2

3.2 Failure-Driven Residual Patching

  • A linear probe distinguishes nominal pre-onset from failure-associated action-hidden states, which directly condition action chunks and provide an interface for local residual patches.The pretrained backbone and action projection remain frozen.Source: §3.2 · p. 3

3.3 Preserve–Redirect–Trust Learning

  • Preserve–Redirect–Trust uses reliable rollouts and nominal failure prefixes for preservation, onset and post-onset states for redirection, demonstration actions for patches, and bounded interventions.For redirection, each failure-associated representation uses a local success prototype formed from nearby reliable representations.Source: §3.3 · pp. 3–4

4 Experiments

FailPatch improves success rates across simulation and real-world evaluations, including clean-to-random generalization, with efficient adaptation. Ablations and mechanistic analyses examine its components and localized interventions.

4.1 Simulation Experiments

  • 11.00 percentage points: FailPatch raises average clean success from 49.50% to 60.50% over the 50-Task Base across four RoboTwin tasks.It also consistently outperforms LoRA SFT, Full SFT, and Demo-Only REB.Source: §4.1 · p. 5
  • 9.50 percentage points: under clean-to-random generalization, FailPatch reaches 37.50% average success, the highest rate.It performs best on three tasks and ties for the highest rate on Stack Bowls.Source: §4.1 · p. 5, Table 2 · p. 5
  • FailPatch trains 0.52% of the VLA in approximately 3 hours, achieving a higher average success rate with 40.1× fewer trainable parameters and completing training 6× faster than PPO on the same hardware.Source: §4.1 · p. 6

4.2 Real-World Experiments

  • 16.7 percentage points: on three real-world tasks, FailPatch raises average success from 41.1% to 57.8% over the Three-Task Base.The evaluation reports success rates over 30 trials per task.Source: §4.2 · p. 6, Table 3 · p. 6

4.3 Ablation Studies

  • Redirect gives the largest standalone ablation gain; Preserve complements it, and adding Trust yields 60.50% performance.A parameter-matched single expert or removal of the Null expert reduces success; manual failure-onset labels yield comparable performance to VLM annotations.Source: §4.3 · p. 6, Table 4 · p. 6

4.4 Mechanistic Analysis

  • Action hidden representations distinguish nominal from failure-associated chunks most clearly, with balanced accuracy of 0.74 and ROC-AUC of 0.83.Shuffled-label controls remain near chance.Source: §4.4 · p. 6, Figure 5 · p. 7
  • The Null route dominates most chunks, while residual-expert probability peaks near failure onset; patching then reduces distance to the local success prototype.The routing returns toward Null after the localized correction.Source: §4.4 · p. 7, Figure 6 · p. 7
  • Residual experts learn distinct intervention directions, and a representative rescue succeeds through a small local trajectory adjustment near the target block.The baseline misses the grasp after a misaligned approach, while FailPatch corrects the approach at the same stage.Source: §4.4 · p. 7, Figure 7 · p. 7, Figure 8 · p. 7

5 Conclusion

FailPatch adapts pretrained VLA policies by separating action supervision from execution-reliability supervision. Simulation and real-robot experiments report performance gains with parameter-efficient adaptation, while the approach relies on offline VLM failure-onset annotations.

5 Conclusion

  • FailPatch uses successful demonstrations to ground executable residual patches and deployment trajectories to determine when to preserve or correct the frozen policy.It applies a Null-gated Residual Expert Bank in action hidden space through the Preserve–Redirect–Trust objective, without corrective-action labels or auxiliary modules at deployment.Source: §5 · p. 7
  • Experiments in simulation and on real robots demonstrate performance gains with high parameter efficiency and substantially reduced training time..Source: §5 · p. 7
  • FailPatch relies on offline VLM-based failure-onset annotations; reducing this dependence through self-supervised or uncertainty-aware localization remains future work.Source: §5 · p. 7

Overview

The supplement provides implementation and evaluation details omitted from the main paper, spanning method, annotation, experiments, ablations, mechanistic analysis, real-world evaluation, and limitations.

Overview

  • Additional method details cover the frozen policy, Residual Expert Bank, local success prototypes, optimization settings, and training and inference procedures.Source: Overview · p. 11
  • Supplementary sections describe offline VLM annotation and human agreement, simulation and real-world protocols, and experimental compute cost.Source: Overview · p. 11
  • Additional ablations and mechanistic analysis examine prototypes, failure-data efficiency, hyperparameters, objective components, model architecture, and the linear-probe protocol.The supplement also includes a Limitations section discussing FailPatch’s primary limitation.Source: Overview · p. 11

S1 Additional Method Details

FailPatch adds a trainable residual bank at the frozen VLA’s action-hidden interface and uses deployment-derived reliability supervision to preserve or redirect representations. Task-specific success prototypes and jointly trained Preserve, Redirect, and Trust branches guide the corrections.

S1.1 Backbone and Action-Hidden Interface

  • FailPatch reads three 224 × 224 RGB views and task instructions; action-token states have shape 50 × 1024 immediately before the frozen action projection.The backbone, visual encoder, language model, action expert, and action projection remain frozen during adaptation.Source: S1.1 Backbone and Action-Hidden… · p. 11

S1.2 Residual Expert Bank

  • The bank combines one parameter-free Null route with eight SwiGLU bottleneck experts, and execution selects a route using sparse Top-1 routing.Zero-initialized expert outputs initially preserve the base representation; the bank and router total about 17.3M trainable parameters.Source: S1.2 Residual Expert Bank · p. 11

S1.3 Local Success Prototypes

  • For each failure-associated representation, FailPatch retrieves 20 nearest neighbors from a task-specific pool of normalized reliable-success representations and takes their coordinate-wise median.Failed-rollout prefixes are used for Preserve supervision but excluded from the prototype pool.Source: S1.3 Local Success Prototypes · p. 11

S1.4 Optimization Details

  • Preserve supervises reliable-success and failed-rollout pre-onset chunks, while Redirect and Trust supervise failure-onset and later chunks using local success prototypes.Training also uses action-supervised demonstrations; the first 5k of 30k steps disable the Null route, and Trust penalizes hidden-space intervention magnitude.Source: S1.4 Optimization Details · pp. 11–12

S1.5 Training and Inference Procedure

S2 Offline VLM Annotation

An offline VLM annotates rollout reliability and failure onset from temporally ordered observations, enabling chunk-level labels for training. Human comparisons report close onset agreement and similar performance with human-labeled onsets.

S2.1 Input and Output Format

  • The annotator receives a temporally ordered contact sheet, task instruction, and terminal outcome, then returns failure onset, confidence, failure type, notes, and reliable-success status.For successful rollouts, onset and failure type are null; for failed rollouts, reliable-success is null.Source: S2.1 Input and Output Format · p. 12

S2.2 Verbatim Annotation Prompt

S2.3 Reliable-Success Filtering

  • Successful rollouts with substantive intermediate deviations are excluded from reliable-success pools, while short-lived motion jitter alone does not trigger exclusion.Examples include an incorrect object choice followed by switching back, visible collision, failed placement followed by retry, and a long stall followed by recovery.Source: S2.3 Reliable-Success Filtering · p. 12, Table 6 · p. 12

S2.4 Human Verification

  • 80% of VLM onset labels fall within one action chunk of human labels, and 95% fall within two chunks.Mean absolute error is 0.74 chunks; the mean signed error is +0.14 chunks, indicating a small tendency to predict onset later than humans. The human-onset ablation reports 61.00% average success versus 60.50% with GPT-5.5 onsets.Source: S2.4 Human Verification · p. 14, Table 7 · p. 14

S3 Experimental Details

The experimental protocol evaluates four long-horizon RoboTwin tasks under clean and randomized settings, using shared seeds and adaptation data across supervised baselines. It also specifies a separate PPO setup and reports training-time and rollout accounting conventions.

S3.1 Simulation Tasks

  • Simulation tasks use 50-action policy chunks and terminate when task-specific success predicates are met, with a maximum of 24 chunks for 1200-step tasks and 14 for 700-step tasks.Success is checked after each environment step; Table 8 gives the exact predicates.Source: S3.1 Simulation Tasks · p. 14, Table 8 · p. 15

S3.2 Per-Seed Object Randomization

S3.3 Clean-to-Random Domain Shift

  • Clean-to-random evaluation reuses the clean-adapted checkpoint and adds visual and scene randomization while keeping the camera configuration unchanged.Figure 10 shows representative sequences; the random setting changes backgrounds, clutter, table height, and lighting in addition to object variation.Source: S3.3 Clean-to-Random Domain Shift · p. 14, Figure 10 · p. 15

S3.4 Evaluation Protocol

S3.5 Supervised Baselines

  • Supervised adaptation baselines share demonstrations, normalization, augmentation, and optimization settings; Demo-Only REB uses FailPatch’s router and experts without deployment supervision.Outcome-conditioned baselines append success labels at episode or chunk level; LoRA SFT and Full SFT update approximately 50.0M and 3.35B parameters, respectively.Source: S3.5 Supervised Baselines · p. 14

S3.6 PPO Baseline

  • PPO starts from the same 50-Task Base, uses sparse terminal success rewards, and updates 694M action-expert parameters through 100 updates of fresh on-policy rollouts.Evaluation uses deterministic flow-ODE inference and the same seeds as supervised methods.Source: S3.6 PPO Baseline · p. 14

S3.7 Compute and Runtime

  • Reported adaptation times are wall-clock times on four NVIDIA H20 GPUs, excluding FailPatch’s one-time offline annotation and rollout-feature caching.FailPatch requires no environment interaction after its fixed deployment buffer is collected.Source: S3.7 Compute and Runtime · pp. 14–15

S4 Additional Ablation Studies

The ablations examine prototype choices, hyperparameter settings, objective and architecture components, and the amount of failure data. Results favor the default configuration, the full objective and architecture, and additional failure experience, with diminishing gains beyond 30 episodes per task.

S4.1 Prototype Construction

  • Prototype ablations vary neighbor count, aggregation, feature normalization, and local versus task-level prototypes while retaining the default pool and retrieval setup as the reference.The default uses 20 nearest task-specific reliable-success representations, normalized features, Euclidean retrieval, and coordinate-wise median aggregation.Source: S4.1 Prototype Construction · pp. 15–17, Table 12 · p. 16

S4.2 Hyperparameter Sensitivity

S4.3 Per-Task Objective and Architecture Ablations

  • Redirect gives the largest standalone improvement over Demo-Only REB, Preserve further complements it, and both a single expert and a bank without Null underperform the full model.The comparisons support diverse intervention directions and learned abstention; Table 14 reports objective, architecture, and annotation ablations.Source: S4.3 Per-Task Objective and… · p. 17, Table 14 · p. 17

S4.4 Failure-Data Efficiency

  • 49.25% average success with no failed rollouts rises to 53.0%, 58.5%, and 60.5% with 10, 30, and 50 failed rollouts per task, respectively.Thirty rollouts improve success by 9.25 percentage points over zero and come within 2.0 points of the full-data result; gains diminish beyond 30 episodes per task.Source: S4.4 Failure-Data Efficiency · p. 17

S5 Additional Mechanistic Analysis

The analysis defines rollout-level classes for probing reliable versus failure-associated states, then evaluates probes and varies the amount of failure data across simulation tasks.

S5 Additional Mechanistic Analysis

  • Pre-onset chunks form the nominal class, while onset and post-onset chunks from the same failed rollouts form the failure-associated class.The train/validation/test split is 80/10/10 at rollout level, keeping chunks from one trajectory in a single partition.Source: S5.1 Linear-Probe Protocol · p. 17
  • Each feature space uses an ℓ2-regularized logistic-regression probe with C=1.0 after training-split StandardScaler fitting; action predictions flatten the full 50 × 32 chunk.Source: S5.1 Linear-Probe Protocol · p. 17
  • Table 15 lists the language instructions and success criteria used for the three physical tasks.Source: Table 15 · p. 18
  • Figure 11 varies failed deployment episodes independently across four simulation tasks, with 50 episodes corresponding to the full FailPatch setting.The figure reports average success rate across those tasks.Source: Figure 11 · p. 18
  • Figure 5 confidence intervals are 95% bootstrap intervals from 1,000 rollout-level resamples, shuffled-label controls repeat ten times, and Table 16 reports aggregate test performance.Table 16 is identified as the aggregate linear-probe results table; the supplied passage does not give its values.Source: S5.1 Linear-Probe Protocol · p. 18, Table 16 · p. 18

S6 Real-World Experimental Details

The real-world setup uses a dual-arm platform with head- and wrist-mounted cameras and replans during chunked action execution. The task protocol specifies demonstrations, rollout collection, resets, and independent evaluation trials.

S6.1 Platform and Control

  • The physical platform uses two six-degree-of-freedom arms and three cameras; the policy predicts 50-step dual-arm action chunks, executes 25 steps, then replans from updated observations.Execution is subject to joint limits, velocity clipping, collision monitoring, and a hardware emergency stop; inference runs on one RTX 4090 GPU.Source: S6.1 Platform and Control · p. 18
  • Figure 12 identifies the head-mounted D435i, dual arms, and two wrist cameras in the real-robot setup.Source: Figure 12 · p. 18

S6.2 Tasks and Success Criteria

  • Table 15 lists the exact language instructions and success criteria for the three physical tasks.Source: S6.2 Tasks and Success Criteria · p. 18
  • One operator collects 100 demonstrations per task, and frozen-base-policy rollouts are collected until each task has 50 successful and 50 failed trajectories.The 300 demonstrations jointly train the Three-Task Base and are reused for adaptation; no human intervention is allowed after a trial begins, and trials reset independently.Source: S6.2 Tasks and Success Criteria · p. 18
  • Each task and method is evaluated in 30 independent trials in separate blocks, with every trial starting from a fresh reset; the study uses a fixed 30k-step checkpoint.Source: S6.2 Tasks and Success Criteria · pp. 18–19

S7 Limitations

FailPatch relies on an offline VLM to localize failure onset from RGB observations, which can be ambiguous in cases involving occlusion or unobserved force and contact information.

S7 Limitations

  • Failure-onset localization depends on an offline VLM interpreting RGB observations, leaving ambiguity when causal errors are occluded or depend on unobserved force and contact information.The human-annotated-onset ablation suggests modest localization errors have limited performance impact; proposed directions include proprioceptive or force signals and self-supervised or uncertainty-aware localization.Source: S7 Limitations · p. 19

Related papers

Loading 2609.34175v1…