PaperlayerView exact revision on arXiv

Source-linked AI summary

CARE: Experience-Guided Atomic Corrective Execution for Vision-Language-Action Policies

Junlan Xiao, Junwei Jiang, Zaibin Zhang, Yifan Wang, Zhongbo Zhang, Huchuan Lu, Lijun Wang

arXiv:2609.24118v1cs.RO

TL;DR

VLA policies are often trained and evaluated on nominal executions, leaving recovery from intermediate failures underrepresented. CARE learns corrective behaviors from stage-conditioned failures and uses 3D monitoring to select atomic corrections; it improves task success across simulation and real-world evaluations.

  • Problem

    Most VLA policies are trained primarily on expert demonstrations and evaluated from nominal initial states, while execution failures can produce underrepresented intermediate states.

    Source: §I · p. 1

  • Method

    CARE models stage-conditioned deviations from failed rollouts, synthesizes corrective demonstrations from them, and uses stage-wise planning with 3D monitoring to select atomic corrections.

    Source: §I · pp. 1–2

  • Results

    CARE gains an average of 14.5 task-success points in simulation and 15.9 points in the real world, with a 7.5-point recovery-success gain on FSR-Bench.

    Source: §I · p. 2

  • Takeaways & Limitations

    Across simulation, recovery-specific, and real-world evaluations, CARE consistently improves recovery and robustness across diverse VLA backbones.

    Source: §VI · p. 7

Abstract

from arXiv · show

Vision-Language-Action (VLA) policies achieve strong performance in robotic manipulation but remain brittle once execution deviates from nominal trajectories. We propose CARE (Corrective Atomic Robotic Execution), a framework that improves recovery by learning from failures encountered during execution. Instead of generating corrective data from manually designed or random perturbations, CARE collects failed rollouts, models stage-conditioned post-failure deviations, and uses the resulting empirical distributions to synthesize representative failure states and corrective demonstrations. At inference time, CARE combines stage-wise planning with physically grounded 3D monitoring to trigger atomic adjustments or re-operations while preserving task progress. We further introduce the Failure State Recovery Benchmark (FSR-Bench), which evaluates recovery from intermediate failure states under local deviations and structural anomalies. Experiments across multiple VLA backbones, simulation benchmarks, and real-world dual-arm tasks show consistent improvements, with average task-success gains of 14.5 points in simulation and 15.9 points in the real world. Code, models, and data are available at https://github.com/xiaojunlan/care

I. INTRODUCTION

VLA policies are often trained and evaluated on nominal executions, leaving recovery from intermediate failures underrepresented. CARE learns corrective supervision from execution failures and invokes atomic corrections through 3D monitoring; FSR-Bench evaluates recovery from intermediate failure states.

I. INTRODUCTION

  • Failures such as slipping, tipping, misplacement, or unsynchronized arms can move execution into states rarely represented in training data, motivating recovery from intermediate failure states.Most policies are trained primarily on expert demonstrations and evaluated from nominal initial states.Source: §I · p. 1

C. Robotic Manipulation Benchmarks

  • FSR-Bench evaluates recovery independently from nominal task execution, using intermediate failure states that include local deviations and structural failures requiring multi-step or coordinated recovery.It fills a gap in benchmarks that primarily assess execution from predefined initial states.Source: §I · p. 2, C. Robotic Manipulation Benchmarks · p. 2

I. INTRODUCTION

  • CARE yields average task-success gains of 14.5 points in simulation and 15.9 points in the real world, plus a 7.5-point recovery-success gain on FSR-Bench.The evaluations span multiple benchmark–backbone settings, RoboTwin 2.0, RoboFactory, FSR-Bench, and real-world dual-arm tasks.Source: §I · p. 2

A. Robotic Self-correction Systems

  • Prior recovery methods use rollback or replanning, corrective demonstrations around perturbed states, or other interventions, but do not explicitly model stage-conditioned post-failure deviations for synthesis.The introduction frames execution failures as potential structured supervision rather than exceptions to remove.Source: §I · p. 1, A. Robotic Self-correction Systems · p. 2

B. Vision-Language-Action Models

  • CARE is designed to be orthogonal to continuous and tokenized VLA architectures, and is instantiated on π0 and π0-FAST.Existing VLA approaches improve generalization and action modeling, while failure recovery remains relatively underexplored.Source: B. Vision-Language-Action Models · p. 2

A. Overview

  • CARE models stage-conditioned deviations from failed executions, uses them to synthesize corrective demonstrations, and combines 3D monitoring with atomic-stage planning at inference.Nominal and corrective instructions are handled by the same VLA executor.Source: A. Overview · p. 2, §I · p. 2

1) Failure Modeling:

  • For each task and atomic stage, CARE characterizes relative translation and rotation deviations at stage-critical events, including grasp closure and placement release.The model fits scalar deviation components to candidate distributions and selects fits using AIC and the KS test.Source: 1) Failure Modeling · pp. 2–3

2) Corrective Data Synthesis:

CARE synthesizes corrective data from stage-relevant deviations and uses a fixed stage plan to select atomic instructions during execution. A shared VLA executor carries out nominal and corrective instructions.

2) Corrective Data Synthesis:

  • At stage k, CARE samples a deviation and perturbs the stage-relevant relative pose to create a new failure state.The system rolls the state forward under physical dynamics; simulation uses an IK/motion-planning oracle for demonstrations, while real-world demonstrations use human teleoperation.Source: 2) Corrective Data Synthesis · p. 3
  • Corrective demonstrations are short segments that end once the atomic recovery target is reached.Synthesized failures are newly generated physical states rather than direct replays of collected failures.Source: 2) Corrective Data Synthesis · p. 3

C. Atomic Corrective Execution Framework

  • A VLM planner decomposes the task into atomic stages, while a 3D geometric monitor selects whether to continue, adjust, or re-operate.The planner and monitor select instructions; the same VLA executor generates all low-level actions.Source: C. Atomic Corrective Execution Framework · p. 3

1) VLM Planner:

  • The planner is queried once at initialization; its stage plans and candidate instructions remain fixed while CARE updates the active stage and selects an instruction online.This separates semantic task decomposition from high-frequency visuomotor control and avoids repeated VLM calls during recovery.Source: 1) VLM Planner · p. 3

2) 3D Geometric Monitor:

CARE monitors stage-relevant geometry to choose nominal or corrective instructions, then executes them with a shared VLA policy. FSR-Bench tests recovery from intermediate failures across local and structural scenarios.

2) 3D Geometric Monitor:

  • The 3D monitor checks point-cloud predicates at stage-critical transitions and tracks gripper states, using geometric relations such as gripper–object alignment and object–target displacement.It is a geometric checker rather than a learned recovery policy or VLM-based judge.Source: 2) 3D Geometric Monitor · pp. 3–4
  • Consistent geometry retains the nominal instruction, in-stage deviations trigger adjustment, and failed outcomes trigger re-operation; the monitor retries until the stage terminates or the retry budget is exhausted.Repeated monitor–execute–verify transitions allow multi-step recovery over the fixed plan.Source: 2) 3D Geometric Monitor · p. 4

3) VLA Executor:

  • The VLA executor receives the current observation and selected atomic instruction, using the same policy for nominal, adjustment, and re-operation instructions.Observations include robot state and global- and wrist-view RGB images.Source: 3) VLA Executor · p. 4

D. FSR-Bench

  • FSR-Bench has 36 recovery scenarios across five tasks: 21 Easy local failures and 15 Hard structural failures requiring multi-step or coordinated recovery.Easy examples include grasp failure and placement offset; Hard examples include tipping, accidental drop, and target occupation.Source: D. FSR-Bench · p. 4, Figure 3 · p. 4
  • Benchmark training uses uniformly sampled perturbations, while evaluation failures are generated from stage-conditioned deviations observed in failed nominal-policy rollouts and physically rolled forward.Each evaluation episode begins from a new failure state rather than a replay, and methods share the same corrective training data and evaluation initializations.Source: D. FSR-Bench · pp. 4–5
  • FSR-Bench evaluation uses 100 randomized trials per task–regime pair and computes recovery success over trials from the corresponding failure scenarios.Evaluation also varies object geometry, table texture, lighting, distractors, and object placement.Source: D. FSR-Bench · p. 5

IV. EXPERIMENTS

Experiments compare CARE with VLA baselines across simulation settings and evaluate recovery from intermediate failures on FSR-Bench.

A. Experimental Setup

  • Evaluation covered seven hard RoboTwin bimanual tasks, three RoboFactory multi-arm tasks, five FSR-Bench tasks, and four real-world dual-arm tasks; short and long horizons mean <200 and 200–400 steps.Source: A. Experimental Setup · p. 5
  • Simulation CARE data used 50 corrective trajectories per error type synthesized from 100 preliminary rollouts; real-world tasks used 20 additional trajectories per error type and 100 evaluation trials per task.Source: A. Experimental Setup · p. 5

B. Main Results in Simulation

C. Results in FSR-Bench

D. Main Results in Real World

On the SO-101, CARE outperforms FailSafe across two VLA backbones; ablations also favor corrective data, experience-guided sampling, and 3D monitoring.

D. Main Results in Real World

  • On SO-101, CARE raises π0 average success from 30.8% to 50.0% and π0-FAST from 18.8% to 31.3%, exceeding FailSafe by 13.7 and 7.3 points, respectively.Success rates are reported over 100 trials per task.Source: Table IV · p. 6, D. Main Results in Real World · p. 6

A. Ablation of Components

  • 23.2 points on π0 and 19.8 on π0-FAST are full CARE’s gains over baselines; against data-only, it adds 14.0 and 10.8 points.Data alone adds 9.2 points on π0 and 9.0 on π0-FAST, versus 4.2 and 2.0 points from Execution alone.Source: Table V · p. 7, A. Ablation of Components · p. 7

B. Ablation of Corrective Data Collection

  • With matched perturbation ranges, trajectory counts, and training settings, experience-guided sampling improves average success over uniform sampling by 12.2 points for π0 and 8.8 for π0-FAST.The comparison changes only the sampling distribution.Source: Table VI · p. 7, B. Ablation of Corrective Data… · p. 7

C. Ablation of Monitoring Mechanisms

  • 93.8% versus 71.4% average accuracy and 0.24 s versus 3.40 s monitor latency favor CARE’s 3D monitor across three simulated and two real-world tasks.The latency comparison includes perception, point-cloud construction, and predicate evaluation; the reported speedup is 14.1×.Source: Table VII · p. 7, C. Ablation of Monitoring Mechanisms · p. 7

VI. CONCLUSION

  • Across simulation, recovery-specific, and real-world evaluations, CARE improves recovery and robustness over diverse VLA backbones; future work will integrate reinforcement learning to further enhance recovery.The conclusion describes CARE as combining experience-guided corrective data synthesis with 3D-monitor-driven atomic corrective execution and introduces FSR-Bench.Source: §VI · p. 7

VIII. EXPERIENCE-GUIDED CORRECTIVE DATA SYNTHESIS DETAILS

CARE models failure deviations conditioned on atomic stage, fits empirical priors, and uses them to synthesize physical failure states and short corrective demonstrations.

VIII. EXPERIENCE-GUIDED CORRECTIVE DATA SYNTHESIS DETAILS

  • From 100 preliminary rollouts, CARE records stage failures and models grasp- or placement-specific translation and rotation errors as p(d | k), rather than one task-level prior.Grasp deviations are measured at gripper closure; placement deviations are measured at object release.Source: §VIII · p. 10
  • For the tested tasks, deviations use three translation variables and a yaw rotation variable; candidate distributions are selected using goodness-of-fit statistics and AIC.The candidate families include Gaussian, Beta, Gamma, Weibull, Log-Normal, and Uniform distributions.Source: §VIII · p. 10
  • After sampling a deviation and perturbing the stage-relevant pose, CARE rolls the system forward under physical dynamics and uses the resulting state for expert correction.These synthesized states are newly generated physical states, not direct replays of collected failures.Source: §VIII · p. 10
  • Corrective demonstrations stop once the current atomic recovery target is reached, such as after stable grasp recovery, rather than continuing through the full task.The short-horizon design reduces collection cost while preserving corrective-behavior supervision.Source: §VIII · p. 10
  • FSR-Bench trains recovery with uniform perturbations but samples evaluation initial states from empirical stage-conditioned failure distributions, avoiding training-test coupling.This protocol differs from the experience-guided synthesis used in standard CARE experiments.Source: §VIII · p. 10

A. Offline VLM planner

The offline VLM planner converts the task into fixed atomic-stage plans, specifying nominal actions, monitored geometry, termination conditions, and stage-specific correction options.

A. Offline VLM planner

  • Queried once at initialization, the planner uses the task instruction and initial global image to decompose the task into K ordered atomic stages.Its inputs also include task and agent descriptions, the atomic action pool, and deviation statistics.Source: A. Offline VLM planner · pp. 10–11
  • Each stage plan assigns nominal actions and observable termination conditions, with geometric guidance and corrective instructions tailored to relevant deviations.The planner separates intra-stage adjustments from stage-level re-operations such as re-grasping or re-placing.Source: A. Offline VLM planner · p. 11
  • Plans respect object dependencies, reachability, and inter-arm coordination, then remain fixed during low-level execution.The planner assigns instructions from the atomic action pool and specifies monitored cues and stage termination conditions.Source: A. Offline VLM planner · pp. 10–11

B. Event-triggered 3D geometric monitor

CARE’s 3D monitor checks stage-relevant geometric predicates at execution events and selects atomic adjustments or re-operations when needed. Its skill-level templates are reused across tasks, with stage-specific cues and calibrated tolerances.

B. Event-triggered 3D geometric monitor

  • At triggered checks, masks and depth are fused into point clouds for the gripper, manipulated object, and target region.Geometric perception is invoked at stage-critical transitions or shortly before a predicted transition when adjustment is enabled.Source: B. Event-triggered 3D geometric monitor · p. 11
  • The monitor reuses skill-level geometric predicate templates across tasks, supplying stage-relevant cues and rollout-calibrated tolerances rather than task-specific learned classifiers.The same VLA executor subsequently runs nominal, adjustment, and re-operation instructions.Source: B. Event-triggered 3D geometric monitor · p. 11
  • During grasping and approach, the monitor projects local geometry onto a plane orthogonal to the estimated approach vector and checks grasp conditions.For placement, the effective grasp center and target position are projected onto the support plane.Source: B. Event-triggered 3D geometric monitor · p. 11
  • Active predicates select an intra-execution adjustment or a post-stage re-operation, with geometric tolerances calibrated from stage-conditioned rollout statistics.The empty-grasp threshold is set by the parallel-jaw gripper’s mechanical closing limit.Source: B. Event-triggered 3D geometric monitor · pp. 11–12

X. FSR-BENCH DETAILS

FSR-Bench measures recovery from intermediate failure states across local and structural anomalies. It includes 36 scenarios across five tasks, with separate Easy and Hard regimes.

X. FSR-BENCH DETAILS

  • FSR-Bench contains 36 recovery scenarios across five tasks: 21 Easy local failures and 15 Hard structural failures.Easy cases typically admit one corrective operation; Hard cases require multi-step correction or coordinated multi-arm recovery.Source: §X · p. 12
  • Each episode starts from an intermediate failure state and evaluates whether the policy reaches a scenario-specific recovery target within a predefined horizon.Recovery is evaluated independently from nominal task execution.Source: §X · p. 12
  • Table VIII lists the maximum recovery steps for each task, while Table IX maps failure-state types to tasks.The larger Hard-regime budgets reflect the additional atomic operations often required by structural anomalies.Source: Table VIII · p. 12, Table IX · p. 12, §X · p. 12

SKILLET PLACEMENT RECOVERY; BHR: BLOCK HANDOVER

This material details FSR-Bench scenarios and evaluation, online corrective interventions, event-triggered monitoring, and the rollout-budget ablation. Together, it describes how recovery cases are generated and assessed and how CARE’s corrective behaviors are used.

SKILLET PLACEMENT RECOVERY; BHR: BLOCK HANDOVER

A. Training and test distributions

  • Training perturbations are sampled uniformly within each scenario’s predefined range, while test failures are generated by injecting deviations sampled from stage-conditioned empirical failure distributions.Test episodes also randomize geometry, texture, lighting, distractors, and object placement; methods share the same pre-generated failure initializations.Source: A. Training and test distributions · p. 13

B. Evaluation protocol

  • Each task–regime pair uses 100 evaluation trials, with recovery success measured by reaching the shared scenario-specific target within its prescribed horizon.Methods use the same pre-generated initializations for each task and regime.Source: B. Evaluation protocol · p. 13

A. Analysis of corrective interventions

  • Successful recovery episodes use both intra-execution adjustments and post-stage re-operations, so the figure shows how corrective primitives are invoked online.This intervention-level view complements rather than replaces the component ablation of corrective data and atomic corrective execution.Source: A. Analysis of corrective interventions · p. 13

B. Event-triggered monitoring protocol

  • Monitoring checks occur at stage-critical events; stages supporting adjustment also receive a check shortly before a predicted transition, using an action-horizon window of 50.Stages without gripper-state-associated boundaries are checked once per 50 executed actions.Source: B. Event-triggered monitoring protocol · p. 13

C. Ablation on the Number of Experience-Guided Rollouts

  • 51.5% average success rate with 100 rollouts, compared with 45.0% with 50 and 51.0% with 150, supports 100 as a practical collection–stability trade-off.For Put Object Cabinet, at 100 rollouts the JS divergences for dx, dy, dz, and dr are 0.0224, 0.0200, 0.0374, and 0.0245; at 150 they are 0.0145, 0.0113, 0.0141, and 0.0174.Source: C. Ablation on the Number of… · p. 13

D. Ablation on Corrective Recovery Strategies

The recovery-strategy ablation reports limited improvement from rollback relative to CARE, while the monitor evaluation reports accuracy on a diverse-object task. The section also specifies data and optimization budgets across benchmark settings.

E. Accuracy of the 3D Monitor in Instruction Judgement

  • 96% monitoring accuracy is reported for Put Object Cabinet using the same skill-level predicate templates across many object categories, without a task-specific learned classifier.The additional monitoring analysis extends to object diversity and real-world dual-arm manipulation.Source: E. Accuracy of the 3D Monitor in… · p. 14, Table XII · p. 15

A. Data budgets and VLA training parameters

  • For standard simulation tasks, CARE uses 150 nominal expert demonstrations and 50 experience-guided additional trajectories per error type, with distributions estimated from 100 preliminary rollouts.π0 and π0-FAST are fine-tuned for 40,000 steps with batch size 32 and action horizon 50; FSR-Bench uses 50 uniformly sampled corrective trajectories per recovery type and 20,000 training steps.Source: A. Data budgets and VLA training… · pp. 14–15

D. Ablation on Corrective Recovery Strategies

  • CARE achieves substantially larger gains than rollback, which provides only limited improvement on three simulation tasks with the π0 backbone.Rollback restores the final state of the previous atomic stage and resumes nominal execution without an explicit corrective instruction for the current failure.Source: D. Ablation on Corrective Recovery… · pp. 13–14, Table XI · p. 15

B. Real-world setup and evaluation

The real-world evaluation uses a dual-arm SO-101 platform and a matched data-budget protocol. Evaluation spans simulation, recovery tasks, real-world tasks, and visualized nominal and corrective executions.

B. Real-world setup and evaluation

  • The real-world platform has 12 DoF across two SO-101 arms, one global RGB camera, and one wrist RGB camera per arm.Each task uses 50 nominal demonstrations, 20 additional trajectories per error type, 100 preliminary executions to estimate failure distributions, and 100 evaluation trials.Source: B. Real-world setup and evaluation · p. 15

C. Evaluation coverage and action vocabulary

XIII. VISUALIZATION

  • Place Dual Shoes is visualized as three nominal atomic actions, followed by corrective executions shown separately for Stages 1–3.The corrective stages are illustrated in Figs. 12–14.Source: §XIII · p. 16, Figure 11 · p. 18, Figure 12 · p. 18
  • Training-configuration tables are provided for π0-FAST, π0, DP, DP3, ACT, and RDT-1B.The listed configurations cover RoboTwin and RoboFactory for π0-FAST, π0, and DP, and RoboTwin for DP3, ACT, and RDT-1B.Source: Table XV · p. 16, Table XVI · p. 16, Table XVII · p. 16
  • FSR-Bench visualizations cover 36 failure states, including 21 easy and 15 hard cases spanning local deviations and structural failures.The states are shown across two figure parts.Source: Figure 15 · p. 21, Figure 16 · p. 22
Loading 2609.24118v1…