PaperlayerView exact revision on arXiv

Source-linked AI summary

RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correction

Zheyuan Hu, Robyn Wu, Naveen Enock, Jasmine Li, Riya Kadakia, Zackory Erickson, Aviral Kumar

arXiv:2509.07953v1cs.ROcs.LG

TL;DR

Long-horizon imitation can plateau despite extensive demonstrations, as compounding errors and stochasticity challenge task completion. RaC adds a post-pretraining phase that trains on human interventions combining recovery and correction. Across real-world bimanual tasks, it outperforms full-demonstration and HG-DAgger-style collection, and success improves with more recovery maneuvers at test time.

  • Problem

    Imitation policies can plateau below perfect completion on long-horizon tasks, where compounding errors and stochasticity persist despite extensive demonstrations.

    Source: §1 · p. 1

  • Method

    RaC adds a post-pretraining phase that trains on human intervention trajectories pairing recovery to familiar states with corrective sub-task progress.

    Source: §1 · pp. 1–2

  • Results

    Across three real-world bimanual tasks, RaC achieves higher success rates and steeper scaling trends than full demonstrations or HG-DAgger-style interventions.

    Source: §1 · p. 3

  • Takeaways & Limitations

    RaC policies’ task success improves as they execute more recovery maneuvers at test time.

    Source: §1 · p. 3

    The stated task structure applies to most real-world long-horizon tasks where earlier sub-task progress matters later and aggregated teleoperator data is diverse, especially early in task solving.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

Modern paradigms for robot imitation train expressive policy architectures on large amounts of human demonstration data. Yet performance on contact-rich, deformable-object, and long-horizon tasks plateau far below perfect execution, even with thousands of expert demonstrations. This is due to the inefficiency of existing ``expert'' data collection procedures based on human teleoperation. To address this issue, we introduce RaC, a new phase of training on human-in-the-loop rollouts after imitation learning pre-training. In RaC, we fine-tune a robotic policy on human intervention trajectories that illustrate recovery and correction behaviors. Specifically, during a policy rollout, human operators intervene when failure appears imminent, first rewinding the robot back to a familiar, in-distribution state and then providing a corrective segment that completes the current sub-task. Training on this data composition expands the robotic skill repertoire to include retry and adaptation behaviors, which we show are crucial for boosting both efficiency and robustness on long-horizon tasks. Across three real-world bimanual control tasks: shirt hanging, airtight container lid sealing, takeout box packing, and a simulated assembly task, RaC outperforms the prior state-of-the-art using 10$\times$ less data collection time and samples. We also show that RaC enables test-time scaling: the performance of the trained RaC policy scales linearly in the number of recovery maneuvers it exhibits. Videos of the learned policy are available at https://rac-scaling-robot.github.io/.

1. Introduction

RaC addresses long-horizon imitation’s performance plateau by adding human intervention data that teaches recovery, retries, and correction after pre-training. On three real-world bimanual tasks, it outperforms full-demonstration and HG-DAgger-style collection and supports test-time scaling through recovery maneuvers.

1. Introduction

  • Imitation policies plateau below perfect completion as compounding errors and stochasticity challenge long-horizon tasks, even after thousands of demonstrations.The introduction cites roughly 75% success for placing a t-shirt on a hanger despite over 5000 demonstrations.Source: §1 · p. 1
  • RaC fine-tunes after imitation pre-training on intervention trajectories that pair recovery to familiar states with correction that advances the sub-task.The protocol also ends the episode after the intervention segment.Source: §1 · pp. 1–2
  • RaC explicitly trains recovery, retries, and adaptation alongside successful task execution, rather than relying on incidental recovery in teleoperation data.Only 3.68% of episodes in the cited DROID dataset contain recovery behavior.Source: §1 · p. 2
  • RaC policies show test-time scaling: task success improves as deployed rollouts execute more recovery maneuvers.This lets the policy retry multiple times.Source: §1 · p. 3
  • Across three real-world bimanual tasks, RaC achieves higher success rates and steeper scaling trends than full demonstrations or HG-DAgger-style interventions.The introduction also reports up to 1 order of magnitude greater data efficiency than these approaches.Source: §1 · pp. 2–3

2. Related Work

Related work establishes the data and intervention context for RaC: scaling imitation has produced strong skills but often requires many expert demonstrations, while existing interventions emphasize correction. RaC instead scales trajectory-level recovery and correction data for long-horizon tasks.

Scaling data in robotic learning

  • Robot imitation scaling has yielded strong generalist and specialist policies, but complex dexterous tasks can require thousands of high-quality expert demonstrations per skill.Prior scaling studies largely vary environmental diversity in short-horizon tasks, whereas RaC studies trajectory-level maneuvers for long-horizon tasks.Source: §2 · p. 3

Human-in-the-loop imitation learning

  • Unlike HG-DAgger-style approaches that treat interventions as expert solutions, RaC uses recovery segments that may undo sub-task progress and still improve scaling.This challenges the view that only task-optimal interventions are useful.Source: §2 · p. 3

Shared autonomy

  • RaC uses off-the-shelf VR joysticks with a lightweight software modification for immediate intervention, avoiding pose alignment with the robot end effectors.Earlier interfaces can be difficult for dexterous skills or require specialized equipment.Source: §2 · pp. 3–4

Recovery and correction in imitation learning

  • Prior recovery and correction methods use engineered rewind, simulation augmentation, policy changes, or residual learning; RaC treats recovery as a skill learned from human demonstrations.Brandfonbrener et al. studies a similar collection strategy through offline reinforcement learning, while RaC focuses on imitation-learning scaling.Source: §2 · p. 4

3. Background and Robot Setup

The paper frames data scaling around full demonstrations, recovery segments, and correction segments, then contrasts batched collection with iterative interventions. Its robot setup uses two arms and synchronized camera, state, and action streams.

3. Background and Robot Setup

  • The bimanual setup uses two 7-DoF xArm-7 manipulators with soft grippers, three RGB cameras, and synchronized robot state and action streams at 60Hz.RMPFlow provides inverse-kinematic motion generation with real-time collision avoidance and smooth arm motions.Source: §3 · p. 4, Figure 2 · p. 4
  • Figure 3 contrasts correction from out-of-distribution states with RaC’s recovery to a familiar state followed by correction.RaC’s sequence densifies coverage over familiar states and also teaches recovery toward a broader region of initial states.Source: Figure 3 · p. 5
  • The data comprises full successful demonstrations, recovery segments that return from failure to in-distribution states, and corrections that complete the current sub-task.The stated objective is to improve task-success scaling with data-collection budget.Source: §3 · p. 5
  • Collection starts with a Round 0 full-demonstration dataset and policy; later data can be gathered in a single batched expert round or iterative intervention-and-training rounds.The batched protocol adds K×R0 frames, while iterative collection performs K alternating intervention and training rounds.Source: §3 · p. 5

4. RaC: Scaling Recovery and Correction for Imitation Learning

RaC trains policies on recovery and correction behaviors, pairing resets to familiar states with task progress. The section describes how interventions are collected, guided, and used to train the policy.

4. RaC: Scaling Recovery and Correction for Imitation Learning

  • RaC pairs recovery to familiar in-distribution states with corrective task progress, teaching retry behaviors that address under-covered failures and compounding errors in long-horizon imitation.The paper argues that scaling full demonstrations leaves failure states under-covered, while recovery-and-correction trajectories promote recovery followed by retry.Source: §4 · pp. 5–6, §4.1 · p. 7

4.1. Understanding the Role of Recovery and Correction Segments in Imitation

  • When valid initial states are broad but goal states narrow, resetting to familiar states can be easier to learn than completing a sub-task, and repeated retries reduce the chance of never succeeding.The paper links recovery’s relative sample efficiency to the verification-generation gap and describes retries as exponentially reducing the probability of never succeeding on a sub-task.Source: §4.1 · p. 6

4.2. Scaling Recovery and Correction Segments in Human Teleoperation

  • RaC pairs each recovery segment with a correction segment, then terminates the episode after intervention to balance the skills taught and allocate collection toward early sub-tasks.The protocol avoids collecting later sub-tasks from states induced jointly by the policy and human operator, which may differ from the policy’s own state distribution.Source: §4.2 · p. 7

4.3. Shared Autonomy Interface and Guidance for Teleoperators

  • The VR interface uses a clutch for policy-to-human handover, while a gripper-visitation heatmap guides operators toward familiar in-distribution recovery regions.Relative pose deltas in a local frame remove the need for global posture alignment during takeover.Source: §4.3 · p. 8, Figure 5 · p. 8

4.4. Policy Architecture and Training via Imitation Learning

  • The policy uses a 300-million-parameter multimodal diffusion transformer with flow matching to fit action chunks from full demonstrations, successful own rollouts, and human intervention segments containing recoveries and corrections.Inference takes 10 Euler integration steps from t=0 to t=1; it runs once every 0.5 seconds, executing the first half of each action chunk before replanning.Source: §4.4 · p. 9

5. Experimental Evaluation of RaC

The evaluation compares RaC with batched full demonstrations and HG-DAgger across three real-world long-horizon tasks and a simulated assembly task. RaC achieves strong results with less data, while its recovery behavior supports robustness and test-time scaling.

5.1. Evaluation Domains and Task Setups

  • The study evaluates four bimanual dexterous manipulation tasks—three in the real world and one in simulation—requiring extended-horizon control and sequential completion of interdependent subtasks.The real-world tasks are shirt-hanging, airtight-container-lid-sealing, and clamshell-takeout-box-packing. Evaluations use 60 trials for real-world tasks and 100 for simulation.Source: §5.1 · pp. 10–11

5.2. Main Results: One Order of Magnitude Improvement in Data Efficiency

  • 5 hours of training data suffice to surpass 75% full-task success, while shirt-hanging results match or exceed prior work with an order of magnitude less data.Prior shirt-hanging results used thousands of expert demonstrations or more than 100 hours of teleoperation; Table 1 compares similar tasks.Source: §5.2 · pp. 11–12, Table 1 · p. 12
  • RaC achieves at least 2× better data efficiency than batched data collection on shirt-hanging, consistently outperforms HG-DAgger, and has a steeper scaling curve than either baseline.Scaling batched collection across all real-world tasks was infeasible, so that baseline was evaluated on shirt-hanging only.Source: §5.2 · p. 12

5.3. Examining the Properties of Policies Learned via RaC

  • RaC reduces the fraction of rollouts making little progress across successive rounds, whereas batched full-demonstration training does not show the same consistent trend, especially in simulation.The evaluations start from a broad set of initial configurations.Source: Figure 9 · p. 12, §5.3 · p. 13
  • Task success is linearly related to recovery frequency: policies with more recovery segments at deployment achieve higher overall performance.The analysis compares recovery-segment counts and task success across policy rounds.Source: §5.3 · p. 13, Figure 10 · p. 13
  • Successful RaC rollouts tend to be longer than those of other methods, and longer RaC rollouts correlate with better average performance and more successful rollouts.On the simulation task, successful HG-DAgger rollouts have the second-highest median length; full-demonstration policies have the shortest median.Source: §5.3 · pp. 13–14, Figure 11 · p. 13

5.4. Ablation Studies for the RaC Data Collection Protocol

  • RaC maintains a roughly 1:1 to 1:2 ratio of recovery to corrective frames, while conventional intervention data shifts toward corrective frames, reaching 1:3 and 1:10 ratios.The comparison covers four rounds in the simulation task.Source: §5.4 · p. 14, Figure 12 · p. 14
  • In simulation, terminating an episode after intervention yields more effective performance scaling than continuing the policy rollout after the intervention concludes.The authors hypothesize that continuing mixes human- and policy-influenced states, making later trajectory portions out-of-distribution for the learned policy.Source: §5.4 · pp. 14–15, Figure 12 · p. 14

6. Discussion, Conclusion, and Future Work

RaC scales the type of training data by pairing recovery and correction behaviors in human interventions. The authors report improved data efficiency and robust long-horizon performance, and identify online RL and VLA models as directions for future study.

6. Discussion, Conclusion, and Future Work

  • RaC scales not only data quantity but also data type by explicitly pairing recovery and correction behaviors collected through human interventions.The authors report that this helps policies mitigate compounding errors and retry after failures.Source: §6 · p. 15
  • Across the reported experiments, RaC yields robust policies on long-horizon, contact-rich tasks with substantially less data and better data efficiency than the compared approaches.The authors also report that more recovery segments and longer action times correlate with higher performance.Source: §6 · p. 15
  • Future directions include using RaC policies to initialize online RL fine-tuning and applying RaC to generalist vision-language-action models.The authors hypothesize that recovery could support structured exploration and provide stitching points for value-based training; whether recovery systematically emerges in VLA models remains unclear.Source: §6 · p. 15

A. Policy Architecture and Training Details

The experiments use a shared policy architecture and training configuration. The described multimodal DiT processes observation and action modalities with separate transformer weights that interact through attention.

A. Policy Architecture and Training Details

B. Example Rollouts on Various Tasks

The example rollouts illustrate recovery followed by task-specific correction in the three real-world tasks. The captions describe backward gripper motions as recovery and renewed insertion or grasping as correction.

B. Example Rollouts on Various Tasks

  • For shirt-hanging, recovery drives the gripper and hanger backward, and correction reinserts the hanger.Source: Figure 13 · p. 24
  • For airtight-container-lid-sealing, the caption describes recovery as driving the gripper and hanger backward and correction as reinserting the hanger.Source: Figure 14 · p. 24
  • For takeout-box-packing, recovery drives the gripper backward and correction regr asps the spatula.Source: Figure 15 · p. 24

C. Bimanual Manipulation Tasks Evaluation Protocols

Each policy is evaluated on 60 trials per task, with randomized initial conditions described for shirt hanging and lid sealing. The protocol also specifies confidence intervals for progress scores and binary full-task success rates.

C. Bimanual Manipulation Tasks Evaluation Protocols

  • 60 shirt-hanging trials combine five shirts, three initial orientations, and four uniformly randomized hanger locations.The setup uses child-size polo shirts, a child-size hanger, and a rack, matching ShirtEasy from ALOHA Unleashed.Source: C. Bimanual Manipulation Tasks… · p. 25
  • 60 lid-sealing trials uniformly randomize among five lid placements and 12 container locations.Source: C. Bimanual Manipulation Tasks… · p. 25
  • 60 takeout-box trials randomize burger locations over the board’s right half, while the box pile and spatula vary slightly in front of the board.Source: C. Bimanual Manipulation Tasks… · p. 25
  • 95% confidence intervals use task-progress scores for scaling curves and Wilson score intervals for binary full-task success rates.Progress scores are bounded by each task’s maximum number of subtasks.Source: C. Bimanual Manipulation Tasks… · p. 25

D. Comparison to Prior Works on the Shirt-Hanging Task

The shirt-hanging comparison uses ALOHA Unleashed’s ShirtEasy results and an estimated dataset duration, alongside Seed GR-3 results adjusted for differences in task and metric. The estimated GR-3 full-task success rate is derived from its published Sankey diagram.

D. Comparison to Prior Works on the Shirt-Hanging Task

  • 75% full-task success was reported for ALOHA Unleashed’s ShirtEasy with Diffusion Policy trained on ShirtEasy and ShirtMessy data.The ShirtEasy dataset contains 5,345 trajectories, estimated here as approximately 89 hours assuming one minute per trajectory.Source: D. Comparison to Prior Works on the… · p. 26
  • Seed GR-3 [7] differs in its final step, requiring mobile-base rotation from the table to the drying rack, while other subtasks remain largely consistent.Its reported average task progress gives partial credit for completed subtasks, unlike the full-success metric in Table 1.Source: D. Comparison to Prior Works on the… · p. 26
  • 7/11 ≈ 0.636 is the estimated Seed GR-3 [7] full-task success rate, calculated from the final-subtask bar and start location in its Figure 10 Sankey diagram.ALOHA Unleashed [50] does not report task-progress scores, so the comparison estimates GR-3’s full-task success rate from the diagram.Source: D. Comparison to Prior Works on the… · p. 26
Loading 2509.07953v1…