Source-linked AI summary
When, Where, and What to Correct: Human-Guided Adaptive Residual Correction for Robot Policies
Anukriti Singh, Maxime Caniot, Pratap Tokekar, Firas Abi-Farraj
TL;DR
Imitation policies can fail in localized precision-critical regions, while full-policy updates or unrestricted residual exploration treat those failures as global problems. ARC uses sparse human corrections to structure residual learning on a frozen policy, with task reward determining the executed correction. Across three real-robot precision-manipulation tasks, average task success rises from 33.3% to 75.0%.
Problem
Imitation policies may remain brittle in localized precision-critical regions, and directly imitating an intervention does not establish whether it leads to task success.
Source: §1 · pp. 1–2Method
ARC uses sparse human corrections to define when residual control activates and where it searches, then uses task reward to train the residual on a frozen policy.
Source: §1 · p. 2Results
33.3% to 75.0% average task success across three real-robot precision-manipulation tasks.
Source: §1 · p. 2Takeaways & Limitations
Sparse corrections can localize and constrain autonomous post-training without updating the base policy or serving as final action targets.
Source: §1 · p. 2, §5 · p. 7The gate requires task- and operator-specific correction examples, and may vary with how different humans choose when to intervene.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Imitation policies that reliably execute most of a manipulation trajectory can still fail in localized regions requiring last-mile precision. Updating the full policy or exploring residual actions everywhere treats this local failure as a global learning problem. We introduce Adaptive Residual Correction (ARC), a lightweight post-training module that uses reinforcement learning (RL) to learn a residual correction on top of a frozen policy while using sparse human corrections to structure autonomous exploration. Rather than treating human corrections as imitation targets, ARC uses pre-intervention states to determine 'when' residual control should become active and correction geometry to determine 'where' in residual action space autonomous refinement should explore; task reward determines 'what' residual is ultimately executed. Across three precision-manipulation tasks on a real robot, ARC improves average task success from 33.3% to 75.0%. The human corrections define task-dependent 2-3 dimensional subspaces of the seven-joint residual space that retain 96.0-97.7% of squared correction magnitude. Together, these results show that sparse human corrections can localize and constrain autonomous post-training without requiring the observed correction itself to serve as the final action target.
1 Introduction
ARC uses sparse human corrections to focus residual learning in time and action space while keeping the base policy frozen. Across three real-robot precision-manipulation tasks, it improves average task success from 33.3% to 75.0%.
1 Introduction
- Imitation policies can remain brittle around short precision-critical segments, where small execution errors and distribution shift matter.Source: §1 · pp. 1–2
- Sparse corrections localize both intervention timing and residual directions; deployment must recognize robot-controlled states where a human would soon intervene.Source: §1 · pp. 1–2
- ARC learns a pre-intervention gate and corrective basis from approximately 8–10 correction rollouts, then uses separate autonomous experience and task reward to learn residuals.Source: §1 · p. 2
- Unlike correction-supervised methods [10] [11], ARC uses human residuals to define autonomous learning support rather than as final actor targets.Source: §1 · p. 2
- 33.3% to 75.0% average task success: ARC improves outcomes across three real-robot precision-manipulation tasks.Source: §1 · p. 2
2 Related Work
Related work covers residual post-training, learning from corrections, and methods that structure refinement. ARC combines temporal and action-space support from sparse corrections while using task reward to determine the final residual.
Residual refinement
Learning from corrections
3 Adaptive Residual Correction (ARC)
Adaptive Residual Correction (ARC) keeps the base policy frozen and uses sparse human corrections to localize when residual control activates and where it explores. Autonomous task reward then determines the residual executed within that support.
3.1 Problem Formulation
- ARC adds a gated residual in a human-derived corrective basis to the frozen base action, using a binary gate and low-dimensional coefficients mapped into seven-joint actions.The gripper remains controlled by the base policy.Source: §3.1 · p. 3
3.2 WHEN: Localizing Residual Refinement
- The pre-intervention gate predicts an upcoming correction from robot-controlled states, using the five states before each correction bout as positive examples.Correction-active frames are excluded; deployment thresholds and temporal hysteresis produce the binary gate, which leaves the base action unchanged when closed.Source: §3.2 · p. 3
3.4 WHAT: Reward-Driven Residual Learning
- Autonomous exploration selects a corrective-basis direction and sign per rollout, applying a fixed residual only while the gate is open.This creates coherent action–outcome variation while restricting perturbations to the selected refinement region.Source: §3.4 · p. 4
- ARC learns residual coefficients from autonomous task reward rather than human-action targets, scales them by human correction magnitudes, and stores executed residuals in replay.The critic learns from actions actually executed after controller limits; human correction trajectories are not inserted into replay.Source: §3.4 · p. 4
3.3 WHERE: Human-Supported Corrective Action Space
- 97.3%, 96.0%, and 97.7% of squared correction magnitude are retained by 3-, 2-, and 2-dimensional corrective subspaces for tray, marker, and cup, respectively.The basis uses leading eigenvectors of the uncentered correction second moment and is selected using captured magnitude and leave-one-episode-out stability.Source: §3.3 · p. 4, Figure 3 · p. 5
4 Experimental Evaluation
The evaluation tests whether ARC improves last-mile precision, what sparse corrections reveal about when and where to refine, and whether human corrections should determine the executed residual. Across three real-robot tasks, it compares ARC with frozen base policies and correction-supervised alternatives.
4.1 Experimental Setup
- The real-robot evaluation used frozen pretrained ACT policies, approximately 55 demonstrations per task on average, and approximately 8–10 correction rollouts per task before two autonomous collect–reward–update rounds.The study compared Base ACT, direct supervised prediction of human residuals, and ARC over 12 trials per method and task.Source: §4.1 · p. 5
4.2 Q1: Can ARC Improve Last-Mile Precision?
- ARC raised tray-stacking success from 58.3% (7/12) to 100.0% (12/12), marker-insertion success from 25.0% (3/12) to 58.3% (7/12), and cup-stacking success from 16.7% (2/12) to 66.7% (8/12).The base policies reached the relevant manipulation phase, but small execution errors became consequential during precision-critical segments.Source: §4.2 · p. 5
4.4 Q3: Should Human Corrections Define WHAT to Execute?
- Within the same corrective support, supervised prediction reduced held-out human-residual error from 1.98° to 1.90° (4.0%), while its median max-joint residual magnitude was 0.52° versus ARC’s 0.14° (3.7×).On non-trivial states, PC1 sign agreement was 38%; 22% of supervised versus 0% of ARC predictions reached at least half the allowed bound.Source: Figure 4 · p. 6, §4.4 · p. 6
4.3 Q2: What Do Human Corrections Tell Us About WHEN and WHERE?
- Pre-intervention gates activated on all 10 autonomous base-policy rollouts, whereas gates trained on states within correction bouts activated on none; residual control remained active for only 8.5% of marker and 21.3% of cup timesteps.The marker gate achieved AUROC 0.853 and AUPRC 0.137, against a positive-class prevalence of 0.037, and identified 17/36 held-out correction onsets; the cup gate achieved AUROC 0.862 and AUPRC 0.340.Source: §4.3 · p. 6
- Corrections occupied three directions for tray stacking and two for marker insertion and cup stacking, retaining 97.3%, 96.0%, and 97.7% of squared correction magnitude, respectively.Worst leave-one-episode-out subspace similarities were 0.993, 0.995, and 0.999, respectively.Source: §4.3 · p. 6
4.4 Q3: Should Human Corrections Define WHAT to Execute?
- Direct seven-joint supervision achieved 3/12, 5/12, and 3/12 successes on tray, marker, and cup, versus ARC’s 12/12, 7/12, and 8/12.Because this comparison differs in both residual support and learning objective, the authors also compare policies matched in support and residual bounds; supervision still does not recover ARC’s residual behavior.Source: §4.4 · p. 6
5 Conclusion and Discussion
ARC uses sparse human corrections to identify when residual control is needed and where in action space to refine, while task reward determines the executed residual. Its real-robot evaluation spans three tasks, and its gate currently depends on human correction timing.
5 Conclusion and Discussion
- ARC’s targeted post-training improves last-mile precision across three real-robot tasks without updating the base policy or revisiting its original demonstrations.Sparse corrections identify when residual control is needed and where refinement occurs; autonomous task reward determines what residual is executed.Source: §5 · p. 7
- ARC’s demonstrated scope is post-training on three precision-manipulation tasks on a real robot.The proposed monitor is described as a direction that could reduce reliance on task-specific correction timing, not as an evaluated component.Source: §5 · p. 7
- Training the gate requires task- and operator-specific correction examples, and the gate may vary with how different humans choose when to intervene.The paper identifies a continuously operating vision-language-model monitor as a possible replacement or augmentation for the learned gate.Source: §5 · p. 7