PaperlayerView exact revision on arXiv

Source-linked AI summary

Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections

Xiaomeng Xu, Yifan Hou, Chendong Xin, Zeyi Liu, Shuran Song

arXiv:2506.16685v5cs.ROcs.LG

TL;DR

Real-world DAgger must collect corrections that reflect the running policy and update policies effectively from new data. CR-DAgger combines compliant on-policy intervention with a force-aware residual policy; across four contact-rich tasks, it improves base-policy success by 64% and outperforms retraining and finetuning under the same data budgets.

  • Problem

    Real-world DAgger requires informative correction data aligned with the running policy and effective ways to update policies using that data.

    Source: §1 · p. 1

  • Method

    CR-DAgger pairs a compliant interface for on-policy delta-action corrections with a residual policy that uses force feedback to predict residual motions and target forces.

    Source: §1 · p. 1

  • Results

    The system improves base-policy success rate by 64% and outperforms retraining from scratch and finetuning under the same data budgets across four contact-rich tasks.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The authors evaluate practical DAgger design choices and CR-DAgger against alternatives on four real-world contact-rich manipulation tasks.

    Source: §6 · p. 11

    Learning corrections efficiently with 50–100 episodes does not provide extreme robustness under large variations, which requires more information about the problem dynamics.

    Source: §5 · p. 10

Abstract

from arXiv · show

We address key challenges in Dataset Aggregation (DAgger) for real-world contact-rich manipulation: how to collect informative human correction data and how to effectively update policies with this new data. We introduce Compliant Residual DAgger (CR-DAgger), which contains two novel components: 1) a Compliant Intervention Interface that leverages compliance control, allowing humans to provide gentle, accurate delta action corrections without interrupting the ongoing robot policy execution; and 2) a Compliant Residual Policy formulation that learns from human corrections while incorporating force feedback and force control. Our system significantly enhances performance on precise contact-rich manipulation tasks using minimal correction data, improving base policy success rates by 64% on four challenging tasks (book flipping, belt assembly, cable routing, and gear insertion) while outperforming both retraining-from-scratch and finetuning approaches. Through extensive real-world experiments, we provide practical guidance for implementing effective DAgger in real-world robot learning tasks. Result videos are available at: https://compliant-residual-dagger.github.io

1 Introduction

CR-DAgger addresses the challenges of collecting informative human corrections and updating policies from them with a compliant intervention interface and a compliant residual policy. On four contact-rich tasks, it improves base-policy success by 64% and outperforms retraining from scratch and finetuning under the same data budgets.

1 Introduction

  • DAgger correction collection is challenging because demonstrators may deviate from the running policy, while takeover can introduce force discontinuities.The introduction identifies the lack of precise, instantaneous intervention interfaces as part of the problem.Source: §1 · p. 1
  • The compliant intervention interface collects gentle, accurate delta-action corrections on-policy without interrupting execution.It uses compliance control and allows smooth transitions between correction and no-correction modes.Source: §1 · pp. 1–2
  • The compliant residual policy uses force feedback to predict residual motions and target forces, including for a position-only base policy.The paper reports that it learns correction strategies from data collected through the intervention interface.Source: §1 · pp. 1–2
  • The system improves base-policy success rate by 64% and outperforms retraining from scratch and finetuning under the same data budgets on four contact-rich tasks.The tasks are book flipping, belt assembly, cable routing, and gear insertion.Source: §1 · pp. 1–2
  • The authors offer practical DAgger guidance based on real-world studies, including choices such as batch size and sampling strategy.The paper also analyzes correction-data quality against traditional manipulation data collection.Source: §1 · p. 2
  • The extended work adds gear insertion and cable routing experiments and analyzes intention misinterpretation caused by robot tracking errors.It also reports a solution based on reducing tracking error spatially and temporally.Source: §1 · p. 2

2 Related Work

Related work treats human correction collection and policy updating as distinct challenges, then positions CR-DAgger as a practical residual-learning approach that uses compliance control and additional sensory input.

Human-in-the-Loop Corrections for Robot Policy Learning

  • Unlike takeover corrections, the proposed kinesthetic system permits delta corrections while the policy runs and records force feedback; both data types are reported as crucial to learned-policy success.Source: §2 · p. 2

Improving Pretrained Robot Policies with New Data

  • Retraining becomes impractical when large-scale demonstration data is inaccessible, while fine-tuning with only new data is a common alternative.Source: §2 · p. 2
  • Real-world residual-policy learning has required many samples, intermediate scene representations, or consistent visual observations between training and testing.Source: §2 · p. 2
  • The paper presents practical data collection and residual-policy learning for long-horizon contact-rich manipulation, using little real-world correction data and integrating sensory modalities absent from the original model.Source: §2 · p. 3

Compliance and Compliance Control

  • Compliance describes how motion responds to force, and compliance control can use impedance control for back-drivable robots or admittance control with external force sensors for robots that cannot be back-driven.Source: §2 · p. 3

3 CR-DAgger Method

CR-DAgger improves pretrained robot policies with small amounts of human correction data through an on-policy compliant interface and a residual policy that learns corrective behavior. Its method addresses both the quality of corrections and how to incorporate them into a policy.

3.1 Compliant Intervention Interface

  • Takeover transitions can introduce unintended actions, distribution shift, teleoperation adjustment errors, and missing information about intended actions under human force.The interface instead collects on-policy corrections while the demonstrator can sense the policy's intention through haptic feedback.Source: §3.1 · p. 3
  • The interface lets humans apply force-based delta corrections while the base policy continues running, recording correction timing and force feedback.A handle supports direct intervention; an ATI 6-D force sensor records contact forces, and a keyboard key marks correction starts and ends.Source: §3.1 · pp. 3–4
  • Tracking error can reverse the recorded correction direction; velocity tracking and latency-based look-ahead reduce the largest error from 30 mm to below 5 mm during fastest motion.The velocity reference is computed by finite differences, and the look-ahead interval is estimated from hardware latency.Source: §3.1 · p. 4

3.2 Compliant Residual Policy

  • Retraining is costly and may require unavailable initial data, while finetuning can be unstable under distribution mismatch; CR-DAgger trains a residual on correction data to refine base actions and add force actions.The residual can add inputs and outputs without changing the fixed architecture of the base policy.Source: §3.2 · pp. 4–5
  • The residual policy uses visual, proprioceptive, and force feedback to predict delta poses and target forces, which an admittance controller uses to execute compliant actions.It predicts five action frames, corresponding to 0.1 seconds at 50 Hz.Source: §3.2 · p. 5
  • The residual has about 2 MB of trainable weights and trains from 50–100 demonstrations; its force input and higher-frequency inference support force-aware, reactive correction.It can add force awareness to a position-based pretrained policy, with high-frequency force feedback particularly relevant during contact events.Source: §3.2 · p. 5
  • Training uses no-correction examples labeled with zero residual actions and oversamples data shortly after correction starts to improve residual quality and overall success rate.A few trajectories are also collected with the demonstrator holding the handle throughout, including when corrections are small or zero.Source: §3.2 · p. 5

4 Evaluation

The evaluation tests CR-DAgger on four contact-rich tasks and compares correction-collection and policy-update methods. Compliant residual policies improve on base policies, with results also showing the effects of force, correction style, batch size, sampling, and tracking error.

4 Evaluation

  • Evaluations use 50–100 correction episodes per collection method after deploying base policies on book flipping, belt assembly, cable routing, and gear insertion.Base policies are trained with 150–405 demonstrations, and updated policies are evaluated on the same test cases.Source: §4 · pp. 5–6, §4.1 · p. 6

4.2 Base Policy and its Failure Modes

  • The book-flipping base policy succeeds in 40% of trials, while the belt-assembly base policy succeeds in 15.6%; force input raises belt success to 43.8%.The book task requires precise insertion and forceful pushing; the belt task requires force-position coordination amid ambiguous visual alignment.Source: §4.2 · p. 6, §4.1 · p. 6

4.4 Key Findings from Comparisons

  • 60% and 50%: Compliant Residual policies trained on On-Policy Delta data improve base-policy success rates on two tasks, respectively.The book-flipping policy learns to pitch its fingers down before insertion; the belt-assembly policy corrects belt height when misaligned with the pulley slot.Source: §4.4 · p. 8, Figure 9 · p. 9
  • Compliant Residual exceeds the best position-only baseline by 45% on book flipping and 53% on belt assembly, using force feedback and predicting contact forces.On the final book-flipping stage, it predicts large pushing forces and achieves 100% success, while Residual falls from 70% to 35%.Source: §4.4 · p. 8
  • On-Policy Delta yields 45% higher book-flipping success than Take-over-Correction, with smoother actions and correction data better aligned to base-policy training data.Figure 10 compares velocity magnitudes around correction transitions; Figure 11 compares fingertip-trajectory distributions.Source: §4.4 · p. 8, Figure 10 · p. 9, Figure 11 · p. 9
  • Retraining produces stable motions but limited gains, while finetuning performs worst and falls below the base policy on both tasks.Retraining reports a 1.67% success-rate drop on book flipping and an 18.8% improvement on belt assembly; finetuning reports drops of 30% and 15.6%, respectively. KL regularization reduces noisy behavior but does not bring overall success above other baselines.Source: §4.4 · pp. 8–9

4.5 Ablations

  • Batch size 50 stabilizes residual training; a single batch reaches 100% on book flipping, while belt assembly performs best with two batches rather than three.The three-batch belt policy is slightly worse than the two-batch policy, so additional correction data does not guarantee better performance.Source: §4.5 · p. 10, Figure 13 · p. 10, Figure 12 · p. 10
  • Sampling densely after intervention starts gives the best performance, whereas denser sampling around the start adds mostly negative pre-intervention examples; tracking error can also misrepresent corrections on gear insertion.Reducing tracking error lets the collected corrections better reflect human intent; the described issue did not affect book flipping or belt assembly before the fixes.Source: §4.5 · p. 10

5 Limitation and Future Work

The paper identifies boundaries in when residual learning is recommended, its robustness, and the effort required by the data-collection interface.

Base/residual trade-off

  • The authors recommend starting residual correction collection when the base policy has at least 10%–20% success; theoretical guidance for allocating effort between base and residual improvements remains future work.The recommendation is based on their experiments.Source: §5 · p. 10

Incorporate more data

  • Learning with 50–100 episodes is efficient, but extreme robustness under large variations requires more information about problem dynamics.The authors also note that a shallow MLP action head may struggle with complex tasks involving more distinctive action multimodalities.Source: §5 · pp. 10–11

More flexible/intuitive data collection interface

  • Kinesthetic teaching provides richer, higher-quality data than teleoperation, but may require more collection labor because demonstrators must grasp the robot handle.The authors propose developing similarly intuitive, smooth teleoperation and interfaces for higher-DoF manipulation as future directions.Source: §5 · p. 11

6 Conclusion

The paper evaluates practical DAgger design choices and presents CR-DAgger, combining a Compliant Intervention Interface for collecting human corrections with a Compliant Residual Policy for improving the base policy.

6 Conclusion

  • CR-DAgger combines a Compliant Intervention Interface for collecting human correction data with a Compliant Residual Policy for improving the base policy.Source: §6 · p. 11
  • The authors evaluate practical design choices for DAgger in real-world robot learning.Source: §6 · p. 11
  • The designs are compared with a variety of alternatives on four contact-rich manipulation tasks.Source: §6 · p. 11

Funding

The work received support from several named organizations and awards; the authors clarify that the views expressed are their own.

Funding

  • The work was supported in part by NSF awards #2143601, #2037101, and #2132519, a Sloan Fellowship, Toyota Research Institute, and Samsung.Source: Funding · p. 11
  • Google and TRI provided UR5 robot hardware.Source: Funding · p. 11
  • The authors state that the views and conclusions are theirs and do not necessarily represent their sponsors’ official policies.Source: Funding · p. 11

A.1 Controller Architecture

The software uses separate base-policy, residual-policy, and admittance-controller loops, alongside standalone hardware-driver loops.

A.1 Controller Architecture

  • The base policy loop runs the diffusion policy at about 1 Hz, predicting 32 action frames for 3.2 seconds of future robot positions.The implementation is not optimized for computation speed.Source: §A.1 · p. 13
  • The residual policy loop runs at approximately 50 Hz, predicts five frames of delta actions for 0.1 seconds, and adds them to corresponding base-policy actions before execution.The loop rate is limited by the current implementation and could be improved.Source: §A.1 · p. 13
  • The 500 Hz admittance controller implements 6D Cartesian compliance, tracking reference positions when forces are absent and yielding under external force like a spring around the base-policy position.Source: §A.1 · p. 13
  • Standalone hardware-driver loops maintain device communication and buffers for action and feedback.Source: §A.1 · p. 13
  • The authors chose admittance control because their Universal Robot has high position-tracking accuracy and is not back-drivable.Source: §A.1 · p. 13

A.2 Stage-Wise Success Rate

Book-flipping success rates are reported across three key stages, with Fig. 14 providing a stage-wise breakdown of task success rates.

A.2 Stage-Wise Success Rate

  • Fig. 14 reports book-flipping success rates across three key stages.Source: §A.2 · p. 13
  • Fig. 14 is a more detailed version of Fig. 9.Source: §A.2 · p. 13
  • The figure reports task success rates by stage.Source: §A.2 · p. 13

A.3 Correction Data Decomposition

The section describes two ways to train residual policies in regions with little correction data and how their usefulness varies with base-policy stability.

A.3 Correction Data Decomposition

  • Two strategies address low-correction regions: train on uncorrected online data with zero residual labels, or mark 15 of 50 correction episodes as fully corrected.The second strategy includes trajectories even when the demonstrator's correction is small or zero.Source: §A.3 · p. 13
  • The zero-label strategy works better with a more stable, higher-success base policy; whole-trajectory correction works better otherwise.The experiments use the first strategy for the book task, though the passage ends before specifying the full strategy assignment.Source: §A.3 · p. 13
  • Figure 14 organizes stage-wise success rates by task in rows and cumulative stage in columns.Source: Figure 14 · p. 14

A.4 Comparison of Policy Structures

Alternative policy structures were tested on belt assembly, where their success-rate improvements over the original MLP were inconclusive. The section also reports compute hardware and task-specific tools, with other sensor types noted as future directions.

A.4 Comparison of Policy Structures

  • The probabilistic MLP samples actions from a Gaussian distribution, while the Flow Matching Transformer predicts action velocities for ODE-based sampling.Both alternatives were trained on the same belt-assembly data and under configurations matching the original MLP.Source: §A.4 · p. 14
  • 100% success for probabilistic MLP and 90% for Flow Matching Transformer versus 96.8% for original MLP; improvements were inconclusive.Each alternative-structure ablation was evaluated over 10 rollouts; the authors hypothesize expressive structures may benefit tasks with more distinctive action multimodality.Source: §A.4 · p. 14

A.5 Experiments Compute Resources

  • Training and deployment used a desktop with an NVIDIA GeForce RTX 4090 GPU.Source: §A.5 · p. 14

A.6 Hardware Design

  • Task-specific tooling included a fork-shaped book-flipping tool and a WSG-50 gripper with fin-ray fingers for belt assembly.Future directions include automatic manipulator design and incorporating capacitive force/torque or vision-based tactile sensors.Source: §A.6 · p. 14
Loading 2506.16685v5…