Source-linked AI summary
RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulation
Philipp Wu, Yide Shentu, Qiayuan Liao, Ding Jin, Menglong Guo, Koushil Sreenath, Xingyu Lin, Pieter Abbeel
TL;DR
Passive demonstrations are simple to collect, but policies trained on them can struggle with errors during online execution, while interactive teaching is not well supported by existing manipulation interfaces. RoboCopilot pairs Human-Gated DAgger with compliant bilateral teleoperation for bimanual manipulation. Simulation and hardware experiments report benefits from interactive data, including increasing success rates and decreasing intervention needs as data accumulates.
Problem
Policies trained on passive demonstrations can fail to recover from online errors under covariate shift, while manipulation interfaces make human intervention challenging.
Source: §2.1 · p. 2, §1 · p. 2Method
RoboCopilot combines Human-Gated DAgger, which collects human corrections during policy rollouts, with compliant bilateral teleoperation for bimanual manipulation.
Source: §1 · p. 2, §3 · p. 4, §4.2 · p. 5Results
Interactive data collection improves over passive collection, with real-world success rates increasing and intervention needs decreasing as more interactive data is collected.
Source: §5.2 · p. 7, §5.1 · p. 7Takeaways & Limitations
The need for human intervention can serve as a practical signal for when to stop collecting new data.
Source: §5.2 · p. 7The system depends on skilled operators for continual correction, which may limit scalability where expertise is unavailable or tasks are too difficult to teleoperate.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Learning from human demonstration is an effective approach for learning complex manipulation skills. However, existing approaches heavily focus on learning from passive human demonstration data for its simplicity in data collection. Interactive human teaching has appealing theoretical and practical properties, but they are not well supported by existing human-robot interfaces. This paper proposes a novel system that enables seamless control switching between human and an autonomous policy for bi-manual manipulation tasks, enabling more efficient learning of new tasks. This is achieved through a compliant, bilateral teleoperation system. Through simulation and hardware experiments, we demonstrate the value of our system in an interactive human teaching for learning complex bi-manual manipulation skills.
1 Introduction
Passive human demonstrations are simple to collect, but they do not let a policy learn corrective behavior during deployment, and existing teleoperation interfaces make intervention difficult. RoboCopilot combines interactive imitation learning with a compliant bimanual teleoperation system for teaching complex manipulation skills.
1 Introduction
- Learning from demonstrations is simple to collect: humans provide trajectories, and a policy trains to mimic their actions.With enough demonstrated data, the policy can learn to perform the task like the human demonstrator.Source: §1 · p. 1
- Existing teleoperation systems make intervention difficult: relative-pose controllers can be unintuitive, while synchronizing exoskeletons with policy rollouts can be potentially unsafe.Interactive corrective data can address covariate shift, but manipulation interfaces have not made this teaching approach easy to use.Source: §1 · p. 2
- RoboCopilot adapts Human-Gated DAgger and builds compliant bimanual hardware for seamless human takeover during policy execution.The system is used to teach long-horizon, contact-rich bimanual mobile manipulation skills.Source: §1 · p. 2
2 Related Work
Passive behavior cloning learns from human trajectories but can struggle with covariate shift and requires task-dependent amounts of data. Interactive learning gathers corrective demonstrations, while RoboCopilot’s teleoperation system is designed to support timely human takeover.
2.1 Imitation Learning from Passive Human Demonstrations
- Policies trained from passive data can fail to recover from online errors under covariate shift, and demonstration needs vary by task.Prior examples range from ten trajectories for simple picking to a thousand for other tasks.Source: §2.1 · p. 2
2.2 Interactive Imitation Learning
- Interactive imitation lets experts correct rollout mistakes; RoboCopilot follows HG-DAgger, where the operator chooses when to take or release control.Figure 2 depicts warm-up demonstrations, policy deployment, intervention, and fine-tuning from new demonstrations.Source: §2.2 · pp. 2–3, Figure 2 · p. 3
2.3 Robot Teleoperation Systems
- Teleoperation interfaces trade off simplicity, force feedback, kinematic constraints, and synchronization; RoboCopilot builds a compliant bimanual system for interactive teaching.Joystick and VR controllers lack force feedback, while exoskeleton-based systems can lose synchronization during policy execution.Source: §2.3 · p. 3
3 Continual Interactive Imitation Learning
The interactive learning loop warms up a policy with human demonstrations, collects human interventions during policy rollouts, and fine-tunes after each collection round. The paper uses this continual process despite catastrophic-forgetting risk, then reports that retraining from scratch on the interactive data provides the best deployment policy.
3 Continual Interactive Imitation Learning
- Each round adds human intervention data to the dataset and fine-tunes the policy using all collected data.The loop begins with K human trajectories to warm up the policy; the policy uses a diffusion network, and required interventions decrease as it improves.Source: §3 · p. 4
- Continual fine-tuning risks catastrophic forgetting, but online samples address covariate shift, expose current failure modes, and reduce overall training time.The human can intervene to teach corrections and provide targeted demonstrations based on observed failures.Source: §3 · p. 4
- After interactive teaching, retraining from scratch on the collected data avoids non-stationary training and is reported to provide the best deployment policy.This restarts the learning algorithm using the latest dataset.Source: §3 · p. 4
4 Teleoperation System
RoboCopilot combines a mobile bimanual robot with a compliant teleoperation workflow for interactive teaching. Its device design, active bilateral control, and policy-synchronized takeover support human intervention and data collection during execution.
4 Teleoperation System
- The teleoperation system combines a mobile bimanual robot, a teleoperation device, and a workflow for interactive teaching across real-world tasks.The authors describe the custom robot hardware as low cost.Source: §4 · p. 4
4.1 Teleoperation Device
- The leader device is a low-cost approximate kinematic replica fitted with a Quest 2 controller, while manipulation trajectories are difficult to relabel because of high-dimensional actions and complex dynamics.Figure 3 compares Aloha and GELLO handheld interfaces with RoboCopilot’s layout and input key map.Source: Figure 3 · p. 4, §4.1 · p. 5, §4 · p. 5
4.2 Active Control
- Active gravity compensation reduces operator effort, and bilateral feedback lets users feel scaled-down forces from the follower arm.The α and β coefficients keep the device from transmitting the arm’s full inertia and can be varied across situations.Source: §4.2 · p. 5
4.3 Human-in-the-loop Interactive Learning
- During policy execution, the synchronized teleoperation device can take over at any time; stiff leader gains minimize state difference, while lower teleoperation gains ease operator forces.Dedicated buttons support saving data and resetting the robot, helping operators control what data is collected.Source: §4.3 · p. 5
- Table 1 reports that leveraging human intervention during data collection significantly improves over behavior-cloning baselines on the robomimic benchmark.The comparison includes human- and expert-data behavior cloning, continual DAgger, and batched DAgger, with three seeds.Source: Table 1 · p. 6
5 Experiments
Experiments evaluate interactive imitation learning in simulation and on real robots, across industrial manipulation and a long-horizon kitchen task. Across these settings, interactive data collection improves policy performance or data efficiency relative to passive collection.
5 Experiments
- The experiments compare Offline BC, Continual DAgger, and Batched DAgger, using a pretrained expert policy as a reproducible human substitute in simulation.Source: §5 · p. 6, §5.1 · p. 6
5.2.1 Part Picking
- For part picking, the evaluation covers four beam configurations; the table caption reports higher success at matched trajectory counts and fewer human-operated timesteps for interactive data.Source: Table 2 · p. 7, §5.2.1 · p. 8
5.1 Simulation
- In simulation, Continual DAgger improved over behavior-cloning baselines, while Batched DAgger generally outperformed the continually trained variant.Source: §5.1 · pp. 6–7
5.2 Real World
- In real-world experiments, success rates increased as interactive data accumulated, while human interventions decreased and could signal when to stop collecting.Source: §5.2 · p. 7
5.2.2 Part Transport
- For part transport, Continual DAgger was more data-efficient than Offline BC through intelligent resets, while Batched DAgger achieved the best performance.Source: §5.2.2 · p. 8
5.2.3 Kitchen
- With 30 trajectories, Batched DAgger matched or outperformed Offline BC with 45 trajectories across subtasks, with the largest gap on full-task completion.Source: §5.2.3 · pp. 8–9
6 Discussion
The discussion describes RoboCopilot’s limitations and concludes that it supports interactive imitation learning for bimanual manipulation. The toy-kitchen task illustration shows a four-stage sequence for preparing a tomato.
6 Discussion
- Human oversight may limit scalability where skilled operators are unavailable or tasks are too difficult to teleoperate; custom hardware costs and gearbox backlash also constrain deployment and precision.Source: §6 · p. 9
- The toy-kitchen task proceeds from opening and holding a cabinet door to transferring a tomato, placing it in a pot, and turning the corresponding dial.Source: Figure 5 · p. 9
- RoboCopilot demonstrates human-in-the-loop interactive imitation learning for bimanual manipulation, with future work aimed at scalability and ease of deployment.Source: §6 · p. 10
A Additional Real-world Task Visualization
The figure illustrates an autonomous industrial-part transport trajectory from pickup to placement.
A Additional Real-world Task Visualization
- The robot picks up a heavy part with both hands, moves toward the tote, and drops the part; wrist-camera views occupy the top rows and a third-person view the bottom row.Source: Figure 6 · p. 16
B Simulation Task Visualizations
The figure depicts three Robomimic manipulation tasks used in the study.
B Simulation Task Visualizations
- The tasks are can-to-bin placement, square-object insertion onto a pole, and two-arm transport of a tool.Source: Figure 7 · p. 16
C Teleoperation Capabilities
RoboCopilot improves teleoperation through active joint motor selection, inertia compensation, bilateral feedback, and a fully compliant arm. It demonstrates these capabilities across contact-rich kitchen and whole-body tasks.
Teleoperation Capabilities
- Active joint motor selection provides inertia compensation and bilateral feedback, extending teleoperation beyond unilateral devices.The compliant arm supports contact-rich bimanual tasks without compromising safety or risking hardware damage.Source: C Teleoperation Capabilities · p. 17
- The robot performs serving hot water, preparing food, unloading a dishwasher, and opening doors under human teleoperation.The caption describes its compliant design as supporting safe handling of contact-rich interactions.Source: Figure 8 · p. 17
- The demonstrated tasks span daily kitchen activities and whole-body control.The figure caption presents them as examples of the system’s teleoperation capabilities.Source: Figure 8 · p. 17
D Robot Hardware Details
RoboCopilot is a mobile, 20-degree-of-freedom bimanual robot designed around safety, ease of use, and accessibility. Its backdrivable QDD actuators support compliant interaction, while a scaled kinematic replica provides the teleoperation interface.
Design Principles
- Safety, ease of use, and accessibility guide the hardware design, with compliance supporting safe interaction and seamless human takeover.The system is intended to be cost-efficient and simple while handling a wide range of daily tasks.Source: D Robot Hardware Details · p. 18
Robot Configuration
- The robot has two 7-DOF arms with parallel-jaw grippers, an actuated torso, and an omnidirectional base, totaling 20 degrees of freedom.Two wrist-mounted RealSense D405 cameras provide observations.Source: D Robot Hardware Details · p. 18
Compliance
- Backdrivable quasi-direct-drive transmissions provide compliance; external forces reach the motor and can be detected through motor-current measurements.The design uses off-the-shelf mini-cheetah-style transmissions for cost and simplicity.Source: D Robot Hardware Details · p. 18
Actuator Selection
- The arm uses 20 Nm, 9:1 shoulder actuators; 12 Nm, 40:1 upper-arm and elbow actuators; and 3 Nm, 10:1 lower-arm and gripper actuators.The actuator configuration gives an approximate continuous payload of 1 kg, and the eight QDD actuators per arm cost less than $2,000.Source: D Robot Hardware Details · p. 18, Table 4 · p. 18
Arm Design
- The teleoperation device is a 70%-scale kinematic replica, mounted at a 45-degree angle; the sixth-to-seventh joint spacing limits scaling at that degree of freedom.That joint remains unscaled, yet the authors report the device is still capable and intuitive.Source: D Robot Hardware Details · p. 19, Figure 9 · p. 19, Table 5 · p. 19
Gripper Design
- The high-speed gripper uses an offset slider-crank linkage and layer-jammed Finray-like fingers for simple, robust grasping of everyday-sized objects.It achieves a continuous maximum gripping force of 25 N and opens or closes in under one second.Source: D Robot Hardware Details · p. 20
Mobility
- An omnidirectional mobile base and vertically moving gantry expand workspace for bimanual manipulation, including picking objects from the ground.The self-contained platform is battery-powered and includes an onboard computer and router for Ethernet connectivity without Internet access.Source: D Robot Hardware Details · p. 20