PaperlayerView exact revision on arXiv

Source-linked AI summary

I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation

Wensi Ai, Christopher Luey, Karthik Dharmarajan, Evans Han, Yunfan Jiang, Stef Ren, Jianwen Xie, Jiajun Wu, Li Fei-Fei, Ruohan Zhang

arXiv:2610.17748v1cs.ROcs.AI

TL;DR

Offline imitation policies can degrade outside their training distribution, and long-horizon whole-body manipulation makes exhaustive demonstration coverage difficult. I3L combines JoyLo+ coordinated takeover with iterative deployment and retraining; across eight tasks, it achieves at least 90% success and outperforms demonstration-only training by 35.6 percentage points on average under the same trajectory budgets.

  • Problem

    Offline imitation policies remain vulnerable to distribution shift and compounding errors, while systematic evidence for corrective learning across diverse long-horizon whole-body mobile manipulation tasks remains limited.

    Source: §I · p. 1

  • Method

    I3L combines policy-tracking JoyLo+ leaders and touch-triggered whole-body takeover with repeated deployment and retraining to target updated-policy failures.

    Source: §I · p. 2, §1 · p. 2

  • Results

    35.6 percentage points: four-round HG-DAgger outperforms demonstration-only training by this average across eight tasks under the same trajectory budgets, with I3L achieving at least 90% success on every task.

    Source: B. Q1: Targeted Corrections Improve… · p. 5

  • Takeaways & Limitations

    The results show that targeted deployment-time corrections can close the last-mile gap more effectively than additional demonstrations under the same trajectory budgets.

    Source: B. Q1: Targeted Corrections Improve… · p. 5

    Correction-learning strategy comparisons are reported in controlled simulation, where multi-round HG-DAgger achieves the highest final mean success.

    Source: §V · pp. 7–8

Abstract

from arXiv · show

Policies trained offline from teleoperated demonstrations can degrade outside their training distribution. This gap is particularly acute in long-horizon whole-body mobile manipulation, where long horizons amplify compounding errors and coordinated arm, torso, and base motion makes exhaustive demonstration coverage impractical. We introduce I3L, an integrated hardware and learning system that combines whole-body correction with iterative policy refinement to address this last-mile reliability gap. Its low-cost, open-source JoyLo+ interface uses policy-tracking leader arms and touch-triggered takeover to enable coordinated corrections across the body parts, while iterative deployment and retraining allow operators to target failures of the updated policy. We evaluate I3L across three robot embodiments and eight simulated and real-world tasks, including four long-horizon whole-body mobile manipulation tasks. Controlled simulation experiments show that four-round HG-DAgger achieves the highest final success on both tasks. Under this protocol, I3L improves autonomous success over the base policies by an average of 68.3 percentage points and achieves at least 90% success on every task, outperforming demonstration-only training by 35.6 percentage points on average with the same number of training trajectories. Averaged across tasks, takeover duration per episode decreases by 72% over four rounds. Together, these results show that I3L helps close the last-mile gap between offline training and reliable deployment and provides practical insights into collecting and using corrections for long-horizon whole-body mobile manipulation. Videos, visualizations, and appendix are at https://i-3l.github.io/.

I. INTRODUCTION

Offline imitation policies remain vulnerable to distribution shift and compounding errors, especially in whole-body mobile manipulation, where a few critical failures can block task completion. I3L combines JoyLo+ whole-body takeover with iterative policy refinement, and evaluates corrective-learning strategies across long-horizon tasks.

I. INTRODUCTION

  • Whole-body mobile manipulation couples over twenty actuated degrees of freedom with a mobile base, and failures at a few critical stages can prevent full-task completion.The introduction argues that targeted corrections at bottleneck states can be more valuable than demonstrations of already-mastered behavior.Source: §I · p. 1
  • I3L pairs JoyLo+ policy-tracking leaders and touch-triggered takeover with repeated deployment and retraining to target failures of the updated policy.The system is evaluated on four long-horizon real-world BEHAVIOR tasks, alongside controlled comparisons of collection schedules and learning strategies.Source: §I · p. 2

1. An integrated system for iterative whole-body cor-

  • JoyLo+ uses capacitive-touch-triggered switching and motorized leader arms to coordinate arm, torso, and base corrections during whole-body takeover.The interface is low-cost and open-source, and extends GELLO and JoyLo.Source: §1 · p. 2

3. Studies of corrective learning and operator inter-

  • Four-round HG-DAgger achieves the highest final mean success on two simulation tasks under the same trajectory budgets.The study also reports declining cumulative intervention share across operators with varied teleoperation experience.Source: §3 · p. 2

II. RELATED WORK

  • Systematic evidence remains limited on iterative correction across diverse long-horizon whole-body mobile manipulation tasks, which I3L studies while tracking intervention burden across subtasks and body parts.Related work includes corrective-learning studies on mobile part transport, a four-stage kitchen task, and clothes folding.Source: §II · p. 2

III. METHOD

I3L combines JoyLo+ hardware for whole-body policy intervention with a process for recording and using corrections. The interface tracks policy motion during autonomous execution and supports coordinated takeover, while recording labels distinguish intervention from other rollout frames.

A. JoyLo+: A Hardware Interface for Whole-Body Policy Intervention

  • JoyLo+ is an actuated teleoperation interface for collecting data and intervening during autonomous policy execution.R1 Pro uses two seven-joint leaders for bimanual mobile manipulation, while Franka uses one seven-joint leader for single-arm tabletop manipulation.Source: A. JoyLo+: A Hardware Interface for… · p. 3
  • During rollouts, actuated leaders track policy arm targets, and gravity compensation supports operator guidance during intervention.Touch sensing enables takeover without a separate button press or leader-arm movement; release thresholds and a short delay reduce unintended returns to policy control.Source: A. JoyLo+: A Hardware Interface for… · p. 3

B. Intervention Detection and Recording

C. Iterative Interactive Imitation Learning

I3L aggregates active corrections with prior demonstrations and retrains between deployment rounds, allowing operators to target remaining policy failures. The experiments span simulated and real-world tasks, with controlled algorithm comparisons in simulation and real-world evaluation across household tasks.

C. Iterative Interactive Imitation Learning

  • Under HG-DAgger filtering, original demonstrations and active corrections from all rounds provide training data, and the retrained policy is redeployed to collect corrections for its remaining failures.Each round starts from the same initial diffusion-policy weights or pretrained VLA checkpoint.Source: C. Iterative Interactive Imitation… · p. 4

A. Experimental Setup

  • The evaluation covers eight tasks with 29 subtasks across three robot embodiments and simulated and real-world domains.The four real-world R1 Pro tasks involve sequential object interactions and coordinated mobile manipulation, averaging approximately 83 seconds per execution.Source: A. Experimental Setup · pp. 4–5
  • Two simulation tasks use scripted oracles with cuRobo to generate demonstrations and corrective trajectories for controlled comparisons of collection schedules and learning strategies.The comparisons include residual learning, BC with all data, SIRIUS, and HG-DAgger.Source: A. Experimental Setup · p. 4
  • Real-world evaluation includes two precision-oriented, contact-rich Franka tasks and four long-horizon R1 Pro household tasks; all real-world conditions use 25 rollouts.Simulation evaluation averages five seeds with 50 rollouts per seed; diffusion policies and the R1 Pro VLA use their respective training schedules and checkpoints.Source: A. Experimental Setup · pp. 4–5

B. Q1: Targeted Corrections Improve Task Performance

Four-round HG-DAgger improves full-task success more than adding the same number of demonstrations, with particularly strong results on long-horizon real-world tasks. Iterative collection improves success under fixed collection budgets while intervention burden declines, though remaining bottlenecks vary by task.

  • Across eight tasks, four-round HG-DAgger improves success by 68.3 percentage points over base policies, 35.6 points more than 100 additional demonstrations, and reaches at least 90% success on every task.The comparison uses the same additional trajectory budget for demonstrations and corrections.Source: B. Q1: Targeted Corrections Improve… · p. 5, Table II · p. 5
  • On four long-horizon real-world BEHAVIOR tasks, corrections improve success over base policies by 81 percentage points on average and outperform additional demonstrations by 30 points.Source: B. Q1: Targeted Corrections Improve… · p. 5
  • With equal collection budgets, iterative HG-DAgger improves success over a fixed-base-policy batch by 3.6 points on Pick-and-Place Strawberries and 10.0 on Open Fridge and Pick Up Radio.Updating the policy changes the visited state distribution, allowing later corrections to address failures of the updated policy.Source: C. Q2: Iteration Improves Performance… · p. 5
  • Across rounds, takeovers per episode decrease by 63.4%, intervention-frame share by 69.5%, and takeover duration by 72.0%; final policies achieve at least 90% autonomous success on every task.Source: C. Q2: Iteration Improves Performance… · p. 6
  • Remaining corrections are task dependent: basket pickup stays difficult in Boxing Books, while late-stage intervention persists when picking and placing the second, smaller bread in Prepare Breakfast.Source: C. Q2: Iteration Improves Performance… · p. 6

D. Q3: Corrections Span the Whole Body

JoyLo+ interventions engage different combinations of base, torso, and arm motion across four long-horizon real-world tasks. The correction-learning comparison evaluates HG-DAgger filtering against alternatives using data collected and accumulated across rounds.

D. Q3: Corrections Span the Whole Body

  • Base activations account for 34.2% of body-part votes across four tasks, and torso activations account for 8.2% in the three tasks where they occur.Arm correction shares vary by task; multiple body parts can contribute within one takeover segment.Source: D. Q3: Corrections Span the Whole Body · p. 7

E. Q4: Correction-Learning Algorithm Comparison

1) Residual Policy:

The section compares how iterative data-use strategies affect policy success, reports operator intervention trends, and notes evaluation and measurement boundaries. Four-round HG-DAgger leads the controlled comparisons, while residual-policy iteration provides little final benefit over single-round collection.

1) Residual Policy:

  • Residual corrections are predicted from images, proprioception, and the frozen base policy’s action chunk, but iterative residual training shows little final benefit over single-round collection.Human demonstrations receive zero residual targets, and correction timesteps receive 50% sampling probability.Source: 1) Residual Policy · p. 7, 3) SIRIUS · p. 7

3) SIRIUS:

  • Four-round HG-DAgger achieves the highest final mean success on both simulation tasks, while the benefits of additional rounds depend on how collected data is used.Residual-policy iteration adds little final benefit over single-round collection on either task.Source: 3) SIRIUS · p. 7, §V · p. 8

2) BC (all data):

  • BC (all data) finishes close to HG-DAgger on Open Fridge but performs poorly on Strawberries, consistent with learning suboptimal on-policy behavior.This strategy trains on all collected data, including potentially suboptimal on-policy data.Source: 3) SIRIUS · p. 7, 2) BC (all data) · p. 7

3) SIRIUS:

  • SIRIUS assigns 50% sampling probability to intervention frames, up to 50% to demonstrations, and the remainder to on-policy data; its results fluctuate on Strawberries and trail single-round collection on Open Fridge.Its iterative-collection benefit is less consistent than HG-DAgger’s in these comparisons.Source: 3) SIRIUS · p. 7

F. Q5: Operator Study

  • Across three operators with varied experience, cumulative intervention share declined, and takeover-free episodes reached 40–55% of each operator’s cumulative collection by Round 3.Each operator collected 25 episodes per round over three correction rounds.Source: F. Q5: Operator Study · p. 7

V. CONCLUSION AND LIMITATIONS

  • Across eight tasks, I3L achieves at least 90% autonomous success and outperforms demonstration-only training by an average of 35.6 percentage points under the same trajectory budgets; mean takeover duration per episode decreases 72% across four rounds.Source: §V · p. 7
  • The evaluation covers eight tasks, with controlled algorithm comparisons in simulation, selected-protocol real-world validation, and a three-operator study; total operator time remains unquantified.The reported burden measures active takeover while operators continue supervising execution.Source: §V · p. 8

APPENDIX A ROBOT HARDWARE, OBSERVATIONS, AND CONTROL

Appendix A specifies embodiment-specific command layouts, calibration and home settings, and camera inputs for the real and simulated robots. These details distinguish the coordinated whole-body R1 Pro setup from the single-arm and simulation configurations.

A. Action Representations

  • The Franka uses a seven-joint arm command plus gripper, while the A1 simulation uses six arm targets and a gripper command.The R1 Pro simulation uses its 23-dimensional layout, with local-frame base increments and absolute torso and arm angles.Source: A. Action Representations · p. 10
  • The real R1 Pro command has 23 entries for base, torso, two arms, and two grippers; base velocities are clipped, and small commands are set to zero.Joint targets are absolute angles, and gripper values range from −1 open to +1 closed.Source: A. Action Representations · p. 10, Table A1 · p. 10

B. Home Poses and Kinematic Calibration

  • Home and calibration settings are robot- and task-specific, including leader offsets for Franka insertion and randomized initial base pose for simulated R1 Pro Fridge/Radio.The real R1 Pro initializes torso joints at zero and its base with zero velocity, not a fixed position or heading.Source: B. Home Poses and Kinematic Calibration · pp. 10–11, Table A2 · p. 10

C. Camera Layout and Image Formation

D. Frequencies, Timing, and Input Devices

The appendix documents control and input timing, JoyLo+ leader actuation, and touch-triggered takeover hardware. The physical interfaces use policy tracking for autonomous motion and compliant hand guidance during intervention.

D. Frequencies, Timing, and Input Devices

  • The real Franka environment action loop runs at 30 Hz with a 100 Hz wrapper, and real R1 Pro targets 30 Hz with 100 Hz joint interpolation; simulation applies actions and renders at 30 Hz with 120 Hz physics.In simulation, keyboard input selects the scripted correction oracle.Source: D. Frequencies, Timing, and Input… · pp. 11–12

A. Mechanical Structure and Actuation

  • JoyLo+ uses a 100 Hz policy-tracking loop in autonomous operation and switches leader motors to current control with gravity compensation for hand guidance.The R1 Pro configuration also imposes per-motor tracking-current caps.Source: Appendix B · p. 12, A. Mechanical Structure and Actuation · p. 12
  • Franka and R1 Pro leaders use different motor arrangements, and the R1 Pro uses mirrored dual leaders for coordinated arm control.Franka hand guidance combines gravity compensation with velocity-proportional assistance; R1 Pro gravity compensation accounts for mirrored mounting.Source: A. Mechanical Structure and Actuation · p. 12

B. Touch Hardware, Wiring, and Firmware

  • On real robots, grip electrodes connect to an MPR121 touch sensor read by an ESP32-S3; touching an enabled grip activates takeover, while simulations use keyboard input.The R1 Pro’s two grips share one sensor input, so touching either grip triggers the same signal.Source: B. Touch Hardware, Wiring, and Firmware · pp. 12–13, Figure A1 · p. 12
  • Touch thresholds are calibrated from untouched and firmly held samples, and the host retains human authority briefly after grip release.The release delay is 0.5 s for Franka and 0.3 s for R1 Pro.Source: B. Touch Hardware, Wiring, and Firmware · p. 13

C. Authority Switching and Policy Resumption

The appendix explains how authority transfers between policy and operator, and how simulated corrections are collected with scripted oracles. It also details the task setup and the scope of the physical leader cost estimates.

C. Authority Switching and Policy Resumption

  • For real Franka, takeover discards pending policy plans, holds while awaiting a fresh prediction, and resumes from corrected observations rather than executing an obsolete plan.The leader target is reset to its measured pose before position tracking resumes.Source: C. Authority Switching and Policy… · p. 13
  • On the real R1 Pro, grip contact or active JoyCon input transfers authority over both arms, grippers, torso, and base together; after release, the policy replans from the corrected state.At takeover entry, commands transition linearly from policy to human control over 2 s.Source: C. Authority Switching and Policy… · p. 13

C. Demonstration and Correction Collection

D. Bill of Materials and Cost Scope

B. Task Initialization

APPENDIX D DATA RECORDING AND PROCESSING

The appendix describes how intervention-aware episodes are recorded and labeled, how supervision sources are sampled for training, and how task intervals and intervention rates are annotated.

A. Recorded Data and Intervention Labels

  • Rollouts record camera observations, proprioception, executed and policy-proposed actions, and intervention labels, preserving both autonomous and takeover segments.Franka recordings use two RGB views and 8-dimensional actions; R1 Pro recordings use three views and 23-dimensional actions.Source: A. Recorded Data and Intervention Labels · p. 14
  • Four labels distinguish original demonstrations, active interventions, the 15 frames before takeover, and autonomous execution.Active-intervention labels take precedence when pre-intervention windows overlap.Source: A. Recorded Data and Intervention Labels · p. 14

B. Training-Data Preparation

  • HG-DAgger samples demonstration and intervention starts in proportion to their eligible counts, whereas all-data BC preserves the observed proportions of all four supervision categories.SIRIUS reweights available demonstration, intervention, and autonomous data, caps the demonstration fraction at 0.5, and assigns unavailable-category weight elsewhere as specified.Source: B. Training-Data Preparation · pp. 14–15

C. Subtask Annotation

  • Human annotators mark named subtask start and end frames for analysis, while separate intervention labels select training data.Figure 5 boundaries use the mean normalized subtask end position across episodes in a reference round; intervention rates use recorded labels.Source: C. Subtask Annotation · p. 15
  • Intervention-rate curves divide normalized episode progress into 50 bins and pool intervention and total frames across episodes.Takeovers shorter than 15 frames are excluded from intervention counts but remain in the denominator; rates are Gaussian-smoothed with a three-bin standard deviation.Source: C. Subtask Annotation · p. 15

APPENDIX E POLICY LEARNING AND DEPLOYMENT

This appendix specifies policy training and deployment for the evaluated embodiments, including diffusion and vision-language-action policies, and defines evaluation and intervention-burden measures.

APPENDIX E POLICY LEARNING AND DEPLOYMENT

  • A1 and Franka use task-trained diffusion policies, while R1 Pro uses pretrained π0.5; each round retrains on accumulated data from the same initialization.Final checkpoints are evaluated after 200k diffusion-policy steps or 40k VLA steps.Source: Appendix E · p. 15

A. Diffusion Policy

  • Diffusion policies use ResNet-18 image encoders and a conditional 1D U-Net to predict normalized action sequences from images and proprioception.At inference, 16 DDIM steps denoise the sequence; the first predicted action is discarded, then eight actions execute before replanning.Source: A. Diffusion Policy · p. 15

B. Vision–Language–Action Policy

  • The R1 Pro VLA pipeline combines three camera views, robot state, actions, and a task prompt, normalizes inputs, and predicts 30 actions while executing 15 before replanning.Torso and arm targets are represented as differences from current joint positions during training and restored to absolute targets for deployment.Source: B. Vision–Language–Action Policy · p. 16
  • Evaluation success is the fraction of autonomous episodes completing the full task; simulation reports means and sample standard deviations across five seeds with 50 rollouts per seed.Real-world results report successes out of 25 episodes per condition.Source: A. Autonomous Evaluation and… · p. 16

B. Intervention Burden and Aggregation

  • Intervention-burden measures aggregate retained intervention frames within annotated subtask intervals, while the operator study tracks cumulative intervention share through each round.Episode takeover duration is the retained intervention-frame count divided by the recorded frame rate; cumulative share pools timesteps across episodes for each operator.Source: B. Intervention Burden and Aggregation · p. 16

C. Whole-Body Activation Analysis

The appendix defines how body-part activation is counted during takeovers, describes correction-learning baselines, and gives the protocol for the operator study.

C. Whole-Body Activation Analysis

  • A body part counts as active when any coordinate crosses its activity threshold for three consecutive frames, and each part receives at most one vote per takeover segment.Multiple body parts may contribute to a segment; each part’s share is its activation count divided by the total across body parts.Source: C. Whole-Body Activation Analysis · pp. 16–18

D. Baseline Configurations

  • The collection-schedule comparison contrasts 100 correction episodes against a fixed base policy with four rounds of 25 episodes and policy updates between rounds.The algorithm study compares HG-DAgger, all-data BC, SIRIUS, and residual learning; the residual is added to a frozen base policy.Source: D. Baseline Configurations · p. 18

E. Operator Study Protocol

  • Three lab participants, including two with prior teleoperation experience and one novice, each collected three rounds of 25 retained correction episodes on Franka Mug Hanging.Participants began from the same base-policy checkpoint and retrained separately; failed attempts were excluded.Source: E. Operator Study Protocol · p. 18

Related papers

Loading 2610.17748v1…