PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Stage Cable Routing through Hierarchical Imitation Learning

Jianlan Luo, Charles Xu, Xinyang Geng, Gilbert Feng, Kuan Fang, Liam Tan, Stefan Schaal, Sergey Levine

arXiv:2307.08927v5cs.ROcs.AI

TL;DR

Multi-stage cable routing is difficult because imperfect primitives make naïve sequential execution increasingly unreliable. The paper uses vision-based imitation learning at both low-level motor control and high-level primitive sequencing, enabling recovery and correction during routing. The system routes cables through clips, including challenging arrangements, and can be fine-tuned for novel situations, although its absolute success rate remains below industrial requirements.

  • Problem

    Multi-stage cable routing combines deformable-object manipulation, visual feedback, and long-horizon sequencing, while naïve composition causes whole-task success to deteriorate across imperfect stages.

    Source: §I · pp. 1–2

  • Method

    The system trains visual imitation policies for low-level clip insertion and high-level primitive selection, using observations, execution history, retries, and corrective actions.

    Source: §I · p. 2, B. High-Level Primitive Selection Policy · pp. 6–7

  • Results

    The hierarchical system routes cables through clip sequences and adapts to entirely novel situations using ten additional demonstrations, outperforming flat and naïve scripted alternatives.

    Source: §I · p. 3, §VII · p. 12

  • Takeaways & Limitations

    Recovery-capable hierarchical control makes temporally extended cable routing more practical without requiring every low-level primitive to succeed consistently.

    Source: §VII · p. 12

    Absolute success remains imperfect for industrially relevant applications, and the authors identify larger datasets and more diverse primitives as possible improvements.

    Source: §VII · p. 12

Abstract

from arXiv · show

We study the problem of learning to perform multi-stage robotic manipulation tasks, with applications to cable routing, where the robot must route a cable through a series of clips. This setting presents challenges representative of complex multi-stage robotic manipulation scenarios: handling deformable objects, closing the loop on visual perception, and handling extended behaviors consisting of multiple steps that must be executed successfully to complete the entire task. In such settings, learning individual primitives for each stage that succeed with a high enough rate to perform a complete temporally extended task is impractical: if each stage must be completed successfully and has a non-negligible probability of failure, the likelihood of successful completion of the entire task becomes negligible. Therefore, successful controllers for such multi-stage tasks must be able to recover from failure and compensate for imperfections in low-level controllers by smartly choosing which controllers to trigger at any given time, retrying, or taking corrective action as needed. To this end, we describe an imitation learning system that uses vision-based policies trained from demonstrations at both the lower (motor control) and the upper (sequencing) level, present a system for instantiating this method to learn the cable routing task, and perform evaluations showing great performance in generalizing to very challenging clip placement variations. Supplementary videos, datasets, and code can be found at https://sites.google.com/view/cablerouting.

I. INTRODUCTION

The paper addresses multi-stage cable routing by combining learned low-level manipulation primitives with learned high-level sequencing that can recover from failures. Its hierarchical system uses visual demonstrations to route cables through clips and generalizes to novel clip arrangements through fine-tuning.

I. INTRODUCTION

  • Naïve sequencing makes complete-task failure probabilities compound across stages, motivating controllers that recover instead of requiring every primitive to succeed initially.The cable task combines deformable-object manipulation, visual feedback, and sequential decisions across multiple clips.Source: §I · pp. 1–2
  • The cable-routing system requires sequentially routing a cable through three clips by executing a chain of manipulation primitives.A successful rollout ends after the policy routes the cable through clip 3 and triggers go_next for the final time.Source: Figure 1 · p. 1, §I · p. 1, Figure 3 · p. 2
  • The high-level policy uses wrist and side-camera observations plus primitive history, while the low-level policy maps wrist-camera observations and robot state to robot actions.This decomposition separates categorical primitive selection from Gaussian motor-action prediction and supports flexible data collection and training.Source: Figure 2 · p. 2, §I · pp. 2–3
  • Hierarchical imitation learning combines learned or scripted low-level primitives with a learned high-level policy that selects recovery actions during multi-stage cable routing.The high-level controller can retry clipping, reduce slack, and decide when to advance, compensating for imperfect lower-level skills.Source: §I · pp. 2–3
  • The method outperforms flat end-to-end and naïve scripted sequencing approaches and adapts to novel situations using ten additional demonstrations.The paper emphasizes both absolute performance and fine-tuning for entirely new clip arrangements.Source: §I · p. 3

II. RELATED WORK

The paper situates hierarchical imitation learning among work on learned visuomotor control, skill composition, and deformable-object manipulation. It distinguishes its approach by learning both control levels from visual inputs and targeting multi-clip cable routing in unseen scenarios.

II. RELATED WORK

  • The method uses a fixed primitive set and trains a high-level model to select skills, with design choices for low-level invariance, data collection, and rapid out-of-distribution fine-tuning.These choices extend hierarchical imitation to complex multi-stage cable routing.Source: §II · p. 3
  • Unlike planning approaches that depend on accurate state estimators or environment models, this method learns both high-level sequencing and low-level control from visual inputs.Deformable cables and occlusions make manually constructing effective state estimators especially challenging in this setting.Source: §II · p. 3
  • Prior deformable-object methods often rely on domain-specific models or additional sensing, whereas this work applies learning-based visual control to cable routing across multiple clips.The paper identifies multi-clip routing in unseen scenarios as an underexplored application.Source: §II · pp. 3–4

III. SYSTEM AND TASK SETUP

The system addresses multi-stage cable routing by combining vision-based low-level skills with higher-level sequencing that can recover from mistakes. The low-level clip-insertion policy is trained from successful and recovery demonstrations and designed to generalize across clip configurations.

III. SYSTEM AND TASK SETUP

  • Multi-stage cable routing is difficult because deformable cable behavior, clip interactions, and changing cable contact points require closed-loop visual control.The robot must route a cable through clips in varying positions and orientations while grasping it away from the point contacting the clip.Source: §III · p. 4

IV. CABLE ROUTING VIA HIERARCHICAL IMITATION LEARNING

  • A hierarchical policy can compensate for imperfect skills by retrying, repositioning the cable, and deciding dynamically when to advance to the next clip.Naive primitive sequencing does not provide the same corrective behavior.Source: §IV · p. 4
  • The system combines scripted pickup and cable-moving behaviors with a learned single-clip insertion primitive and a learned higher-level selector.The insertion policy is designed to be invariant to clip position and orientation, while the selector integrates primitives for the complete multi-stage task.Source: §IV · p. 4

A. Low-Level Clip Routing Policy

  • The single-clip policy is trained from 1,442 trajectories, including about 800 successful trials and about 600 intentional failure-and-recovery trials.Randomized robot and clip poses expose the policy to varied configurations, while recovery demonstrations target robustness to local mistakes.Source: A. Low-Level Clip Routing Policy · p. 5, Figure 5 · p. 5
  • Wrist-camera observations, gripper-frame commands, and RandAugment improve generalization to variation in clip pose and runtime image conditions.The observation includes two wrist-camera images and the TCP pose; augmentation uses two sequential transforms with strength nine.Source: A. Low-Level Clip Routing Policy · pp. 5–6, Figure 6 · p. 6

B. High-Level Primitive Selection Policy

The high-level policy selects among learned and scripted primitives using global visual observations and recent primitive history. Its primitives support pickup, insertion, cable reshaping, and progression through the ordered clips.

B. High-Level Primitive Selection Policy

  • The high-level selector combines three camera embeddings, end-effector height, and up to six previous primitive choices to choose the next primitive.History helps the policy make corrective decisions after failures and maintain an appropriate sequence for the current clip.Source: B. High-Level Primitive Selection Policy · pp. 6–7
  • The primitive set contains learned clip insertion plus scripted Pickup, Perturb_cable, and Go_next behaviors.The modular design allows these scripted solutions to be replaced by other implementations.Source: B. High-Level Primitive Selection Policy · p. 6
  • Pickup detects the cable segment for the current clip with the top camera and converts its image location into a robot-frame grasp pose.A fixed table-height z-position is used instead of the RGB-D depth map because depth was inaccurate for the thin cable.Source: B. High-Level Primitive Selection Policy · p. 6, Figure 7 · p. 6
  • Perturb_cable changes difficult cable configurations by grasping a specified point and applying a predefined motion that reduces slack.In practice, stretching the cable along one direction was sufficient to escape pathologically difficult shapes.Source: B. High-Level Primitive Selection Policy · pp. 6–7, Figure 8 · p. 7
  • Go_next releases the cable and moves near the next clip with a random offset sampled from the low-level policy’s training distribution.This places the end effector within the relative-pose distribution used to train the single-clip skill.Source: B. High-Level Primitive Selection Policy · p. 7
  • The high-level policy is behaviorally cloned from about 250 keyboard demonstrations, augmented by labeling three to five adjacent frames around each primitive decision.Its categorical output is trained with cross-entropy loss, while image encoders are reused from the low-level policy.Source: B. High-Level Primitive Selection Policy · p. 7

C. Interactive Finetuning

Interactive finetuning uses human overrides during high-level policy execution to correct primitive selections and improve performance in novel scenarios.

C. Interactive Finetuning

  • The high-level policy is finetuned with human supervision, overriding incorrect primitive selections until each episode completes successfully.The procedure is designed to detect non-recoverable choices immediately, before the robot executes them.Source: C. Interactive Finetuning · p. 7
  • Up to ten corrected trials are collected per new scenario, and the MLP is finetuned only on this newly collected data.The previous high-level dataset is not reused, and learning-rate warm-up is applied.Source: C. Interactive Finetuning · p. 7

V. DATA COLLECTION AND OPEN-SOURCE RELEASE

The study collects demonstrations for both single-clip primitives and high-level multi-stage routing across randomized clip arrangements, then releases the resulting data and setup information.

A. Data Collection

  • The experiments collect a total of 1442 single-clip cable-routing demonstrations, varying each of three clips within specified randomization-area constraints while one cable end remains fixed to the table.Source: A. Data Collection · p. 7
  • Three clips are randomized within 12.5 cm by 15 cm regions spaced 7.5 cm apart, with rotations ranging from 0 to 45 degrees.These placement constraints define the randomized collection regions shown in Figure 9.Source: Figure 9 · p. 8
  • High-level demonstrations cover tasks with one, two, or three clips, recording observations while experts select primitives until routing is complete.Adjacent states are additionally labeled with the primitive selected at the expert’s decision point.Source: A. Data Collection · p. 8

B. Open-Source Dataset

The project releases datasets and reproducibility materials while evaluating hierarchical imitation learning against flat alternatives, design variants, and generalization scenarios.

B. Open-Source Dataset

  • The released data include 1,442 low-level trajectories and 257 high-level trajectories, plus CAD files for the 3D-printable clips.Low-level trajectories contain camera images, robot state, and commanded actions; high-level trajectories record observations and selected primitive indices.Source: B. Open-Source Dataset · p. 8
  • High-level training data can be collected by having a human select each primitive after the previous primitive finishes until the cable passes through all clips.This process is illustrated in Figure 10.Source: Figure 10 · p. 8

VI. EXPERIMENTAL EVALUATION

  • The experiments compare hierarchical performance with flat imitation learning and evaluate the system’s design choices, generalization, and finetuning efficiency.The evaluation explicitly asks whether hierarchy outperforms other imitation-learning approaches and how well the policy handles novel arrangements.Source: §VI · p. 8

A. Experiment Setup

The evaluation tests low-level routing across clip placements and cable shapes, measures complete-task success for high-level policies, and compares the learned primitive with scripted and ablated alternatives.

A. Experiment Setup

  • Full cable-routing trials count as successful only when the cable is correctly routed through every clip.The high-level policy is evaluated over multiple complete-task runs.Source: A. Experiment Setup · p. 9

B. Evaluating the Low-Level Single Clip Routing Policy

  • Low-level policies are tested over five clip positions, two cable shapes, and five rollouts per combination, totaling 50 attempts per policy.Trials begin with the end-effector about 5 cm in front of the clip and use a 50-timestep rollout.Source: B. Evaluating the Low-Level Single… · p. 9
  • The learned low-level policy achieves 46% overall success, succeeding on 18/25 easier-shape trials and 5/25 harder-shape trials.The harder shape curves away from the clip opening and is more difficult to align.Source: B. Evaluating the Low-Level Single… · p. 9, Figure 11 · p. 9
  • The learned policy has twice the success rate of a scripted strategy that uses privileged clip position and orientation but does not visually perceive the cable.The learned policy’s visual feedback supports closed-loop insertion rather than fixed waypoint execution.Source: B. Evaluating the Low-Level Single… · p. 9, Table I · p. 9
  • Image augmentation, eye-in-hand views, and correction demonstrations improve robustness, while removing image augmentation or using an external side view causes substantial performance degradation.Decoupled ResNet encoders provide only marginal benefit, whereas excluding correction demonstrations impairs recovery retries.Source: B. Evaluating the Low-Level Single… · p. 9, Table II · p. 9

C. Evaluating the High-Level Policy for Cable Routing

The hierarchical policy maintains reasonable performance as routing extends from one to three clips and substantially outperforms flat and manually scripted alternatives. History-dependent high-level sequencing supports recovery from failures that individual primitives cannot avoid.

In-distribution evaluation

  • The hierarchical policy maintains reasonable performance from one through three clips rather than suffering exponential degradation as task length increases.The evaluation uses randomized in-distribution clip configurations and counts a trial successful only when every clip is routed.Source: C. Evaluating the High-Level Policy… · pp. 9–10, Table III · p. 10

Baseline comparison

  • Our full method significantly outperforms end-to-end baselines and a non-learned state machine on the full multi-stage routing task.The state machine includes scripted retries and perturbation, while the learned policy selects recovery actions dynamically.Source: Table IV · p. 10, C. Evaluating the High-Level Policy… · pp. 10–11
  • Flat BC and Behavior Transformer each achieve 0% success across 24 trials, while ACT also fails to complete the task despite improving action smoothness.Observed failures include missed clips, dropped cables, and failure to grasp or route the cable.Source: C. Evaluating the High-Level Policy… · p. 10
  • Learned high-level recovery strategies achieve more than double the success rate of the hand-designed state machine.The results support explicit hierarchical sequencing over flat, semihierarchical, and manually scripted alternatives for this long-horizon task.Source: C. Evaluating the High-Level Policy… · p. 11

History ablation

Failure analysis

  • Over 95% of failures occur when the policy predicts go_next before the cable is actually routed, especially when the cable lies behind the clip.The authors hypothesize that visual features distinguishing an inserted cable from one behind the clip are insufficiently distinctive.Source: C. Evaluating the High-Level Policy… · p. 11

D. Interactive Fine-Tuning to Quickly Improve Performance

Interactive fine-tuning rapidly improves the high-level policy on both out-of-distribution and in-distribution clip arrangements. With only ten demonstrations, the system adapts to challenging geometry, including a previously unseen fourth clip.

Fine-tuning approach

Out-of-distribution fine-tuning

  • With ten demonstrations, fine-tuning doubles performance on three-clip out-of-distribution configurations and enables adaptation to a four-clip arrangement where zero-shot transfer fails.The OOD configurations vary clip positions and orientations beyond training ranges; the fourth clip was absent from the original high-level-policy data.Source: D. Interactive Fine-Tuning to Quickly… · p. 11, Figure 14 · p. 11, Figure 15 · p. 12

In-distribution fine-tuning

VII. DISCUSSION AND FUTURE WORK

The paper argues that hierarchical imitation learning makes multi-stage cable routing more practical by enabling recovery at both primitive and sequencing levels. It demonstrates routing across clip sequences and rapid fine-tuning, while acknowledging that absolute success remains below industrially relevant reliability.

Discussion

  • The system combines learned low-level primitives with a learned high-level policy that retries or corrects mistakes during multi-stage cable routing.The low-level primitive corrects small errors, while the high-level policy selects recovery actions and when to advance.Source: §VII · p. 12
  • The approach prevents multi-stage performance from mechanically declining exponentially by allowing each stage to compensate for errors.The evaluation supports this principle on cable-routing tasks and harder arrangements that can be fine-tuned with as few as 10 additional trials.Source: §VII · p. 12

Limitations and future work

  • Absolute success remains imperfect for industrially relevant applications, and future improvements may require larger datasets, more diverse primitives, or reinforcement learning.This limitation is stated despite performance exceeding baselines without learned policies at both hierarchy levels.Source: §VII · p. 12

APPENDIX

The system expresses observations and actions in an end-effector-centered coordinate frame, then maps policy outputs back to the robot base frame for control. This view-invariant design supports generalization across clip placements and robustness through randomized initial poses.

A. View-invariant coordinate system

  • End-effector-centered coordinates help the policy generalize to new clip placements by preserving spatial relationships rather than absolute positions.Randomizing the end-effector’s initial pose is equivalent from the policy’s perspective to physically moving the goal clip.Source: A. View-invariant coordinate system · p. 12
  • The wrist camera provides observations in the end-effector frame, while the policy uses only end-effector height alongside its 4DoF action space.The action space includes XYZ translation and rotation about the z-axis; height is rotationally invariant for this space.Source: A. View-invariant coordinate system · p. 12
  • The policy outputs a 4D twist in the current end-effector frame, which is converted to the robot base frame before being sent to the Franka controller.The conversion uses an Adjoint mapping determined by the current homogeneous transformation matrix.Source: A. View-invariant coordinate system · pp. 12–13
  • The Adjoint map relates twists expressed in the current end-effector frame and the robot base frame, using the skew-symmetric matrix formed from the translation vector.The mapped twist is the quantity sent to the Franka controller.Source: A. View-invariant coordinate system · pp. 12–13

B. Neural Network Details

The implementation combines ResNet18 visual backbones with separate low-level routing and high-level primitive-selection policies. Training uses distinct hyperparameter configurations for low-level policies, high-level policies, Behavior Transformer, and action chunking.

B. Neural Network Details

  • Low-level routing uses ResNet18 features, projected height, a two-layer MLP, and a TanhGaussian policy that outputs mean and variance.The height is projected to 128 dimensions, concatenated with visual embeddings, and processed by MLP layers with 256 nodes each.Source: B. Neural Network Details · p. 13
  • The high-level primitive-selection policy reuses frozen ResNet18 backbones and combines three image embeddings, height, and a learned 6 × 4 primitive-sequence embedding.A 256-unit MLP with softmax activation produces the primitive-selection output.Source: B. Neural Network Details · p. 13

C. Training Hyperparameters

  • Training hyperparameters are separately specified for low-level routing policies, high-level primitive-selection policies, Behavior Transformer, and action chunking.The supplied appendix identifies dedicated tables for each configuration.Source: Table VII · p. 13, Table VIII · p. 13, Table IX · p. 13

Related papers

Loading 2307.08927v5…