PaperlayerView exact revision on arXiv

Source-linked AI summary

Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration

Edward Johns

arXiv:2105.06411v2cs.ROcs.LG

TL;DR

The paper addresses how to learn novel robot manipulation tasks from a single human demonstration without prior object knowledge. It estimates an interaction-start bottleneck from self-supervised visual data, reaches it through a coarse linear approach, and replays demonstrated velocities for interaction. Experiments show diverse everyday tasks can be learned while retaining a stable, interpretable controller.

  • Problem

    Existing imitation-learning approaches commonly require multiple demonstrations, repeated environment resetting, or prior task knowledge.

  • Method

    The method models imitation as bottleneck state estimation, trains the estimator by moving an end-effector camera around the object, and replays demonstrated velocities after reaching the estimated state.

  • Results

    Real-world experiments show the method learns a diverse range of novel everyday tasks from a single human demonstration.

  • Takeaways & Limitations

    Using machine learning only for pose estimation yields an analytical controller described as stable and interpretable while enabling complex interaction trajectories without explicit policy learning.

Abstract

from arXiv · show

We introduce a simple new method for visual imitation learning, which allows a novel robot manipulation task to be learned from a single human demonstration, without requiring any prior knowledge of the object being interacted with. Our method models imitation learning as a state estimation problem, with the state defined as the end-effector's pose at the point where object interaction begins, as observed from the demonstration. By then modelling a manipulation task as a coarse, approach trajectory followed by a fine, interaction trajectory, this state estimator can be trained in a self-supervised manner, by automatically moving the end-effector's camera around the object. At test time, the end-effector moves to the estimated state through a linear path, at which point the original demonstration's end-effector velocities are simply replayed. This enables convenient acquisition of a complex interaction trajectory, without actually needing to explicitly learn a policy. Real-world experiments on 8 everyday tasks show that our method can learn a diverse range of skills from a single human demonstration, whilst also yielding a stable and interpretable controller.

I. INTRODUCTION

The paper targets visual imitation learning that minimizes both human interaction and prior task knowledge. It proposes a coarse-to-fine, state-estimation approach that learns novel manipulation tasks from one demonstration while retaining an analytical controller.

  • Existing demonstration-only methods require many demonstrations, while exploration- and transfer-based methods require resetting, task-specific apparatus, or prior task-family knowledge.
  • The method separates manipulation into a coarse approach trajectory and a fine interaction trajectory, using a linear path for approach and a demonstration for interaction.
  • It defines the interaction-start state as the object’s bottleneck, namely the end-effector pose observed when interaction begins.
  • A bottleneck pose is estimated from visual observations, after which the original demonstration’s end-effector velocities are replayed without explicitly learning a policy.
  • The proposed controller uses machine learning for pose estimation rather than policy learning, yielding a stable and interpretable analytical controller.

II. RELATED WORK

Prior visual imitation methods learn through behavioral cloning, exploration, transfer, or visual servoing, but each commonly depends on substantial data, task knowledge, resetting, or restrictive interaction conditions.

  • The paper focuses on learning new tasks from visual observations without manually engineered state spaces, task-specific pose estimators, or fiducial markers.
  • Behavioral cloning maps observations to actions but commonly requires supervised action data and explicitly learns an end-to-end policy.
  • Exploration-based methods bootstrap from demonstrations through environment interaction, using mechanisms such as residual policies, replay buffers, or restricted exploration.
  • Transfer-learning methods adapt knowledge from similar tasks or task families, making efficient adaptation dependent on prior task-related training.
  • Visual-servoing approaches align observations with goals or demonstrations, but reported alternatives require manually defined interaction controllers or close initial alignment.

III. METHOD

The method models manipulation as a coarse approach to a virtual bottleneck followed by fine interaction, estimating that bottleneck from images and replaying demonstrated local velocities after reaching it.

  • The bottleneck is a virtual frame where the end-effector should be when object interaction begins, and all coarse trajectories converge there.
  • Because only the approach destination matters, the robot reaches the bottleneck through a simple linear path using inverse kinematics.
  • Local end-effector velocities recorded from the bottleneck onward can be replayed during testing for the interaction trajectory.
  • Training collects images and bottleneck poses while the end-effector camera moves around the object, then trains f(I) to predict the bottleneck pose.
  • A second network g(I) performs a last-inch correction after the coarse approach, while the demonstration begins near the object to limit interaction-trajectory offset.

C. Self-Supervised Learning

Self-supervision collects image–pose pairs by moving the end-effector around the object, while constrained pose sampling simplifies prediction and supports specialized near-bottleneck correction.

  • The dataset D contains images I paired with bottleneck poses TEB, which are calculated from forward kinematics and the demonstration-defined bottleneck.
  • D is collected automatically from random poses above the object along straight trajectories toward sampled poses near the bottleneck.
  • Simplifying the prediction space: The pose estimator’s prediction space is reduced to two horizontal translations and one vertical-axis rotation by constraining end-effector orientation and height.
  • Simplifying the prediction space: Dataset E fixes the end-effector height at the bottleneck and restricts motion to horizontal translation and vertical-axis rotation, specializing g(I) near the bottleneck.
  • Training algorithm: Training begins by recording one demonstration’s initial pose and velocities, defining the bottleneck orientation and height, and collecting image–pose samples along randomized trajectories.

D. Sequential State Estimation

Sequential state estimation fuses multiple bottleneck-pose predictions collected during the approach trajectory. The method compares batch averaging and filtering, with uncertainty modeled through several alternatives.

  • The method assumes the object is stationary, so sequential predictions estimate the same bottleneck pose.
  • Batch: Batch estimation averages all predictions observed so far using inverse-variance weighting under Gaussian uncertainty.
  • Filtering: Filtering combines the previous estimate with the current prediction and their associated uncertainties.
  • Filtering: The filtering initialization sets the first estimate to the first prediction and its uncertainty to the Prior uncertainty, forming a zero-process-noise Bayes filter.
  • Estimating Uncertainty: The sequential methods require Gaussian uncertainty for each prediction, estimated using Dropout, a learned uncertainty function, or validation error.

E. Task Execution

During testing, the robot approaches the estimated bottleneck along a linear path, refines the pose near the bottleneck, and then replays the demonstration’s interaction velocities.

  • The robot moves along a linear approach path toward the latest estimated bottleneck pose.
  • The bottleneck pose is computed from forward kinematics and the bottleneck prediction, with g(I) providing a final estimate at bottleneck height.
  • After moving to the refined bottleneck pose, the robot rotates by R and executes the demonstration velocities U.
  • Algorithm 2 summarizes testing as repeated image capture, prediction, sequential updating, linear motion, final correction, rotation, and velocity execution.

A. Implementation Details

The implementation uses convolutional networks for pose and uncertainty prediction, with mean squared error training and validation-based early stopping on a 7-DOF robot.

  • Both f(I) and g(I) use four convolutional layers followed by four fully connected layers with ReLU and Dropout.
  • Training uses mean squared error with fixed translation-rotation balancing coefficients and early stopping on a 20% validation subset.
  • Real-world experiments use a 7-DOF robot.

B. Target Reaching

Target-reaching experiments evaluate bottleneck-pose estimation methods across seven objects and compare their approach-trajectory errors. The results favor explicit sequential state estimation, constant uncertainty, and filtering methods in the reported comparisons.

  • The target-reaching task evaluates approach trajectories using seven random objects and multiple artificial bottlenecks.
  • Eleven bottleneck-pose estimation methods include oracle, single-image, best-image, visual-servoing, batch, and filtering variants.
  • Table I reports mean, minimum, and maximum position errors in mm and orientation errors in degrees, averaged across five poses per object and seven objects.
  • The best multiple-prediction fusion methods typically outperform basic visual servoing, supporting explicit state estimation over implicit deep-learning estimation.
  • Methods using constant Prior uncertainty typically outperform Dropout and Predicted uncertainty, while filtering performs at least as well as batch estimation.

C. Imitation Learning

The experiments evaluate the framework on eight everyday manipulation tasks using single demonstrations, comparing coarse-trajectory methods and last-inch correction. Full filtering with correction achieves encouraging success rates across tasks, while performance varies with precision, visual texture, and interaction demands.

  • Tasks: The evaluation covers 8 everyday tasks spanning varied manipulation challenges.Tasks include placing, inserting, twisting, lifting, hammering, scooping, and plugging actions.
  • Evaluation: Results report success rates over 20 object poses per task, with the best result per table column highlighted.The task success criteria were based on the final object configuration and judged by eye.
  • Results: The full implementation combines filtering with last-inch correction and learns a range of tasks from a single demonstration.Three tasks succeeded in all 20 tested object poses.
  • Results: Sequential state estimation outperforms basic visual servoing, and last-inch correction significantly increases success rates.The comparison includes visual servoing, filtering with prior uncertainty, and residual reinforcement learning methods for the coarse trajectory.
  • Task difficulty: Task difficulty depends on required precision and the suitability of object shape and texture for 2D image-based pose estimation.Knife is difficult because its thin slot and low-texture rack complicate orientation prediction.
  • Task difficulty: Plug succeeds in 9 out of 20 socket poses despite contact-rich insertion, while Hammer and Scoop benefit from replaying demonstration velocities at high interaction speeds.Plug performance would likely be lower without compliant sliding, whereas Hammer and Scoop suit direct velocity replication.

V. CONCLUSIONS

The conclusion presents Coarse-to-Fine Imitation Learning as a state-estimation framework that separates approach from interaction. It learns varied real-world tasks from one demonstration while retaining an analytical, stable, and interpretable controller.

  • Contribution: The framework models manipulation as a coarse approach trajectory followed by a fine interaction trajectory, with imitation learning formulated as state estimation.This structure supports learning novel real-world everyday tasks from a single human demonstration.
  • Controller: A key component replays the demonstration’s velocities to acquire complex motion without explicitly learning a policy.The conclusion contrasts this analytical controller with end-to-end policy-learning methods.
  • Outcome: The controller is described as analytical, stable, and interpretable, and the framework learns a range of novel real-world tasks from one demonstration.The paper contrasts this controller characterization with many end-to-end visual imitation methods.
  • Future work: Future work includes 3D computer vision for bottleneck pose estimation, closed-loop interaction control, and multi-stage tasks.These directions extend the pose estimator, interaction controller, and task scope.
Loading 2105.06411v2…