PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control

Chen Wang, Rui Wang, Ajay Mandlekar, Li Fei-Fei, Silvio Savarese, Danfei Xu

arXiv:2103.00375v2cs.ROcs.AIcs.LG

TL;DR

Imitation learning from demonstrations can struggle to generalize to unseen configurations, while human hand-eye coordination directs movement toward task-relevant objects regardless of their absolute locations. HAN learns 3D keypoints and selects an attended location to guide end-effector commands, training end-to-end from human teleoperated demonstrations. HAN achieves strong zero-shot generalization to unseen initial configurations; with 50 demos, it outperforms BC-states in extrapolation, with a 0.6 versus 0.2 success rate.

  • Problem

    Imitation learning from demonstrations can struggle to generalize to unseen configurations, while human hand-eye coordination directs movement toward task-relevant objects regardless of their absolute locations.

    Source: §I · p. 1

  • Method

    HAN learns 3D keypoints and selects an attended location to guide end-effector commands, training end-to-end from human teleoperated demonstrations.

    Source: §I · pp. 1–2

  • Results

    HAN achieves strong zero-shot generalization to unseen initial configurations; with 50 demos, it outperforms BC-states in extrapolation, with a 0.6 versus 0.2 success rate.

    Source: §I · p. 2, C. Results · p. 6

  • Takeaways & Limitations

    The results support learning visuomotor policies that inherit spatial invariance from hand-eye coordination and generalize to unseen configurations.

    Source: §I · p. 2, §VI · p. 7

Abstract

from arXiv · show

Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. However, IL methods often fail to generalize to new scene configurations not covered by training data. On the other hand, humans can manipulate objects in varying conditions. Key to such capability is hand-eye coordination, a cognitive ability that enables humans to adaptively direct their movements at task-relevant objects and be invariant to the objects' absolute spatial location. In this work, we present a learnable action space, Hand-eye Action Networks (HAN), that can approximate human's hand-eye coordination behaviors by learning from human teleoperated demonstrations. Through a set of challenging multi-stage manipulation tasks, we show that a visuomotor policy equipped with HAN is able to inherit the key spatial invariance property of hand-eye coordination and achieve zero-shot generalization to new scene configurations. Additional materials available at https://sites.google.com/stanford.edu/han

I. INTRODUCTION

HAN learns to guide robot movements through task-relevant 3D locations, approximating human hand-eye coordination. Across manipulation tasks, it achieves zero-shot generalization to unseen configurations.

I. INTRODUCTION

  • Human hand-eye coordination directs movements toward task-relevant objects and makes them invariant to objects’ absolute spatial locations.During tea serving, attention shifts from the teapot to the mug as the task progresses.Source: §I · p. 1, Figure 1 · p. 1
  • The paper asks whether imitation learning can recover coordinated hand-eye movements from teleoperated demonstrations, addressing poor generalization to unseen scene configurations.The introduction notes that image-to-action policies may rely on spurious pixel-action correlations in complex multi-object scenes.Source: §I · p. 1
  • HAN learns 3D keypoints, switches attention among them, and generates end-effector commands from the selected location in an end-to-end imitation-learning network.The network is trained with behavior cloning and does not require direct keypoint supervision.Source: §I · p. 2
  • HAN inherits spatial invariance from human hand-eye coordination and achieves strong zero-shot generalization to unseen manipulation configurations.The reported tasks include grasping, stacking, and tool manipulation.Source: §I · p. 2

II. RELATED WORKS

Prior work addresses imitation learning, visual attention, spatially invariant action spaces, keypoint representations, and reference-frame selection. The paper distinguishes HAN through learned 3D attention locations that directly guide continuous robot actions.

Imitation learning

  • Image-to-action imitation policies can generalize poorly from spurious pixel-action connections; online teachers can help but require on-demand expert labels and remain limited by dataset coverage.The related-work discussion identifies behavioral cloning as the imitation-learning formulation studied here.Source: §II · p. 2

Visual attentions for policy learning

  • Visual-attention methods extract task-relevant features, but connecting 2D visual features to 3D robot actions remains unclear; this work uses attended spatial locations to guide actions.The approach is trained end-to-end from human demonstrations.Source: §II · p. 2

Spatially-invariant action spaces

  • Spatial action maps share convolution kernels across locations, but prior methods are tied to top-down views and high-level actions, limiting fine-grained manipulation such as tool use.HAN instead learns spatially invariant policies with continuous end-effector control.Source: §II · p. 2

Keypoint representations for control

  • Keypoint methods [21] [22] often use keypoints as features or regularizers, whereas this work grounds 3D keypoints to guide actions without added supervision.The demonstrations provide continuous control actions through teleoperation.Source: §II · p. 2

Reference frame selection for control

  • Reference-frame selection methods may assume salient objects or ground-truth positions; this work learns action attention directly from vision, where demonstrations may contain spatial hints.The passage also notes visual observation as a more generic input for manipulation tasks.Source: §II · pp. 2–3

III. PROBLEM DEFINITION

The paper formulates robot manipulation as an MDP and uses behavior cloning to train policies from demonstrations. It targets zero-shot generalization to new initial-state distributions, with actions limited to relative position changes and a binary gripper command.

III. PROBLEM DEFINITION

  • The policy observes each state, chooses an action, and receives the next state and reward under the MDP formulation.Source: §III · p. 3
  • Behavior cloning trains a policy to clone demonstrated actions by minimizing squared action error over the demonstration dataset.Source: §III · p. 3
  • The target is zero-shot generalization to new initial-state distributions without additional training data.Source: §III · p. 3
  • The policy outputs relative xyz end-effector position changes and a binary gripper action; full 6-DoF control is deferred to future work.Source: §III · p. 3

IV. METHOD

HAN is a differentiable action space for imitation learning that uses task-relevant spatial locations to guide end-effector actions. Its 3D visual attention network turns image regions into robot-frame keypoints that can guide movements.

IV. METHOD

  • HAN combines 3D visual attention, attention switching, and action targeting to guide end-effector actions from attended spatial locations.The components are trained end-to-end from demonstrations using behavior cloning.Source: §IV · p. 3

A. 3D Visual Attention Network

B. Attention Switching Network

The attention switching network produces one 3D target location from candidate keypoints. Its learned weighting lets the target attend to different scene locations as task stages change.

B. Attention Switching Network

  • Multi-stage tasks can require shifting attention from one object to another, such as from a teapot handle to a mug.Source: B. Attention Switching Network · pp. 3–4
  • HAN’s architecture uses the selected target keypoint as the basis for a local action target, which is shifted by a learned offset before action generation.Source: Figure 2 · p. 4
  • The network combines candidate keypoints with learned confidence weights to produce a single 3D location for the next action.The weights are generated from region features by an MLP and softmax normalization.Source: B. Attention Switching Network · p. 4

C. Action Target Network

The action target network maps attended keypoints to end-effector actions using a constrained local target. This design balances expressive trajectory fitting with retaining a meaningful connection between the keypoint and action.

C. Action Target Network

  • A flexible network mapping from target keypoints to actions can ignore the keypoint and rely on other inputs instead.Source: C. Action Target Network · p. 4
  • The network predicts a 3D offset from the attended keypoint to set a local action target relative to the end effector.The constrained target is intended to balance nonlinear trajectory fitting against over-parameterization.Source: C. Action Target Network · p. 4
  • A learned scalar gain modulates action magnitude according to the target’s position relative to the end effector.Source: C. Action Target Network · p. 4
  • The MLP predicts offset, gain, and gripper command from keypoint and end-effector positions plus local and region visual features.Source: C. Action Target Network · p. 4
  • Offset coordinates are constrained to [-0.1, 0.1] and gain to [0, 1], which the authors report worked well across evaluated tasks and prevents degenerating into ignoring the target keypoint.Source: C. Action Target Network · p. 4

D. Deep Imitation Learning

HAN is trained end-to-end with behavior cloning on demonstrations. Its loss combines action error and directional alignment.

V. EXPERIMENTS

The experiments test whether HAN improves task performance and zero-shot generalization, approximates hand-eye coordination, and benefits from its design components. The evaluation uses three manipulation tasks and tests new initial configurations beyond those used for demonstrations.

V. EXPERIMENTS

  • The experiments ask whether HAN improves performance and zero-shot generalization, approximates hand-eye coordination, and benefits from its components.Source: §V · p. 5

A. Task Setup

  • The study evaluates lifting, stacking, and tool use, from basic grasping to multi-stage manipulation.Source: §V · p. 5, A. Task Setup · p. 5
  • Demonstrations use interpolation-region initializations, while evaluation includes both interpolation and extrapolation regions.The evaluated workspace and robot setup uses RGB images and depth values for 3D keypoint generation.Source: A. Task Setup · p. 5, Figure 3 · p. 5
  • Stacking requires switching attention from the red block to the green plate; tool use requires grasping a tool, hooking and retrieving a cube, then inserting it into a ring.The tool-using task is long-horizon and requires manipulating one object with another.Source: A. Task Setup · p. 5

B. Learning Setup and Baselines

The learning setup compares expert demonstrations with human teleoperation data and tests HAN against image-based, state-based, and HAN ablation baselines. Human demonstrations are noisy and can include actions that do not follow the assumed hand-eye coordination pattern.

  • Demonstrations come from a noisy hard-coded expert policy or human teleoperation through RoboTurk.The expert policy includes added Gaussian noise; the human demonstrations use a smartphone interface.Source: B. Learning Setup and Baselines · p. 5
  • Rollout snapshots show predicted keypoints, action targets, and highest-confidence 2D bounding boxes for Stacking and Tool-using.Some action targets can be occluded by scene objects.Source: Figure 4 · p. 6
  • Human actions can be suboptimal and include curvy approaches whose sideways motion cannot be explained by attention.This tests whether HAN can recover hand-eye coordination behavior from noisy samples.Source: B. Learning Setup and Baselines · p. 6
  • The baselines include an MLP using ground-truth poses, an image-based imitation model, and HAN variants that change its action-target formulation or attention components.The variants use an MLP action-target network, omit region-based attention, or remove constraints on the offset and scalar gain.Source: B. Learning Setup and Baselines · p. 6

C. Results

With 50 demonstrations, HAN outperforms a ground-truth-state baseline in extrapolation on Lifting, with similar trends in Stacking and Tool-using. Qualitative rollouts show stagewise attention shifts, and HAN retains the strongest performance among image-based methods with human demonstrations.

  • 0.6 vs. 0.2: with 50 demonstrations, HAN outperforms BC-states in extrapolation; similar trends hold in Stacking and Tool-using.BC-states has access to ground-truth state.Source: C. Results · p. 6
  • In Stacking, the predicted keypoint switches from the grasped red block to the green plate, while Tool-using rollouts attend to the relevant object at each stage.Source: C. Results · p. 6
  • With human demonstrations, HAN has the smallest performance drop among image-based methods and outperforms the other baselines in interpolation and extrapolation.The results verify that HAN can learn hand-eye coordination behavior from human data.Source: C. Results · pp. 6–7

VI. CONCLUSION

The conclusion presents HAN as a differentiable imitation-learning policy with a novel action space that solves challenging manipulation tasks and generalizes to unseen situations. The work focuses on imitation learning, with reinforcement learning and rotational actions identified as possible future extensions.

  • HAN solves challenging manipulation tasks and generalizes to unseen situations, a property the authors say is absent from existing imitation-learning algorithms.Source: §VI · p. 7
  • HAN is a fully differentiable policy network with a novel action space for learning robotic manipulation through imitation learning.Source: §VI · p. 7
  • The work focuses on imitation learning; reinforcement learning may also benefit, and extending HAN to rotational actions remains future work.Source: §VI · p. 7

Related papers

Loading 2103.00375v2…