PaperlayerView exact revision on arXiv

Source-linked AI summary

From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation

Yuzhe Qin, Hao Su, Xiaolong Wang

arXiv:2204.12490v2cs.ROcs.CVcs.LG

TL;DR

Dexterous manipulation learning needs efficient, intuitive demonstrations that can support different robot hands and transfer beyond simulation. The paper introduces single-camera customized-hand teleoperation, translates demonstrations offline, and uses them for imitation-augmented learning, obtaining better simulated performance and more robust real-world transfer.

  • Problem

    High-DoF dexterous manipulation makes reinforcement learning sample-inefficient, while vision-based teleoperation requires difficult retargeting and robot-specific data collection.

  • Method

    The system reconstructs a customized simulated robot hand for each operator, collects demonstrations with an iPad, translates trajectories offline to specified robot hands, and augments RL with behavior cloning.

  • Results

    The method significantly improves dexterous manipulation over baselines across specified robot hands and yields more robust real-world transfer than pure RL.

  • Takeaways & Limitations

    A single demonstration-collection process can generate imitation data for multiple robot hands while supporting human-like skills and improved robustness against the Sim2Real gap.

Abstract

from arXiv · show

We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand. We introduce a novel single-camera teleoperation system to collect the 3D demonstrations efficiently with only an iPad and a computer. One key contribution of our system is that we construct a customized robot hand for each user in the physical simulator, which is a manipulator resembling the same kinematics structure and shape of the operator's hand. This provides an intuitive interface and avoid unstable human-robot hand retargeting for data collection, leading to large-scale and high quality data. Once the data is collected, the customized robot hand trajectories can be converted to different specified robot hands (models that are manufactured) to generate training demonstrations. With imitation learning using our data, we show large improvement over baselines with multiple complex manipulation tasks. Importantly, we show our learned policy is significantly more robust when transferring to the real robot. More videos can be found in the https://yzqin.github.io/dex-teleop-imitation .

I. INTRODUCTION

The paper addresses inefficient and difficult dexterous manipulation learning by introducing single-camera teleoperation with customized robot hands and offline translation to multiple specified robots. Demonstrations improve simulated manipulation and produce more robust real-world transfer than pure RL.

  • High-DoF joints and discontinuous contact make reinforcement-learning policies sample-inefficient for dexterous manipulation.
  • Vision-based teleoperation is more flexible and scalable than VR or wired-glove systems, but requires motion retargeting and calibration.
  • The system constructs a customized simulated robot hand matching each operator’s hand geometry and kinematics, avoiding online retargeting during teleoperation.
  • Offline retargeting converts one customized-hand trajectory collection into demonstrations for Schunk, Adroit, and Allegro robot hands.
  • 60 successful Relocate demonstrations per hour were collected with the customized hand versus around 10 by directly operating the Allegro Hand in simulation.
  • Imitation-trained policies significantly outperform baselines on specified robot hands and transfer more robustly to real and unseen objects than pure RL.

II. RELATED WORK

Prior work establishes dexterous manipulation, imitation learning, and vision-based teleoperation as complementary directions, while this paper combines them into an end-to-end multi-robot learning framework. Its distinctive setup uses a single iPad camera, customized hands, offline translation, and demonstration-augmented policy learning.

  • Dexterous Manipulation: RL has shown promise for complex in-hand manipulation but suffers from poor sample efficiency in high-dimensional dexterous hands.
  • Imitation Learning from Demonstrations: Human demonstrations constrain exploration toward expected human-like behavior and reduce sample-efficiency demands in imitation learning.
  • Imitation Learning from Demonstrations: Unlike prior motion-imitation work that follows one expert demonstration, this framework trains policies across different environment configurations.
  • Vision-based Teleoperation: Compared with vision-based teleoperation using four calibrated RealSense cameras, this work requires only a single iPad camera and supports multiple specified robot hands.
  • The framework combines customized-hand teleoperation, multi-robot demonstration translation, and demonstration-augmented reinforcement learning.
  • Sim2Real experiments with an Allegro Hand on XArm-6 show that demonstrations significantly increase policy robustness against the Sim2Real gap.

IV. CUSTOMIZED HAND TELEOPERATION

The teleoperation setup uses an iPad and laptop to stream and process the operator’s hand while displaying a physical simulation. It supports customized-hand visualization and compares degrees of freedom across robot models.

  • The system streams RGB-D video at 25 fps and processes each frame in less than 30 ms, matching the camera frequency.
  • Its three components are a physical simulator, a hand detector, and a GUI for visualizing the simulation environment.
  • Table I compares finger degrees of freedom across robot models, including the customized hand.
  • The hardware setup consists of an iPad and a computer, with the iPad providing the teleoperation capture device.
  • Customized robot hands are generated from different human hands, with red lines depicting their kinematics trees.

A. Task Description

The study evaluates three dexterous manipulation tasks with randomized object or environment conditions. A hand detector estimates the operator’s wrist, pose, and shape parameters from RGB-D input for teleoperation.

  • Relocate: Relocate requires picking up an object and matching a randomized target pose using three YCB objects.
  • Flip: Flip requires rotating a mug 90 degrees without pushing it away, testing directional force control under randomized conditions.
  • Open Door: Open Door requires grasping and rotating a handle to unlock the door, then pulling it open with randomized door positions.
  • The hand detector outputs wrist pose, hand pose parameters, and hand shape parameters from RGB-D frames.
  • MediaPipe crops the detected hand region, and FrankMocap estimates pose and shape parameters using the SMPL-X representation.

C. Customized Robot Hand

The system builds a user-specific customized robot hand in simulation from estimated hand geometry and matching kinematics. Demonstrations are later retargeted offline to specified robot hands using keypoint and temporal-consistency objectives.

  • C. Customized Robot Hand: The customized hand reconstructs each user’s hand geometry and creates an articulated simulator model with the same kinematics structure.Primitive shapes such as boxes and capsules support efficient collision detection and stable simulation.
  • C. Customized Robot Hand: Pose estimates control the customized hand through filtered position targets for each joint.A confidence score derived from hand-shape estimation modulates the controller under unreliable camera views.
  • C. Customized Robot Hand: The collected trajectory is translated into demonstrations for three different robot hands after teleoperation.The figure depicts camera input, hand-pose detection, customized-hand simulation, and retargeted demonstrations on other hands.
  • C. Customized Robot Hand: When perception error is large, the controller reduces stiffness to prevent abrupt motion.The joint torque uses PD parameters, with stiffness adjusted according to confidence in the estimated pose.
  • C. Customized Robot Hand: Offline retargeting converts customized-hand trajectories into specified robot-hand trajectories using keypoint alignment and joint-change regularization.The optimization uses forward kinematics, initializes from the previous customized-hand state, and enforces joint limits for each specified robot.

B. Action Computation

The action-computation stage converts specified-robot pose trajectories into control actions for demo-augmented policy learning. It filters the trajectory before computing joint torques through inverse dynamics.

  • B. Action Computation: Specified-robot pose trajectories are converted into finger-joint actions, such as joint torques or motor-control commands.The procedure follows DexMV’s action-estimation approach.
  • B. Action Computation: The pose trajectory first passes through a first-order low-pass filter before torque computation.Filtering is applied before using the robot’s manipulator equation for inverse dynamics.
  • B. Action Computation: The translated demonstrations depict different robot hands executing the same task after pose-trajectory conversion.The visualization includes Schunk, Allegro, and Adroit hands in the translated demonstrations.

VI. DEMONSTRATION-AUGMENTED POLICY LEARNING

The paper uses demonstration-augmented policy learning rather than naive behavior cloning for randomized dexterous tasks. DAPG combines an RL policy-gradient objective with an imitation objective based on demonstrations.

  • VI. DEMONSTRATION-AUGMENTED POLICY LEARNING: Retargeted demonstrations are used for imitation learning on the dexterous manipulation tasks.The demonstrations provide the input to the policy-learning stage after retargeting.
  • VI. DEMONSTRATION-AUGMENTED POLICY LEARNING: DAPG is adopted because randomized initial and target poses make naive behavior cloning difficult.The method incorporates demonstrations directly into reinforcement learning.
  • VI. DEMONSTRATION-AUGMENTED POLICY LEARNING: DAPG combines the vanilla policy-gradient objective with an imitation objective using demonstration data.The two objectives are weighted by hyperparameters during training iterations.

VII. EXPERIMENT

The experiments evaluate teleoperation efficiency and imitation learning across multiple dexterous tasks, robot hands, and simulation settings. Customized-hand demonstrations improve data collection and generally outperform reinforcement-learning baselines.

  • Teleoperation User Study: 17 operators compared customized, Schunk, Adroit, and Allegro hands on Relocate and Open Door tasks.The customized hand used direct pose targets, whereas the other hands required online motion retargeting.
  • Teleoperation User Study: 60 successful Relocate demonstrations per hour were collected with the customized hand versus 10 with the Allegro hand.Operators also reported smoother control because customized-hand teleoperation removed variable online retargeting delays of 76 ± 65 milliseconds.
  • Experimental Setup: Learning curves were compared across RL and DAPG for four robot hands, with shaded regions representing standard deviation over three random seeds.The experiments also varied friction, object density, controller parameters, and demonstration count.
  • Task Learning Comparison: Imitation learning outperformed the RL baseline for most tasks and robot hands.The evaluation covered Relocate with three objects, Flip a mug, and Open Door across specified robot hands.
  • Task Learning Comparison: DAPG produced more natural Open Door behavior than pure RL, grasping the handle rather than relying on forceful friction-based opening.Open Door with the Allegro hand was the reported exception where imitation learning did not outperform RL.

C. Ablation Study

The ablation study examines how dynamics parameters and demonstration quantity affect DAPG on Relocate with a tomato soup can and Schunk hand.

  • C. Ablation Study: DAPG learning was tested under changes to friction, object density, PD controller parameters, and demonstration count.The demonstrations remained fixed across the ablation conditions.
  • C. Ablation Study: The learning curve remained robust to friction changes in the tested Relocate setting.The passage relates this result to the grasping requirements of a two-finger parallel-jaw gripper.

D. Real-World Robot Experiments

Real-world experiments transfer policies to an Allegro hand mounted on an XArm-6 and compare imitation learning with pure RL on Relocate and Flip. The imitation-trained policies show a larger real-world advantage than in simulation and can generalize to novel objects.

  • D. Real-World Robot Experiments: Real-world evaluation used an Allegro hand attached to an XArm-6 on Relocate and Flip tasks.A matching XArm6+Allegro simulation model was also constructed for comparison.
  • D. Real-World Robot Experiments: The real-world gap between imitation learning and pure RL was larger than the gap observed in simulation.The authors conjecture that more human-like manipulation is more robust to the Sim2Real gap.
  • D. Real-World Robot Experiments: Imitation learning generalized Relocate policies to novel objects not seen during training.The evaluation sampled known and novel objects using the same object sets for each policy.
  • D. Real-World Robot Experiments: The object-shape geometry was not included in the policy input; only the 6D object pose was used.The authors connect this setting to multi-finger robustness under changes in object shape.
  • D. Real-World Robot Experiments: RL often used unstable two-finger contact for Relocate, whereas imitation learning used all four fingers and stably grasped the object.This behavioral difference explains the observed real-robot contrast in the visualized example.
  • D. Real-World Robot Experiments: For Flip, RL pushed the mug swiftly in simulation, while imitation learning inserted a finger and rotated the wrist in a human-like sequence.The pushing strategy could hardly succeed with the real robot, whereas the demonstration-like behavior achieved much better results.

VIII. CONCLUSION

The paper presents a single-camera teleoperation system with customized robot hands for collecting demonstrations and transferring them across robot platforms. The demonstrations improve dexterous manipulation and real-world robustness while requiring data collection only once.

  • VIII. CONCLUSION: The system uses a customized robot hand to provide an intuitive interface for collecting human hand manipulation data.The same collected data can support multiple robot hands and improve real-world deployment robustness.
  • VIII. CONCLUSION: Data collection needs to be conducted only once to generate demonstrations for multiple robots.The conclusion links this reuse to improvements in dexterous manipulation and robustness after deployment.
Loading 2204.12490v2…