Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Sridhar Pandian Arunachalam; Sneha Silwal; Ben Evans; Lerrel Pinto

Source-linked AI summary

Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Sridhar Pandian Arunachalam, Sneha Silwal, Ben Evans, Lerrel Pinto

arXiv:2203.13251v1Mar 24, 2022cs.ROcs.AIcs.CVcs.LG

TL;DR

Dexterous manipulation is difficult to learn efficiently because multi-finger control requires high-dimensional actions and collecting demonstrations is challenging. DIME uses a single RGB camera to teleoperate an Allegro hand, then applies imitation learning in simulation and on hardware, where it solves flipping, spinning, and rotating tasks. The framework’s scope is limited by RGB depth ambiguity and weaker real-robot performance on some tasks such as spinning.

Problem
High-dimensional multi-finger control and difficult demonstration collection make efficient learning for dexterous manipulation challenging.
Method
DIME uses a single RGB camera to detect and retarget a human operator’s fingertips, then trains dexterous policies with imitation learning.
Results
DIME solves flipping, spinning, and rotating tasks on simulated and real Allegro hands using imitation-learning policies.
Takeaways & Limitations
Inexpensive demonstrations collected with DIME are compatible with standard imitation-learning methods for dexterous manipulation.
Takeaways & Limitations
RGB-based demonstration collection requires a specified z-plane, and some real-robot tasks such as spinning have low success rates.

Abstract

from arXiv · show

Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature. Perhaps one of the most powerful techniques to learn complex manipulation strategies is imitation learning. However, collecting and learning from demonstrations in dexterous manipulation is quite challenging. The complex, high-dimensional action-space involved with multi-finger control often leads to poor sample efficiency of learning-based methods. In this work, we propose 'Dexterous Imitation Made Easy' (DIME) a new imitation learning framework for dexterous manipulation. DIME only requires a single RGB camera to observe a human operator and teleoperate our robotic hand. Once demonstrations are collected, DIME employs standard imitation learning methods to train dexterous manipulation policies. On both simulation and real robot benchmarks we demonstrate that DIME can be used to solve complex, in-hand manipulation tasks such as 'flipping', 'spinning', and 'rotating' objects with the Allegro hand. Our framework along with pre-collected demonstrations is publicly available at https://nyu-robot-learning.github.io/dime.

I. INTRODUCTION

DIME addresses the difficulty of learning dexterous manipulation by combining inexpensive single-camera demonstrations with imitation learning. It supports dexterous tasks in simulation and on a real Allegro hand.

I. INTRODUCTION: High-dimensional, contact-rich robot actions make dexterous control difficult, while model-free RL often requires reward design, millions of samples, and weeks of training.
I. INTRODUCTION: Imitation learning is sample-efficient but demonstrations for dexterous hands are difficult to obtain with kinesthetic teaching, cyber gloves, or multi-camera visual piloting.
I. INTRODUCTION: DIME introduces a framework that collects and learns from visual demonstrations for dexterous manipulation.The framework uses a single RGB camera and is designed for multi-fingered robotic hands.
I. INTRODUCTION: DIME teleoperates a multi-fingered hand from a single RGB camera by detecting fingertip positions, retargeting them, and controlling the robot.
I. INTRODUCTION: DIME studies imitation in simulation and on hardware, using model-free RL with demonstrations in simulation and nearest-neighbor learning on the real robot.Simulation training takes about 2 days, while nearest-neighbor learning can achieve high performance immediately on hardware.
I. INTRODUCTION: The framework solves dexterous flipping, spinning, and rotating tasks and provides demonstrations compatible with state-of-the-art imitation learning algorithms.

II. RELATED WORK

Prior work obtains robot demonstrations through physical guidance, assistive devices, gloves, or camera rigs, but these approaches face dexterity, cost, calibration, or registration challenges. DIME builds on this literature with single-camera hand tracking.

II. RELATED WORK: Kinesthetic training is difficult for multi-fingered hands because their action space is substantially larger than that of robotic arms.
II. RELATED WORK: CyberGloves precisely measure hand motion but are expensive and require calibration before operation.
II. RELATED WORK: DexPilot uses an RGBD camera rig for dexterous demonstrations, whereas DIME alleviates the need for multiple cameras and associated registration and calibration.
II. RELATED WORK: Single-camera hand-pose estimation is advantageous over gloves and calibrated camera rigs, but occlusions can make parts of the hand unobservable.The authors report that operators avoid occlusion regions in their experiments.

C. Imitation Learning

DIME uses teleoperated demonstrations to train dexterous policies in simulation or on hardware. The framework pairs simulation with sample-complex policy-gradient methods and real-robot learning with non-parametric action prediction.

C. Imitation Learning: DIME uses non-parametric nearest-neighbor imitation to map observations to actions because of its simplicity, bounded action space, and empirical manipulation performance.
C. Imitation Learning: Prior reinforcement-learning work motivates combining demonstrations with RL because pure RL can require simple behaviors, perfect models, or extensive simulated time.
C. Imitation Learning: DIME has two phases: RGB-based teleoperation of an Allegro hand followed by imitation learning from the collected demonstrations.
C. Imitation Learning: The teleoperation pipeline estimates hand pose, retargets fingertip locations, uses inverse kinematics, and provides visual feedback to the operator.
C. Imitation Learning: Simulation learning supports sample-complex policy-gradient methods that can correct for noise in demonstrations, while real-robot learning uses more sample-efficient methods.

B. Hand Pose Mapping

DIME maps single-camera hand observations to robot fingertip targets using 2.5D landmarks and a simple planar retargeting scheme. Calibration uses reference poses and interpolation before control is applied.

B. Hand Pose Mapping: A single RGB camera and MediaPipe extract 2.5D hand landmarks, which DIME maps directly to 3D robot fingertip positions.
B. Hand Pose Mapping: Because absolute depth is unavailable, DIME treats fingertips as lying on a plane above the palm and keeps robot fingertips at a fixed height.
B. Hand Pose Mapping: The index, middle, and ring fingers are mapped along the y plane, while the pinky provides finer control and the thumb controls robot xy movement within a quadrilateral bound.
B. Hand Pose Mapping: Calibration requires moving both hands through reference positions at finger extremes, then linearly interpolating intermediate target 3D positions.
B. Hand Pose Mapping: The robot computes joint angles with inverse kinematics and combines gravity compensation with PD control, running control at 300Hz and streaming desired positions at 30Hz.

D. Demonstration Collection

DIME collects demonstrations through real-time hand-pose detection and fingertip-based control, then applies imitation-learning algorithms in simulation. DAPG augments policy gradients with demonstration-derived behavior-cloning updates.

DIME runs hand-pose detection and mapping in real time on a single desktop computer.
Demonstrations record Allegro Hand images, object state, and target fingertip locations at 5 Hz for state- and image-based imitation.The action space consists of four desired fingertip positions relative to the wrist.
DAPG combines standard policy gradients with a weighted behavior-cloning gradient derived from demonstration and policy state-action distributions.Its exponential weighting decays to zero during training.
Simulation success curves show that behavior cloning finetuned with reinforcement learning, including BCRL and DAPG, achieves high success across all tasks.

F. Imitation on the Robot

For real-robot imitation, DIME uses non-parametric nearest-neighbor learning to avoid the multi-day online training required by simulation-oriented DAPG. The experiments evaluate whether demonstrations can support dexterous behaviors.

DIME turns to sample-efficient non-parametric imitation for real robots because DAPG would require several days of online training.
Nearest-neighbor learning predicts actions by averaging outputs from the k closest training inputs.
The robot experiments ask whether DIME collects high-quality demonstrations and whether those demonstrations train dexterous behaviors.

A. Dexterous Manipulation Tasks

The evaluation covers three Allegro Hand tasks requiring coordinated multi-finger contact: flipping a rectangular object, spinning a knob, and rotating a cube. Simulation environments use dense distance-based rewards for reinforcement-learning optimization.

The benchmark includes flipping, spinning, and rotating tasks designed to reflect challenges in robot dexterity.
Flipping succeeds when a rectangular object lands within 2 cm of the palm center within 1 minute.The task requires precise coordination because uneven finger movements can cause the object to fall.
Spinning succeeds after 120 degrees of rotation for a three-pronged knob within 1 minute.The hardware observation excludes the knob’s rotation, unlike the simulation observation.
Rotating succeeds after 90 degrees of cube rotation in the plane within 1 minute.The hand must maintain multi-finger contacts while correcting deviations from the palm center.
MuJoCo simulation environments use dense rewards linear in the object’s distance from its target pose.

B. Demonstration Collection

DIME collects demonstrations with a single-camera teleoperation setup and evaluates both demonstration difficulty and learned robot behavior. Non-parametric INN performs best across tasks, while VINN solves two tasks and BC solves none.

DIME collects 30 demonstrations per task, while simulation uses 10 demonstrations per task.Average teleoperation times are 30, 120, and 150 seconds for flipping, spinning, and rotating, respectively.
INN outperforms parametric behavior cloning across all three real-robot tasks.The quantitative experiments use ten trials per algorithm and task.
VINN solves the flipping and rotating tasks, whereas behavior cloning is unable to solve any task.The comparison is visualized across the three tasks.
Single-camera occlusions can cause MediaPipe inaccuracies, so operators adjust their hands using continuous visual feedback from the robot hand.
Four untrained volunteers required 210 seconds on average for their first rotating demonstration, 40% longer than a trained expert.

1) Behavior Cloning (BC & VBC):

The paper compares supervised behavior cloning with non-parametric nearest-neighbor imitation for learning dexterous manipulation policies from demonstrations.

1) Behavior Cloning (BC & VBC):: Behavior cloning uses a parametric neural network to predict actions from states through supervised training.
1) Behavior Cloning (BC & VBC):: The real-robot experiments evaluate both vision-based and state-based observations for behavior cloning and nearest-neighbor imitation.
1) Behavior Cloning (BC & VBC):: Nearest-neighbor imitation matches observations to demonstration examples and applies the action associated with the best match.
1) Behavior Cloning (BC & VBC):: The simulation methods compare randomly initialized model-free RL with policies initialized from demonstrations or behavior cloning.

4) Behavior Cloning with RL finetuning (BCRL):

BCRL combines behavior cloning with model-free reinforcement learning to refine demonstration-derived policies in simulation, producing smoother and more capable behaviors than PPO alone on several tasks.

4) Behavior Cloning with RL finetuning (BCRL):: The RL-based algorithms are evaluated only in simulation because real-robot training would require substantial time and could create safety risks.
4) Behavior Cloning with RL finetuning (BCRL):: BCRL and DAPG produce similar results, while PPO exhibits more erratic movements and fails to rotate the cube successfully.
4) Behavior Cloning with RL finetuning (BCRL):: Demonstration-based policies are qualitatively smoother than teleoperated demonstrations, including a learned rotation strategy that lifts the cube between two fingers.
4) Behavior Cloning with RL finetuning (BCRL):: RL finetuning lets behavior-cloned policies account for states outside the demonstrations’ support, whereas pure behavior cloning fails on all tasks.
4) Behavior Cloning with RL finetuning (BCRL):: PPO solves spinning as well as DAPG, but succeeds there through extreme random finger movements rather than the smoother behavior learned with demonstrations.

E. Imitation Learning on Real Allegro Hand

On the real Allegro Hand, nearest-neighbor imitation outperforms parametric behavior cloning across tasks, while visual imitation is strong on flipping and rotation but weaker on spinning.

E. Imitation Learning on Real Allegro Hand: Nearest-neighbor imitation outperforms parametric behavior cloning across all real-robot tasks, with INN achieving perfect success on 90-degree cube rotation.
E. Imitation Learning on Real Allegro Hand: Performance degrades mildly at larger rotation angles when the cube moves outside the manipulable region of the palm.
E. Imitation Learning on Real Allegro Hand: Visual imitation performs strongly on flipping and rotation but underperforms state-based imitation on average.
E. Imitation Learning on Real Allegro Hand: VINN fails to learn good representations for spinning, which the authors attribute to visual complexity and limited demonstrations.
E. Imitation Learning on Real Allegro Hand: The framework remains limited by RGB depth ambiguity requiring a specified z-plane, and some real-robot tasks such as spinning do not achieve high success rates.