Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Tony Z. Zhao; Vikash Kumar; Sergey Levine; Chelsea Finn

Source-linked AI summary

Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Tony Z. Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn

arXiv:2304.13705v1Apr 23, 2023cs.ROcs.LG

TL;DR

Fine-grained manipulation remains difficult for imitation-learning systems requiring high-frequency control and closed-loop feedback. This paper introduces the low-cost ALOHA system and ACT, achieving the highest success rate among compared methods and 80–90% success on several real-world skills with about 10 minutes of demonstrations.

Problem
Existing imitation-learning algorithms perform poorly on fine-grained tasks requiring high-frequency control and closed-loop feedback.
Method
The paper combines a low-cost teleoperation system with ACT, a generative action-sequence model using action chunking and temporal ensembling.
Results
ACT achieves the highest success rate among compared methods across simulated and real tasks, reaching 84–92% success on four reported real-world tasks.
Takeaways & Limitations
The combined low-cost system and ACT learn six real-world fine-manipulation skills, including cup opening and battery slotting, from about 10 minutes of demonstrations.
Takeaways & Limitations
The system remains unable to perform some tasks, including buttoning a dress shirt, and low-cost hardware limits forceful or multi-finger manipulations.

Abstract

from arXiv · show

Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback. Performing these tasks typically requires high-end robots, accurate sensors, or careful calibration, which can be expensive and difficult to set up. Can learning enable low-cost and imprecise hardware to perform these fine manipulation tasks? We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations, collected with a custom teleoperation interface. Imitation learning, however, presents its own challenges, particularly in high-precision domains: errors in the policy can compound over time, and human demonstrations can be non-stationary. To address these challenges, we develop a simple yet novel algorithm, Action Chunking with Transformers (ACT), which learns a generative model over action sequences. ACT allows the robot to learn 6 difficult tasks in the real world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, with only 10 minutes worth of demonstrations. Project website: https://tonyzhaozh.github.io/aloha/

1 Stanford University 2 UC Berkeley 3 Meta · I. INTRODUCTION · II. RELATED WORK

The paper introduces ALOHA, a low-cost bimanual teleoperation and learning system for precise, contact-rich manipulation, and ACT, an imitation-learning method designed to reduce compounding errors. It positions this approach against expensive hardware, classical control, and prior imitation-learning methods for fine manipulation.

1 Stanford University 2 UC Berkeley 3 Meta: <$20k system cost enables bimanual teleoperation with off-the-shelf robots and 3D-printed components for precise, contact-rich, and dynamic tasks.Users backdrive leader robots while follower robots mirror their motion.
I. INTRODUCTION: Fine manipulation requires closed-loop visual feedback, hand-eye coordination, and delicate contact operations where millimeters of error can cause failure.Examples include opening a condiment-cup lid through pinching and prying and slotting a battery.
I. INTRODUCTION: Existing fine-manipulation systems rely on expensive robots and high-end sensors, whereas this work targets accessible, reproducible, and less precise low-cost hardware.Learning is proposed as a way to address the greater sensing and planning challenge created by low-cost hardware.
I. INTRODUCTION: The paper combines a dexterous low-cost teleoperation system for collecting high-quality demonstrations with an imitation-learning algorithm that learns from them.The approach emphasizes the value of human demonstrations for fine manipulation while recognizing that policy performance depends heavily on the training-data distribution.
I. INTRODUCTION: ACT groups action sequences into chunks and is designed to address compounding errors caused by small prediction mistakes in precision tasks with visual feedback.Such mistakes can produce large state deviations and make recovery difficult [47] [64] [29].
I. INTRODUCTION: The central contribution is the synergy of low-cost, high-precision teleoperation and ACT, which learns precise closed-loop behavior and drastically outperforms previous methods across 6 fine-manipulation tasks.The supplied passage identifies the combined system and algorithm as the paper’s key contribution.
II. RELATED WORK: Prior work addresses imitation-learning errors through architectures, objectives, regularization, or on-policy expert corrections, but annotation requirements make such approaches costly.Behavioral cloning maps observations to actions, while compounding errors can drive robots into hard-to-recover states [47] [64].
II. RELATED WORK: Earlier bimanual systems used classical models or learning, and fine-grained tasks often required expensive robots; this work instead targets arms costing around $5k each.Related examples include knot untying, cloth flattening, and needle threading with systems such as the da Vinci surgical robot or ABB YuMi [19] [31].

III. ALOHA: A LOW-COST OPEN-SOURCE HARDWARE SYSTEM FOR BIMANUAL TELEOPERATION

ALOHA is a low-cost, user-friendly, repairable, and easy-to-build bimanual teleoperation system developed for fine manipulation. Built within a 20k USD budget, it supports precise, contact-rich, and dynamic tasks that are unavailable in existing systems costing 5–10× more.

Design principles: The system is designed around five principles: low-cost, versatile, user-friendly, repairable, and easy-to-build.The design targets broad fine-manipulation use with accessible construction and maintenance.
Teleoperation interface: ALOHA uses direct joint-space mapping from a $3300 WidowX leader to the ViperX follower, rather than mapping VR- or camera-captured hand poses into task space.A 3D-printed handle-and-scissor mechanism improves gripper control and teleoperation usability.
Hardware setup: The setup uses four 480×640 RGB Logitech C922x webcams, including wrist-mounted close-up views, with teleoperation and recording at 50Hz.A robot cage made from 20×20mm aluminum extrusions and crossing steel cables supports the hardware.
System capabilities and cost: ALOHA costs within a 20k USD budget, comparable to a single research arm, while enabling precise, contact-rich, and dynamic manipulation tasks unavailable in existing systems costing 5–10× more.Demonstrated skills include threading zip ties, inserting RAM, and juggling a ping pong ball.

IV. ACTION CHUNKING WITH TRANSFORMERS

Action Chunking with Transformers (ACT) is introduced because existing imitation-learning algorithms perform poorly on fine-grained tasks requiring high-frequency control and closed-loop feedback. The method is trained from ALOHA demonstrations using leader joint positions as actions and follower-state plus image observations.

Motivation: ACT addresses poor imitation-learning performance on fine-grained tasks requiring high-frequency control and closed-loop feedback.The section presents ACT as a novel algorithm designed to leverage data collected by ALOHA.
Training Pipeline: Training ACT begins by collecting human demonstrations with ALOHA and recording leader-robot joint positions as the action targets.The training pipeline first gathers demonstrations, then uses the human operator’s leader joint positions as actions.
Training Pipeline: Leader rather than follower joint positions are used because the low-level PID controller implicitly determines applied force from their difference.Observations include the follower robots’ current joint positions and the image feed.

A. Action Chunking and Temporal Ensemble · B. Modeling human data

The method shortens effective imitation-learning horizons through overlapping action chunks and temporal ensembling, while modeling noisy human demonstrations with a conditional variational autoencoder. Together, these mechanisms target compounding errors, jerky execution, trajectory variability, and precision-sensitive behavior.

A. Action Chunking and Temporal Ensemble: Overlapping action chunks reduce the effective horizon of high-frequency trajectories, addressing compounding imitation-learning errors while remaining compatible with pixel-to-action policies.The approach is inspired by neuroscience action chunking, where actions are grouped and executed as one unit.
A. Action Chunking and Temporal Ensemble: The training algorithm reconstructs action sequences with a decoder while regularizing a latent encoder toward a Gaussian prior using a weighted reconstruction-plus-KL objective.The encoder conditions on action sequences and non-image observations, while the decoder predicts action chunks from observations and latent variables.
A. Action Chunking and Temporal Ensemble: At inference, the policy predicts future action chunks from observations, stores their timestep-aligned actions, and selects the current action through temporal aggregation.The inference procedure uses the trained decoder with z = 0 and FIFO buffers for overlapping predictions.
A. Action Chunking and Temporal Ensemble: Querying the policy every timestep creates overlapping chunks, avoiding abrupt observation updates every k steps that can otherwise produce jerky robot motion.Temporal ensembling combines multiple predicted actions at each timestep.
B. Modeling human data: Human demonstrations are noisy because identical observations can yield different valid trajectories, with greater stochasticity in regions where precision matters less.This motivates focusing policy learning on regions requiring high precision.
B. Modeling human data: The policy models demonstration variability as a conditional variational autoencoder that generates action sequences conditioned on current observations.Its encoder is used only during training, while the decoder serves as the policy and is used at test time.
B. Modeling human data: The CVAE objective combines reconstruction loss with Gaussian-prior regularization, whose weight β controls latent information transmission; the objective was essential for precise human-demonstration tasks [23, 62].Higher β transmits less information through z.

C. Implementing ACT · V. EXPERIMENTS · A. Tasks

ACT uses a transformer-based CVAE to generate action sequences, and is evaluated on eight fine-grained bimanual manipulation tasks spanning simulation and real-world ALOHA experiments. The tasks combine delicate coordination, visual-feedback correction, and challenging perception of small or translucent objects.

C. Implementing ACT: The CVAE encoder uses a BERT-like transformer over joint positions and target action sequences, while the decoder generates actions through cross-attention.Four image features form a 1200 × 512 sequence with positional embeddings, augmented by current joint positions and a style variable to produce a 1202×512 encoder input.
C. Implementing ACT: ACT has around 80M parameters, trains from scratch per task in around 5 hours on one 11G RTX 2080 Ti GPU, and infers in around 0.01 seconds.These training and inference procedures are summarized in Algorithms 1 and 2.
V. EXPERIMENTS · A. Tasks: The evaluation covers eight fine-grained bimanual tasks: two simulated MuJoCo tasks and six real-world ALOHA tasks.Task videos are provided on the project website, and Figure 6 and Figure 7 illustrate real-world and simulated task definitions.
A. Tasks: All tasks require coordinated bimanual manipulation, including grasping, inserting, unzipping, cutting, transferring, and placing delicate objects.Examples include opening a randomized ziploc bag, inserting a battery while stabilizing the controller, and hanging tape on a cardboard box.
A. Tasks: Successful execution requires visual feedback to correct millimeter-scale grasp perturbations during multistep coordination, including inserting a 3mm x 25mm loop.The velcro task uses a 2mm x 10-25mm tie and requires both arms to insert one end into the other in mid-air.
A. Tasks: Perception is difficult because transparent or translucent objects, wrinkles, reflections, and small projected areas complicate localization.Challenging examples include the ziploc bag, tape, condiment cup components, and the velcro tie viewed from above.
V. EXPERIMENTS · A. Tasks: ACT significantly outperforms four baselines on success rate across two simulated and two real-world tasks, using scripted or human training data as specified.The remaining three real-world tasks are compared only against the best-performing baseline, BeT.

B. Data Collection

The study collects real-world demonstrations through ALOHA teleoperation, using task-dependent episodes and roughly 10–20 minutes of demonstrations per task. Because human demonstrations are stochastic, successful policies must learn robust coordination rather than memorize fixed motion locations.

Real-world demonstrations: 8–14-second episodes produce 400–700 time steps at 50 Hz, totaling 10–20 minutes of demonstrations per task and 30–60 minutes including resets and teleoperator mistakes.
Real-world demonstrations: 50 demonstrations are collected for each real-world task, except Thread Velcro, which receives 100.The six real-world tasks use ALOHA teleoperation for demonstration collection.
Demonstration variability: Human demonstrations vary across episodes, so the policy must prevent gripper collisions and move the left gripper to grasp the tape instead of memorizing a fixed handover position.The tape handover position differs between episodes because the operator lacks a visual or haptic reference for reproducing it exactly.

C. Experiment Results

ACT achieves the highest success rate among prior imitation-learning methods on the reported simulated and real tasks, including 84% on Cup Open, 64% on Prep Tape, and 92% on Put On Shoe. On Thread Velcro, performance falls to 20% because of premature gripping and imprecise insertion failures.

C. Experiment Results: The comparison evaluates ACT against four prior imitation-learning methods, including BC-ConvMLP and BeT, with hyperparameters tuned using cube transfer.BC-ConvMLP predicts actions from convolutional image features and joint positions, whereas BeT uses Transformer-based visual retrieval and weighted k-nearest-neighbor actions.
C. Experiment Results: ACT achieves the highest success rate compared with all prior methods on two simulated and two real tasks in Table I.Simulated results average three seeds with 50 trials each, while real-world results use one seed and 25 trials.
C. Experiment Results: Thread Velcro success decreases from 92% at the first stage to 20% finally, as premature gripper closure and imprecise insertion cause failures.The right arm fails to grasp the cable-tie tail mid-air, or the insertion misses the loop.

VI. ABLATIONS · A. Action Chunking and Temporal Ensembling

The ablations examine action chunking, temporal ensembling, and conditional-VAE modeling as components addressing compounding errors and noisy, non-Markovian demonstrations. Across four simulated settings, action-chunk length and temporal ensembling are evaluated separately, with temporal ensembling improving BC-ConvMLP by 4% and ACT by 3.3%.

VI. ABLATIONS: The ablations evaluate action chunking, temporal ensembling, and conditional-VAE training to address compounding errors, non-Markovian demonstrations, and noisy human data.The study also includes a user study concerning the necessity of high-frequency control in ALOHA.
VI. ABLATIONS: The study reports results across four settings comprising two simulated tasks with scripted or human demonstrations.These settings support the component ablations and the accompanying user study on high-frequency control.
A. Action Chunking and Temporal Ensembling: Action-chunk length k is varied to test whether chunking explains ACT’s advantage over methods that predict only single-step actions.The experiments disable temporal ensembling and train a separate policy for each k.
A. Action Chunking and Temporal Ensembling: k = 1 represents no action chunking, whereas k = episode_length represents fully open-loop control from the first observation.The robot outputs the entire episode’s action sequence in the fully open-loop condition.
A. Action Chunking and Temporal Ensembling: Chunking performance is summarized by success rates averaged across four settings: two simulated tasks using either human or scripted demonstrations.Figure 8 (a) plots these results while varying k, with the blue line representing AC.
A. Action Chunking and Temporal Ensembling: Temporal ensembling improves BC-ConvMLP by 4% and ACT by 3.3% across four simulated tasks and different action-chunk lengths.The comparisons use separately tuned hyperparameters for policies with and without temporal ensembling.

B. Training with CVAE · C. Is High-Frequency Necessary? · VII. LIMITATIONS AND CONCLUSION

ACT’s CVAE objective is crucial for learning noisy, multimodal human demonstrations, while fine manipulation motivates high-frequency teleoperation. The combined ALOHA–ACT system achieves 80–90% success on selected tasks with about 10 minutes of demonstrations, but cannot perform every skill.

B. Training with CVAE: The CVAE objective models noisy, multimodal human demonstrations, whereas the comparison objective predicts action sequences with L1 loss.The ablation compares ACT trained with CVAE against ACT without the CVAE objective.
B. Training with CVAE: 35.3% versus 2% success shows that removing the CVAE objective severely harms learning from human demonstrations.On scripted data, removing CVAE makes almost no performance difference because the dataset is fully deterministic.
C. Is High-Frequency Necessary?: The high-frequency teleoperation study lowers control frequency from 50Hz to 5Hz using the same hardware setup.The study examines threading a zip cable tie and un-stacking two plastic cups, both requiring millimeter-level precision and closed-loop visual feedback.
VII. LIMITATIONS AND CONCLUSION: The system’s capabilities arise from the synergy between the ALOHA teleoperation system and the ACT imitation-learning algorithm.Together, they enable direct learning of fine manipulation skills in the real world.
VII. LIMITATIONS AND CONCLUSION: 80–90% success with around 10 minutes of demonstrations demonstrates that ALOHA and ACT can learn fine manipulation skills directly in the real world.The reported examples include opening a translucent condiment cup and slotting a battery.
VII. LIMITATIONS AND CONCLUSION: Tasks such as buttoning a dress shirt remain beyond the capability of either the robots or the learning algorithm.The paper refers readers to Appendix F for a more detailed discussion of limitations.

APPENDIX … C. Detailed Architecture Diagram

The appendix shows that ALOHA generalizes to unmodified real-world objects, reproduces 14 of 15 tasks from a system costing over 10× more, and uses a CVAE-based policy architecture with deterministic test-time execution.

A. Comparing ALOHA with Prior Teleoperation Setups: ALOHA demonstrates real-world generality by performing additional teleoperated tasks with unmodified objects taken directly from the real world.These examples are presented in Figure 9 and accompanied by project-website videos.
A. Comparing ALOHA with Prior Teleoperation Setups: ALOHA uses leader-follower teleoperation with joint-space mapping, exploiting the kinesthetic similarity between the two robots and following an approach used in earlier systems and by RE2.The appendix situates this design within teleoperation systems for hazardous-material handling and dexterous manipulation.
A. Comparing ALOHA with Prior Teleoperation Setups: ALOHA recreated 14 of 15 Shadow Teleoperation System tasks using similar objects and comparable time, but could not perform Baoding-ball rotation because it lacks a hand.The Shadow system costs more than 10× ALOHA; the comparison covered tasks including insertion, writing, and cable-tie manipulation.
B. Example Image Observations: ALOHA’s policy observes four camera views—top, front, left wrist, and right wrist—combining static scene views with moving gripper views during execution.The front camera is rotated 90 degrees to capture more vertical space; all cameras use fixed focal length and auto-exposure.
C. Detailed Architecture Diagram: During training, the model samples RGB images, joint positions, and target action sequences, then infers a style variable z with a CVAE encoder before predicting actions.The encoder receives a learned [CLS] token and embedded joint positions along with the visual inputs.
C. Detailed Architecture Diagram: Each camera image is processed by ResNet18, flattened, linearly projected, and augmented with 2D sinusoidal position embeddings before camera features are concatenated for the policy.The positional embeddings preserve spatial information in the visual feature sequence.
C. Detailed Architecture Diagram: At test time, the CVAE encoder is discarded; setting z to the zero vector yields deterministic actions from incoming image and joint observations.The zero vector is the mean of the unit Gaussian prior used during training, benefiting policy evaluation.

D. Experiment Details and Hyperparameters · E. User Study Details

The experiments carefully tune baseline hyperparameters and document implementation choices, while the user study evaluates ALOHA with six inexperienced or experienced participants on high-precision, visually guided tasks.

D. Experiment Details and Hyperparameters: Baseline hyperparameters were carefully tuned and reported in Tables III–VII.The study compares methods using explicitly documented experimental settings.
D. Experiment Details and Hyperparameters: For BeT, increasing history length from 10 to 100 greatly improved performance, while larger hidden dimensions generally helped.The original BeT history length was 10, whereas the tuned setting used 100.
D. Experiment Details and Hyperparameters: For VINN, k was adaptively selected using the lowest validation loss, following the original paper.VINN also incorporated joint-position differences alongside visual-feature similarity.
E. User Study Details: The user study included 6 computer-science graduate students: 4 men and 2 women aged 22–25.Three participants had prior robot teleoperation experience and three did not; none had used ALOHA before.
E. User Study Details: The 5Hz ALOHA version read leader-robot states at 5Hz, interpolated joint-space positions, and sent them to the robot at 50Hz.This implementation enabled the user-study control loop despite the lower-rate leader readings.
E. User Study Details: The selected tasks emphasized high precision and closed-loop visual feedback, with object images provided in Figure 12.The study included concrete task geometry, including a 4mm x 1.5mm cable-tie hole and a 0.8mm x 3.5mm cable tie.

F. Limitations

ALOHA is limited by hardware that cannot dexterously coordinate multiple fingers or exert high forces, and by ACT’s difficulty learning behaviors requiring ambiguous visual cues or complex mid-air manipulation. The reported failures include unwrapping candies and opening flat ziploc bags.

Hardware Limitations: ALOHA struggles with tasks requiring multiple fingers from both hands or high forces, including push-tab pill bottles, heavy objects, sealed water bottles, and tightly pressed marker caps.These limitations arise from the hardware’s coordination and force capabilities.
Policy Learning Limitations: ACT failed to learn candy unwrapping despite 50 demonstrations, because the wrapper seam can appear anywhere and is difficult for humans and the policy to identify.The policy reliably picked up the candy in preliminary trials but often attempted to pry at nonexistent seams.
Policy Learning Limitations: ACT also struggled to open a flat ziploc bag: after consistently picking it up, the policy had difficulty with the three subsequent mid-air manipulation steps.The task requires sequentially adjusting the bag and coordinating both grippers to grasp and pull the opening region.