Source-linked AI summary
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
Zipeng Fu, Tony Z. Zhao, Chelsea Finn
TL;DR
Mobile manipulation requires bimanual whole-body coordination beyond the tabletop focus of much imitation-learning work. The paper introduces Mobile ALOHA for collecting such demonstrations and co-trains with static ALOHA data. Across complex tasks, co-training improves success and data efficiency with limited mobile demonstrations.
Problem
Most imitation-learning results focus on tabletop manipulation, leaving mobility and dexterity insufficiently addressed for bimanual whole-body tasks.
Method
Mobile ALOHA combines ALOHA arms with a mobile base for whole-body teleoperation, while training aligns mobile data with existing static ALOHA data.
Results
Over 80% success across complex mobile-manipulation tasks is achieved with 50 human demonstrations per task and co-training.
Takeaways & Limitations
Co-training enables data-efficient imitation learning for complex bimanual mobile-manipulation tasks using existing static bimanual demonstrations.
Takeaways & Limitations
The learning results are limited to single-task imitation, and demonstrations come from two expert operators rather than highly suboptimal heterogeneous datasets.
Abstract
from arXiv · showhide
Imitation learning from human demonstrations has shown impressive performance in robotics. However, most results focus on table-top manipulation, lacking the mobility and dexterity necessary for generally useful tasks. In this work, we develop a system for imitating mobile manipulation tasks that are bimanual and require whole-body control. We first present Mobile ALOHA, a low-cost and whole-body teleoperation system for data collection. It augments the ALOHA system with a mobile base, and a whole-body teleoperation interface. Using data collected with Mobile ALOHA, we then perform supervised behavior cloning and find that co-training with existing static ALOHA datasets boosts performance on mobile manipulation tasks. With 50 demonstrations for each task, co-training can increase success rates by up to 90%, allowing Mobile ALOHA to autonomously complete complex mobile manipulation tasks such as sauteing and serving a piece of shrimp, opening a two-door wall cabinet to store heavy cooking pots, calling and entering an elevator, and lightly rinsing a used pan using a kitchen faucet. Project website: https://mobile-aloha.github.io
1. Introduction
The paper targets bimanual mobile manipulation, where useful tasks require coordinated mobility and dexterous whole-body control. It introduces Mobile ALOHA and a co-training strategy that improves data-efficient imitation learning.
- Motivation: Realistic household tasks require coordinated mobility and dexterous manipulation rather than isolated robot skills.Putting away a heavy pot, for example, requires navigating to a cabinet and manipulating it.
- Motivation: $200k+ robots and specialized teleoperation hardware make bimanual mobile manipulation data collection inaccessible to typical research labs.Existing systems may also require additional calibration or multiple control devices.
- System: Mobile ALOHA extends ALOHA with a wheeled base and records base velocity and arm puppeteering data simultaneously.The user backdrives the wheels while controlling both arms, enabling whole-body demonstrations.
- Learning Method: A 16-dimensional action vector concatenates 14-DoF arm joint positions with the mobile base’s linear and angular velocity.This formulation enables direct use of existing deep imitation-learning algorithms with minimal implementation changes.
- Results: Over 80% success with 50 demonstrations per task and a 34% average absolute improvement show that co-training enables data-efficient learning.The system achieves this across complex tasks including cabinet opening, elevator calling, chair pushing, and wine cleanup.
2. Related Work
Prior mobile-manipulation systems rely heavily on model-based engineering or structured learning strategies, while imitation learning broadens robot skill acquisition from demonstrations. This work applies co-training with static bimanual data to mobile manipulation.
- Mobile Manipulation: Model-based mobile-manipulation systems encode substantial human expertise but are difficult to develop and maintain.Minor perception-modeling errors can produce significant control failures.
- Mobile Manipulation: Learning-based mobile-manipulation methods address high-dimensional exploration using skill primitives, decomposed action spaces, or whole-body objectives.These approaches reduce reliance on manually engineered control structures.
- Imitation Learning for Robotics: Behavioral cloning maps observations to actions, with extensions using history, new objectives, regularization, motor primitives, and preprocessing.Imitation learning has been applied across navigation, manipulation, multitask, few-shot, language-conditioned, and video-based settings.
- Imitation Learning for Robotics: Co-training on diverse robot datasets has shown promise for single-arm manipulation and navigation, motivating its use with static bimanual data here.The paper addresses the limited availability of accessible bimanual mobile-manipulation datasets.
3. Mobile ALOHA Hardware
Mobile ALOHA combines a low-cost, repairable bimanual teleoperation setup with a mobile base designed for speed, stability, whole-body control, and untethered operation. Its hardware supports broad household-task interaction within a $32k budget.
- Design Goals: Mobile ALOHA extends ALOHA beyond tabletop manipulation while retaining low-cost, dexterous, and repairable bimanual teleoperation.The platform is designed for household tasks requiring mobility and arm coordination.
- Design Goals: The four design goals are mobility, stability with heavy objects, simultaneous whole-body teleoperation, and untethered operation.The system moves at around 1.42m/s and includes onboard power and compute.
- Mobile Base: The Tracer base provides up to 1.6m/s speed, a 100kg payload, and stability supported by a low-mounted balancing weight.It can traverse 10mm obstacles and 8-degree slopes with load.
- Whole-Body Teleoperation: Simultaneous control of the base and both arms expands the workspace for tasks such as opening a two-door cabinet.The teleoperation design preserves coordinated human motion, including stepping back while opening doors.
- Untethered Hardware: A 1.26kWh battery supplies untethered operation and doubles as a balancing weight, while a consumer laptop handles compute.Three RGB cameras stream 480x640 images at 50Hz.
- Capabilities: $32k buys a platform reaching 65cm–200cm vertically, extending 100cm beyond its base, lifting 1.5kg, and exerting 100N at 1.5m.The platform is demonstrated on housekeeping, cooking, and human-robot interaction tasks.
- Reproducibility: The software and hardware are open-sourced with a tutorial covering 3D printing, assembly, and installation.The tutorial is available through the project website.
4. Co-training with Static ALOHA Data
The paper reduces the need to collect task-specific mobile demonstrations from scratch by co-training with existing static ALOHA data. Static and mobile observations and actions are aligned so standard imitation-learning methods can train jointly.
- Motivation: Task-specific imitation-learning datasets require lengthy demonstration collection on each robot platform and task.Policies trained only on specialized data may not be robust to perceptual perturbations.
- Data Sources: The co-training pipeline leverages 825 static ALOHA demonstrations to improve bimanual arm actions for mobile manipulation.The static dataset spans tasks such as sealing, picking, wrapping, dispensing, and handovers.
- Action Representation: Mobile bimanual actions use 14-dimensional arm targets including gripper actions, while base actions use two linear and angular velocity targets.The two action components are represented separately before being combined for training.
- Training Data: Each observation contains two wrist-camera images, one egocentric top-camera image, and arm joint positions.The co-training objective samples equally from static and mobile datasets.
- Dataset Alignment: Static examples are zero-padded for missing base actions, front-camera inputs are omitted, and actions are normalized using Mobile ALOHA statistics.These steps align the datasets for joint training.
- Policy Methods: The same co-training recipe is combined with ACT, Diffusion Policy, and VINN imitation-learning approaches.This tests the pipeline across multiple base algorithms rather than a single policy family.
5. Tasks
The paper evaluates Mobile ALOHA on seven challenging real-world tasks spanning navigation, bimanual manipulation, precision interactions, and human-robot interaction.
- Task suite: The task suite covers realistic objects, interactions, and base movements across seven mobile manipulation tasks.Six tasks are illustrated in Figure 3, while High Five appears in Appendix A.1.
- Task suite: Wipe Wine requires navigating between a sink and table while coordinating both arms to lift a glass and wipe surfaces.The task is not possible with static ALOHA and would take longer for a single-armed mobile robot.
- Task suite: Cook Shrimp combines pouring oil and shrimp, lifting and angling a pan, flipping shrimp, and serving it into a bowl.The task introduces randomized robot and object configurations and requires coordinated bimanual actions.
- Task demands: Table 1 frames co-training as improving ACT success across seven challenging mobile manipulation tasks, especially precision bottlenecks.The highlighted sub-tasks include Press Button in Call Elevator and Turn on Faucet in Rinse Pan.
- Task suite: Use Cabinet, Call Elevator, and Push Chairs test heavy-object placement, precise elevator interaction, and forceful multi-object pushing.Call Elevator includes a 2cm×2cm button and only 30cm clearance at the elevator door; Push Chairs involves five 5kg chairs.
- Task demands: Open-loop replay achieves zero whole-task success, indicating that successful completion requires closed-loop reactions to execution errors.Base velocity control produces more than 10cm average error during a 180° turn with 1m radius.
6. Experiments
The experiments test whether co-training enables data-efficient mobile manipulation and whether Mobile ALOHA supports multiple imitation-learning methods. Results show broad gains from co-training, including improved data efficiency and compatibility across methods.
- Experimental design: The experiments ask whether small amounts of mobile data plus co-training can learn complex skills and whether ACT, Diffusion Policy, and VINN are compatible.The evaluation uses extensive real-world experiments.
- Experimental design: Action chunking predicts future action sequences, improving trajectory coherence and reducing latency, while base-action delays require asymmetric execution of arm and base actions.The robot executes the first k−d arm actions and last k−d base actions from a chunk of length k.
- Co-training results: Co-training yields at least 80% success on six tasks using 50 demonstrations each, while Cook Shrimp reaches 40% with 20 demonstrations.Co-training improves whole-task success in five of seven tasks, with boosts of 45%, 20%, 80%, 95%, and 80%.
- Co-training results: Co-training improves generalization to unseen configurations, raising Push Chairs success on the fourth and fifth chairs by 15% and 89%, respectively.The co-trained policy also performs better near the boundary of the Wipe Wine glass-randomization region.
- Robustness: The study compares co-training with pre-training, where ACT is first trained on static data and then fine-tuned with Mobile ALOHA data.The comparison is conducted on Wipe Wine.
- Method compatibility: Diffusion Policy matches ACT at 100% on co-trained Push Chairs, reaches 65% on Wipe Wine, and improves with co-training by 30% and 20%, respectively.VINN with chunking reaches 60% on Push Chairs and 15% on Wipe Wine, with jerky chunk transitions among its failure modes.
7. Ablation Studies
The ablations show that co-training improves data efficiency, remains effective across different data mixtures, and outperforms sequential pre-training on Wipe Wine.
- Co-training improves data efficiency on Wipe Wine: 35 in-domain demonstrations achieve 70% success versus 50% for no co-training with 50 demonstrations.The co-trained 35-demonstration policy outperforms the no-co-training policy trained with 50 demonstrations by 20%.
- Co-training remains robust across sampling strategies, with 95%, 95% and 90% success at 30%, 50% and 70% co-training rates.The similar performance reduces manual tuning when incorporating co-training on a new task.
- Pre-training on static ALOHA data provides no improvements over training solely on Wipe Wine data.The authors hypothesize that fine-tuning causes the network to forget its experience on static ALOHA data.
8. User Studies
The user study evaluates how quickly new operators learn unseen teleoperation tasks, while the conclusion and limitations define the system’s accessibility and current operating boundaries.
- User Study: Eight participants completed five consecutive trials of Wipe Wine and Use Cabinet after an expert demonstration.Four participants had no prior teleoperation experience, and none had used Mobile ALOHA before.
- User Study: Completion time fell from 46s to 28s for Wipe Wine and from 75s to 36s for Use Cabinet after five trials.These correspond to decreases of 39% and 52%, respectively.
- Conclusion: Mobile ALOHA combines whole-body teleoperation with co-trained imitation learning and performs tasks using 20 to 50 demonstrations.The system costs under $32k including onboard power and compute, with software and hardware open-sourced.
- Limitations: The 90cm x 135cm footprint can be too narrow for certain paths, while fixed arm height makes lower cabinets, ovens and dishwashers challenging to reach.The authors plan to add more degrees of freedom to the arms’ elevation.
- Limitations: Policy learning is limited to single-task imitation, without autonomous self-improvement or exploration, and demonstrations come from two expert operators.Learning from highly suboptimal, heterogeneous datasets remains future work.
A.1. High Five
The High Five task tests whether Mobile ALOHA can navigate around a kitchen island, respond to an approaching person, and resume movement appropriately.
- High Five: The robot circles a kitchen island and stops to high five when a human approaches from the front.After the high five, it resumes only when the human moves out of its path.
- High Five: Data collection uses different clothes, while evaluation tests unseen people and unseen attire.The task highlights Mobile ALOHA’s potential for studying human-robot interactions despite requiring limited precision.
A.2. Example Image Observations
Wipe Wine observations combine three camera viewpoints arranged sequentially over time, pairing a stationary top view with close-up wrist views.
- Camera Views: Observations come from the top camera, left wrist camera and right wrist camera, shown from left to right columns.The top camera is stationary relative to the robot, whereas wrist cameras are attached to the arms and provide close-up gripper views.
- Temporal Arrangement: Images are arranged sequentially in time from top to bottom.
A.3. Experiment Details and Hyperparameters of ACT, Diffusion Policy and VINN
The experiments document hyperparameter configurations for co-training, ACT, Diffusion Policy, and VINN, alongside open-loop replay-error analysis. Replay of a 300-step demonstration produced substantial end-effector spread from base-motion variability.
- Hyperparameters: Tables 5–9 list hyperparameters for co-training, ACT, Diffusion Policy, BYOL, and VINN with chunking.The tables separately cover co-training, ACT, Diffusion Policy, BYOL as VINN’s feature extractor, and VINN with chunking.
- Open-Loop Replay Errors: A 300-step, 6-second demonstration included a 180-degree turn with an approximately 1-meter radius before the end-effector tapped paper.The replay test evaluates the final tapping position after a mobile trajectory.
- Open-Loop Replay Errors: Replay points were biased left by roughly 10cm and spread along a line of roughly 20cm because of ground-contact and low-level-controller stochasticity.The result describes error when replaying the base velocity profile.
- Open-Loop Replay Errors: The policy corrected replay-induced errors without explicit localization such as SLAM.This correction was reported despite the observed end-effector spread during open-loop replay.