Source-linked AI summary
RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei
TL;DR
Robot learning research has lacked large demonstration datasets because existing collection mechanisms are difficult to scale. ROBOTURK addresses this gap with accessible, cloud-based smartphone teleoperation and crowdsourcing. Its pilot data supported sparse-reward policy learning, while larger demonstration quantities generally improved consistency and performance.
Problem
Existing robot-demonstration mechanisms make it difficult to collect data that is simultaneously natural, plentiful, and suitable for large-scale crowdsourcing.
Method
ROBOTURK uses cloud-based simulation, smartphone motion control, browser video feedback, and scalable crowdsourced teleoperation to collect trajectory-level demonstrations.
Results
Over 2200 successful demonstrations were collected in the pilot, and the data enabled sparse-reward policy learning while larger demonstration quantities generally improved performance.
Takeaways & Limitations
ROBOTURK provides a practical route to collecting thousands of robot-learning demonstrations within days and using them for manipulation policy training.
Takeaways & Limitations
The round assembly comparison is confounded by different training durations, with 48 hours allowed for assembly versus 24 hours otherwise.
Abstract
from arXiv · showhide
Imitation Learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of Reinforcement Learning such as exploration and reward specification. However, research in this area has been limited to modest-sized datasets due to the difficulty of collecting large quantities of task demonstrations through existing mechanisms. This work introduces RoboTurk to address this challenge. RoboTurk is a crowdsourcing platform for high quality 6-DoF trajectory based teleoperation through the use of widely available mobile devices (e.g. iPhone). We evaluate RoboTurk on three manipulation tasks of varying timescales (15-120s) and observe that our user interface is statistically similar to special purpose hardware such as virtual reality controllers in terms of task completion times. Furthermore, we observe that poor network conditions, such as low bandwidth and high delay links, do not substantially affect the remote users' ability to perform task demonstrations successfully on RoboTurk. Lastly, we demonstrate the efficacy of RoboTurk through the collection of a pilot dataset; using RoboTurk, we collected 137.5 hours of manipulation data from remote workers, amounting to over 2200 successful task demonstrations in 22 hours of total system usage. We show that the data obtained through RoboTurk enables policy learning on multi-step manipulation tasks with sparse rewards and that using larger quantities of demonstrations during policy learning provides benefits in terms of both learning consistency and final performance. For additional results, videos, and to download our pilot dataset, visit $\href{http://roboturk.stanford.edu/}{\texttt{roboturk.stanford.edu}}$
1 Introduction
ROBOTURK addresses the difficulty of collecting natural and plentiful robot-manipulation demonstrations by enabling crowdsourced, trajectory-level teleoperation. The platform combines accessible interfaces with policy learning from large demonstration sets.
- Large-scale robotic skill learning needs demonstrations that are both natural and plentiful, but existing game and free-space interfaces each lack one of these properties.Game interfaces can introduce trajectory artifacts, while free-space interfaces such as VR are difficult to scale across remote users.
- ROBOTURK is a crowdsourcing platform for collecting large sets of robot-manipulation task demonstrations through trajectory-level supervision.It is designed to address the challenge of scaling demonstration collection for robot learning tasks.
- ROBOTURK combines remote smartphone-based teleoperation with a cloud-supported system intended to make demonstration collection accessible to many users.Users log on through a website and use a smartphone as a motion controller, while the platform supports simultaneous use by many concurrent users.
- Over 2200 task demonstrations totaling 137 hours of data were collected in 20 hours of system usage with contracted workers.This initial dataset demonstrates the platform’s capacity for rapid, large-scale data collection.
- The collected demonstrations support policy learning on challenging manipulation tasks with sparse rewards, while larger demonstration sets improve learning consistency and final performance.The paper uses demonstration-augmented reinforcement learning to connect collected supervision with policy training.
2 ROBOTURK: Design and Development of the Proposed Platform
ROBOTURK is a cloud-based platform for scalable robot-learning data collection that provides real-time simulated control, broad user access, concurrent sessions, and modular extensibility. Its architecture separates user interfaces, communication, simulation, and teleoperation services.
- ROBOTURK enables collection of thousands of robot-task demonstrations within days through a cloud-based large-scale data collection platform.The platform targets robot learning tasks and is designed for on-demand collection.
- Users control simulated robots in real time while receiving video and haptic feedback.The system provides real-time interaction with simulated domains rather than requiring users to run robot simulation locally.
- The platform lowers access barriers by supporting typical crowdsourcing workers with limited hardware and simple client-side interactions.Users need an ARKit-compatible iPhone and a separate device with a web browser, while simulation runs on cloud machines.
- ROBOTURK supports many simultaneous users through coordinated teleoperation sessions and dedicated server instances.A coordination server creates user sessions and establishes low-latency WebRTC channels between browsers, phones, and teleoperation servers.
- Its modular design decouples task and controller development from the infrastructure, enabling extension to new tasks, simulators, and robots.Each teleoperation server maps phone poses to joint velocities for a simulated robot arm.
- The evaluation uses simulated Block Lifting, Bin Picking, and Nut-and-peg Assembly tasks with a 7-DoF Sawyer robot arm.Lifting is diagnostic, whereas picking and assembly provide more challenging manipulation settings.
3 System Analysis
ROBOTURK evaluates smartphone-based teleoperation across interfaces, simulated manipulation tasks, and adverse network conditions. Its phone interface performs similarly to VR, while poor simulated and real-world networks still permit successful demonstrations, though long-distance operation is slower.
- 3.1 Tasks: ROBOTURK evaluates three simulated manipulation tasks—lifting, picking, and nut-and-peg assembly—and compares interfaces and network conditions.The study uses eight students, each providing demonstrations across eight test conditions.
- 3.2 User Interface Evaluation: The study compares Keyboard, 3D Mouse, VR Controller, and Phone interfaces, spanning axis-aligned, 6-DoF, and free-space control.The phone uses ARKit pose tracking and is designed to combine VR-like control with smartphone accessibility.
- 3.2 User Interface Evaluation: 151.45 seconds and 112.57 seconds are the picking-task mean completion times for Keyboard and 3D Mouse, versus 89.97 seconds for Phone and 79.36 seconds for VR.Keyboard and 3D Mouse are statistically slower than Phone and VR at the 5% significance level; Phone and VR completion-time distributions are not distinguishable.
- 3.3 Robustness to Network Capacity and Delays: 500Kbps and 120ms adverse network conditions produced roughly the same completion-time distributions as the 2.4Mbps, 20ms baseline.The evaluation tested low capacity, high delay, and combined low-capacity/high-delay links.
- 3.4 System Stress Test: Remote Teleoperation from California to China: 24 and 28 seconds are the mean completion-time differences for assembly and picking between China and Oregon servers tested from California.Users completed tasks despite the cross-Pacific delay, but more slowly with the China deployment.
4 Policy Learning from Teleoperated Demonstrations
ROBOTURK demonstrations were collected and used to guide reinforcement learning on sparse-reward manipulation tasks, with larger demonstration sets generally improving performance and consistency.
- Data Collection and Task Setup: The pilot dataset contained over 1000 demonstrations for picking and assembly, collected remotely in 20 hours of platform usage.
- Policy Learning Setup: Policies were trained on simplified can-picking and round-assembly tasks using reinforcement learning from sparse rewards.Training initialized episodes from states sampled from demonstration trajectories and used distributed PPO.
- Results: 1000 demonstrations produced the best mean task performance on both tasks among the evaluated demonstration counts.The comparison included None, 1, 10, 100, and 1000 demonstrations, with policies trained from 10 random seeds per condition.
- Results: Using more demonstrations generally improved average task performance, with the best results obtained from 1000 demonstrations on each task.The maximum possible return for each task was 1000.
- Results: The 1000-demonstration condition showed high average performance, while results had high variance because some policies failed or solved tasks inconsistently.Policies could receive zero return when they failed within the allotted time.
- Results: The performance gap from 1000 demonstrations was larger for can picking than round assembly, partly because round assembly received 48 rather than 24 hours of training.The authors suggest experience diversity may matter more for diverse or compositionally complex tasks.
5 Conclusion
ROBOTURK supports large-scale, on-demand robot-learning data collection and demonstrated remote policy learning from sparse-reward manipulation data.
- ROBOTURK supports large-scale, on-demand data collection for robot-learning tasks, including real-time trans-Pacific communication and control.The authors report no significant difference in performance and task completion times under that communication setting.
- The pilot dataset contained over 2200 successful demonstrations collected in 22 hours on bin-picking and nut-and-peg assembly tasks.These tasks were simplified variants of the original tasks.
- The collected data enabled reinforcement-learning policies to solve simplified manipulation tasks with sparse rewards.
- Performance generally improved with the quantity of demonstrations used during policy learning.The final model controlled the start-state distribution using demonstration trajectories.
A.1 Simulation Environment Details
The evaluation uses three simulated Sawyer-arm manipulation tasks designed to test dexterity and higher-level object understanding across lifting, picking, and assembly.
- Environment Design: Each simulated environment contains a 7-DoF Sawyer robot arm interacting with objects in the workspace.The environments were developed in the MuJoCo physics engine.
- Task Details: The lifting task requires grabbing a cube and lifting it, while picking requires placing each object into its corresponding bin.
- Task Details: The assembly task requires placing round and square nuts onto their corresponding pegs.
- Task Details: Picking and assembly are complex tasks involving grasping, pick-and-place, nut assembly, object-category perception, and geometry understanding.
- Task Motivation: The tasks resemble industrial part-assembly subtasks, with picking analogous to kitting and assembly analogous to placing nuts on designated pegs.
A.2 UI Experiments: Lifting Task
The lifting-task UI study found that keyboard control was slower, while the other three interfaces had roughly equal completion-time performance.
- Results: Users took significantly longer to complete the lifting task with the keyboard interface than with the other three interfaces.
- Results: The other three interfaces performed roughly equally on lifting-task completion time.Figure 8 compares their completion-time distributions.
A.3 UI Experiments: Details
The UI experiments established general performance trends and reported their concrete results in two tables.
- The UI experiments identified general trends in interface performance.
- Table 2 details the concrete results for the lifting-task experiments.
- Table 3 details the concrete results for the picking-task experiments.
A.4 Network Experiments
The network experiments varied bandwidth and delay, including a combined worst-case condition. Despite these degradations, completion-time performance did not differ significantly across conditions.
- Four network conditions combined 2.4Mbps or 500kbps bandwidth with 20ms or 120ms delay.The baseline was 2.4Mbps and 20ms; the both condition combined 500kbps bandwidth with 120ms delay.
- At a 5% significance level, worsened network conditions produced no statistically significant performance difference.Table 6 compares completion-time distributions using the Kolmogorov-Smirnov statistic.
- WebRTC’s adaptive video compression largely maintained quality and minimized delays under poorer network conditions.
A.5 Analysis of object difficulty on the Bin Picking task
The bin-picking analysis compared object and interface difficulty while describing the teleoperation control pipeline and the dataset and learning settings used for evaluation.
- Object difficulty: Milk tended to be the easiest object to grasp across all interfaces, while the other objects were comparably difficult.
- Teleoperation pipeline: The teleoperation server maps phone poses to desired end-effector poses, computes joint targets with inverse kinematics, and controls joint velocities.The controller uses q∗ as the desired joint-position set point and computes velocities from the position error.
- Pilot dataset: 2218 successful demonstrations were included in the dataset, comprising 1171 picking and 1147 assembly demonstrations.These demonstrations were collected during 22 hours of total system usage and corresponded to 137.5 hours of trajectories.
- Policy learning: The policy-learning experiments used 100-step horizons with a sparse reward of 1 only upon task completion.
- Policy learning: Demonstration-based resets occurred with 90% probability, while normal environment resets occurred with 10% probability.