PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity

Ajay Mandlekar, Jonathan Booher, Max Spero, Albert Tung, Anchit Gupta, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei

arXiv:1911.04052v1cs.ROcs.HCcs.LG

TL;DR

Robotics has struggled to match the scale of vision and language datasets: self-supervised data can be noisy, while human demonstrations have been limited in scale and task complexity. This paper extends RoboTurk for remote demonstrations on physical robots, collecting over 111 hours across 54 users and three challenging tasks in one week.

  • Problem

    Robotics lacks a data-collection approach that combines the scale of self-supervision with the quality of human demonstrations, while prior human-supervised work has been limited in scale and task complexity.

    Source: §I · p. 1

  • Method

    The paper extends RoboTurk to collect remote human demonstrations on physical robots and introduces three manipulation tasks requiring reasoning and dexterity.

    Source: §I · pp. 1–2

  • Results

    Over 111 hours of robot manipulation data from 54 users across three tasks were collected in one week, yielding the largest dataset collected via remote teleoperation.

    Source: §I · p. 2, B. Data Collection and Dataset Details · p. 5

  • Takeaways & Limitations

    The three tasks combine reasoning and dexterous manipulation with diverse admissible solutions, and participants adapted quickly to phone control while producing diverse, successful demonstrations.

    Source: §VII · p. 7

Abstract

from arXiv · show

Large, richly annotated datasets have accelerated progress in fields such as computer vision and natural language processing, but replicating these successes in robotics has been challenging. While prior data collection methodologies such as self-supervision have resulted in large datasets, the data can have poor signal-to-noise ratio. By contrast, previous efforts to collect task demonstrations with humans provide better quality data, but they cannot reach the same data magnitude. Furthermore, neither approach places guarantees on the diversity of the data collected, in terms of solution strategies. In this work, we leverage and extend the RoboTurk platform to scale up data collection for robotic manipulation using remote teleoperation. The primary motivation for our platform is two-fold: (1) to address the shortcomings of prior work and increase the total quantity of manipulation data collected through human supervision by an order of magnitude without sacrificing the quality of the data and (2) to collect data on challenging manipulation tasks across several operators and observe a diverse set of emergent behaviors and solutions. We collected over 111 hours of robot manipulation data across 54 users and 3 challenging manipulation tasks in 1 week, resulting in the largest robot dataset collected via remote teleoperation. We evaluate the quality of our platform, the diversity of demonstrations in our dataset, and the utility of our dataset via quantitative and qualitative analysis. For additional results, supplementary videos, and to download our dataset, visit http://roboturk.stanford.edu/realrobotdataset .

I. INTRODUCTION

Robotics has struggled to match the scale of vision and language datasets: self-supervised collection can be noisy, while human demonstrations have been limited in scale and task complexity. This paper extends RoboTurk to collect human demonstrations on physical robots and reports a dataset of over 111 hours from 54 users across three challenging tasks.

I. INTRODUCTION

  • Self-supervised and off-policy methods have collected over 1000 hours for tasks such as grasping, but random controls can produce low-signal data and complex reward functions can be difficult to specify.Human demonstrations implicitly provide successful task executions, avoiding explicit reward-function specification.Source: §I · p. 1
  • Human-supervised imitation learning has succeeded on restricted task instances, but prior approaches were limited in data scale and task complexity.The paper identifies matching self-supervised scale while retaining human-in-the-loop data quality as the ideal approach.Source: §I · p. 1
  • Scaling robotics datasets requires real-time remote interaction robust to actuation and network delays, as well as operation at crowdsourcing scale.Unlike offline data labeling, an expert must demonstrate the task in real time.Source: §I · p. 1
  • RoboTurk is extended from simulation to physical robots, with a framework for robot management, actuation delays, novice-user safety, and alignment of multiple sensor streams.The work introduces three tasks requiring human reasoning and dexterity, and a diverse dataset an order of magnitude larger than the state of the art.Source: §I · pp. 1–2
  • Over 111 hours of demonstrations from 54 users across three challenging tasks were collected in one week, forming the largest robot dataset collected via remote teleoperation.The paper evaluates platform quality, demonstration diversity, and dataset utility through quantitative and qualitative analysis.Source: §I · p. 2

II. RELATED WORK

Prior robotics data collection includes large-scale physical trials, self-supervision, imitation learning, and crowdsourced teleoperation, each with practical constraints. RoboTurk uses smartphone control and cloud integration to support remote participation and diverse strategies, while the paper’s tasks are designed to require both dexterity and planning.

II. RELATED WORK

  • Large-scale physical robot trials have used hundreds of hours of interaction, although most trials were unsuccessful; self-supervision can also have low signal-to-noise ratio from random exploration.Imitation learning can avoid reward specification, but behavioral cloning typically needs substantial data and inverse reinforcement learning many environment interactions.Source: §II · p. 2
  • Virtual-reality teleoperation has been used for data collection and policy learning, but hardware and client-compute requirements have limited deployment on crowdsourcing platforms.Some VR approaches collect data locally or require powerful computers to render high-definition sensor streams.Source: §II · p. 2
  • RoboTurk combines a smartphone-based 6-DoF controller with cloud integration, supporting homogeneous service across client compute resources and demonstrations from multiple users.The paper links crowdsourcing with variation in strategies across people and situations.Source: §II · p. 2
  • The three collection tasks are designed to require fine-grained dexterity and high-level planning, and the setup streams a front-facing robot view to teleoperators.The setup uses three Sawyer arms, each with a front-facing webcam and top-down Kinect depth-camera view.Source: Figure 3 · p. 3

III. SYSTEM DESIGN

RoboTurk enables remote task demonstrations through a browser, smartphone controller, and cloud-hosted simulation. For physical robots, the system adds resource-aware scheduling and mechanisms for latency, safety, and multi-stream data collection.

A. RoboTurk Overview

  • RoboTurk lets remote users collect demonstrations through low-latency teleoperation, using a browser for streamed video and a smartphone as the robot controller.The simulation runs on a cloud server to provide homogeneous service regardless of user compute resources.Source: A. RoboTurk Overview · p. 3, §III · p. 3
  • WebRTC links the user’s phone, browser, and cloud teleoperation server, while the coordination server creates and manages user sessions.The user receives real-time workspace video and sends phone position and orientation changes; each teleoperation server is dedicated to one user.Source: A. RoboTurk Overview · p. 3

B. Extending RoboTurk for use with real robots

  • Physical-robot RoboTurk addresses limited robot availability, hardware and network delays, novice-user safety, and sensor streams emitted at different rates.A mutual-exclusion lock allows only one user to operate each robot at a time; low-pass filtering smooths commands, and nearby teleoperation servers reduce command latency.Source: B. Extending RoboTurk for use with… · pp. 3–4

IV. DATA COLLECTION

The three tasks combine low-level dexterity with high-level reasoning and admit multiple solutions, encouraging users to explore different strategies. Their specific challenges range from finding objects in clutter to stacking vessels and flattening clothing.

A. Task Design

  • Object Search requires finding all objects in a randomly selected target category among clutter and placing them in the matching box, without using an outside staging area.The task combines detecting and searching for targets with digging through clutter, pushing objects aside, grasping, and fitting items into holes.Source: A. Task Design · p. 4
  • The tasks were selected to combine low-level dexterity and high-level reasoning while allowing several admissible solutions for each instance.This multimodal solution space gives users room to try different strategies.Source: A. Task Design · p. 4
  • Tower Creation asks users to stack cups and bowls into the tallest stable tower, requiring physical reasoning about object properties and stack order.Initial configurations vary: ten objects are sampled from 28 bowls in seven varieties and 12 cups in three varieties, then randomized.Source: A. Task Design · p. 4
  • Laundry Layout asks users to straighten a hand towel, pair of jeans, or t-shirt until it lies flat without folds.Each reset places the item in a new configuration, requiring reasoning about its shape and response to contact.Source: A. Task Design · p. 4

B. Data Collection and Dataset Details

The dataset was collected remotely by 54 participants over one week, with more than 111 hours of robot manipulation data. Across Object Search and Laundry Layout, users completed tasks faster with experience while showing more dexterous motion.

A. Quantitative User Performance Analysis

  • Users completed Object Search and Laundry Layout faster with experience, despite a wide range of skill across users.The Tower Creation task was excluded from this experience analysis because most users used the full five minutes to build their tower.Source: Figure 4 · p. 5, Figure 5 · p. 5
  • Phone orientation change increased with experience, consistent with users learning more dexterous motions as task completion times fell.The figures display quartiles, which the authors use to show that the changes were consistent across the population.Source: Figure 4 · p. 5, A. Quantitative User Performance… · p. 5

B. Data Collection and Dataset Details

  • Data were collected remotely using three Sawyer arms, with RGB and depth images at 30 Hz and robot readings at approximately 100 Hz.Participants completed a supervised hour that included a five-minute tutorial, then could collect data without supervision.Source: B. Data Collection and Dataset Details · pp. 4–5

A. Quantitative User Performance Analysis

  • Average exertion, measured by squared L2 norms of phone translations, remained largely invariant as experience increased.This suggests faster task completion did not come from simply executing the same trajectory faster.Source: A. Quantitative User Performance… · p. 5

B. Qualitative User Feedback

The authors used a self-reported NASA TLX survey to assess user workload and describe the analysis as examining dataset utility for several learning applications.

B. Qualitative User Feedback

  • Users rated Tower Creation as requiring the most workload across all NASA TLX metrics.The survey covered mental, physical, and temporal demand, performance, effort, and frustration.Source: B. Qualitative User Feedback · p. 5
  • The NASA TLX form measured six self-reported workload dimensions on a 21-point scale, with higher total scores indicating higher workload.Total workload was computed as the sum of the averages across the dimensions.Source: B. Qualitative User Feedback · p. 5

VI. EXPERIMENTS: DATA ANALYSIS

  • The dataset analysis considers potential applications including multimodal density estimation, video prediction, reward learning, policy learning, and hierarchical task planning.Source: §VI · p. 6

A. Evaluating Object Search Task Complexity

The paper examines task complexity, diversity in human solution strategies, and the use of learned visual embeddings to infer rewards and task progress.

A. Evaluating Object Search Task Complexity

  • Different start and end bin configurations across successful Object Search demonstrations indicate that operators substantially reoriented the contents to find target objects.The figure compares the start and end configurations of the bin.Source: Figure 6 · p. 6, A. Evaluating Object Search Task… · p. 6

C. Inferring a Reward Signal from Demonstrations

  • A modified Time Contrastive Network was trained on Laundry Layout demonstrations using intra-chunk, inter-chunk, and terminal-frame triplet losses.The changes address the need to learn temporal similarity across multi-stage tasks and demonstrations lasting several minutes.Source: C. Inferring a Reward Signal from… · p. 6

B. Diversity of task demonstrations and solution approaches

  • For Laundry Layout, users with similar starting towel configurations used markedly different strategies to flatten the towel.One demonstration placed a towel side in a better configuration before pushing the remaining flap; another used lifting and dragging motions.Source: B. Diversity of task demonstrations… · p. 6, Figure 7 · p. 7
  • Tower Creation demonstrations using the same objects produced different tower structures through different stacking choices.The towers were comparably high, and the dataset includes many such instances of multimodality and creative problem solving.Source: B. Diversity of task demonstrations… · p. 6, Figure 7 · p. 7

C. Inferring a Reward Signal from Demonstrations

  • Negative embedding distance to a successful flat-towel frame provides a visual reward signal that reflects task progress.Frames with a nearly flat towel had smaller distances to the target than frames where the robot held the towel away from the table.Source: C. Inferring a Reward Signal from… · p. 6, Figure 8 · p. 7

D. Behavioral Cloning

Behavioral cloning on Laundry Layout was unsuccessful on the full dataset, though restricting training to a particular demonstration segment yielded some success.

D. Behavioral Cloning

  • Behavioral Cloning mapped RGB images to robot joint positions for Laundry Layout.Source: D. Behavioral Cloning · p. 7
  • Attempts to train on the entire Laundry Layout dataset were unsuccessful, which the authors attribute to the diversity of the demonstrations.Source: D. Behavioral Cloning · p. 7
  • Some success was achieved by restricting training to segments where the arm moves to a towel corner and lifts it; addressing dataset diversity for policy learning was left for future work.Source: D. Behavioral Cloning · p. 7

VII. CONCLUSION

The paper presents three crowdsourcing-suited manipulation tasks and a large teleoperated dataset collected with extensions to RoboTurk. It identifies handling solution diversity in future policy learning as an ongoing challenge.

VII. CONCLUSION

  • Object Search, Tower Creation, and Laundry Layout each require higher-level reasoning and dexterous manipulation, and admit diverse solutions.These properties make the tasks amenable to crowdsourcing.Source: §VII · p. 7
  • The dataset contains over 111 hours of demonstrations from 54 users, collected in one week on three Sawyer arms.The authors describe it as the largest known crowdsourced teleoperated robot manipulation dataset.Source: §VII · p. 7
  • Further work is needed to handle the inherent diversity and multimodality of demonstrated solutions in policy learning.The authors say initial results suggest potential for one-shot imitation learning and direct imitation, but further considerations and innovations are necessary.Source: §VII · p. 7

VIII. APPENDIX

The appendix specifies how instances are assembled for the three tasks and describes a hierarchical Time Contrastive Networks embedding-learning setup for video demonstrations.

A. Additional Task Details

  • Object Search instances randomly select a target category, with all categories present in shuffled clutter and category-specific destination holes.Operators place all three items from the selected category into its corresponding hole.Source: A. Additional Task Details · p. 9
  • Tower Creation instances use 10 randomly arranged items sampled from 28 bowls and 12 cups; Laundry Layout varies items by operator experience.New operators receive hand towels, while more experienced operators receive jeans and T-shirts.Source: A. Additional Task Details · p. 9

B. Reward Learning Experiment Details

  • The embedding model extends Time Contrastive Networks by splitting demonstrations into approximately one-second, 24-frame chunks and applying triplet loss within chunks.The method targets reward functions learned from long video demonstrations.Source: B. Reward Learning Experiment Details · p. 9
  • A second triplet loss encourages chunks close in time to remain close in embedding space, using chunks sampled within temporal radii 3 and 6.This captures relationships at a coarser temporal resolution.Source: B. Reward Learning Experiment Details · p. 9
  • The two loss terms receive equal weights, and optimization uses semi-hard negative mining, Adam, and a learning rate of 1e−3.The embedding network uses a ResNet-18 with 1000-dimensional output followed by fully connected layers of sizes 256 and 32.Source: B. Reward Learning Experiment Details · p. 9

C. Data Diversity

The section presents randomly sampled dataset frames for Object Search, Tower Creation, and Laundry Layout.

C. Data Diversity

Loading 1911.04052v1…