PaperlayerView exact revision on arXiv

Source-linked AI summary

Data Scaling Laws in Imitation Learning for Robotic Manipulation

Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, Yang Gao

arXiv:2410.18647v4cs.RO

TL;DR

The paper asks whether data scaling laws can enable single-task robot policies to generalize across unseen objects and environments, addressing limited evidence for such laws in robotics. It empirically studies imitation-learning policies while varying environments, objects, and demonstrations, using extensive real-world evaluation. The results show approximately power-law scaling with environment and object diversity, and an efficient collection strategy reaches about 90% success on new conditions for two tasks.

  • Problem

    Robotics lacks comprehensive data scaling laws, and many robotic policies lack zero-shot generalization across unseen objects and environments.

  • Method

    The paper conducts an empirical imitation-learning study that varies training environments, objects, and demonstrations, evaluating policies in unseen conditions with normalized tester scores.

  • Results

    Generalization approximately follows power laws with training environments and objects, while diversity matters more than demonstration quantity; an efficient strategy achieves around 90% success across eight new environments and objects for each of two tasks.

  • Takeaways & Limitations

    Data collection for single-task manipulation should prioritize diverse environments and objects, with sufficient demonstrations per setting rather than maximum demonstration volume.

  • Takeaways & Limitations

    The study is limited to single-task imitation learning, uses UMI demonstrations and Diffusion Policy, and does not explore task-level generalization or reinforcement learning.

Abstract

from arXiv · show

Data scaling has revolutionized fields like natural language processing and computer vision, providing models with remarkable generalization capabilities. In this paper, we investigate whether similar data scaling laws exist in robotics, particularly in robotic manipulation, and whether appropriate data scaling can yield single-task robot policies that can be deployed zero-shot for any object within the same category in any environment. To this end, we conduct a comprehensive empirical study on data scaling in imitation learning. By collecting data across numerous environments and objects, we study how a policy's generalization performance changes with the number of training environments, objects, and demonstrations. Throughout our research, we collect over 40,000 demonstrations and execute more than 15,000 real-world robot rollouts under a rigorous evaluation protocol. Our findings reveal several intriguing results: the generalization performance of the policy follows a roughly power-law relationship with the number of environments and objects. The diversity of environments and objects is far more important than the absolute number of demonstrations; once the number of demonstrations per environment or object reaches a certain threshold, additional demonstrations have minimal effect. Based on these insights, we propose an efficient data collection strategy. With four data collectors working for one afternoon, we collect sufficient data to enable the policies for two tasks to achieve approximately 90% success rates in novel environments with unseen objects.

1 INTRODUCTION

The paper asks whether data scaling can produce single-task manipulation policies that generalize zero-shot across unseen objects and environments. It studies this through imitation-learning experiments and finds that diversity, rather than demonstration volume alone, is central to generalization.

  • The study investigates whether robotics has data scaling laws that can guide the construction of large-scale manipulation datasets.
  • It targets zero-shot operation on nearly any object within a category and in any environment, emphasizing real-world rather than controlled-lab generalization.
  • The experiments separately examine environment generalization, object generalization, and demonstration-count effects for single-task policies.
  • Over 40,000 demonstrations and more than 15,000 real-world robot rollouts support the empirical study and its rigorous evaluation protocol.
  • Using four data collectors for one afternoon produced policies achieving around 90% success across eight new environments and objects for each of two tasks.
  • Generalization approximately follows power laws with training diversity, while increasing environments and objects is more effective than simply adding demonstrations.

2 RELATED WORK

Prior work established scaling laws in language, vision, video, and other domains, while robotics has increasingly accumulated larger datasets. Robotic-manipulation research also pursues generalization to unseen objects and environments, but these directions remain distinct from the paper’s data-scaling focus.

  • Scaling laws relate model performance to dataset size, model size, or training compute across language, image, video, and other domains.
  • These laws support performance prediction for larger models and more effective resource allocation.
  • Robotic manipulation is following a broader trend toward scaling up data, including through large multi-task datasets.
  • Robotics research has separately targeted generalization to new object instances and unseen environments.

3 APPROACH

The approach studies single-task behavior-cloning policies across varied real-world environments and objects, varying dataset composition and demonstrations while evaluating only unseen conditions. It uses UMI-collected demonstrations, Diffusion Policy modeling, and a normalized tester score for rigorous comparison.

  • Generalization dimensions: The study defines generalization along environment and object dimensions, covering simultaneous real-world variations rather than isolated factors.
  • Data scaling laws formulation: Data are collected across M environments and N same-category objects, with K demonstrations for each environment-object pair.
  • Data source: UMI provides a portable, intuitive, low-cost hand-held gripper for independently collecting substantial human-demonstration datasets.
  • Policy learning: Diffusion Policy models the collected data, using a CNN-based U-Net and DDIM to support real-time control.
  • Policy learning: A fine-tuned DINOv2 visual encoder is used because its scene-layout and object-boundary features support spatial reasoning for robot control.
  • Policy learning: Temporal ensemble addresses jerky motion caused by discontinuities when successive predicted action sequences switch.
  • Evaluation: Evaluation tests only unseen environments or objects and uses tester-assigned normalized scores instead of sparse success rates.

4 UNVEILING OF DATA SCALING LAWS

The study finds that single-task policy generalization improves mainly with diversity in training objects, environments, and environment-object pairs, following approximate power laws. Demonstrations help initially but provide diminishing returns after coverage is sufficiently diverse.

  • Object generalization: 32 training objects produce normalized scores above 0.9 on unseen objects, while 8 objects already exceed 0.8 for both tasks.Performance improves across all demonstration fractions, and fewer demonstrations per object are needed as object diversity increases.
  • Environment generalization: Increasing training environments improves unseen-environment generalization, even when the total number of demonstrations is held constant.The benefit of using more demonstrations per environment diminishes quickly, with the 50% and 100% curves overlapping.
  • Joint generalization: Increasing training environment-object pairs substantially improves generalization across both novel environments and objects.In this setting, additional demonstrations saturate faster, consistent with greater data diversity from changing both factors simultaneously.
  • Power-law fitting and quantitative analysis: Generalization to new objects, environments, or both scales approximately as a power law with training objects, environments, or environment-object pairs.The analysis defines the optimality gap as 1 −Normalized Score and fits power laws after logarithmic transformation.
  • Efficient data collection strategy: Collecting one unique object in as many diverse environments as possible is recommended; 32 environment-object pairs are generally sufficient for novel environments and unseen objects.Adding multiple objects within an environment helps when environments are few, but the gap becomes negligible around 16 environments.

5 VERIFICATION OF DATA COLLECTION STRATEGY

The paper verifies its efficient data collection strategy on two additional manipulation tasks and reports success rates across eight unseen environments. It also includes model-related experiments on Pour Water.

  • Evaluation: Table 1 reports average success rates and standard deviations across eight unseen environments for all tasks.Per-environment performance is provided in Table 12.
  • Verification: 32 environment–object pairs and 50 demonstrations per environment are collected for Fold Towels and Unplug Charger.Both tasks are evaluated in eight unseen environments.
  • Model experiments: Model-related experiments on Pour Water use a default fully fine-tuned DINOv2-pretrained ViT-L/14 visual encoder and a base-size 1D CNN U-Net action diffusion model.

6 MODEL SIZE AND TRAINING STRATEGY: BEYOND DATA SCALING

The paper extends its analysis beyond data scaling by studying Diffusion Policy model components, including visual-encoder training and parameter scaling for the visual and action models.

  • Model scaling: Model-side experiments examine visual-encoder training strategy and parameter scaling for the visual encoder and action diffusion model.The experiments are conducted on Pour Water using data from 32 environment-object pairs and a selected fraction of valid demonstrations.

7 DISCUSSION, LIMITATIONS, & FUTURE WORKS

The paper argues that diverse environments and objects support zero-shot single-task generalization with modest data collection, while identifying several boundaries for future work. These include task-level generalization, reinforcement learning, data quality and learning algorithms, and validation beyond four tasks.

  • Discussion: Environment and object diversity are presented as more valuable than blindly increasing demonstration quantity for single-task generalization.The paper frames in-the-wild generalization as its ultimate data-scaling goal.
  • Limitations: Task-level generalization remains outside the study because it would require data from thousands of tasks.Future work could use language-conditioned policies for new task instructions.
  • Limitations: The study covers imitation learning with UMI-collected demonstrations modeled by Diffusion Policy, leaving reinforcement learning, data quality, and alternative algorithms for future investigation.
  • Limitations: Resource constraints limit exploration and validation to four tasks, motivating evaluation on larger and more complex task sets.

APPENDICES

The appendices document sampled training and testing environments, task-specific training and testing objects, the meaning of manipulation-object counts, and approaches for collecting human demonstrations.

  • Environment visualizations: Figures 8–11 show sampled training environments for Pour Water, Mouse Arrangement, Fold Towels, and Unplug Charger.Each task uses 12 sampled training environments.
  • Environment visualizations: Figure 12 presents eight unseen testing environments used across all tasks.
  • Object visualizations: Figures 13–16 show task-specific training and testing objects for Pour Water, Mouse Arrangement, Fold Towels, and Unplug Charger.The object sets include unseen testing instances for each task.
  • Object visualizations: A single manipulation object may denote all objects involved in completing a task, such as a bottle and mug for Pour Water.The convention likewise groups mouse and pad, charger and power strip, while towels remain singular.
  • Related work: The related-work appendix distinguishes teleoperation, learning from human video, and hand-held grippers as human-demonstration collection approaches.Hand-held grippers avoid the embodiment gap and support portable, intuitive data collection.

B.2 EXPERIENCE OF USING UMI TO COLLECT DATA

The authors report practical lessons for collecting UMI demonstrations, emphasizing randomized starting conditions, visually rich environments, and appropriately sized objects. They also recommend standardizing collector behavior and avoiding unintended motion or insufficient gripper force.

  • Data collection practices: Randomize the hand-held gripper’s initial height and orientation, and vary object positions within the robot’s reachable range.Without this variation, policies become sensitive to particular initial poses.
  • Data collection practices: Choose environments with sufficient visual features because dark areas or blank walls can cause UMI’s SLAM tracking to fail.Distractors and surface textures can increase visual features and provide data augmentation.
  • Data collection practices: Use appropriately sized manipulation objects because camera-obstructing objects can cause SLAM to misinterpret the camera as stationary.This limitation led the authors to avoid tasks such as opening drawers.
  • Dataset consistency: Standardize behavior patterns and task completion times across collectors to reduce multimodal behavior in the dataset.
  • Dataset consistency: Avoid moving distractor objects and prevent other moving entities from entering the camera’s field of view during collection.
  • Dataset consistency: Apply slight force when closing the gripper to introduce minor deformation.

C POLICY TRAINING

Policy training uses nested datasets to compare scaling fairly, adjusts training epochs so policies with different dataset sizes can converge, and applies a minor observation-history modification. Evaluation covers four manipulation tasks with task-specific scoring and success thresholds.

  • Training setup: Larger training datasets always contain smaller ones, keeping data distributions consistent across dataset sizes.For environment generalization, n environments include all m environments when m < n.
  • Training setup: Training epochs are adjusted to provide larger datasets with more optimization steps.The smallest dataset receives 800 epochs and 5.3 × 10^4 steps, while the largest receives 75 epochs and 5 × 10^5 steps.
  • Policy architecture: The implementation follows Diffusion Policy and UMI, increasing the observation horizon for selected tasks.For Pour Water, adding a history step 0.25 seconds earlier improves performance without much additional cost.
  • Pour Water: Pour Water requires grasping a bottle, pouring into a mug, and placing it on a red coaster, with success requiring scores of at least 2, 3, and 2.
  • Mouse Arrangement: Mouse Arrangement requires picking up a mouse and placing it on a mouse pad, with success requiring a score of 3 followed by at least 2.Varying mouse geometry and color requires adaptation of the grasping strategy.
  • Towel Folding: Towel Folding requires grasping all towel layers and folding rightward, with success requiring a first-step score of 3 and a second-step score of at least 2.
  • Unplug Charger: Unplug Charger requires securely grasping and pulling out a charger without collision, then placing it to the right of the power strip.

E.1 COMPARISON OF EVALUATION METRICS

The study compares tester-assigned scores with validation-set action MSE as evaluation metrics. MSE can track real-world performance in some settings, but its inconsistent relationship with closed-loop behavior makes human scores primary.

  • Metric comparison: Tester-assigned scores are the primary metric, while validation-set MSE is evaluated as an objective alternative using 30 human demonstrations per environment or object.MSE averages squared differences between policy-predicted and human actions at each timestep.
  • Metric comparison: Pearson’s r = −0.98 and Spearman’s ρ = −1.00 show a strong inverse MSE–normalized-score relationship when generalizing across environments and objects on Pour Water.
  • Metric comparison: Object generalization on Pour Water shows a weaker relationship: increasing training objects to 16 raises MSE, with Pearson’s r = −0.73.
  • Metric comparison: LoRA has lower MSE than full fine-tuning, 0.0049 vs. 0.006, but lower real-world normalized scores, 0.72 vs. 0.9.
  • Evaluation workflow: The evaluation workflow averages normalized scores over 40 trials across eight unseen environments, using five trials per environment.
  • Evaluation workflow: Policies can be compared within the same evaluation batch, but comparisons across different batches are invalid because environments and initial object positions vary.

G.1 DATA SCALING LAWS ON MSE

The appendix tests whether the paper’s data-scaling relationships also appear in action MSE and examines generalization while approximately holding demonstration counts constant. MSE generally follows weaker power-law trends, while diversity continues to improve performance and gains saturate around 1,600 demonstrations.

  • MSE scaling: Power-law relationships generally hold for MSE, but weaker scaling trends appear than in the tester-assigned-score results.Most absolute correlation coefficients are relatively large, yet all are smaller than in the corresponding main-text analysis.
  • MSE scaling: Log-transformed MSE data are fit with linear models, and Figure 20 reports power-law fits on logarithmic axes.
  • Metric limitation: MSE cannot fully capture closed-loop visuomotor performance, so tester-assigned scores remain the primary evaluation metric.The authors use MSE mainly as a debugging tool when human evaluation is impractical.
  • Controlled demonstration counts: The approximately constant-demonstration plots connect original points with similar demonstration counts rather than rerunning experiments.The points are not exactly equal in demonstration count; the 1× line is excluded because results become unstable at larger environment or object counts.
  • Controlled demonstration counts: When total demonstrations are approximately controlled, increasing training environments or objects still improves generalization performance.Figure 23 shows this effect most clearly for environment-object pairs.
  • Controlled demonstration counts: Around 16×, or approximately 1,600 demonstrations, appears sufficient for performance saturation, after which further increases yield minimal improvement.
Loading 2410.18647v4…