PaperlayerView exact revision on arXiv

Source-linked AI summary

MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar

arXiv:2302.12422v2cs.RO

TL;DR

Long-horizon imitation learning is limited by the cost of collecting sufficient demonstrations. MIMICPLAY learns goal-conditioned latent plans from inexpensive human play and uses a small amount of robot teleoperation data for low-level control, outperforming prior methods across 14 real-world tasks while retaining clear scope limitations.

  • Problem

    Long-horizon imitation learning requires costly, labor-intensive demonstrations, limiting efficient robot training for complex manipulation tasks.

  • Method

    MIMICPLAY learns goal-conditioned latent plans from human play data and trains a low-level robot controller on a small amount of teleoperated demonstrations.

  • Results

    MIMICPLAY outperforms prior methods by over 50% in 14 challenging long-horizon manipulation tasks.

  • Takeaways & Limitations

    Human play data can provide scalable high-level planning information while robot demonstrations provide low-level control skills.

  • Takeaways & Limitations

    The current latent plan is learned from scene-specific human play data, and evaluated tasks are limited to tabletop settings.

Abstract

from arXiv · show

Imitation learning from human demonstrations is a promising paradigm for teaching robots manipulation skills in the real world. However, learning complex long-horizon tasks often requires an unattainable amount of demonstrations. To reduce the high data requirement, we resort to human play data - video sequences of people freely interacting with the environment using their hands. Even with different morphologies, we hypothesize that human play data contain rich and salient information about physical interactions that can readily facilitate robot policy learning. Motivated by this, we introduce a hierarchical learning framework named MimicPlay that learns latent plans from human play data to guide low-level visuomotor control trained on a small number of teleoperated demonstrations. With systematic evaluations of 14 long-horizon manipulation tasks in the real world, we show that MimicPlay outperforms state-of-the-art imitation learning methods in task success rate, generalization ability, and robustness to disturbances. Code and videos are available at https://mimic-play.github.io

1 Introduction

MIMICPLAY addresses the high cost of collecting long-horizon demonstrations by combining scalable human play data for planning with limited robot teleoperation data for control. It reports state-of-the-art performance across 14 real-world tasks, including generalization and disturbance robustness.

  • Motivation: Long-horizon imitation learning remains costly because complex tasks require labor-intensive demonstrations across broad initial and goal conditions.Existing imitation learning methods have largely focused on short-horizon primitives.
  • Related directions: Hierarchical imitation learning separates high-level planning from low-level visuomotor control, while play-data methods use unguided interactions to scale training.Play data is collected without specific task goals or guidance.
  • Proposed paradigm: Human play data is faster and easier to collect than robot teleoperation data, enabling broader coverage of situations and behaviors.MIMICPLAY uses this data for high-level plans and a small amount of demonstrations for low-level manipulation policies.
  • Method: MIMICPLAY trains a goal-conditioned latent planner from human hand trajectories and conditions a low-level robot controller on those plans.The planner provides 3D guidance about what to do and where to interact, while the controller supplies fine-grained actions.
  • Results: MIMICPLAY achieves state-of-the-art performance on 14 real-world long-horizon tasks, with reported generalization to novel tasks and robustness against disturbances.The framework also supports prompting robot motion with human videos.

2 Related Work

Related work scales imitation learning through hierarchy, play data, and human video pretraining, but prior approaches commonly face expensive robot data collection or domain-transfer challenges. MIMICPLAY instead assigns planning and control to complementary data sources.

  • Imitation learning: Deep imitation learning methods handle high-dimensional observations and closed-loop control but generally require many human demonstrations.Traditional trajectory methods are more sample-efficient but less flexible for these settings.
  • Play data: Human play data can be collected without robot hardware and may substantially reduce the need for on-robot demonstration data.This motivates using human interactions as a scalable source for learning high-level behavior.
  • Hierarchical imitation learning: Prior hierarchical methods learn planning and control from expensive teleoperated robot demonstrations, whereas MIMICPLAY uses human play for planning and robot demonstrations for control.The paper presents this division as strengthening planning while keeping demonstration demand low.
  • Human videos: Internet-scale human-video representation pretraining can face large domain gaps and difficulty transferring to specific manipulation tasks.The cited comparison concerns methods such as R3M and MVP.
  • Learning from play data: Robot play-data methods offer diverse behavior but require laborious teleoperation, reported as 4.5 or 6 hours in prior work.MIMICPLAY replaces this with human hand interactions that are faster to collect.

3 MimicPlay

MIMICPLAY learns 3D-aware, goal-conditioned latent plans from human play and uses them to guide a multi-task robot controller trained with limited teleoperation data. Its design combines multimodal trajectory modeling, visual-gap reduction, video prompting, and hierarchical control.

  • Human play data: Human play provides rich state transitions and implicit knowledge of object affordances while avoiding task labeling and environment resetting.The paper describes free one-handed interactions such as opening an oven or moving a pan.
  • Overview: Figure 2 uses human play to train a goal-conditioned trajectory generator, then freezes that planner while training a teleoperation-conditioned low-level controller.At test time, either a human or robot task video supplies a single prompt for latent-plan generation.
  • Human play data: The training dataset contains 10 minutes of human play video per task environment, approximately equivalent to 3 hours of robot teleoperation video.This comparison concerns collection duration rather than identical task content.
  • 3D-aware latent plans: The planner reconstructs future 3D human hand trajectories conditioned on goals, using latent plans to encode motion-planning information.Hand trajectories are treated as signals of the operator’s underlying intentions.
  • Latent plan learning: MIMICPLAY formalizes planning as goal-conditioned 3D trajectory generation, with visual observations encoded into low-dimensional latent plan vectors.A decoder uses the plan and hand location to predict the 3D hand trajectory.
  • Multimodal modeling: A Gaussian Mixture Model captures multimodal human motion strategies that simple trajectory regression cannot fully represent.Different strategies may achieve the same task goal, so the model represents multiple trajectory components.
  • Visual-gap handling: The planner reduces human–robot visual mismatch by minimizing KL divergence between human- and robot-domain feature distributions without requiring paired videos.The final planner loss combines the GMM objective with a weighted KL-divergence term.
  • Video prompting: MIMICPLAY uses one-shot human or robot videos as goal-specification prompts to generate robot-executable latent plans.The planner therefore provides an interface for prompting long-horizon manipulation with human motion videos.

4 Experiment Setups

The experiments evaluate multi-task imitation learning across 14 long-horizon tasks, using several goal-conditioned and play-data baselines. Ablations and quantitative comparisons are conducted in the Study Desk environment and across the evaluation suite.

  • Environments and Tasks: The benchmark contains 14 tasks across six environments, covering tool use, articulated-object handling, and deformable-object manipulation.Task horizons range from 2000 to 4000 action steps, corresponding to 100–200 seconds at 20Hz.
  • Environments and Tasks: The Study Desk evaluation tests compositional generalization on three unseen tasks with increasing difficulty.Easy combines two trained tasks, while Medium and Hard require novel motions for unseen subgoal compositions.
  • Ablations: Ablation results are reported for the Study Desk environment using 20 demonstrations.Table 2 presents the ablation evaluation results.
  • Multi-task Learning: Quantitative multi-task learning results are reported in Table 3.The table summarizes quantitative evaluation results for multi-task learning.
  • Baselines: Five baselines are evaluated: GC-BC with RNNs, GC-BC with transformers, C-BeT, LMP, and R3M-BC.The baselines span goal-conditioned behavior cloning, robot play-data methods, and pretrained visual representations.

5 Results

MIMICPLAY improves long-horizon imitation learning through human-play latent plans, hierarchical control, and strong generalization and disturbance robustness. Ablations show benefits from human-play scale, multimodal plan modeling, and visual-gap alignment.

  • Human play data: More than 23% improvement over Ours (0% human) in long-horizon task settings demonstrates the value of human play data for task success and sample efficiency.The improvement holds across all trained tasks in the reported ALL evaluations, using only 10 minutes of cheap, unlabelled human play data.
  • Hierarchical policy: More than 15% improvement over prior end-to-end methods shows that the two-stage hierarchical policy is more effective for long-horizon tasks.The comparison is reported for Ours (0% human) in the long-horizon ALL settings.
  • Multi-task learning: The smallest multi-task performance drop occurs with MIMICPLAY, although the whiteboard task falls from 0.5 to 0.2.The authors hypothesize that the whiteboard drop reflects shorter demonstrations and an imbalanced training dataset.
  • Generalization: More than 35% improvement over all baselines on unseen tasks shows stronger generalization to new subgoal compositions.The method extracts latent plans from human play data and uses them to guide low-level robot policies.
  • Human-video prompting: Human-video prompting retains competitive performance with robot-oracle-video prompting across three Study Desk tasks.The paper attributes this interface to a joint latent plan space integrating human motion and robot skills.
  • Robustness: Real-time re-planning at 17Hz enables recovery from disturbances absent from teleoperated demonstrations.In the cloth-folding example, the robot refolds a towel after a person unfolds it.

6 Conclusion and Limitations

MIMICPLAY combines cost-effective human play data with small-scale robot demonstrations to learn hierarchical imitation policies. It reports over 50% improvement over prior methods across 14 challenging long-horizon manipulation tasks, while remaining limited by scene-specific training data and tabletop settings.

  • Limitations: The high-level latent plan is currently learned from scene-specific human play data, limiting scalability.
  • Limitations: The evaluated tasks are limited to tabletop settings, leaving mobile manipulation as an extension target.
  • Conclusion: MIMICPLAY learns goal-conditioned latent plans from human play data and uses robot demonstrations to train a low-level controller.The planner predicts future 3D human hand trajectories from goal images, while the controller generates robot actions from the latent plans.
  • Conclusion: Over 50% improvement over prior arts is reported across 14 challenging long-horizon manipulation tasks.

A Implementation details

The implementation collects unlabeled human play videos and teleoperated robot demonstrations, then trains and deploys a plan-guided controller. Training uses future goal images and 3D end-effector information, while inference advances through task-video frames.

  • Data collection: Human play data are recorded for 10 minutes per scene at 60 frames per second without cutting or labeling.Each sequence contains around 36k frames, and an off-the-shelf multi-view tracker detects the 3D hand trajectory.
  • Data collection: Robot teleoperation data use RoboTurk, with 20 demonstrations per task and a 40-demonstration dataset for sample-efficiency testing.The robot arm operates at 17–20Hz and the gripper at 2Hz.
  • Video prompting: During training, the planner uses a goal image H steps after the observation, with H uniformly sampled from [200,600].This interval corresponds to 10–30 seconds in wall-clock time, and the current robot end-effector location supplies 3D positional information.
  • Video prompting: During inference, task videos provide goal images beginning at frame 200 and advancing by one frame after each step.The experiments use i=1 for this frame progression.
  • Testing: Real-time deployment maps raw images to 6-DoF end-effector and gripper commands at 17Hz using Operational Space Control.

B Experiment setups

The experiments evaluate MIMICPLAY on diverse long-horizon manipulation tasks, including compositional generalization, and compare it with prior imitation-learning methods and targeted ablations. The setup distinguishes partial subgoal completion from full-task success.

  • Environments: The benchmark contains six environments and 14 tasks spanning contact-rich, articulated-object, high-precision, and deformable-object manipulation.Examples include whiteboard cleaning, oven opening, flower insertion, lamp-button pressing, and cloth folding.
  • Tasks: Subgoal tasks measure first-subgoal completion, whereas Long horizon tasks require completing the full multi-subgoal task.
  • Tasks: Compositional generalization tasks test novel goal sequences absent from training, with Easy, Medium, and Hard settings based on unseen subgoal transitions.Easy concatenates trained tasks; Medium includes one unseen transition; Hard includes two.
  • Baselines: MIMICPLAY is compared with five prior approaches, including recurrent and transformer goal-conditioned behavior cloning, C-BeT, and LMP.
  • Ablations: Ablations vary human-play-data usage, task-specific versus shared models, GMM plan learning, and KL loss for the human–robot visual gap.Variants include 0%, 50%, and full human data, plus Ours (single), Ours (w/o GMM), and Ours (w/o KL).

C Supplementary Experiment Results

Supplementary experiments examine simulation performance, latent-plan structure, visual alignment, and generalization behavior. They show benefits from hierarchical planning, human play data, transformer policies, and KL-based feature alignment under the reported comparisons.

  • Simulation results: In LIBERO simulation, MIMICPLAY’s hierarchical policy-learning framework outperforms baselines using five seeds and 100 testing trials per method.Because human play data are unavailable in simulation, the Ours (0% human) variant and baselines use teleoperated robot play data.
  • Trajectory visualization: Human-play training generates trajectories closer to ground truth on unseen subgoal transitions than the Ours (0% human) variant.Without human play data, the model overfits training transitions and may output an incorrect box-opening plan after lamp deactivation when pen pickup is required.
  • Transformer architecture: Transformer-based GC-BC exceeds recurrent GC-BC by more than 30% in the 40-demos Subgoal setting.
  • Transformer architecture: At 20 demonstrations, transformer and recurrent GC-BC performance becomes similar, indicating greater data requirements for end-to-end vision-based transformer training.
  • Latent-plan visualization: Latent plans for the same task tend to cluster in t-SNE space, supporting task distinction in the learned plan representation.
  • Visual-gap analysis: KL divergence reduces the visual gap between human and robot feature embeddings, while the KL variant covers 23% more area than the baseline.
  • Data collection: Human play collection involves unguided single-hand interaction, whereas robot demonstrations use phone teleoperation to complete predefined subgoals.

E Details of the task designs

The evaluation defines long-horizon tasks through specified initial states and ordered subgoals, spanning kitchen, tabletop, and household-object manipulation. Tasks include sequential object placement, tool use, ingredient assembly, erasing, and cloth folding.

  • Each long-horizon task is successful only when all predefined subgoals are completed in the correct order.
  • Kitchen tasks: Kitchen tasks require ordered interactions such as opening appliances, moving containers, manipulating food, and placing objects on designated surfaces.
  • Household-object tasks: Household-object tasks vary initial object locations and lamp or box states while requiring sequences involving books, pens, lamps, and boxes.
  • Other manipulation tasks: Additional tasks require arranging flowers in a vase, erasing a marked curve, assembling a sandwich, or folding a cloth twice.
  • Other manipulation tasks: The cloth task specifies two successive folds, reducing the cloth first to 1/2 and then to 1/4 of its original size.

F Training hyperparameters

Training hyperparameters are reported separately for the latent planner, robot policy, and GC-BC baseline. The planner and policy configurations are identified by their respective model components.

  • The latent planner P and robot policy π use separate hyperparameter tables, while GC-BC is documented in a third table.
  • Hyperparameters prefixed with GMM correspond to the MLP-based GMM model, and those prefixed with GPT correspond to the transformer architecture.

G Network Architecture

The described policy architecture represents sequences of multimodal embeddings and uses a transformer to autoregressively predict trajectory embeddings. A separate dataset-visualization figure is referenced in this section.

  • The GC-BC baseline is represented by a separate hyperparameter table rather than an architecture description in the supplied passages.
  • Transformer-based policy network: The transformer policy represents T time steps as a sequence of w, e, and p embeddings before processing them with self-attention and feed-forward layers.
  • Transformer-based policy network: Given T−1 embedded time steps, the transformer autoregressively generates the next trajectory-prediction embedding x_T.
  • Figure 9 is identified as a visualization of the dataset.
Loading 2302.12422v2…