PaperlayerView exact revision on arXiv

Source-linked AI summary

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, Chelsea Finn

arXiv:2202.02005v1cs.ROcs.LG

TL;DR

Vision-based robots struggle to generalize to novel manipulation tasks. BC-Z studies scalable interactive imitation learning with language or human-video task specifications, achieving generalization to unseen tasks without robot data for those tasks.

  • Problem

    Zero-shot generalization to new vision-based manipulation tasks remains challenging across diverse skills and objects.

  • Method

    BC-Z combines demonstrations and human interventions with a policy conditioned on language or human-video task embeddings.

  • Results

    The system performs 24 unseen manipulation tasks without robot demonstrations for those tasks.

  • Takeaways & Limitations

    Scaling simple imitation learning to 100 training tasks can facilitate generalization to entirely new tasks without additional robot data for those tasks.

  • Takeaways & Limitations

    Performance on novel tasks varies significantly.

Abstract

from arXiv · show

In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning. We approach the challenge from an imitation learning perspective, aiming to study how scaling and broadening the data collected can facilitate such generalization. To that end, we develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions and can be conditioned on different forms of information that convey the task, including pre-trained embeddings of natural language or videos of humans performing the task. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%, without any robot demonstrations for those tasks.

1 Introduction

BC-Z addresses zero-shot generalization in vision-based manipulation, where robots must perform diverse unseen tasks from arbitrary commands. It studies whether interactive imitation learning and broader data collection can support this capability.

  • Zero-shot generalization to new vision-based manipulation tasks remains difficult, especially across diverse skills and objects.
  • BC-Z combines shared-autonomy teleoperation with task conditioning from language instructions or human videos.Shared autonomy collects demonstrations and corrective interventions, while continuous task specifications can support novel-task commands.
  • 25,877 robot demonstrations and 18,726 human videos cover 100 manipulation tasks collected across 12 robots and 7 operators.The robot data totaled 125 hours of robot time.
  • 24 unseen manipulation tasks were evaluated without robot demonstrations for those tasks.

2 Related Work

Prior imitation-learning research has generalized across objects, configurations, scenes, and goals, but BC-Z focuses on the harder problem of generalizing to new manipulation tasks across a broad task suite.

  • Prior work has achieved one-shot or zero-shot generalization to novel objects, configurations, scenes, and goals.
  • Existing systems condition on robot demonstrations, human videos, language instructions, or goal images to adapt to new scenarios.
  • DAgger reduces distribution shift but is difficult for robotic manipulation because experts must annotate actions while not controlling the robot.
  • BC-Z targets novel-task generalization across 100 challenging tasks involving 7-DoF control at 10 Hz.

3 Problem Setup and Method Overview

BC-Z learns a conditional visuomotor policy that maps images and task commands to robot actions. Its encoder-control decomposition supports language or video task embeddings and auxiliary supervision.

  • The policy maps RGB images and task commands to 7-DoF actions comprising end-effector pose and parallel-jaw gripper control.
  • Training data combines direct teleoperation demonstrations with human-in-the-loop corrections through shared autonomy.Interventions provide iterative policy improvement and a continuous performance signal.
  • An encoder q(z|w) converts the language or video command into an embedding, and a control layer π maps the image and embedding to an action.

4 Data Collection and Workflow

The workflow is designed to collect diverse demonstrations, corrective feedback, and scalable evaluations for zero-shot task generalization. It uses VR teleoperation, shared autonomy, and randomized household-object scenes.

  • The system requires diverse data collection, corrective feedback, and evaluation across many tasks to support zero-demonstration generalization.
  • VR teleoperation uses handheld controllers and a 10 Hz closed-loop control interface to demonstrate tasks.
  • Robots interact with 6 to 15 household objects in randomized poses across 100 pre-specified tasks spanning 9 underlying skills.
  • Shared-autonomy collection begins with expert demonstrations, then lets the learned policy act while a human supervises and takes over when needed.
  • Evaluation and additional data collection compete for limited robot time as the desired degree of generalization increases.

5 Learning Algorithm

The learning algorithm combines task-conditioned policy training with language or human-video encoders, using auxiliary objectives and action representations designed to improve generalization and control behavior.

  • Task-conditioned data: The dataset pairs each task’s expert demonstrations with a language command and, where available, a human video command.
  • Language and video encoders: Language commands are encoded into 512-dimensional task vectors using a pretrained multilingual sentence encoder.
  • Language and video encoders: Human-task videos are encoded with a ResNet-18 model trained end-to-end alongside the control policy using paired videos and robot demonstrations.
  • Language and video encoders: An auxiliary cosine language-regression loss aligns video embeddings with language embeddings because end-to-end video encoding otherwise overfits scenes and generalizes poorly.
  • Policy architecture: The policy processes camera images with a ResNet18 torso and two-layer action heads, conditioning delta XYZ, delta axis-angle, and gripper predictions on task embeddings through FiLM layers.
  • Policy training: Actions are defined as state differences to target poses N > 1 steps ahead, with adaptive N selected from arm and gripper motion to reduce tiny actions and dithering.

6 Experimental Results

The experiments test BC-Z on single-task imitation, zero-shot and few-shot generalization to unseen tasks, encoder-versus-policy bottlenecks, and key design choices. BC-Z generalizes across diverse held-out tasks, while multi-task training, adaptive state differences, and HG-DAgger materially support performance.

  • 6.1 BC-Z on Single-Task Imitation Learning: BC-Z reaches a 3.4 picks-per-minute pick-rate and 87% training-scene versus 94% held-out-scene success in single-task evaluation.These results validate performance on both training and held-out scenes.
  • 6.2 Evaluating Zero-Shot and Few-Shot Task Generalization: 29 held-out tasks test language-conditioned zero-shot and video-conditioned few-shot generalization across within-family and mixed object sets.Four tasks use objects from the 79-task family, while 25 mix objects between the 21-task and 79-task families.
  • 6.2 Evaluating Zero-Shot and Few-Shot Task Generalization: 32% average success and non-zero success on 24 held-out tasks demonstrate language-conditioned zero-shot generalization, including an average 44% success among those non-zero tasks.Video conditioning generalizes to nine novel tasks with non-zero success, particularly without novel object combinations.
  • 6.2 Evaluating Zero-Shot and Few-Shot Task Generalization: “Last-centimeter” errors, including gripper failures and near misses, are the most common reported source of generalization failures.These errors involve failing to close or release the gripper, or narrowly missing the target during release.
  • 6.3 Ablation Studies and Comparisons: Training-task performance with one-hot and language conditioning is similar, suggesting the latent language space is sufficient and control-layer generalization is the larger bottleneck.The comparison separates limitations in the task encoder from limitations in the control policy.
  • 6.3 Ablation Studies and Comparisons: 5% success for the single-task baseline shows that pooling data across tasks is important, while adaptive state differences and HG-DAgger improve performance.Using N = 1 future states causes noise fitting, slower movement, and state drift; HG-DAgger outperforms expert-demonstration cloning with the same total data.

7 Discussion

The study finds that large-scale interactive imitation learning can generalize to entirely new manipulation tasks, while performance varies significantly across novel tasks. It also identifies HG-DAgger and frozen, pre-trained language embeddings as important components for good performance.

  • 100 training tasks were sufficient to enable generalization to 29 unseen vision-based manipulation tasks without additional robot data for those tasks.The evaluation covered unseen tasks involving varied objects and scenes.
  • HG-DAgger was important for achieving good performance in the experiments.
  • Frozen, pre-trained language embeddings were effective task conditioners without additional training.
  • Novel-task performance varies significantly, although less successful tasks often show behavior consistent with partial task understanding.Examples include reaching for the correct object or producing a semantically related motion.
  • The system uses VR teleoperation with direct demonstrations and human interventions that correct the learned policy during operation.The operator can switch between autonomous and manual control using the teleoperation interface.

B.1 Inter-task Variability

The paper studies how variability across objects, scenes, robots, and demonstrations affects imitation learning. It combines adaptive action featurization, task conditioning, video processing, and distributed data collection to address this variability.

  • Inter-task Variability: Object, background, robot, and instance variation increase the sample complexity required to reach a target performance level.The data collection protocol spans multiple robots and locations and uses physically varied object instances.
  • Inter-task Variability: 37 expert demonstrations achieved 97% success in simulation when scene variation was absent.This result illustrates the effect of removing scene variability within a sub-task.
  • Featurization Details: Actions are represented as state differences, with an adaptive temporal offset that enlarges motion steps while slowing near gripper changes or contact.The offset increases until gripper change or arm-joint movement exceeds specified thresholds.
  • Featurization Details: The deterministic policy predicts XYZ displacement, axis-angle displacement, and gripper angle using Huber losses for motion and log loss for gripper angle.
  • Video Conditioning: Video-conditioned performance improves when multiple video embeddings are averaged before conditioning the policy.At evaluation, embeddings from 10 new human videos are averaged for a novel task.
  • Video Conditioning: Human videos are encoded with augmentation and processed as 20-frame inputs arranged into a 4x5 image array for a 2D ResNet-18.This arrangement enables temporal processing through 2D convolutions without tuning a new architecture.

E.2 Ablation on Video Encoder Batch

The video-encoder ablation compares task-based batch sampling with alternative compositions while holding the architecture, data, hyperparameters, and training time fixed. Task-based sampling gives the best performance and helps organize embeddings by semantic task meaning.

  • Batch Construction: Sampling one human video and one robot video per sampled task balances tasks and modalities within each batch.Every task appears equally often, and each batch is 50% human videos and 50% robot videos.
  • Ablation Results: Task-based sampling gives the best video-encoder performance among the tested batch compositions.The ablation controls architecture, dataset, hyperparameters, and training time while changing only batch sampling.
  • Training Objective: The video encoder is trained from paired human and robot examples for the same task, with both video types matched to a language embedding.The encoder uses a cosine loss together with the end-to-end behavior-cloning loss.
  • Embedding Visualization: Adding a language loss helps prevent embeddings from grouping tasks by object family rather than semantic meaning.The similarity matrix compares mean embeddings across tasks and groups the first 21 tasks separately from the remaining tasks.

H Single-Task Validation on Bin-Emptying

The bin-emptying validation shows that a deterministic policy can solve a task whose demonstrations contain multiple valid object-order behaviors. Training data and noise may help the policy resolve this multimodality.

  • Single-Task Validation: 3–4 objects per minute were cleared by the policy trained on 30 hours of expert demonstrations.A human teleoperator requires about 43 seconds to demonstrate the task.
  • Single-Task Validation: The bin-emptying task is inherently multimodal because demonstrations grasp objects in arbitrary order.
  • Single-Task Validation: The deterministic unimodal policy nevertheless solves the bin-emptying task.The authors hypothesize that noise and dataset variety may break symmetry, such as by favoring the nearest object.
  • Training Procedure: The BC-Z training procedure computes task vectors from language or video commands and uses them to condition the policy.The procedure samples task-specific videos and applies the resulting task vector during policy training.
  • Embedding Analysis: The embedding visualization groups tasks by similarity, and language supervision helps organize them by semantic meaning rather than object family.

I Single-Task Validation on Door Opening

The door-opening validation tests BC-Z under randomized real and simulated conditions, using diverse demonstrations and sim-to-real adaptation. Evaluation emphasizes performance in unseen meeting rooms and safe task completion.

  • Data and setup: 12,000 real and 36,000 simulated demonstrations train door-opening policies across varied meeting-room layouts.The real demonstrations span 24 rooms, while simulation covers 5 rooms with both right- and left-swing doors.
  • Data and setup: Randomized arm configurations and robot poses require the policy to infer the arm’s position relative to each door.The policy predicts base forward and yaw velocity plus a binary termination action.
  • Sim-to-real transfer: RetinaGAN adapts simulated images toward real appearance, enabling simulated door-opening demonstrations to support sim-to-real transfer.The model is trained separately on simulated and real images using unsupervised object-detection consistency.
  • Evaluation: Policies are evaluated in four holdout meeting rooms containing two right-swing and two left-swing doors.The robot begins from a marked location, moves to a randomized pose within a 25x25 cm area, and then executes the task.
  • Evaluation: Door opening is counted as successful only when the door is sufficiently open for the robot to enter, while environment collisions count as failure.The evaluation protocol therefore combines task completion with collision-based safety criteria.

L Negative Results

The negative-results appendix records empirical alternatives that failed to improve performance, while identifying representation and checkpoint-selection observations that shaped the final system. These findings are explicitly anecdotal rather than systematic evaluations.

  • Action representations: Decomposing XYZ prediction into direction and magnitude did not outperform direct XYZ prediction and led to the adaptive algorithm used in final results.The decomposition was introduced to address the small-action problem identified in the paper.
  • Architecture and conditioning: Removing spatial softmax layers increased performance, whereas proprioceptive and previous-pose conditioning did not improve performance.The spatial-softmax observation concerns both the policy and video encoder; the conditioning result may reflect causal confusion between inputs and expert actions.
  • Video representations: Using 40 instead of 20 video frames did not improve video-encoder performance and slowed training, while unrelated human-video negatives did not help contrastive learning.The negative examples were considered too easy to embed far from the other videos.
  • Model and representation alternatives: Pre-training ResNet on ILSVRC2012, enlarging the architecture beyond ResNet-18, and using mixture density networks did not improve performance.The tested larger architectures included ResNet-34 and larger models, while the mixture density network used 10 Gaussians.
  • Action representations: Delta axis-angle representations outperformed absolute axis-angle, absolute quaternion, and delta quaternion representations on manipulation tasks.This representation choice was associated with better empirical results among the tested alternatives.
  • Evaluation and training: Validation error correlated with task success, but models with similar validation errors could have substantially different task success.Checkpoint selection was therefore challenging, and additional evaluation might have produced higher reported performance for specific checkpoints.
  • Video representations: Pre-trained video embeddings generalized to some held-out tasks but performed below BC-Z’s learned video embedding, while TecNets performed worse overall.TecNets were especially worse on tasks mixing objects from the 21-task and 79-task families, and two runs with different random seeds showed instability.
Loading 2202.02005v1…