PaperlayerView exact revision on arXiv

Source-linked AI summary

Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments

Haritheja Etukuru, Norihito Naka, Zijin Hu, Seungjae Lee, Julian Mehu, Aaron Edsinger, Chris Paxton, Soumith Chintala, Lerrel Pinto, Nur Muhammad Mahi Shafiullah

arXiv:2409.05865v1cs.ROcs.LG

TL;DR

Robot policies often require environment-specific fine-tuning, unlike zero-shot vision and language models. RUMs combine demonstration collection, multimodal imitation learning, and mLLM-based verification and retrying to deploy policies zero-shot. They achieve 90% average success in 25 novel environments, with data diversity and recovery among the reported lessons.

  • Problem

    Robot policies commonly need fine-tuning in each new environment, unlike vision and language models that can be deployed zero-shot.

    Source: §1 · p. 2

  • Method

    RUMs combine quickly collected demonstrations, multimodal imitation learning, and mLLM-based introspection and retrying for zero-shot deployment.

    Source: §1 · p. 2, §2.4 · p. 6

  • Results

    90% average success rate in 25 novel environments was achieved by RUMs trained with about 1,000 demonstrations.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Data quantity and quality matter more than architecture choice, while diversity matters more than raw data volume for generalization.

    Source: §1 · pp. 2–3

    The robot is assumed to start in the task space facing the objective, and retries assume errors leave the task somewhat in-distribution and allow an easy reset.

    Source: §5 · pp. 13–14

Abstract

from arXiv · show

Robot models, particularly those trained with large amounts of data, have recently shown a plethora of real-world manipulation and navigation capabilities. Several independent efforts have shown that given sufficient training data in an environment, robot policies can generalize to demonstrated variations in that environment. However, needing to finetune robot models to every new environment stands in stark contrast to models in language or vision that can be deployed zero-shot for open-world problems. In this work, we present Robot Utility Models (RUMs), a framework for training and deploying zero-shot robot policies that can directly generalize to new environments without any finetuning. To create RUMs efficiently, we develop new tools to quickly collect data for mobile manipulation tasks, integrate such data into a policy with multi-modal imitation learning, and deploy policies on-device on Hello Robot Stretch, a cheap commodity robot, with an external mLLM verifier for retrying. We train five such utility models for opening cabinet doors, opening drawers, picking up napkins, picking up paper bags, and reorienting fallen objects. Our system, on average, achieves 90% success rate in unseen, novel environments interacting with unseen objects. Moreover, the utility models can also succeed in different robot and camera set-ups with no further data, training, or fine-tuning. Primary among our lessons are the importance of training data over training algorithm and policy class, guidance about data scaling, necessity for diverse yet high-quality demonstrations, and a recipe for robot introspection and retrying to improve performance on individual environments. Our code, data, models, hardware designs, as well as our experiment and deployment videos are open sourced and can be found on our project website: https://robotutilitymodels.com

1. Introduction

Robot policies commonly require environment-specific fine-tuning, unlike zero-shot vision and language models; building zero-shot robot policies also faces data, diversity, hardware, and recovery challenges. RUMs combine scalable demonstration collection, multimodal imitation learning, and mLLM-based verification and retrying, with reported success in novel environments and lessons about data and policy support.

1. Introduction

  • Robot policies commonly need environment-specific fine-tuning, while zero-shot deployment raises challenges in data scale, behavioral diversity, hardware variation, and error recovery.The paper contrasts this with zero-shot deployment in vision and language.Source: §1 · p. 2
  • RUMs are a framework for focused utility policies that deploy zero-shot in novel environments without further training or fine-tuning.The system combines quickly collected data, multimodal behavior learning, and mLLM self-critique with retries.Source: §1 · p. 2
  • The reported lessons are that data quantity and quality matter more than architecture choice, and data diversity matters more than raw volume for generalization.Self-critique by an independent model and retrying can improve performance in individual environments.Source: §1 · pp. 2–3
  • 90% average success rate in 25 novel environments was achieved with RUMs trained using about 1,000 demonstrations.The introduction also reports 2,950 real-world rollouts across homes in New York City, Jersey City, and Pittsburgh.Source: §1 · p. 3
  • Multi-modal policies provided a 74.4% zero-shot success rate, and the mLLM-based self-critique and retrying system further improved the success rate by 15.6%.Source: §1 · p. 3

2. Robot Utility Models

Robot Utility Models use a full-stack imitation-learning system, with tools and techniques for data collection, training, inference, and deployment. The system is designed to scale diverse demonstrations and support zero-shot deployment in new environments.

2. Robot Utility Models

  • The system combines imitation learning with tools and techniques for data collection, model training, inference, and deployment.The paper describes this as a full-stack approach to scaling imitation learning for zero-shot deployment.Source: §2 · p. 3

2.1. Data collection tool

  • Stick-v2 is a portable handheld iPhone Pro tool designed to collect accurate demonstrations quickly across environments.It combines an iPhone Pro with a $25 bill of materials; the iPhone captures RGB and depth and provides synchronized pose data without camera calibration.Source: §2.1 · pp. 3–4, Figure 2 · p. 4

2.2. Collected datasets

  • The five datasets cover door opening, drawer opening, object reorientation, tissue pickup, and bag pickup, with about 1,000 demonstrations across roughly 40 environments per task.Door opening uses 1,200 demonstrations and drawer opening 525; test environments and objects are held out from data collection.Source: §2.2 · pp. 4–5, Figure 3 · p. 5

2.3. Model training

  • The authors train multiple policy classes on each task’s data, with VQ-BeT and Diffusion Policy performing best among those tested.The policies share a vision encoder and transformer-based trunk, and training uses 500 epochs with hyperparameter sweeps.Source: §2.3 · p. 5

2.4. Retrying with GPT-4o feedback

  • An external multimodal LLM verifies task success from robot-observation summaries and triggers a reset and retry with a new initial robot state when it detects failure.The retry process continues until success or timeout.Source: §2.4 · p. 6, Figure 4 · p. 6

2.5. Deployment Details

  • Deployment primarily uses Hello Robot Stretch with an iPhone wrist camera, while the system also supports robot arms with relative 6D pose control and other wrist cameras.The hardware setup relies on a flexible two-finger gripper, a wrist camera with sufficient field of view, and a six-degree-of-freedom arm.Source: §2.5 · p. 6, Figure 5 · p. 7

3. Capabilities of Robot Utility Models

The evaluation tests zero-shot task solving on unseen environments and objects, compares training components and data properties, examines self-critique, and assesses transfer to new embodiments.

3.1. Zero-shot evaluation of RUMs on unseen environments

  • 90% overall success rate across novel environments, with individual-task success rates ranging from 84% to 94%.The evaluation also reports some success in every environment.Source: Figure 6 · p. 8, §3.1 · p. 8

3.2. Effect of policy architecture and training method on RUMs

  • VQ-BeT and Diffusion Policy perform comparably on most tasks, while ACT and MLP-BC are not far behind; VQ-BeT is used for final models.VQ-BeT was selected over Diffusion Policy for higher performance and lower robot-CPU latency.Source: §3.2 · p. 8, Figure 7 · p. 9, Figure 8 · p. 9

3.3. Effect of scaling datasets on RUMs

  • Diffusion Policy performs better on smaller datasets but saturates as data scales, while VQ-BeT outperforms it at the 900–1,200-demonstration limit.VQ-BeT performance continues to improve on the largest tested datasets.Source: Figure 9 · p. 10, §3.3 · p. 10

3.4. Importance of data diversity in training RUMs

  • Diverse datasets are more valuable than uniform ones, especially for reorientation across many unseen environments and objects.For door opening, both diverse- and uniform-environment policies performed well.Source: Figure 10 · p. 10

3.5. Impact of using expert demonstrations on training policies

  • With roughly 500 demonstrations, expert data outperforms non-expert data on every task, while co-training can sometimes help and sometimes hurt.Expertise is defined as experience deploying Dobb·E policies on the robot.Source: §3.5 · p. 11

3.6. Effects of introspection and retrying with self-critique in RUMs

  • Self-critique and retrying improve performance by 15.6% on average; successful tasks take 1.31 tries on average, while false positives average 4.8% across five tasks.False positives constitute half of the total errors.Source: Figure 11 · p. 11, §3.6 · p. 11

3.7. Transferring RUMs to different embodiments

  • On a different robot embodiment and camera setup, RUMs transfer with roughly a 10% performance drop in both cases without corrective mLLM feedback.The transfer test uses the UFactory xArm 7 rather than the Hello Robot Stretch setup.Source: §3.7 · p. 12

4. Related works

Prior robot-model work has pursued broad generalization through large-scale data, efficient collection, and modular systems, while RUMs target zero-shot deployment without environment-specific fine-tuning. Related work also explores language-model feedback for robot decision-making.

Embodiment Transfer

  • Figure 12 reports that RUMs transfer to different embodiments with minimal performance loss when run without corrections.The figure caption identifies the comparison as performance across embodiments.Source: Figure 12 · p. 12

Large Scale Data Collection

  • Robot demonstration collection spans exoskeletons, controllers, VR, phones, and devices that map demonstrator movement to the robot, each balancing intuitiveness, efficiency, and force feedback.RUMs use a data-collection device building on prior work that combines simple-controller versatility with physical end-effector intuitiveness.Source: §4 · p. 12

Pretrained Robot Models

  • Robot foundation models face limited datasets, cross-domain discrepancies, and challenges in action tokenization, while RUMs aim to generalize without fine-tuning for each new environment or embodiment.Prior approaches include internet-scale training followed by task-specific fine-tuning; the cited RUM framing does not expect data from deployment environments.Source: §4 · p. 13

Large Models Feedback and Improvement

  • Language models have been used for robot feedback, affordance judgments, outcome imagination, planning, reward shaping, and long-horizon reasoning; this work uses mLLM feedback as a reset signal.The reset signal is used in open-ended environments.Source: §4 · p. 13

5. Limitations and Conclusion

The paper identifies hardware constraints, a separate-navigation assumption, and assumptions about task-space errors and resetting as limitations of current RUM deployments. It suggests modular navigation and failure-recovery training as possible directions for improvement.

Limitations

  • Two-fingered grippers like Stick-v2 cannot open doors with round doorknobs, while flexible fingertips make heavy objects difficult to manipulate.The authors identify improved gripper and fingertip design as a direction for addressing these hardware limitations.Source: §5 · p. 13
  • The system assumes navigation is separate and that the robot begins in task space facing the objective.The authors suggest combining with modular navigation work such as Liu et al. (2024b) to address this issue.Source: §5 · pp. 13–14
  • Retrying assumes model errors leave the task space somewhat in-distribution and permit an easy reset to the initial state.The authors suggest adding failure-recovery behavior to training data so robots can recover more naturally from such cases.Source: §5 · p. 14

A.1. Detailed Results from Experiments with Self-critique and Retrying

The supplied passage identifies a table of detailed RUM success statistics for the evaluation environments but does not provide its values.

A.1. Detailed Results from Experiments with Self-critique and Retrying

  • Table 1 presents detailed RUM success statistics for the evaluation environments.The caption does not state the individual values or comparisons.Source: Table 1 · p. 21

A.2. Evaluation Environments

The appendix figures show evaluation environments for reorientation, drawer and door opening, tissue pickup, and bag pickup.

A.2. Evaluation Environments

  • Figure 13 shows evaluation environments for reorientation, drawer opening, and door opening.The caption identifies the pictured environments by task.Source: Figure 13 · p. 22
  • Figure 14 shows evaluation environments for tissue pickup and bag pickup.The caption identifies the pictured environments by task.Source: Figure 14 · p. 23

A.3. Multimodal Large Language Model Prompts for Success Verification

The appendix gives task-specific prompts for an mLLM to verify whether the robot completed each task, with responses constrained to Yes or No.

A.3. Multimodal Large Language Model Prompts for Success Verification

  • The door prompt checks whether the arm opens the door and is still grasping its handle at the final timestep.The drawer prompt checks that the handle is grasped and the drawer is open in the last timestep.Source: §A.3 · p. 23
  • The reorientation prompt checks whether the gripper reorients the object upright and whether it is upright in the last frame.Source: §A.3 · p. 24
  • The pickup prompts check whether the gripper grasps the tissue or bag and still holds it in the final timestep.Each prompt requests only a Yes or No response.Source: §A.3 · pp. 23–24

A.4. Evaluation Schedule

The evaluation schedule uses ten runs to assess positional generalization, varying robot starting positions and, for object-centric tasks, object positions.

A.4. Evaluation Schedule

  • The evaluation uses 10 runs to examine Robot Utility Models’ positional generalization capabilities.Source: §A.4 · p. 24
  • Robot starting positions are marked by pale blue dots, with the robot facing the object at varying offsets from the target.Source: Figure 15 · p. 24
  • Object positions are also randomized for reorientation, bag pickup, and tissue pickup tasks.Source: Figure 15 · p. 24

A.5. Bill of Materials

The bill of materials assumes the user already owns an iPhone Pro and puts the setup cost slightly below $25.

A.5. Bill of Materials

  • The complete setup costs slightly below $25, assuming the user already owns an iPhone Pro.Source: §A.5 · p. 25
  • The stated total excludes the cost of the iPhone Pro because ownership is assumed.Source: §A.5 · p. 25
  • The bill of materials is organized into tables for the Stick-v2 main body, gripper tips, and phone holder.Source: Table 2 · p. 25, Table 3 · p. 25, Table 4 · p. 26

A.6. Deploying on Stretch’s Default D405 Camera

The D405 image is affine-transformed to approximate the iPhone camera’s viewpoint, but close-range focus differences accompany a performance drop. The appendix also describes task-specific failures during mLLM-guided retries.

A.6. Deploying on Stretch’s Default D405 Camera

  • The affine transform maps the D405 image to pixel coordinates matching the iPhone camera, but RUM performance drops, likely because close-range images differ in focus.The authors hypothesize that the cameras’ different focal lengths cause the close-range difference and suggest image augmentations as a possible remedy.Source: §A.6 · p. 26, Figure 16 · p. 26

A.7. Failure Modes

  • Reorientation retries can become impossible when the object rolls away, falls, is pushed out of reach, or is rotated sideways before grasping.On out-of-distribution surfaces, visually estimating the surface size can also lead to placement misses or robot-surface collisions.Source: §A.7 · p. 27
  • Bag-pickup feedback has a 4% improvement from GPT, with false positives often arising because the robot’s wrist or head camera cannot clearly show whether the bag was picked up.The authors suggest a better third-person workspace view could help address this issue.Source: §A.7 · p. 28

Related papers

Loading 2409.05865v1…