PaperlayerView exact revision on arXiv

Source-linked AI summary

Octo: An Open-Source Generalist Robot Policy

Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, Sergey Levine

arXiv:2405.12213v2cs.ROcs.LG

TL;DR

Robot policies need to generalize across embodiments and interfaces while remaining easy to adapt, but existing systems often restrict inputs or finetuning. Octo addresses this with an open-source transformer policy trained on 800k demonstrations, supporting language and goal-image conditioning and adaptation to new observation and action spaces. Across 9 robotic platforms, it serves as a versatile initialization for out-of-the-box control and finetuning, while its ablations analyze architectural and data choices.

  • Problem

    Robot policies must handle diverse embodiments, sensors, action spaces, tasks, environments, and compute budgets, while existing generalist policies often restrict observations or finetuning.

  • Method

    Octo is an open-source transformer policy pretrained on 800k Open X-Embodiment demonstrations, using adaptable tokens and heads for language, goal images, observations, and actions.

  • Results

    Across 9 robotic platforms, Octo supports out-of-the-box multi-robot control and effective finetuning to unseen observation and action spaces.

  • Takeaways & Limitations

    Octo provides a versatile initialization for robotic control and releases the model, training pipeline, checkpoints, and finetuning tools for broad use.

Abstract

from arXiv · show

Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models.

I. INTRODUCTION

Octo addresses the challenge of building broadly applicable robot policies by combining diverse pretraining data with an architecture that supports varied interfaces and efficient adaptation. The resulting open-source policy is designed for zero-shot use across configurations and finetuning to unseen robot setups.

  • Training policies separately for each robot and task demands substantial data collection and typically yields narrow generalization.
  • Generalist robot policies must accommodate different embodiments, sensors, action spaces, task specifications, environments, and compute budgets.
  • Existing generalist policies have commonly restricted input observations, weak support for downstream finetuning, or limited public availability.
  • Octo uses a transformer mapping arbitrary task and observation tokens to action outputs, supporting camera configurations, robot platforms, language commands, and goal images.
  • 800k robot demonstrations from Open X-Embodiment train Octo, whose components combine into an open-source generalist manipulation policy.
  • Experiments across 9 robots show strong out-of-the-box multi-robot control and effective finetuning to unseen observation and action spaces.

II. RELATED WORK

Prior work established cross-domain robot policies but often constrained their interfaces or availability. Octo combines scalable transformer design, multimodal tokenization, and adaptable inputs and outputs to broaden downstream use.

  • Earlier generalist policies broadened robot and task generalization, but many focused on restricted observations, limited finetuning, or non-public models.
  • Octo trains across multiple robot embodiments, increasing the effective training set and supporting finetuning to varied robot setups.
  • Octo’s design combines diffusion action decoding, action chunks, and scalable vision-transformer-inspired training for cross-embodied policy learning.
  • Architecture: The architecture tokenizes language, goal images, and camera observations into a shared sequence processed by a transformer and readout heads.
  • Architecture: Block-wise attention supports causal observation processing while masking absent modalities, allowing heterogeneous task specifications.
  • Architecture: New task inputs, observations, loss functions, and action heads can be added during finetuning while retaining pretrained transformer weights.

DATASET SAMPLING WEIGHTS

The training mixture draws from 25 diverse Open X-Embodiment datasets, with batch contributions weighted primarily by dataset size and adjusted to balance size and diversity.

  • 25 Open X-Embodiment datasets with image observations, end-effector actions, and diverse behaviors form the curated training subset.
  • Each dataset’s average fraction of a training batch is visualized, with weights based on sample counts and small adjustments for dataset size and diversity.
  • The architecture is intended to support changing observations or tasks without reinitializing most of the model.

B. Training data

Octo is trained on heterogeneous robot data and uses diffusion-based action decoding, multimodal conditioning, and a standardized finetuning recipe. The pipeline handles missing modalities and supports efficient adaptation to target domains.

  • Training data: The training mixture combines 25 datasets spanning robot embodiments, sensors, scenes, tasks, and language-label availability.
  • Training data: Missing camera channels are zero-padded and gripper actions are aligned across datasets, while the authors identify mixture quality as an area for further analysis.
  • Training objective: A conditional diffusion head predicts continuous, multimodal action distributions using one transformer forward pass followed by denoising within the head.
  • Training objective: The diffusion head is trained by adding Gaussian noise to actions and learning to reconstruct the original actions with a DDPM objective.
  • Finetuning: Finetuning uses about 100 target-domain trajectories, 50k steps, cosine learning-rate decay with warmup, and updates the full model.
  • Training procedure: Octo uses two observation-history frames, hindsight goal relabeling, image augmentation, and random language or goal-image dropout for flexible conditioning.

E. Model Checkpoints & Code

The paper releases the resources needed to use, finetune, reproduce, and pretrain Octo.

  • The release includes pretrained Octo-Small and Octo-Base checkpoints with 27M and 93M parameters.
  • Finetuning scripts and the complete Octo pretraining pipeline are provided in JAX.
  • Standalone Open X-Embodiment data loaders support both JAX and PyTorch.

IV. EXPERIMENTS

Experiments evaluate Octo across nine robot-learning setups and compare zero-shot control and finetuning under varied tasks, robots, observations, and action spaces. The evaluations show strong out-of-the-box control and data-efficient adaptation, while highlighting degradation on novel behaviors.

  • Evaluation setups: Nine robot-learning setups across four institutions test diverse objects, long-horizon tasks, precise manipulation, and new interfaces.The evaluations include new force-torque observations, joint-position action spaces, and robot embodiments.
  • Zero-shot evaluation: Out-of-the-box, Octo controls multiple robots in environments represented in pretraining data.The zero-shot comparison covers language-specified tasks across multiple robot embodiments and setups.
  • Finetuning evaluation: Octo enables finetuning with approximately 100 target demonstrations per domain and outperforms training from scratch and pretrained visual representations.Success rates are averaged over 20 trials using the same finetuning hyperparameters; new observations and action spaces are included.
  • Zero-shot evaluation: Octo achieves a 29% higher average success rate than RT-1-X across the evaluated zero-shot language tasks.Each robot was tested on two language tasks with 10 trials per task and varying initial conditions.
  • Zero-shot evaluation: Octo performs similarly to RT-2-X on the tested WidowX and RT-1 Robot tasks.For WidowX, RT-2-X results were reported from prior work because its model was not openly available.
  • Zero-shot evaluation: Goal-image conditioning yields a 25% higher success rate than language conditioning on the WidowX tasks.In BridgeV2, zero-shot performance remains high on novel objects but degrades in new scenes and more strongly for novel behaviors.

B. Octo Enables Data-Efficient Learning in New Domains

Octo supports data-efficient finetuning across new observations, action spaces, and robot embodiments, outperforming alternative initialization strategies across six evaluation setups. Its ablations identify transformer architecture, broad training data, diffusion decoding, and larger model scale as important design choices.

  • Finetuning results: 52%: Octo outperforms the next best baseline on average across six evaluation setups after finetuning to new domains.The same finetuning recipe and hyperparameters were used across all evaluation tasks.
  • Finetuning results: Octo accommodates force-torque inputs, joint-position control, and new single- and dual-arm robot embodiments during finetuning.These include Berkeley Insertion, Berkeley Pick-Up, Berkeley Coke, and Berkeley Bimanual.
  • Ablations: The ViT architecture, diffusion action head, and wide training-data mixture produce the best performance in the WidowX ablations.Table II averages success rates over 40 trials across two language-conditioned and two goal-conditioned tasks.
  • Ablations: Performance improves as the training mixture expands from fewer datasets, while the broader mixture is associated with better policy performance.The comparison includes an 11-dataset mixture and a target-robot-only baseline.
  • Ablations: Zero-shot performance improves with model size, with Octo-Tiny, Octo-Small, and Octo-Base containing 10M, 27M, and 93M parameters, respectively.The Base model is also reported to be more robust to initial scene configurations and less prone to early grasp attempts than the Small model.

V. DISCUSSION AND FUTURE WORK

Octo is a large transformer policy pretrained on 800k robot trajectories and designed to support diverse robotic control setups. The authors identify training-data coverage and broader robot scope as important directions for improvement.

  • Octo was pretrained on 800k robot trajectories and demonstrated across varied tasks as a generalist manipulation policy.Its design supports finetuning to new inputs and action spaces.
  • Only 27% of pretraining data contains wrist-camera information, and only 56% contains language annotations.The authors link these coverage gaps to weaker wrist-camera processing and differences between language- and goal-conditioned performance.
  • The authors identify expanding beyond optimal demonstrations as a natural avenue for improving Octo.The current model is trained by imitation on optimal robot demonstrations.
  • Octo was trained and evaluated only on single- and dual-arm manipulators, leaving navigation and mobile manipulation as unaddressed robot categories.The authors describe expanding to these robots as a high-opportunity direction.
  • Future improvements include better language conditioning, stronger wrist-camera support, and data beyond optimal demonstrations.The authors present Octo as a launchpad for efficient learning of new tasks and broader generalization.

APPENDIX A CONTRIBUTIONS

The appendices document Octo’s contributors, pretrained-model inference example, training-data mixture, optimization settings, tokenization, and model variants. These materials cover both reproducibility and practical use.

  • Contributions: The contribution statement assigns model development, evaluation, benchmark design, infrastructure, and writing responsibilities across the project team.The listed contributors include researchers responsible for training, evaluations, robot experiments, and project guidance.
  • Inference: A pretrained Octo model can be loaded and queried with an image observation and language task using a short JAX-based program.The example loads a released checkpoint, prints its input-output specification, creates a text task, and samples actions.
  • Training setup: Octo’s pretraining mixture uses datasets from Open X-Embodiment, with sampling weights based mostly on dataset size and some manual adjustments.The detailed mixture is listed in Table III.
  • Training setup: Training follows documented vision-transformer practices using AdamW, inverse-square-root learning-rate decay, warm-up, and standard image augmentation.Hyperparameters are listed in Table IV.
  • Model architecture: Images are encoded by a shallow convolutional stack into flattened patches, while language annotations are represented by 16 tokens from pretrained T5-base.Architecture details for model variants are listed in Table V.
  • Action head: The diffusion action head uses a three-layer MLP with residual connections, layer normalization, a DDPM objective, cosine noise schedule, and 20 diffusion steps.

APPENDIX E THINGS THAT WORKED AND DID NOT WORK (YET)

The ablations identify design choices that improve Octo’s zero-shot or fine-grained control, alongside action-head and encoder choices that remain problematic. Several gains involve history, action chunking, image resolution, data shuffling, and representation choices.

  • One frame of history improved zero-shot evaluations, while longer histories showed no benefit on the evaluated tasks.Other tasks may benefit from additional history.
  • Action chunking produced more coherent policy movements, whereas temporal ensembling added no benefit beyond receding-horizon control.
  • 16 × 16 image patches improved performance over 32 × 32 patches, particularly for grasping and other fine-grained tasks.The smaller patches quadruple the number of tokens and increase compute complexity.
  • A shuffle buffer of up to 500k, with frames shuffled before image decoding, addressed training-data shuffling challenges across 25 datasets.Small buffers and trajectory-level interleaving substantially hurt zero-shot performance.
  • MSE action heads produced slow hedging policies, while discrete action heads were decisive but often lacked grasping precision.
  • ResNet encoders did not scale as well on larger datasets, although they outperformed the ViT architecture when trained from scratch on around 100 demonstrations.
  • Relative gripper actions tended to reduce unnecessary opening and closing because most training data represents no gripper change.
  • Adding proprioceptive observations generally worsened performance, potentially because of causal confusion between states and target actions.

APPENDIX F EXPERIMENTAL SETUPS

The experiments evaluate Octo on multiple robot setups and task types, including WidowX, UR5, and RT-1 systems. The benchmark spans language- and goal-conditioned manipulation with novel object positions and other generalization axes.

  • WidowX BridgeV2: WidowX BridgeV2 uses a 6-DOF robot, a third-person camera, end-effector position-delta actions, and two language-conditioned placement tasks.The tasks place a carrot on a plate and an eggplant in a pot while varying object positions across 10 trials per task.
  • Evaluation scope: The evaluation covers 9 real robot setups across 4 institutions in zero-shot and finetuning scenarios.
  • UR5: UR5 experiments use a third-person camera and end-effector position deltas for tabletop tasks involving toy placement and table wiping.
  • RT-1 Robot: RT-1 experiments evaluate picking varied objects and opening or closing cabinet drawers using third-person images and end-effector position deltas.
  • Ablation setup: Model ablations use WidowX with two language-conditioned and two goal-conditioned tasks, including objects unseen in the Bridge dataset.The setup also analyzes generalization across several axes.

C. Finetuning Evaluations

The finetuning evaluations test Octo across varied tasks, observation modalities, action spaces, robot embodiments, and generalization axes. Ablations identify effective design choices, while zero-shot analysis shows strong generalization to novel objects and environments but difficulty with unseen skills.

  • Evaluation setup: Octo is evaluated on manipulation tasks spanning diverse cameras, proprioception and force/torque sensing, action spaces, control rates, and robot embodiments.The tasks include coffee-pod placement, peg insertion, block and can pickup, and bimanual uncapping.
  • Evaluation setup: The Berkeley Peg Insertion task uses a 1.5mm matching tolerance and force/torque measurements for contact-rich insertion.The policy must search for the matching opening through contact.
  • Model ablations: The ablation study reports best performance with the ViT architecture, diffusion action head, and wide training data mixture.Evaluations average success rates over 40 trials across two language-conditioned and two goal-conditioned tasks.
  • Zero-shot generalization: Octo performs best on in-distribution tasks and generalizes well to novel objects and environments, but struggles with skills absent from the WidowX embodiment.The zero-shot analysis averages success rates over 20 trials across two tasks.
  • Evaluation setup: For bimanual uncapping, Octo receives a newly initialized action head mapping to ALOHA’s 14-dimensional action space.The setup uses two ViperX arms, although pretraining used only single-arm robot data.
Loading 2405.12213v2…