Source-linked AI summary
Robotic Control via Embodied Chain-of-Thought Reasoning
Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, Sergey Levine
TL;DR
Learned robot policies and standard VLAs can struggle to generalize or reason in a way grounded in sensory observations and robot state. The paper introduces ECoT, which trains VLAs on synthetic embodied reasoning chains, and reports a 28% absolute OpenVLA success-rate improvement across challenging generalization tasks without additional robot training data.
Problem
Standard VLAs typically map observations directly to actions, while naive CoT is insufficiently grounded in sensory observations and robot state.
Method
ECoT trains VLAs to interleave semantic plans and sub-tasks with embodied reasoning about motions, object bounding boxes, and gripper positions using synthetic supervision.
Results
28% absolute success-rate improvement is reported for OpenVLA across challenging generalization tasks without additional robot training data.
Takeaways & Limitations
ECoT also makes policy failures more interpretable and enables natural-language correction of faulty reasoning chains.
Takeaways & Limitations
ECoT inference remains speed-limited because its reasoning increases predicted tokens per timestep from 7 for OpenVLA to 350.
Abstract
from arXiv · showhide
A key limitation of learned robot control policies is their inability to generalize outside their training data. Recent works on vision-language-action models (VLAs) have shown that the use of large, internet pre-trained vision-language models as the backbone of learned robot policies can substantially improve their robustness and generalization ability. Yet, one of the most exciting capabilities of large vision-language models in other domains is their ability to reason iteratively through complex problems. Can that same capability be brought into robotics to allow policies to improve performance by reasoning about a given task before acting? Naive use of "chain-of-thought" (CoT) style prompting is significantly less effective with standard VLAs because of the relatively simple training examples that are available to them. Additionally, purely semantic reasoning about sub-tasks, as is common in regular CoT, is insufficient for robot policies that need to ground their reasoning in sensory observations and the robot state. To this end, we introduce Embodied Chain-of-Thought Reasoning (ECoT) for VLAs, in which we train VLAs to perform multiple steps of reasoning about plans, sub-tasks, motions, and visually grounded features like object bounding boxes and end effector positions, before predicting the robot action. We design a scalable pipeline for generating synthetic training data for ECoT on large robot datasets. We demonstrate, that ECoT increases the absolute success rate of OpenVLA, the current strongest open-source VLA policy, by 28% across challenging generalization tasks, without any additional robot training data. Additionally, ECoT makes it easier for humans to interpret a policy's failures and correct its behavior using natural language.
1 Introduction
The paper introduces ECoT to address the limits of direct VLA action prediction and insufficiently grounded CoT reasoning. ECoT improves OpenVLA generalization and supports natural-language diagnosis and correction of failures.
- Motivation: Reactive robot policies map observations directly to actions, which can struggle to generalize to novel scenes or unfamiliar objects.The paper motivates reasoning before acting as an alternative to simply executing learned skills.
- Motivation: VLAs leverage pretrained vision-language models but typically predict actions without intermediate reasoning.This leaves their reasoning process between observation and action unspecified.
- Approach: ECoT interleaves semantic sub-task planning with embodied reasoning about object boxes, robot state, and low-level movement primitives before action prediction.The method is designed for VLA policies rather than relying on naive step-by-step prompting alone.
- Results: 28% absolute success-rate improvement is reported for OpenVLA across challenging generalization tasks without additional robot training data.The evaluations cover new objects, scenes, viewpoints, and instructions.
- Results: ECoT makes policy failures more interpretable and enables humans to correct behavior by modifying reasoning chains with natural-language feedback.The paper presents this as an additional benefit beyond raw task success.
2 Related Work
Related work pursues robot generalization through diverse robot datasets and Internet-pretrained vision-language models. VLAs provide a simple, scalable way to integrate such models into learned robot policies.
- Scaling robot learning: Large and diverse robot datasets support generalist policies that can follow natural-language prompts across manipulation tasks and sometimes multiple robot embodiments.This line of work treats dataset diversity as a route toward greater robustness and generalization.
- Vision-language models for robot generalization: Vision-language models have been integrated into robot learning to generate goals, provide rewards, learn visual state representations, and support VLA policies.VLAs are highlighted as a simple and scalable alternative for using pretrained vision-language models in robot control.
3 Preliminaries: Vision-Language-Action Models
VLAs fine-tune pretrained vision-language models to map image observations and task instructions to robot actions. OpenVLA uses pretrained visual encoders and a Llama 2 7B backbone, with actions represented as discrete tokens.
- VLA formulation: VLAs directly fine-tune a pretrained vision-language model to predict the next robot action from the current image and task instruction.Continuous actions are represented through autoregressively generated action tokens.
- Action representation: Continuous robot actions are discretized independently by dimension into one of 256 vocabulary bins.This converts low-level continuous control values into tokens compatible with the vision-language model.
- OpenVLA: OpenVLA combines pretrained SigLIP and/or DinoV2 visual features with a Llama 2 7B language-model backbone.The model maps visual and task tokens to target action tokens during training.
4 Embodied Chain-of-Thought Reasoning for Visuomotor Policies
ECoT trains VLAs to reason through task plans, current sub-tasks, movements, and grounded scene features before producing actions. Synthetic supervision is generated from existing robot datasets, while inference efficiency is addressed by reusing higher-level reasoning across steps.
- Designing reasoning steps: ECoT explicitly trains VLA policies to perform embodied reasoning about plans, sub-tasks, motions, and visual features before action prediction.The approach is motivated by the limited reasoning ability of relatively small VLA language-model backbones under prompting alone.
- Reasoning chain: The reasoning chain progresses from task rephrasing and high-level planning to selecting the current sub-task and predicting low-level movement commands.These stages connect semantic task structure to the robot state and required action.
- Embodied grounding: Grounded steps predict the robot end-effector position and object names with bounding-box pixel coordinates to focus attention on the image.These features complement linguistic sub-task reasoning with spatially grounded scene information.
- Limitations: The chosen reasoning tasks and fixed ordering do not exhaustively cover alternatives, leaving automated task selection and ordering for future work.The authors identify alternative reasoning structures as an important direction.
- Synthetic data generation: Synthetic ECoT supervision combines scene descriptions, open-vocabulary detections, robot-derived movement primitives, gripper positions, and language-model-generated plans.The pipeline processes existing trajectories rather than requiring direct human annotation of every transition.
- Synthetic data generation: The pipeline labels the Bridge v2 dataset, containing more than 2.5M transitions, over seven days.Gemini generates the final reasoning chains from task instructions, scene descriptions, and per-step movement primitives.
- Efficient inference: ECoT inference increases tokens per timestep from 7 for OpenVLA to 350, motivating synchronous and asynchronous reuse of higher-level reasoning.The proposed strategies trade off control speed and performance by keeping selected reasoning components fixed across multiple steps.
5 Experiments
Experiments evaluate ECoT on challenging manipulation generalization, policy interpretation and correction, inference efficiency, and transfer across robot embodiments. ECoT improves OpenVLA performance, supports language-based correction, and can reason about unseen embodiments, while inference speed and domain gaps remain practical considerations.
- 5 Experiments: Experiments test ECoT across generalization, interpretability, human correction, runtime efficiency, and embodiment-transfer questions.The evaluation uses a WidowX arm, 314 trials per approach, and tasks involving spatial relations, unseen objects, and unseen instructions.
- 5.2 Embodied Chain-of-Thought Reasoning Improves Policy Generalization: ECoT achieves the highest aggregate success rate, improving over OpenVLA by 22% in-distribution and 34% out-of-distribution.Table 1 reports aggregate absolute improvements over Octo, OpenVLA, RT-2-X, and naïve CoT across two camera-view settings.
- 5.2 Embodied Chain-of-Thought Reasoning Improves Policy Generalization: ECoT substantially outperforms OpenVLA across all but one generalization evaluation despite identical VLM bases and robot fine-tuning data.It also surpasses RT-2-X in the tested tasks, although RT-2-X uses 10 additional robot datasets and a 55B versus 7B network.
- 5.3 Diagnosing Policy Failures Through Inspecting Reasoning Chains: Embodied reasoning grounds sub-task plans in object bounding boxes, gripper positions, and low-level actions, while reasoning chains expose mistakes behind downstream failures.A qualitative failure occurs when the hammer is identified as a screwdriver; the authors caution that reasoning inspection is not bullet-proof because final actions may deviate from plans.
- 5.4 Chain-of-Thought Reasoning Enables Interactive Policy Correction: ECoT gains 48% success from a single natural-language intervention on the most challenging evaluation tasks.The intervention is incorporated into corrected reasoning chains, whereas vanilla OpenVLA and RT-2-X benefit less from the same procedure.
- 5.6 Additional Analysis: ECoT reasoning transfers to other robot embodiments without embodiment-specific reasoning annotations, but suffers in SIMPLER from the real-to-sim domain gap.Prompting with “TASK:” enables recognition of grippers, objects, positions, and future gripper movements despite different appearances and camera setups.
6 Discussion and Limitations
ECoT improves OpenVLA performance without additional robot training data, but its fixed reasoning structure and runtime constrain effectiveness and scalability.
- Performance: 69% aggregate success is achieved by the ECoT base policy on the harder out-of-distribution view setting.The same trial subset reports 29% for OpenVLA (Bridge), 46% for RT-2-X, and 14% for Octo.
- Limitations: ECoT always executes reasoning steps in a fixed order rather than adapting the chain structure to the task.The authors suggest selecting only a subset of reasoning steps based on robot and scene state.
- Limitations: ECoT execution speed remains limiting, despite runtime optimizations that improve achievable control frequencies.The authors identify improved LLM throughput as a route toward higher-frequency control tasks.
A Grounding DINO Detections and Prismatic Descriptions
This section presents qualitative examples of scene descriptions, object localization, task prompts, and successful or failed reasoning chains used for grounding and evaluation.
- Visual grounding: Prismatic VLM generates captions of Bridge observations, while Grounding DINO generates associated object bounding boxes.
- Task prompts: Example task prompts specify object manipulation goals such as moving a watermelon to a towel or placing a purple object in a container.
- Examples: Figures provide example starting scenes and qualitative examples of both successful and failed chain-of-thought reasonings.
B List of Movement Primitives
Movement primitives are assigned from short-horizon robot-state changes, producing compact labels for directional motion, tilt, rotation, and gripper actions.
- Label construction: Movement labels compare the current robot state with its position four steps ahead and threshold axis differences at 0.03.
- Label vocabulary: Labels encode forward, backward, left, right, up, and down motion together with tilt, rotation, and gripper opening or closing.
- Label distribution: 36 = 729 possible labels are technically available, but only 54 occur in more than 0.1% of cases.
- Label distribution: Stop is the most frequent movement label at 26.9%, followed by close gripper at 10.8% and open gripper at 7.2%.
C Prompts
The prompts construct trajectory annotations that connect task goals and high-level plans to step-level actions, movements, and scene features for VLA reasoning.
- Scene descriptions: Scene descriptions are generated by asking for objects and spatial relations, optionally prefixed with the robot task instruction.
- Trajectory representation: Trajectory features represent consecutive robot states with movement labels, and each dictionary entry describes the move about to be executed.
- Prompt design: The data-generation prompts request descriptive reasoning for each trajectory step, including the task, remaining plan, movements, and action justification.
- Task reasoning: Task-level prompts require an overview of the activity, interacting objects, relative locations, high-level movements, step intervals, and justifications.
- Step reasoning: Step-level reasoning records the remaining objective, current progress, relevant objects, and next-step plan from available features.
- Output format: Reasoning strings are organized with tags for the remaining task, plan, current subtask, subtask justification, movement, and movement justification.
- Prompt sources: Prompts are supplied for Gemini data labeling, OpenVLA language conditioning, and ChatGPT-based human interventions.