PaperlayerView exact revision on arXiv

Source-linked AI summary

Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models

Yifu Yuan, Yaoting Huang, Xianze Yao, Yutong Li, Shuoheng Zhang, Linqi Han, Pengyi Li, Jiangeng Sun, Wenting Jia, Zhao Zhang, Yuhao Liu, Ruihao Liao, Yucheng Hu, Qiyu Wu, Yuxiao Li, Zibin Dong, Fei Ni, Yan Zheng, Shuyang Gu, Yi Ma, Hongyao Tang, Han Hu, Jianye Hao

arXiv:2606.11324v2cs.ROcs.AIcs.LG

TL;DR

Existing embodied models fragment cognition, planning, correction, and grounding across capabilities and models, limiting progress toward unified physical intelligence. Embodied-R1.5 unifies these capabilities in an 8B-parameter model with automated data construction, balanced reinforcement learning, and a Planner-Grounder-Corrector loop, achieving state-of-the-art results on 16 of 24 embodied VLM benchmarks and outperforming strong VLA baselines across four manipulation suites.

  • Problem

    Existing embodied models cover fragmented capability subsets or rely on separate models, while heterogeneous multi-task training faces convergence difficulties.

  • Method

    Embodied-R1.5 unifies embodied cognition, planning, correction, and grounding in one 8B-parameter model using automated data construction, balanced reinforcement learning, and a Planner-Grounder-Corrector loop.

  • Results

    16 of 24 embodied VLM benchmarks achieve state-of-the-art performance, while light action-data fine-tuning outperforms strong VLA baselines across 4 manipulation benchmark suites.

  • Takeaways & Limitations

    Zero-shot real-robot experiments support generalization across instruction following, affordance grounding, articulated object manipulation, and long-horizon closed-loop execution.

  • Takeaways & Limitations

    The PGC closed-loop framework has been validated on tabletop manipulation, leaving mobile manipulation and navigation with longer horizons and richer environments for future work.

Abstract

from arXiv · show

We introduce Embodied-R1.5, a unified Embodied Foundation Model (EFM) that integrates comprehensive embodied reasoning capabilities, spanning embodied cognition, task planning, correction, and pointing, within a single architecture toward general physical intelligence. Leveraging three automated data construction pipelines to significantly expand the data coverage of critical capabilities, we build a large-scale data system of over 15B tokens, and design a multi-task balanced RL recipe to alleviate heterogeneous task conflicts. We further introduce a Planner-Grounder-Corrector (PGC) closed-loop framework that enables a single model to autonomously execute and self-correct over long-horizon tasks. With only 8B parameters, Embodied-R1.5 achieves SOTA on 16 out of 24 embodied VLM benchmarks, surpassing leading models like Gemini-Robotics-ER-1.5 and GPT-5.4. Benefiting from the internalized embodied capabilities, Embodied-R1.5 can be fine-tuned into a VLA with only a small amount of data, outperforming leading VLA models like $π_{0.5}$ across 4 popular manipulation benchmark suites. We further conduct extensive zero-shot real-robot experiments, validating performance in instruction following, affordance grounding, articulated object manipulation, and long-horizon complex tasks, demonstrating strong generalization to the physical world. We open-source model weights, datasets, training code, and EmbodiedEvalKit, an evaluation framework tailored for embodied tasks, to facilitate future research in EFMs.

1 Introduction

Embodied-R1.5 addresses the challenge of grounding intelligence in the physical world by unifying cognition, planning and correction, and pointing within one Embodied Foundation Model. With 8B parameters, it achieves broad benchmark leadership and supports autonomous long-horizon execution through closed-loop planning, action, and self-correction.

  • Motivation: Grounding intelligence in the physical world remains a central challenge, and embodied reasoning aims to bridge the seeing-to-doing gap through perception, spatial geometry, and task arrangement.The introduction frames embodied reasoning as requiring models to perceive the physical world and reason about spatial geometry and task arrangement.
  • Bottlenecks: Existing embodied models are fragmented across cognition, planning and correction, or grounding, with some multi-dimensional efforts relying on separate models of different scales.This fragmentation is identified as one of three fundamental bottlenecks to realizing a unified EFM.
  • Unified EFM: Embodied-R1.5 unifies cognition, spatial reasoning, planning, correction, and pointing within a single model, supported by three automated data production pipelines.The capability taxonomy groups these dimensions into Cognition & Spatial Reasoning, Planning & Correction, and Pointing & Location, totaling over 15B tokens.
  • Results: 16 out of 24 Embodied VLM benchmarks achieve SOTA with only 8B parameters, while the model averages 70.4% across 21 main accuracy-based benchmarks.It surpasses Gemini-Robotics-ER-1.5 by 17.0% and GPT-5.4 by 21.7%.
  • Downstream transfer: Downstream action learning outperforms π0.5 by over 20% on Simpl­erEnv Google Robot Visual Matching and ManipLLM by 11% on PartNet-Mobility.The introduction attributes these results to internalized embodied reasoning substituting for action-data scaling and also reports zero-shot real-robot coverage of instruction following, affordance grounding, articulated manipulation, and long-horizon tasks.
  • Closed-loop autonomy: The PGC framework enables autonomous planning, execution, and self-correction on long-horizon complex tasks without human intervention.The system is presented as a unified embodied capability system with closed-loop autonomy.

2 Unified Embodied Capabilities & Architecture

Embodied-R1.5 unifies three complementary embodied-reasoning dimensions—cognition and spatial reasoning, planning and correction, and pointing and location—within one model. Its 8B-parameter VLM represents outputs as plain-text token sequences, including normalized coordinates, trajectories, and free-form reasoning.

  • Unified capability organization: Three embodied capabilities form a progressive reasoning chain from perception to decision to execution within a single model.Unification allows information to flow freely across dimensions without external communication.
  • Embodied Cognition & Spatial Reasoning: Embodied cognition and spatial reasoning provide semantic and spatial understanding of the physical world, including geometric relations and dynamic interaction possibilities.The dimension covers spatial relations such as orientation, support, containment, occlusion, and viewpoint-dependent visibility.
  • Embodied Planning & Correction: Embodied planning and correction span task execution from pre-execution planning through runtime monitoring and post-failure recovery.They include long-horizon task decomposition and next-step planning conditioned on current progress and observation.
  • Embodied Pointing & Location: Embodied pointing and location connect embodied reasoning to physical execution, with strengthened open-vocabulary pointing and physical affordance understanding.The model also treats coordinates as symbols that can be repeatedly referenced and reasoned about.
  • Model architecture: 8B parameters balance strong reasoning capability with practical deployment cost in Embodied-R1.5’s VLM architecture.Outputs use plain-text token sequences: coordinates are normalized to [0, 1000], trajectories are ordered coordinate sequences, and reasoning is free-form text.

3 Training Data Construction

Embodied-R1.5 builds a balanced embodied data system exceeding 15B tokens by integrating existing resources with three automated pipelines targeting capability gaps. Its construction includes structured error-correction data, expanded affordance and trajectory supervision, and general-data regularization.

  • Data system: 34 datasets and more than 15B tokens form the unified embodied data system, combining restructured open-source resources with three automated capability-gap pipelines.The corpus is designed for both large scale and balance across capability dimensions.
  • Error correction: Six QA types organize planning and execution failures across detection, localization, and correction for closed-loop autonomous execution.The cognitive levels correspond to discovering anomalies, diagnosing their source, and generating repairs.
  • Pointing data: ∼400K samples comprise ER1.5-Pointing, expanding automated supervision for functional affordances and trajectory annotations after quality filtering and coordinate normalization.The dataset covers diverse affordance and trajectory annotations, using motion-semantic supervision that remains effective in low-fidelity simulation.
  • Data balancing: General visual cognition, logical reasoning, and instruction-following data regularize the mixture to prevent catastrophic forgetting of general visual understanding.The overall data mixture is balanced across capability categories.

4 Training Strategy

Embodied-R1.5 uses supervised fine-tuning to establish a multi-task foundation, followed by reinforced fine-tuning with verifiable rewards to refine capabilities. Its RL strategy addresses intra- and inter-task imbalance through difficulty-aware data selection, global normalization, adaptive reasoning, and heterogeneous reward designs.

  • Two-stage training: SFT builds foundational capability across heterogeneous outputs, while RFT uses verifiable rewards to refine the model, particularly boosting pointing.The full data system is used for one SFT epoch, while low-resource, high-value tasks are reserved for RL.
  • Supervised fine-tuning: Starting from Qwen3-VL-8B-Instruct, full-parameter SFT jointly optimizes reasoning, coordinates, and trajectories as token sequences with a shared causal-language-modeling objective.The vision encoder is trained jointly with the LLM to adapt visual features to embodied scenarios.
  • Multi-task RL balancing: The RL recipe addresses intra-task difficulty bias and inter-task reward-scale imbalance using medium-difficulty filtering, degenerate-group masking, and mixed-batch reward normalization.The method retains ∼200K medium-difficulty samples and computes advantages with within-group means and entire-batch standard deviations.
  • Reward design: Five reward families match output structures: exact matching, IoU, point distance, trajectory RMSE, and semantic similarity.Continuous rewards use piecewise-linear decay to provide partial credit and denser gradient signals than binary rewards.
  • Task-specific rewards: Point localization uses average nearest-neighbor distance with R=φ(d_nn; 40, 150), point-in-region checks when applicable, and an additional count penalty δ_c=0.3.Trajectory rewards align samples by linear interpolation and combine 2D RMSE with depth MAE using R=0.5 R_2D + 0.5 R_depth.

5 Closed-Loop PGC Autonomy Framework

The PGC framework uses one Embodied-R1.5 model as a unified inference service for planning, spatial grounding, and asynchronous correction during long-horizon execution. Its lightweight harness schedules these roles and manages memory without multi-model cascading or multi-agent orchestration.

  • Unified closed-loop architecture: One Embodied-R1.5 instance asynchronously serves Planner, Grounder, and Corrector roles, eliminating multi-model cascading and multi-agent orchestration.The framework is designed as a minimalist closed-loop system for grounding the three capability dimensions in physics during long-horizon execution.
  • Planning and execution: The Planner decomposes user instructions into structured sub-task sequences and replans after each completed sub-task using the latest observation.Inputs include the user instruction, current image observation, and optional external context such as a task SOP document.
  • Planning and execution: For each sub-task, the Grounder adaptively selects pointing interfaces and produces spatial commands for a low-level skill executor.For example, pressing a tea dispenser switch uses OFG to localize the functional part and VTG to plan the pressing trajectory.
  • Asynchronous correction: The Corrector asynchronously assesses execution from current observations and historical memory, while shared model context improves error attribution and accuracy.It can directly use reasoning established during planning and spatial understanding from grounding, avoiding information loss in cascaded systems.
  • Unified closed-loop architecture: The PGC framework is a lightweight stateless harness that schedules control flow and manages memory rather than containing the embodied capabilities itself.Planning, grounding, and correction are provided by the shared Embodied-R1.5 instance.

6 EmbodiedEvalKit

EmbodiedEvalKit addresses unreliable cross-paper comparison by standardizing embodied evaluation, which existing VLM evaluation tools and ad-hoc scripts cannot adequately support. Its four-layer modular design unifies benchmark data loading and model inference across backends.

  • Motivation: Existing VLM evaluation frameworks target VQA and captioning, while embodied evaluation requires parsing coordinates, bounding boxes, and trajectories.Embodied tasks also involve differing output formats and grounding requirements.
  • Contribution: Ad-hoc scripts and non-overlapping benchmark subsets make cross-paper comparison unreliable; EmbodiedEvalKit provides a unified open-source framework for embodied VLMs.The framework is designed specifically to bridge these evaluation gaps.
  • Architecture: EmbodiedEvalKit uses a four-layer modular design whose data layer standardizes diverse benchmarks in HuggingFace Parquet format and whose inference layer supports multiple model backends.The inference layer supports vLLM, HuggingFace Transformers, and API backends through a model-agnostic interface.

7 Experiments

Embodied-R1.5 achieves state-of-the-art results across embodied benchmarks, while its internalized embodied reasoning transfers effectively to VLA adaptation and zero-shot physical manipulation. Experiments also show robust affordance grounding, correction under perturbations, and gains from reinforced fine-tuning.

  • Embodied benchmark results: 16 out of 24 embodied benchmarks achieve state-of-the-art performance for Embodied-R1.5 across cognition, planning, correction, and pointing.It records dimension averages of 70.2 for Cognition, 65.3 for Planning, and 72.8 for Pointing.
  • VLA adaptation: 92.4% overall on Google Robot Visual Matching establishes Embodied-R1.5-VLA as the top performer in that cross-embodiment setting.On LIBERO-Plus, it reaches 76.0%, outperforming OpenVLA-OFT at 69.6% by +6.4pp and leading on 6 of 7 shift types.
  • Affordance generalization: 0.94 on Kettle, 0.92 on Oven, and 0.90 on Washing Machine demonstrate affordance prediction for unseen articulated-object categories from visual appearance alone.The evaluation covers 30 articulated-object categories and diverse primitives including opening, pressing, pulling, and rotating.
  • Zero-shot manipulation: 65.0% average success rate is achieved with zero-shot manipulation, approaching fine-tuned 𝜋0.5 at 72.9% and surpassing fine-tuned 𝜋0 at 15.0% and RDT at 12.0%.On Click Bell, the model reaches 99% success versus 66% for fine-tuned 𝜋0.5, without in-domain demonstrations.
  • Training ablations: +3.8pp on Pointing, +2.7pp on Planning, and +1.3pp on Cognition are the gains from RFT over an SFT-only variant.Pointing benefits most because geometric rewards provide precise signals, while Planning and Cognition use noisier LLM-as-judge rewards.

8 Related Work

Related work has progressed from EFMs targeting individual embodied capabilities to models that unify reasoning, grounding, and self-evolution. Embodied-R1.5 further unifies these capabilities in one model and addresses heterogeneous training interference with balanced multi-task reinforcement learning.

  • Early Embodied Foundation Models targeted individual capabilities, including spatial reasoning, physical scene understanding, planning, and embodied grounding.
  • Recent models broadened unification across driving, manipulation, embodied reasoning, grounded pointing, and iterative self-evolution.Examples include MIMO-Embodied, ACE-Brain-0, Pelican-VL, RynnBrain, and HY-Embodied.
  • Embodied-R1.5 unifies all three capability dimensions within a single model and introduces multi-task balanced RL to resolve heterogeneous training interference.

9 Conclusion

Embodied-R1.5 unifies three embodied reasoning dimensions in a single 8B-parameter EFM, supported by a 15B-token data corpus, balanced RL, and PGC closed-loop execution. Its VLA extension and zero-shot real-robot experiments demonstrate broad manipulation capability, while native 3D perception and tighter reasoning–action coupling remain open directions.

  • Unified EFM: Embodied-R1.5 integrates three embodied capability dimensions within a shared 8B-parameter Transformer, replacing prior fragmented multi-model systems.The paper presents a unified Embodied Foundation Model and a complete EFM recipe.
  • Training recipe: 15B tokens and three automated data construction pipelines support the EFM, alongside a multi-task balanced RL recipe.The recipe is designed around expanded data coverage and balanced multi-task training.
  • Closed-loop execution: The Planner-Grounder-Corrector framework lets a single model orchestrate the full autonomy stack for closed-loop execution.This framework is presented as a central mechanism for internalizing embodied reasoning and autonomous execution.
  • VLA transfer: 92.4% on SimplerEnv Google Robot Visual Matching marks one result for Embodied-R1.5-VLA, which consistently outperforms π0.5 across 4 manipulation benchmark suites.The VLA extension uses the internalized embodied capabilities and a small amount of data, as described in the paper context.
  • Real-robot evaluation: Zero-shot real-robot experiments cover instruction following, affordance grounding, articulated object manipulation, and long-horizon closed-loop execution.These experiments support the paper’s thesis that embodied reasoning can partially substitute for large-scale action pretraining in the evaluated settings.
  • Limitations and future work: Native 3D perception and tighter coupling between reasoning tokens and action generation remain promising directions, while PGC has been validated on tabletop manipulation.The proposed 3D inputs include point clouds and depth maps for cluttered and occluded scenes.

Contributions … A.3 Embodied Correction Data

Embodied-R1.5 combines contributions spanning dataset construction, evaluation, robotic experiments, embodied cognition, planning, and correction. Its data composition integrates multi-view and 3D reasoning, robot-centric scene cognition, long-horizon planning, and automated failure-correction data.

  • Contributions: Yifu Yuan led the methodology, training dataset, implementation, training, inference, analysis, EmbodiedEvalKit design, and manuscript drafting.Yifu Yuan, Hongyao Tang, and Yi Ma served as co-project leads overseeing project execution.
  • Contributions: Contributors evaluated models, constructed and cleaned data, and conducted real-world robotic platform setup and experiments.The passage names separate contributor groups for evaluation, data work, and robotic experiments.
  • A Data Composition Details: The appendix documents the full data composition details described in Section 3.This appendix covers embodied cognition, planning, and correction data composition.
  • A.1 Embodied Cognition & Spatial Reasoning Data: Multi-view spatiotemporal reasoning data combines VLM-3R, Cambrian-S, and SAT datasets to strengthen embodied perception across temporal inputs and diverse viewpoints.The data targets spatial reasoning and scene cognition for embodied VLMs.
  • A.1 Embodied Cognition & Spatial Reasoning Data: DepthLM data provides sensor-level ground-truth depth from ScanNet++, Taskonomy, HM3D, and Matterport3D for 3D environmental understanding.The data is used to construct depth question-answer pairs from indoor 3D datasets.
  • A.1 Embodied Cognition & Spatial Reasoning Data: Approximately 106K Robo2VLM and RoboBrain 2.0 samples train first-person robot-view scene cognition for manipulation scenarios.The samples cover scene-understanding question-answering from the robot’s manipulation viewpoint.
  • A.1 Embodied Cognition & Spatial Reasoning Data: ER1.5-Spatial contributes ∼20K tabletop samples generated by an automated 3D scene-annotation pipeline from Fractal, BridgeData V2, and DROID.The pipeline converts a single RGB image into a structured 3D semantic scene graph for programmatic spatial-reasoning QA generation.
  • A.2 Embodied Planning Data: Planning data spans long-horizon robot visual QA and first-person action-sequence prediction using RoboVQA, EO-Data, AgiBot-World, Cosmos-Reason, EgoPlan-IT, and EgoRe.The sources cover execution-state assessment, task objectives, decomposition, next-step planning, and prediction from observed manipulation progress.

A.4 Embodied Pointing Data · A.5 General Knowledge Data · B Automatic Data Construction Pipeline

Embodied-R1.5 treats pointing as a signature capability, expanding data for functional affordance and trajectory generation while retaining broad vision-language reasoning. Its automated construction pipelines cover spatial reasoning, correction, functional affordance, and trajectory data.

  • A.4 Embodied Pointing Data: Pointing is Embodied-R1.5’s signature capability, with automated pipelines expanding functional affordance and trajectory-generation data beyond existing open-source resources.The data are organized around four pointing capabilities defined in Section 2.1.
  • A.4 Embodied Pointing Data: REG data train precise localization of target objects from natural-language descriptions using RefCoco, SAM2, Pixmo-Points, CoSyn-Point, LVIS, RoboRefit, and Ref-L4.These sources provide large-scale referring-expression grounding supervision.
  • A.4 Embodied Pointing Data: RRG data support object placement by identifying suitable free regions and assessing feasibility through robotic, spatial, inherited, extended, and human-video grounding sources.Regular Rearrangement simulation adds pattern-completion tasks requiring spatial reasoning and pointing.
  • A.4 Embodied Pointing Data: OFG data localize functional object parts for grasping strategies using open-source affordance annotations, simulation-based extraction, and articulated-object data from ManiSkill.The cited resources include HandAL, PACO-LVIS, InstructPart, PRISM, and PartNet-Maniskill.
  • A.4 Embodied Pointing Data: VTG data depict feasible robot-centric end-effector and object-centric manipulation trajectories, automatically extracted from self-constructed simulations and human interaction data.Trajectory annotations represent expected motion paths as 2D or 3D feasibility curves.
  • A.5 General Knowledge Data: General knowledge samples preserve vision-language capability and prevent catastrophic forgetting across VQA, captioning, reasoning, and multimodal instruction-following tasks.Sources include LLaVA, HONEY, MM-IF, and EUCLID.
  • B Automatic Data Construction Pipeline: The appendix details three automated construction pipelines: ER1.5-Spatial for spatial reasoning, ER1.5-Correction for failure-aware correction, and functional affordance and trajectory data pipelines.These pipelines correspond to the automated data construction framework described in Section 3.

B.1 Pipeline 1: 3D Scene Annotation for Spatial Reasoning Data

The automated pipeline constructs the ∼20K-sample ER1.5-Spatial dataset by converting single RGB tabletop-operation images into structured 3D semantic scene graphs. These annotations support programmatic generation of spatial reasoning QA covering relations, distances, scene cognition, and appearance.

  • Pipeline overview: ∼20K samples comprise ER1.5-Spatial, built by transforming single RGB images into structured 3D semantic scene graphs for tabletop spatial reasoning.The pipeline aggregates object categories, spatial positions, and inter-object relations before generating QA pairs.
  • Pipeline overview: The pipeline targets tabletop manipulation rather than room-level navigation by automatically inferring complete 3D scene information and generating fine-grained spatial reasoning QA.Its input images depict tabletop operation scenes with object categories, spatial positions, and inter-object relations.
  • Annotation pipeline: Each image yields semantic labels, 2D masks, a world-frame point cloud, per-instance 3D bounding boxes, and a camera-to-world transformation through staged processing with embedded quality control.The stages include semantic understanding, geometry estimation, 2D segmentation, 3D lifting, and coordinate normalization.
  • Geometry estimation: Metric-scale MoGe-2 jointly predicts absolute metric depth, surface normals, and camera intrinsics from each RGB image, enabling subsequent 3D reconstruction.The predicted depth is expressed in meters, while normals and intrinsics support geometric lifting and reconstruction.
  • Coordinate normalization: RANSAC-based horizontal-plane alignment places the table at z=0 and aligns the z-axis upward, while scenes failing the 7 cm object-bottom sanity check are removed.The alignment estimates the dominant table-plane normal and height, then verifies that most object bounding-box bottoms lie near z=0.
  • Spatial reasoning QA generation: Dense per-scene metadata enables programmatic QA generation across spatial relations, distance metrics, scene cognition, and appearance.The metadata combines the image, normalized labels, masks, 3D point-cloud segments, intrinsics, bounding boxes, and world transformation.

B.2 Pipeline 2: Failure-Aware Data Construction for Correction Data … C.3 Spatial Reasoning Examples

Embodied-R1.5 expands embodied training data through failure-aware correction, automated affordance grounding, and trajectory extraction pipelines. Qualitative visualizations further cover zero-shot manipulation, pointing, spatial reasoning, and explicit embodied chain-of-thought reasoning.

  • B.2 Pipeline 2: Failure-Aware Data Construction for Correction Data: ∼800K+ samples form ER1.5-Correction, a failure-aware QA dataset spanning simulation and real-world scenarios for failure perception and autonomous correction.The dataset organizes failures along stage and cognitive-level dimensions.
  • B.2 Pipeline 2: Failure-Aware Data Construction for Correction Data: Planning failures target five error types through step deletion, duplication, swapping, object replacement, and action/location replacement perturbations.These operators are applied to correct subtask plans to construct incorrect plans.
  • B.2 Pipeline 2: Failure-Aware Data Construction for Correction Data: Execution failures are constructed by truncating videos, replacing object or action descriptions, and injecting physical perturbations during ManiSkill execution.These strategies create mismatches between instructions and observed video content.
  • B.2 Pipeline 2: Failure-Aware Data Construction for Correction Data: Quality control verifies failure validity, boundary alignment, semantic plausibility, and detection-sample balance before final dataset assembly.The final statistics include BridgeData ∼802K samples across 19 QA subtypes and ManiSkill ∼3.5K samples across 11 tasks.
  • B.3 Pipeline 3: Functional Affordance & Trajectory Data Construction: Automated functional affordance and trajectory pipelines generate data for Object Functional Grounding and Visual Trace Generation.The pipelines address expensive part-level annotation and missing explicit visual trajectory labels.
  • B.3.1 Object Functional Affordance Data: OFG annotations combine simulation-rendered articulated objects with human hand-object contact regions to supervise functional parts and grasp affordances.Simulation maps semantic part labels to image coordinates, while interaction data indicates where and how objects should be grasped for manipulation intents.
  • B.3.2 Trajectory Data: VTG constructs robot-centric end-effector traces and object-centric motion traces in 2D and 3D from robot demonstrations, interaction videos, and simulation.The traces undergo noise, motion, and physical-plausibility filtering, coordinate normalization, and pairing with natural-language instructions.
  • C Qualitative Visualizations: Qualitative visualizations cover zero-shot RoboTwin manipulation, embodied pointing, embodied spatial reasoning, and explicit chain-of-thought reasoning across embodied capabilities.The visualizations correspond to figures 15–19 in the appendix.

C.4 Chain-of-Thought Reasoning Examples

The examples show Embodied-R1.5 reasoning from task instructions and scene relations to 2D target points for manipulation. They also illustrate pointing, spatial reasoning, feasibility planning, state verification, correction, and action understanding across visual inputs.

  • Zero-shot manipulation: In a zero-shot RoboTwin visualization, Embodied-R1.5 predicts pointing locations that are converted into robot actions through unified motion logic without RoboTwin training data.The model identifies grasp points, place targets, and functional affordances across diverse manipulation tasks.
  • Pointing and manipulation reasoning: The pointing examples cover referring-expression grounding, free-space identification, and functional-part localization across manipulation scenes.The qualitative examples include locating points relative to bordered, highlighted, or outlined objects and planes, as well as identifying object regions.
  • Spatial reasoning: The spatial reasoning examples address multi-view spatiotemporal reasoning, depth and metric understanding, and object-level spatial relations.These capabilities are presented as qualitative visualizations under diverse embodied scenes.
  • Pointing and manipulation reasoning: Embodied-R1.5 converts manipulation instructions into 2D target locations while explicitly reasoning about object positions and destinations.Examples cover moving a blue cube onto a yellow cube, placing markers inside containers, stacking blocks, and positioning a can near a bottle.
  • Chain-of-thought reasoning: The chain-of-thought examples span pointing over single images, feasibility planning over egocentric video, state verification and correction, and action understanding over frame sequences.Final decisions are emitted within an <answer> tag.
Loading 2606.11324v2…