PaperlayerView exact revision on arXiv

Source-linked AI summary

Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks

Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Weiming Lu, Peng Li, Yueting Zhuang

arXiv:2503.21696v2cs.CLcs.CV

TL;DR

Embodied reasoning remains difficult because interactive tasks require long-horizon multimodal interaction, spatial and temporal reasoning, and self-reflection beyond the logical deduction emphasized in mathematical reasoning. Embodied-Reasoner addresses this gap with synthesized Observation-Thought-Action trajectories and a three-stage training pipeline, and it outperforms advanced visual reasoning models, especially on complex tasks.

  • Problem

    Existing deep-thinking models have shown strong results in mathematics and coding, but their effectiveness in embodied tasks requiring continuous image-action interaction and long-horizon reasoning remains largely unexplored.

  • Method

    The paper synthesizes coherent Observation-Thought-Action trajectories and trains Embodied-Reasoner through imitation learning, rejection sampling tuning, and reflection tuning.

  • Results

    +9% success rate and +12% search efficiency over OpenAI o1, o3-mini, and Claude-3.7-Sonnet-thinking are reported across 809 tasks in 12 novel scenarios.

  • Takeaways & Limitations

    Embodied-Reasoner shows stronger embodied reasoning and search behavior, with particularly large advantages on complex long-horizon tasks.

Abstract

from arXiv · show

Recent advances in deep thinking models have demonstrated remarkable reasoning capabilities on mathematical and coding tasks. However, their effectiveness in embodied domains which require continuous interaction with environments through image action interleaved trajectories remains largely -unexplored. We present Embodied Reasoner, a model that extends o1 style reasoning to interactive embodied search tasks. Unlike mathematical reasoning that relies primarily on logical deduction, embodied scenarios demand spatial understanding, temporal reasoning, and ongoing self-reflection based on interaction history. To address these challenges, we synthesize 9.3k coherent Observation-Thought-Action trajectories containing 64k interactive images and 90k diverse thinking processes (analysis, spatial reasoning, reflection, planning, and verification). We develop a three-stage training pipeline that progressively enhances the model's capabilities through imitation learning, self-exploration via rejection sampling, and self-correction through reflection tuning. The evaluation shows that our model significantly outperforms those advanced visual reasoning models, e.g., it exceeds OpenAI o1, o3-mini, and Claude-3.7 by +9\%, 24\%, and +13\%. Analysis reveals our model exhibits fewer repeated searches and logical inconsistencies, with particular advantages in complex long-horizon tasks. Real-world environments also show our superiority while exhibiting fewer repeated searches and logical inconsistency cases.

1. Introduction

Embodied-Reasoner extends deep-thinking models to interactive embodied tasks that require long-horizon multimodal interaction, spatial and temporal reasoning, and self-reflection. It combines synthesized reasoning trajectories with a three-stage training pipeline and shows stronger performance, especially on complex tasks.

  • Research Gap: Embodied tasks require continuous visual feedback, coherent reasoning over image-action histories, and contextually consistent actions across long horizons.These requirements differ from single-turn question answering and remain challenging for current multimodal and visual reasoning models.
  • Research Gap: Embodied scenarios require situation analysis, spatial reasoning, temporal recall, planning, reflection, and verification rather than logical deduction alone.The model must infer likely object locations, plan exploration paths, recall previous cues, and adapt after interaction outcomes.
  • Approach: The paper introduces Embodied-Reasoner, which generates diverse thinking processes interleaved with observations and actions to reason from interaction history and spatial layout.Its data engine synthesizes situational analysis, spatial reasoning, self-reflection, task planning, and verification trajectories.
  • Approach: 9.3k task instructions paired with interactive trajectories cover 64k images across 107 indoor scenes, 2,100 objects, and 2,600 containers for model training.The trajectories support four high-level embodied task categories in AI2-THOR: Search, Manipulation, Transportation, and Composite Tasks.
  • Results: +9% success rate and +12% search efficiency over OpenAI o1, o3-mini, and Claude-3.7-Sonnet-thinking are reported across 809 tasks in 12 novel scenarios.The performance gap widens on complex composite tasks, where the model outperforms the second-best model by +39.9%.
  • Results: The model exhibits more consistent reasoning and efficient search by generating more reasoning tokens for complex tasks and avoiding repetitive exploration through temporal reasoning.The reported contributions include improvements over state-of-the-art models, particularly for complex long-horizon tasks.

2. Observation-Thought-Action Corpora

The paper constructs an Observation-Thought-Action corpus for embodied tasks by synthesizing valid instructions, graph-derived action sequences, exploratory searches, and context-aware thinking between observations and actions.

  • Task Design: The task environment emphasizes high-level planning and reasoning for search, manipulation, transportation, and composite tasks rather than low-level motor control.The robot begins with limited visibility in an unknown room, and composite tasks chain multiple transport operations.
  • Instruction Synthesis: Instruction synthesis uses task templates, metadata-based constraints, code filtering, and style and difficulty diversification to generate valid task commands.Constraint checking prevents references to nonexistent objects or illegal actions, while sequentially combining simple tasks creates composite tasks.
  • Action Sequence Synthesis: An affiliation graph links objects through relations such as containment, enabling the engine to derive minimum key-action sequences for task completion.For a hidden object, the graph supports navigation through parent containers before opening, picking up, and placing the object.
  • Action Sequence Synthesis: Additional search processes are inserted into key-action sequences so trajectories model gradual exploration of unfamiliar environments before locating targets.The engine adds exploratory navigation and observation actions after failed searches, making trajectories more realistic and reasonable.
  • Thought Interleaving: Each action produces a first-person visual observation, and generated thoughts are inserted between observations and actions to form an interleaved context.The resulting sequence is represented as Observation-Thought-Action, with thoughts selected from five patterns: situation analysis, task planning, spatial reasoning, self-reflection, and double verification.
  • Thought Interleaving: Thought generation uses the latest observation, previous interaction trajectory, and upcoming action to produce rationales that remain logically consistent with earlier thoughts.This links each reasoning step to both current visual evidence and the trajectory history.

3. Training Recipe for Embodied-Reasoner

Embodied-Reasoner is trained progressively through imitation learning, self-exploration with rejection sampling, and reflection tuning for self-correction. The pipeline moves from basic interaction skills toward adaptive search and recovery from abnormal or erroneous actions.

  • Training Recipe: The three-stage recipe combines imitation learning, rejection sampling tuning, and reflection tuning to bootstrap a VLM into an embodied interactive model with deep thinking.The stages progressively target interaction, exploration, and reflection capabilities.
  • Imitation Learning: In stage one, Qwen2-VL-7B-Instruct is fine-tuned on synthesized instruction trajectories to understand interleaved image-text context and output reasoning and action tokens.The initial trajectories contain limited searching or only key actions, establishing basic interaction ability.
  • Motivation: The training recipe is designed to address limited search behavior after imitation, when the model may fail to continue searching for objects that are not immediately found.The progression adds exploration before introducing self-correction for unreasonable actions and temporary failures.
  • Self-Exploration: Rejection sampling selects successful self-generated trajectories on novel instructions, and 6,246 retained trajectories are used to develop adaptive planning and searching behaviors.The data engine evaluates sampled trajectories as a process-supervision reward model.
  • Self-Correction: Reflection tuning injects navigation and manipulation anomalies into successful trajectories and generates reflective thoughts after incorrect actions or abnormal states.Failed trajectories are revised around their first incorrect action, while erroneous partial trajectories are excluded from the loss.

4. Dataset Statistics

The dataset contains 9,390 trajectory examples across three training stages, broad indoor-scene and object coverage, and diverse reasoning patterns with situation-dependent transitions.

  • Dataset Composition: 9,390 unique instruction-trajectory pairs are distributed across 1,128 imitation, 6,246 exploratory, and 2,016 self-correction trajectories.These counts correspond to the three training stages.
  • Dataset Composition: The dataset spans 107 indoor scenes, 2,100 interactive objects, 2,600 containers, 64K first-person images, and 8M thought tokens.Examples include kitchens and living rooms, with objects such as eggs and laptops and containers such as refrigerators and drawers.
  • Thinking Patterns: Task planning and spatial reasoning are the most frequent thought patterns, occurring 36.6K and 26.4K times respectively.Each trajectory contains about four planning and three reasoning thoughts, while self-reflection occurs about twice per trajectory after failed searches.
  • Thinking Patterns: Thinking-pattern transitions are flexible: task planning is followed by planning 55% of the time and spatial reasoning 45% of the time.Navigation often shifts from action to spatial reasoning at 42%, while failed searches shift from action to self-reflection at 33%.
  • Evaluation Set: The test set contains 809 tasks in 12 novel scenarios, including 25 ultra-long-horizon tasks with four subtasks and 14-27 key actions each.The test tasks are manually designed with annotated key actions and final states.
  • Evaluation Metrics: Success rate measures task completion, search efficiency compares key-action count with predicted-action count, and task completeness measures the proportion of predicted actions that satisfy task requirements.These metrics assess correctness, efficiency, and completeness of generated trajectories.

5. Experiments

Embodied-Reasoner outperforms advanced visual reasoning models, with especially strong results on complex and long-horizon tasks. Its staged training, adaptive reasoning, and reflection reduce inefficient search behaviors, while real-world tests show higher success than o1 and o3-mini.

  • Main Results: +9.6% success rate versus GPT-o1, +24% versus GPT-o3-mini, and +13% versus Claude-3.7-Sonnet-thinking, with higher search efficiency and task completeness.Search efficiency exceeds GPT-o1 by +12%.
  • Main Results: +39.9% on composite tasks versus GPT-4o, while the model lags GPT-o3-mini by 13.4% on simpler search tasks.The paper attributes the simpler-task gap to occasional over-exploration and missed nearby objects.
  • Main Results: 14.7% to 80.9%: three-stage training progressively improves success through imitation learning, rejection sampling, and self-correction trajectories.The stages raise the base model from 14.7% to 25.4%, then 65.4%, and finally 80.9%.
  • Long-Horizon Robustness: Over 60% success rate in most complex-task scenarios, while baseline success rates drop sharply as tasks exceed five key actions.The model remains more robust as task length and interaction demands increase.
  • Long-Horizon Robustness: 1,000 to 3,500 response tokens: Embodied-Reasoner increases reasoning output as tasks become more complex, unlike Gemini-2.0-flash-thinking.The paper links longer analysis and self-reflection with more efficient search paths and fewer redundant actions.
  • Search Behavior: -50% repetitive exploration rate across four tasks, with Embodied-Explorer reaching 26% in composite tasks versus 54% for GPT-o3-mini.The models recall prior observations and reflect on explored actions before forming new plans.
  • Real-world Experiments: 56.7% real-world success, compared with 50% for OpenAI o1 and 43.4% for OpenAI o3-mini.The real-world evaluation covers 30 object-searching tasks across kitchen, bathroom, and bedroom scenes.

6. Related Works

Related work shows that deep-thinking models have advanced text-based reasoning and that newer systems extend this paradigm to multimodal inputs. However, most visual reasoning remains single-round, leaving embodied interactive tasks with continuous image-action interaction insufficiently addressed.

  • Large Language Models Reasoning: Recent o1-style models improve reasoning on college-level mathematics through long thought generation, self-reflection, and self-validation.These systems largely target text-based tasks.
  • Multimodal Reasoning: Multimodal deep-thinking models use long-chain-of-thought post-training or large-scale reinforcement learning, but most operate in single-round image-question-answer settings.This setting differs from embodied tasks that require ongoing interaction.
  • Embodied Agents: Embodied-agent research combines language models with robotic data, visual-language models, function libraries, or traditional reinforcement and imitation learning.These efforts span planning, manipulation, navigation, and generalist robotic strategies.

7. Conclusions

Embodied-Reasoner is designed for interactive search by combining spontaneous search, reasoning, and action with synthesized multimodal trajectories and staged training. Its evaluations and real-world experiments report superior reasoning capabilities.

  • The model is trained on 9,390 Observation-Thought-Action trajectories containing 64K images and 8M thought tokens.
  • A three-stage pipeline uses imitation learning, rejection sampling tuning, and reflection tuning to enhance interaction and reasoning abilities.
  • Extensive evaluations and real-world experiments demonstrate superior reasoning capabilities.

A.1. Real-World Experiments Result

In 30 object-search tasks across three real-world scenes, Embodied-Reasoner generalizes beyond simulator evaluation and achieves higher success rates than three comparison models.

  • 30 real-world object-search tasks span three scenes for evaluating reasoning-model generalization.
  • 12.7% higher success rate than OpenAI o3-mini is reported for Embodied-Reasoner.
  • 13.4% higher success rate than Qwen2.5-VL-72B-Instruct is reported for Embodied-Reasoner.
  • 6.7% higher success rate than OpenAI o1 is reported for Embodied-Reasoner.
  • Repeat Exploration Rate measures revisits to previous locations divided by total explorations.For example, revisiting two locations in five explorations gives 40%.

B.1. Distribution of Task Instructions

The training and test instructions span four task types with varied subtasks and interaction actions, while navigation is the most frequent interaction in both sets.

  • Training task distribution: 9,390 training instructions cover Search, Manipulate, Transport, and Composite tasks with 10 subtask types.
  • Test task distribution: 809 test cases cover four task types and 11 corresponding subtask types.
  • Training interaction distribution: Training trajectories contain eight interaction types: navigate to, pickup, open, close, put in, observe, move forward, and toggle.
  • Training interaction distribution: Navigate to is the most frequent training interaction, occurring over 29k times.
  • Test interaction distribution: Test tasks contain six interaction types: navigate to, pickup, open, close, put in, and toggle.
  • Test interaction distribution: Navigate to also appears significantly more frequently than other test key actions.

B.3. Distribution of Task Length

Trajectory and key-action lengths increase with task complexity: Search tasks are shortest, while Composite tasks have the longest sequences in both training and test distributions.

  • Training trajectories average 7.2 interactions with the environment.
  • Test Composite tasks usually exceed 8 key actions and extend beyond 19.
  • Search training trajectories typically range from 1 to 9 interactions, whereas Composite trajectories usually exceed 8 and extend beyond 23.
  • Transport training trajectories generally range from 3 to 14 interactions.
  • Test Search tasks mainly use 1-2 key actions, while Transport tasks generally use 4-7.
  • The datasets span 107 indoor scenes, over 2,100 interactive objects, and 2,600 containers.

B.5. Description of Sub-task Types

The paper organizes embodied interaction into four task families and multiple subtasks, spanning search, grasping, toggling, and object transfer. It also illustrates task templates, key action sequences, comparative trajectories, and a real-world example.

  • Task Categories: The four task families are Search, Manipulate, Transport, and Composite, with Composite containing Sequential Object Transfer and Long-Term Complex Task.The dataset further divides these families into subtasks based on object exposure, containment, and transfer requirements.
  • Search Subtasks: Exposed Object Search finds surface items without opening containers, whereas Enclosed Object Search requires opening containers to locate the target.Examples include finding an apple on a table versus finding an egg inside a refrigerator.
  • Manipulation Subtasks: Manipulation subtasks include grasping exposed or enclosed objects and toggling exposed objects without opening containers.The examples cover picking up a cup, retrieving a bowl from a cabinet, and starting a coffee machine.
  • Transport Subtasks: Transport subtasks move objects between exposed and enclosed locations, including transfers requiring one or two containers to be opened.The listed variants include exposed-to-exposed, exposed-to-enclosed, enclosed-to-exposed, and enclosed-to-enclosed transfer.
  • Composite Subtasks: Composite tasks include sequential transfers and long-term complex operations that combine multiple ordered object operations.Sequential Object Transfer combines two transfer types, while Long Term Complex Task combines four task types.
  • Examples and Evaluation: The paper provides task templates and constraint checks, key action sequences, and trajectory comparisons between Embodied-Reasoner, GPT-o1, and a real-world deployment.The comparison describes GPT-o1 forgetting an objective and entering repeated action loops, while Embodied-Reasoner completes a real-world milk-placement task through step-by-step planning and reasoning.
Loading 2503.21696v2…