Source-linked AI summary
RoboBrain 2.0 Technical Report
BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Xiansheng Chen, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, Yi Han, Yingbo Tang, Xiangqi Xu, Wei Guo, Yaoxu Lyu, Yijie Xu, Jiayu Shi, Mengfei Du, Cheng Chi, Mengdi Zhao, Xiaoshuai Hao, Junkai Zhao, Xiaojie Zhang, Shanyu Rong, Huaihai Lyu, Zhengliang Cai, Yankai Fu, Ning Chen, Bolun Zhang, Lingfeng Zhang, Shuyi Zhang, Dong Liu, Xi Feng, Songjing Wang, Xiaodan Liu, Yance Jiao, Mengsi Lyu, Zhuo Chen, Chenrui He, Yulong Ao, Xue Sun, Zheqi He, Jingshu Zheng, Xi Yang, Donghai Shi, Kunchang Xie, Bochao Zhang, Shaokai Nie, Chunlei Men, Yonghua Lin, Zhongyuan Wang, Tiejun Huang, Shanghang Zhang
TL;DR
Conventional VLMs focus on general static VQA, while RoboBrain 2.0 addresses embodied spatial, temporal, and long-chain causal reasoning. It combines a modular vision-language architecture with diverse embodied training data and demonstrates strong generalization, including exceptional performance across nine spatial benchmarks.
Problem
Conventional VLMs focus on general static VQA, while RoboBrain 2.0 targets embodied spatial, temporal, and long-chain causal reasoning.
Source: §2 · p. 4Method
Results
Both variants show exceptional performance across nine spatial reasoning benchmarks, and the report describes strong generalization across diverse challenging embodied tasks.
Source: §6.1 · p. 13, §7 · p. 16Takeaways & Limitations
The authors regard RoboBrain 2.0 as a solid foundation toward developing more general embodied AI.
Source: §7 · p. 16The authors found incorrect cases in a benchmark test set and manually screened and excluded them.
Source: §6.1 · p. 14
Abstract
from arXiv · showhide
We introduce RoboBrain 2.0, our latest generation of embodied vision-language foundation models, designed to unify perception, reasoning, and planning for complex embodied tasks in physical environments. It comes in two variants: a lightweight 7B model and a full-scale 32B model, featuring a heterogeneous architecture with a vision encoder and a language model. Despite its compact size, RoboBrain 2.0 achieves strong performance across a wide spectrum of embodied reasoning tasks. On both spatial and temporal benchmarks, the 32B variant achieves leading results, surpassing prior open-source and proprietary models. In particular, it supports key real-world embodied AI capabilities, including spatial understanding (e.g., affordance prediction, spatial referring, trajectory forecasting) and temporal decision-making (e.g., closed-loop interaction, multi-agent long-horizon planning, and scene graph updating). This report details the model architecture, data construction, multi-stage training strategies, infrastructure and practical applications. We hope RoboBrain 2.0 advances embodied AI research and serves as a practical step toward building generalist embodied agents. The code, checkpoint and benchmark are available at https://superrobobrain.github.io.
1 Introduction
RoboBrain 2.0 aims to bridge perception, reasoning, and planning for embodied tasks by combining visual observations with language instructions. It pairs spatial and temporal data contributions with a heterogeneous model and staged training approach.
1 Introduction
- RoboBrain 2.0 targets three embodied-model bottlenecks: limited spatial understanding, weak temporal modeling, and insufficient reasoning chains for dynamic tasks.These gaps hinder spatial-relation and affordance recognition, long-horizon planning and closed-loop control, and alignment of causal logic in instructions with changing environments.Source: §1 · p. 3
- Its data contributions span spatial-task synthesis, multi-robot coordination and planning, closed-loop feedback, and context-conditioned reasoning traces.The spatial pipeline covers pointing, affordance prediction, and trajectory generation; temporal data includes RoboOS coordination templates, externally generated long-horizon trajectories, and randomized failure events, while reasoning VLM traces supervise causal chains across vision, language, and action.Source: §1 · p. 3
- On over 12 public benchmarks, RoboBrain 2.0 achieves state-of-the-art results on 6; its 32B variant mostly leads spatial and temporal benchmarks over open-source and proprietary models.The report evaluates spatial understanding, temporal modeling, and multimodal reasoning.Source: §1 · p. 3
- RoboBrain 2.0 combines a roughly 689M-parameter vision encoder with a 7B or 32B decoder-only language model and a three-stage training curriculum.The stages cover foundational spatiotemporal learning, embodied spatiotemporal enhancement, and chain-of-thought reasoning; FlagScale supports scalable training with hybrid parallelism, memory optimization, high-throughput I/O, and fault tolerance.Source: §1 · p. 3
- The report describes spatial pointing and bounding-box prediction, future-trajectory estimation, scene-graph updating, and interactive long-horizon planning with closed-loop feedback.It presents lightweight 7B and full-scale 32B variants for differing deployment resource constraints.Source: §1 · p. 3, Figure 2 · p. 4
2 Architecture
RoboBrain 2.0 unifies visual and language inputs in a modular encoder-decoder architecture for embodied perception, reasoning, and planning. Its decoder supports varied spatial, textual, and reasoning outputs for tasks from spatial grounding to long-horizon planning.
2 Architecture
- RoboBrain 2.0 combines a tokenizer, vision encoder, MLP projector, and Qwen2.5-VL-initialized language model to process multimodal inputs as a unified token sequence.It supports high-resolution images, multi-view inputs, video frames, language instructions, and scene graphs.Source: §2 · p. 4, Figure 3 · p. 5
2.1 Input Modalities and Tokenization
- Language instructions and scene graphs are tokenized, while multi-view images and video frames are encoded visually and projected into the language model’s token space.Video frames may include timestamp tokens for temporal grounding, and visual tokens from multiple views are concatenated into a unified sequence.Source: §2.1 · p. 5
2.2 Vision Encoder and Projection
- Dynamic-resolution vision processing and frame-wise tokenization encode high-resolution, multi-view, and temporally grounded observations for the language model.The encoder uses adaptive positional encoding and windowed attention; multi-view tokens receive view-specific positional identifiers before fusion.Source: §2.2 · p. 5
2.3 LLM Decoder and Output Representations
- The decoder generates free-form text, spatial coordinates, and optional reasoning traces, supporting tasks from spatial grounding to long-horizon multi-agent planning and causal reasoning.Rotary positional encodings and temporally conditioned tokens help maintain coherence across multi-round perception-action loops.Source: §2.3 · p. 6
3 Training Data
RoboBrain 2.0’s training data spans images, multi-view inputs, videos, scene graphs, and natural-language instructions, organized around general multimodal understanding, spatial perception, and temporal modeling.
3 Training Data
- The dataset combines high-resolution images, multi-view inputs, videos, scene graphs, and language to support embodied spatial understanding, temporal modeling, and long-chain causal reasoning.Figure 4 groups the data into general multimodal understanding, spatial perception, and temporal modeling, with examples including long-horizon planning, closed-loop feedback, spatial prediction, and multi-agent collaboration.Source: §3 · p. 6, Figure 4 · p. 6
3.1 General MLLM VQA
- The general VQA collection contains 873K high-quality samples from LLaVA-665K [33] and LRV-400K [32], covering standard VQA, region queries, OCR, and visual dialogue.After filtering, 531K samples remain from LLaVA-665K and 342K from LRV-400K; the latter uses GPT-4 to generate image-conditioned instructions across 16 vision-language tasks.Source: §3.1 · pp. 6–7
3.2 Spatial Data
- Spatial perception data supports object localization and pointing through LVIS grounding conversations, filtered Pixmo-Points annotations, and RoboPoint [77] reference-point QA.The collection includes 86K grounding conversations; filtering Pixmo-Points [13] yields 190K QA pairs across 64K images, while RoboPoint contributes 347K QA annotations across 288K images, sampled to at most ten points per question.Source: §3.2 · p. 7
- Affordance data covers object functionality and placement regions, using PACO-LVIS [51] annotations and RoboPoint [77] spatial-reference labels.The datasets provide 561K object-affordance QA pairs and 320K spatial-reference QA pairs across 270K images, with 14 spatial relationship labels and answers resampled to at most ten points.Source: §3.2 · pp. 7–8
- Spatial understanding comprises 826K samples across 31 concepts, while 802K spatial-referring samples target unambiguous object localization and placement in physical-world tasks.The data uses 2D images converted into pseudo-3D scene graphs, 3D scene-based and embodied videos, hierarchical captions, and occupancy maps encoding positions, orientations, and metric relations.Source: §3.2 · p. 8
3.3 Temporal Data
- Temporal planning data includes EgoPlan-IT [9] task-progress examples, ShareRobot [23] frame-aligned low-level plans, and AgiBot-World [6] step-by-step manipulation plans.ShareRobot provides 1M QA pairs from 51K instances across 102 scenes, 12 robot embodiments, and 107 atomic tasks; AgiBot Planning provides 9,148 QA pairs across 19 tasks, with 4–17 frames per sample.Source: §3.3 · pp. 8–9
- Multi-robot planning data contains 44,142 generated samples covering 1,659 collaboration-task types across household, supermarket, and restaurant settings.Structured scenarios are decomposed into workflow graphs and agent-specific tool plans that translate long-horizon goals into low-level observation-action pairs.Source: §3.3 · p. 9
- The closed-loop interaction dataset provides synthesized observation-thought-action trajectories for long-horizon tasks across 120 indoor environments and more than 4,000 interactive objects.Its AI2Thor [25] pipeline builds scene-appropriate tasks, derives actions from object-affiliation graphs, adds search actions, and interleaves GPT-4o-generated reasoning with observations and actions.Source: §3.3 · p. 9
4 Training Strategy
RoboBrain 2.0 uses three progressively more complex training stages to develop spatial understanding, temporal modeling, and chain-of-thought reasoning, moving from foundational perception toward dynamic reasoning and planning.
4.1 Stage 1: Foundational Spatiotemporal Learning
- Stage 1 trains spatial perception and temporal understanding on multimodal data, grounding objects, spatial relations, and motion events for egocentric video and spatially anchored instructions.Data include dense captioning, object localization, interleaved image-text documents, video QA, and referring-expression comprehension across common physical scenes and interactions.Source: §4.1 · pp. 9–10
4.2 Stage 2: Embodied Spatiotemporal Enhancement
- Stage 2 adds multi-view and egocentric data, instruction-augmented navigation and interaction, and multi-agent scenarios to support long-horizon planning and adaptive decisions.Tasks include viewpoint-aware referring, 3D affordance estimation, and scene-graph construction; extended sequences and multi-camera encoding fuse visual information across viewpoints.Source: §4.2 · p. 10
4.3 Stage 3: Chain-of-Thought Reasoning in Embodied Contexts
- Stage 3 applies CoT-SFT followed by RFT to multi-turn embodied reasoning examples, using GRPO rewards for answer accuracy and format correctness.CoT-SFT uses GPT-4o [22] rationales for 10% of constructed training data; RFT samples an additional 10%, reformats incorrect responses, and optimizes with GRPO [17], following Reason-RFT [62].Source: §4.3 · p. 10
5 Infrastructures
RoboBrain 2.0’s infrastructure targets efficient and reliable training, reinforcement fine-tuning, and inference. It combines multimodal data and parallel-training optimizations with recovery mechanisms and mixed-bit inference quantization.
5.1 Large-Scale Training Infrastructure
- Training uses hybrid parallelism, memory pre-allocation, distributed loading, kernel fusion, and compute-communication overlap alongside flexible multimodal data handling.A custom WebDataset-based format supports text, images, and video, while preserving sample order within each dataset.Source: §5.1 · p. 11
5.1.1 Multi-Dimensional Hybrid Parallelism
- FlagScale addresses multimodal training heterogeneity with non-uniform pipeline parallelism and vision-only recomputation during annealing.The first pipeline stage receives fewer LLM layers, and recomputation is confined to the ViT when vision inputs reach 20,000–30,000 tokens to reduce activation memory while preserving LLM compute efficiency.Source: §5.1.1 · p. 11
5.1.2 Pre-Allocate Memory
- Padding samples to the dataset’s maximum sequence length before training lets tensors reuse pre-allocated memory and reduces allocator fragmentation and OOM errors.This strategy addresses variable input lengths during supervised fine-tuning without emptying the CUDA cache before every forward pass.Source: §5.1.2 · p. 11
5.1.3 Data Pre-Processing
- Preprocessing JSON only takes less than 10 minutes for 320,000 samples, versus over 2 hours when preprocessing both JSON and images.Images stay in their original form and are processed by TaskEncoder using the same preprocessor as Qwen2.5-VL, aligning image handling between preprocessing and training.Source: §5.1.3 · pp. 11–12
5.1.4 Distributed Data Loading
- Restricting data loading to required processes reduces redundant I/O in multidimensional distributed training.Only the first and last pipeline stages load data, and within each tensor-parallel group one GPU loads data for broadcast to the others.Source: §5.1.4 · p. 12
5.1.5 Fault Tolerance
- FlagScale and the custom DataLoader support recovery from hardware or software failures, resuming from the latest checkpoint with data-loading and shuffling states preserved.Errors such as LostCard and KubeNodeNotReady trigger automatic job recovery and restart.Source: §5.1 · p. 11, §5.1.5 · p. 12
5.2 Reinforcement Fine-Tuning Infrastructure
- VeRL supports RoboBrain 2.0 reinforcement fine-tuning with RLVR through scalable, parallel post-training workflows and verified rewards.Based on HybridFlow, VeRL coordinates dataflow with global and distributed controllers and supports algorithms such as GRPO.Source: §5.2 · p. 12
5.3 Inference Infrastructure
- Approximately 30% lower inference latency is achieved by weight-only quantization in end-to-end embodied tasks, with negligible impact on model accuracy.The vision encoder remains full precision; language-module weights use 8-bit integers and activations use 16-bit floating point. FlagScale searches for inference engines and configurations suited to the model and hardware.Source: §5.3 · p. 12
6 Evaluation Results
Using FlagEvalMM [20], the evaluation covers spatial and temporal reasoning benchmarks. RoboBrain-2.0’s leading results vary by task and model size.
6.1 Spatial Reasoning Capability
- RoboBrain-7B-2.0 leads BLINK [15] (83.95) and CV-Bench [67] (85.75), while RoboBrain-32B-2.0 reaches near-SOTA EmbSpatial [14] accuracy (78.57).The BLINK and CV-Bench scores are reported as SOTA; Gemini-2.5-Pro-preview-05-06 scores 78.74 on EmbSpatial.Source: §6.1 · p. 13, Table 2 · p. 13
- RoboBrain-32B-2.0 achieves SOTA scores on RoboSpatial [59] (72.43), RefSpatial-Bench [81] (Loc. 54.00; Pla. 54.00), and SAT [53] (86.67).RoboBrain-7B-2.0 scores 54.23 on RoboSpatial and 36.00 (Loc.) and 29.00 (Pla.) on RefSpatial-Bench.Source: Table 2 · p. 13, §6.1 · p. 14, Table 3 · p. 14
- RoboBrain-32B-2.0 leads Where2Place [77] (73.59 average), ShareRobot-Bench Affordance [23] (35.28 accuracy), and ShareRobot-Bench Trajectory [23] (DFD 0.2368).Where2Place scores are 73.95 Seen and 72.74 Unseen; lower DFD indicates better trajectory performance.Source: §6.1 · pp. 14–15, Table 3 · p. 14
- On VSI-Bench [75], RoboBrain-32B-2.0 scores 42.69, below Gemini-2.5-Flash-preview-04-17 (48.83), while outperforming most general and embodied baselines.RoboBrain-7B-2.0 scores 36.10.Source: §6.1 · p. 14, Table 3 · p. 14
6.2 Temporal Reasoning Capability
- Temporal results vary by task: RoboBrain-32B-2.0 scores 57.23 on Ego-Plan2 [9], while RoboBrain-7B-2.0 scores 72.16 on RoboBench; Multi-Robot Planning [61] reports 81.50 (7B) and 80.33 (32B).The report identifies 32B as SOTA on Ego-Plan2 and 7B as SOTA on RoboBench.Source: §6.2 · pp. 15–16, Table 4 · p. 15
7 Conclusion and Future Works
RoboBrain 2.0 combines modular vision-language architecture and three-stage training to support embodied perception, reasoning, and planning, and achieves state-of-the-art results on most public spatial and temporal reasoning benchmarks. Future work targets VLM-powered action generation and tighter robotics-system integration.
7 Conclusion and Future Works
- RoboBrain 2.0 achieves state-of-the-art results on most public embodied spatial and temporal reasoning benchmarks, outperforming open-source and proprietary models in spatial understanding, closed-loop interaction, and long-horizon planning.Its capabilities include affordance prediction, spatial referring, trajectory forecasting, multi-agent coordination, and scene graph construction and updating.Source: §7 · p. 16
- A modular vision encoder and decoder-only language model support high-resolution image and video comprehension, while three-stage training develops spatiotemporal learning, embodied capabilities, and chain-of-thought reasoning.The report describes the model as a foundation for more general embodied AI, emphasizing tightly integrated perception, reasoning, and planning.Source: §7 · p. 16
- Future work will integrate embodied VLMs into the VLA framework to improve action-generation generality and robustness for complex, open-ended real-world instructions.The intended system combines VLM spatiotemporal perception and high-level reasoning with more precise action execution.Source: §7 · p. 16
- Planned system integration targets serverless deployment, adaptation-free skill registration, and low-latency real-time control, alongside a plug-and-play embodied AI ecosystem for perception, reasoning, and control.RoboBrain 2.0 is released with model checkpoints, training recipes, and evaluation tools to support research and downstream applications.Source: §7 · p. 16
8 Contributions and Author List · Appendix · A Qualitative examples
The section presents qualitative examples of RoboBrain 2.0 across embodied AI tasks, illustrating spatial reasoning, temporal planning, and interactive reasoning, as well as potential for real-world applications.
A Qualitative examples
- The qualitative examples span various embodied AI tasks to illustrate RoboBrain 2.0’s capabilities.The section presents them as examples of the model’s potential for real-world applications.Source: A Qualitative examples · p. 23
- The examples illustrate RoboBrain 2.0’s proficiency in spatial reasoning.Source: A Qualitative examples · p. 23
- The examples illustrate RoboBrain 2.0’s proficiency in temporal planning.Source: A Qualitative examples · p. 23
- The examples illustrate RoboBrain 2.0’s proficiency in interactive reasoning.Source: A Qualitative examples · p. 23
A.1 Examples for Pointing
RoboBrain 2.0 identifies and points to objects specified by simple or complex spatial instructions, with examples spanning object attributes, spatial relationships, affordances, and free space.
A.1 Examples for Pointing
- RoboBrain 2.0 accurately points to objects named directly or specified through spatial relationships, including an orange box and a brown box on a shelf.The examples illustrate both simple object identification and more intricate spatial instructions.Source: §A.1 · p. 23, Figure 5 · p. 23
- Examples show the model pointing to objects or their parts according to queried affordances, and to objects referred to by spatial relations or attributes.These examples are presented in Figures 16 and 17.Source: Figure 16 · p. 34, Figure 17 · p. 35
- Examples also show the model pointing out free space indicated by spatial relations, as well as the referenced objects.This example type is shown in Figure 19.Source: Figure 19 · p. 37
A.2 Examples for Affordance … A.5 Examples for Close-Loop Interaction
RoboBrain 2.0 examples illustrate object-affordance understanding, constrained trajectory planning, multi-step activity decomposition, and feedback-driven interaction. The examples span both individual actions and longer sequences of tasks.
A.2 Examples for Affordance
- The affordance examples show RoboBrain 2.0 identifying an object's functional parts, such as a mug's liquid-holding interior and a handbag's graspable handle.These examples assess understanding of object functionalities and interaction possibilities.Source: §A.2 · p. 40
A.3 Examples for Trajectory
- For trajectory planning, RoboBrain 2.0 generates a smooth path to grasp an apple while accounting for spatial constraints and obstacles.The report describes its planned trajectories as optimal and collision-free.Source: §A.3 · p. 42
- The trajectory example presents path prediction as navigation for a robot arm instructed to grasp an apple.The report characterizes this capability as spatial and temporal reasoning for navigation tasks.Source: §A.3 · p. 42
A.4 Examples for EgoPlan2
- In EgoPlan2, RoboBrain 2.0 decomposes making coffee into actionable steps, from locating a coffee machine and fetching beans to brewing.The example is presented as sequential planning from an egocentric perspective.Source: §A.4 · p. 44
- EgoPlan2 illustrates long-horizon planning by outlining and executing a detailed sequence for preparing coffee.The report links this example to task decomposition and sequential planning.Source: §A.4 · p. 44
A.5 Examples for Close-Loop Interaction
- In the close-loop example, RoboBrain 2.0 adapts actions through real-time feedback while searching for a mug and operating a coffee machine to pour coffee.The task requires repeated navigation and search before completing the pour.Source: §A.5 · p. 47
- The close-loop examples also depict multi-step object-placement tasks, including moving a pen to a box and a pillow to an armchair.The supplied task descriptions include further sequences involving a mug, potato, egg, sink, fridge, and garbage.Source: §A.5 · pp. 47, 49
A.6 Examples for Multi-Robot Planning
RoboBrain 2.0 demonstrates multi-robot coordination across restaurant, household, and supermarket tasks. Its planning assigns tasks and coordinates robot movements toward shared goals.
A.6 Examples for Multi-Robot Planning
- In supermarket restocking, RoboBrain 2.0 assigns tasks, coordinates robot movements to avoid collisions, and aims to achieve the shared goal in a timely manner.The examples are presented as evidence of multi-agent coordination and long-horizon planning.Source: §A.6 · p. 51
- Across three scenarios, RoboBrain 2.0 coordinates robots for burger preparation and delivery, fetching household items, and gift selection and packaging.The examples feature scene-aware task decomposition in the restaurant, commands such as “Give me an orange and a knife” in the household, and gift placement and bag-opening skills in the supermarket.Source: Figure 33 · p. 51, §A.6 · p. 51
A.7 Examples for Synthetic Benchmarks · B Prompts Details
The appendix illustrates RoboBrain 2.0’s performance on synthetic spatial benchmarks and details prompts for spatial reasoning, temporal planning, and multi-robot coordination. The prompts use concise, stepwise visual-spatial reasoning and task-specific answer formats.
A.7 Examples for Synthetic Benchmarks
- RoboBrain 2.0 performs well across synthetic benchmarks, including BLINK depth and spatial relations and CV-Bench 3D scene reasoning.The examples show selected objects or points for the respective benchmarks.Source: §A.7 · p. 52, Figure 34 · p. 52, Figure 35 · p. 53
B Prompts Details
- Prompts require concise 200–500-word, five-stage reasoning in <think> and <answer> tags, using direct visual-spatial language and maintaining the robot’s role.Tasks simulate visual-spatial perception and request step-by-step reasoning.Source: B Prompts Details · p. 54
B.1 Spatial Understanding: Coordinates -- Pointing
- Pointing prompts guide the robot to analyze an outlined plane, verify distinct points lie in vacant regions, and return normalized image coordinates as tuples.The example asks for a list of (x, y) tuples.Source: §B.1 · p. 54
B.2 Spatial Understanding: Coordinates -- Trajectory
- Trajectory prompts ask the robot to predict up to 10 key joint-control points that reach a target while avoiding obstacles, outputting coordinate tuples.The example task is reaching a banana on a plate.Source: §B.2 · p. 54
B.3 Spatial Understanding: Bounding Box -- Affordance
- Affordance prompts require assessing object features, gripper compatibility, task relevance, and reachability before returning an interaction-area bounding box.The example asks for an affordance area for holding a coffee mug.Source: §B.3 · pp. 54–55
B.5 Temporal Understanding: Long-horizon Planning
- Long-horizon planning prompts use action history and the current scene to assess task progress and choose the next action consistent with the goal.In the cooking example, the robot chooses an action such as pouring oil after onions are prepared.Source: §B.5 · p. 55
B.6 Temporal Understanding: Closed Loop Conversation
- Closed-loop conversation prompts combine prior visual inputs and responses with the current view and task goal to select the next action.The example asks what to do after opening the fridge, with an example answer to turn on the coffee machine.Source: §B.6 · p. 55
B.7 Temporal Understanding: Multi-Robot Planning
- Multi-robot planning prompts coordinate robots through global task decomposition and agent-based tool calling, with a task-flow graph as the example output.The prompts are illustrated in Figures 36 and 37.Source: §B.7 · p. 55, Figure 36 · p. 56, Figure 37 · p. 57
B.4 Spatial Understanding: Freeform Q&A -- General Spatial Analysis
RoboBrain answers questions about spatial relationships and action outcomes from one or more images through a staged, visually grounded reasoning process.
B.4 Spatial Understanding: Freeform Q&A -- General Spatial Analysis
- The robot answers questions about spatial relationships or action outcomes using one or more images.Source: §B.4 · p. 55
- Its reasoning identifies prominent scene features and their spatial arrangement, interprets the question’s objective, and analyzes relevant visual elements or projected actions.Source: §B.4 · p. 55
- It synthesizes observations into a hypothesis, evaluates provided options, and derives an answer justified by visual evidence and logical reasoning.Source: §B.4 · p. 55
Related papers
This paper cites
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning2025
- DeepSeek-V3 Technical Report2024
- HybridFlow: A Flexible and Efficient RLHF Framework2024
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations2016
- SAM 2: Segment Anything in Images and Videos2024
- Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs2024
Cited by
- Look, Reason, Draw a Region: Recursive Refinement for Spatial Context Prediction2026
- CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness2026
- Anvil: Scaling Embodied Spatial Intelligence through Knowledge Recirculation2026
- ABot-ER: Unified Embodied Reasoning through Agentic Data-Model Co-Evolution2026
- DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination2026
- Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis2026