PaperlayerView exact revision on arXiv

Source-linked AI summary

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

Kehan Li, Bohan Hou, Minghao Zhu, Tianyi Zhang, Zesen Cheng, Zhikai Wang, Sicong Leng, Xin Li, Xiao Lin, Biying Yao, Minghua Zeng, Jiangpin Liu, Ronghao Dang, Jiayan Guo, Siteng Huang, Haoyu Zhao, Heng Ping, Yaxi Zhao, Tong Zhao, Kexiang Wang, Tong Lu, Shengke Xue, Jiahao Tang, Yulei Wang, Zejing Wang, Jianwei Gao, Shijian Lu, Chengju Liu, Jianfei Yang, Mingxiu Chen, Deli Zhao

arXiv:2607.17977v2cs.RO

TL;DR

Embodied foundation models must connect visual and language understanding to 3D structure, spatial grounding, and executable robot behavior, while RynnBrain 1.0 left manipulation alignment and downstream VLA initialization underexplored. RynnBrain 1.1 addresses these gaps with contact-point prediction, native 3D grounding, and a unified cross-embodiment VLA, achieving state-of-the-art benchmark performance and stronger real-robot policies with joint training.

  • Problem

    Embodied foundation models still need stronger 3D reasoning, physical grounding, manipulation alignment, and downstream VLA initialization than prior multimodal systems provide.

  • Method

    RynnBrain 1.1 scales unified spatio-temporal and physically grounded pretraining across three model sizes, adding contact-point prediction, native 3D grounding, and a masked cross-embodiment action space.

  • Results

    The 122B-A10B model surpasses all evaluated proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench, while joint multi-task and multi-embodiment training improves real-robot performance over per-task policies.

  • Takeaways & Limitations

    RynnBrain 1.1 provides a scalable embodied foundation model family with manipulation-oriented outputs and evidence that shared training transfers across heterogeneous robot embodiments and tasks.

  • Takeaways & Limitations

    Contact-point prediction lacks a standardized metric and is therefore evaluated qualitatively rather than through a reliable functional-validity measure.

Abstract

from arXiv · show

We present RynnBrain 1.1, a family of embodied foundation models spanning 2B, 9B, and 122B-A10B scales. Trained with a unified spatio-temporal and physically grounded framework, RynnBrain 1.1 supports embodied perception, spatial reasoning, localization, and planning. Compared with RynnBrain 1.0, it further introduces contact-point prediction across the model family and native 3D grounding for the 2B and 9B models, yielding representations and outputs that are more directly aligned with robot manipulation. We also develop RynnBrain-VLA with a unified cross-embodiment action space and embodiment-specific masking, and deploy it on Unitree G1, Astribot-S1, and Tianji-Wuji. RynnBrain 1.1 achieves strong results on embodied cognition, localization, and 3D grounding, with the 122B-A10B model outperforming all evaluated proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench. Real-robot experiments show that RynnBrain-initialized policies outperform Qwen-based and representative generalist VLAs, while joint multi-task and multi-embodiment training improves process scores and success rates over per-task training.

1 Introduction

RynnBrain 1.1 extends embodied foundation modeling beyond visual-language understanding by unifying physically grounded perception, spatial reasoning, localization, 3D grounding, and planning across model scales. It adds manipulation-aligned capabilities and a cross-embodiment VLA, achieving strong benchmark and real-robot results.

  • Embodied intelligence requires object perception, spatial reasoning, actionable-region identification, and translation of understanding into executable robotic behaviors.
  • RynnBrain 1.1 spans 2B, 9B, and 122B-A10B models trained under a unified capability definition and training framework.The consistent design supports systematic study of capability changes with parameter scale.
  • RynnBrain 1.1-122B-A10B surpasses all proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench.Evaluations also demonstrate an upward trend in model capabilities as parameter scale increases.
  • RynnBrain-VLA combines a unified cross-embodiment action space with embodiment-specific masking and gains from joint multi-task, multi-embodiment training over separately fine-tuned per-task policies.It delivers strong real-world performance across heterogeneous embodiments, tasks, and control interfaces.

2 Overview

RynnBrain 1.1 is a decoder-only vision–language model family built on Qwen3.5 design principles, with shared architecture and training across 2B, 9B, and 122B-A10B scales. Its infrastructure extends the prior large-scale embodied multimodal training pipeline to this family while using sparse MoE training for the 122B-A10B model.

  • Architecture: RynnBrain 1.1 uses a decoder-only vision–language architecture with a vision encoder, vision–language projector, and Qwen3.5-initialized LLM backbone.The architecture follows Qwen3.5 design principles.
  • Model scales: The release provides three scales—RynnBrain 1.1-2B, RynnBrain 1.1-9B, and RynnBrain 1.1-122B-A10B—with shared architecture and training formulation.This supports systematic study of embodied scaling across model sizes.
  • Training infrastructure: The training infrastructure extends the prior pipeline to the Qwen3.5-based 2B, 9B, and 122B family while keeping the same pipeline across model sizes.The pipeline includes sequence-length-aware load balancing, per-sample loss reduction, ZeRO, gradient checkpointing, and optimized MoE execution.
  • Training infrastructure: The 122B-A10B model uses sparse MoE training with expert parallelism and communication-efficient token dispatching.These choices follow infrastructure principles established in RynnBrain.

3 Pretraining

RynnBrain 1.1 pretraining unifies spatio-temporal memory with physically grounded spatial prediction in a single autoregressive framework. It extends this framework with native 3D grounding for the 2B and 9B models and capability-oriented data spanning cognition, localization, robot perception, and planning.

  • Unified spatio-temporal and physical grounding: The framework jointly models temporal visual information and explicit spatial outputs, including objects, regions, affordances, trajectories, and contact-related predictions.Multimodal inputs and language or spatial targets share one autoregressive prediction interface rather than separate task-specific decoders.
  • Unified spatio-temporal and physical grounding: RynnBrain 1.1 represents images and videos as ordered visual-token sequences with temporal position information, enabling cross-frame modeling of persistence and motion.Static images use T = 1, while videos use uniformly sampled frames with temporal ordering.
  • Native 3D grounding: 2B and 9B pretraining adds native 3D grounding by predicting camera-coordinate boxes with 9 parameters: center, dimensions, and orientation in physical units.The inputs include a natural-language instruction and camera intrinsics; outputs use (cx, cy, cz), (w, l, h), and (pitch, yaw, roll).
  • Capability-oriented data: The pretraining mixture covers general multimodal understanding, embodied cognition, spatio-temporal localization, 3D grounding, robot-oriented perception, and planning.Localization supervision includes object, area, affordance, trajectory, and grasp/contact-related prediction.
  • Robot-oriented perception: Contact-point supervision provides an action-relevant spatial anchor and orientation cue while leaving gripper aperture, collision constraints, and robot kinematics to downstream manipulation policies.This avoids treating a grasp rectangle as a uniquely canonical image-space target whose extent depends on gripper geometry and annotation conventions.

4 Post-training for Vision-Language-Action Model

RynnBrain-VLA combines a diffusion-transformer policy with native conversational inputs, a shared masked action space, and a cross-embodiment deployment interface. Real-Time Chunking and standardized fine-tuning support low-latency control and controlled backbone comparisons.

  • Policy architecture: RynnBrain-VLA uses a single-stream diffusion transformer with dense vision-language-action interaction and flow matching to predict action chunks.Actions are placed at the sequence end to enable KV caching during inference, while inputs follow the VLM’s native conversation format.
  • Unified action space: A shared high-dimensional action space partitions controls into semantically aligned groups, with each embodiment activating only physically available dimensions.The action loss is computed only on active dimensions, enabling trajectories from different robots to be mixed in one training batch.
  • Cross-embodiment deployment: The cross-embodiment framework separates model, control, and embodiment layers, allowing new robots to be added by implementing only the embodiment layer.It is instantiated on the Unitree G1, Astribot-S1, and Tianji-Wuji, with a 32-step model chunk, 30 Hz coordination, and 200 Hz interpolation.
  • Real-Time Chunking: Real-Time Chunking triggers new inference every 5 steps while a 32-step chunk executes, using remaining actions to guide denoising with β = 10.0.The method follows the action-guidance formulation of Black et al. and assigns guidance weights according to which previous actions will be consumed before the new chunk arrives.
  • Fine-tuning setup: Both Qwen-based VLA and RynnBrain-VLA policies are fine-tuned for 60k steps with learning rate 2 × 10−5 and batch size 32 using the same demonstrations and settings.Input images are proportionally resized with the short side set to 384 pixels to reduce downstream-training differences in the backbone comparison.

5 Evaluation

The evaluation assesses RynnBrain 1.1 across embodied cognition and localization benchmarks at multiple model scales, comparing it with prior RynnBrain versions and other embodied or general-purpose models.

  • Evaluation scope: Evaluation begins with embodied cognition and localization benchmarks across three model scales.The paper organizes evaluation into several parts and first assesses the base models.
  • Base-model comparisons: At 2B, RynnBrain 1.1 is compared with RynnBrain 1.0, Cosmos 3-Edge, SenseNova-SI-1.1-InternVL3-2B, Cosmos-Reason2-2B, and Qwen3.5-2B.These comparisons cover the listed embodied-model and general-purpose baselines at the 2B scale.
  • Base-model comparisons: At 4B–9B, comparisons include RynnBrain 1.0, Cosmos 3-Nano, RoboBrain-2.5-4B, Thinker-4B, Molmo2-ER-5B, Pelican-VL-7B, MiMo-Embodied-7B, Cosmos-Reason2-8B, SenseNova-SI1.5-8B, and Qwen3.5.The passage lists the evaluated baselines for this model-scale range.
  • Base-model comparisons: The evaluation also includes comparisons at the largest model scale, though the supplied passage truncates the corresponding baseline list.The available text ends after “Cosmos 3-” and does not specify the remaining comparisons.

5.1 Embodied Cognition Capability

RynnBrain 1.1 is evaluated across diverse embodied cognition and spatial-understanding benchmarks, substantially improving over earlier RynnBrain models, with more systematic gains at 9B than 2B. At 2B, improvements are task-dependent and several RoboSpatial and EmbSpatial results remain below RynnBrain-2B.

  • Evaluation scope: The evaluation spans video-based spatial intelligence, multi-view reasoning, embodied question answering, robot-oriented spatial understanding, and object-centric cognition across eight benchmarks.The benchmarks include VSI-Bench, MMSI, ERQA, RoboSpatial-VQA, MindCube, EmbSpatial, RynnBrain-Object, and RynnBrain-Spatial.
  • Scale-dependent results: At 2B, MMSI rises from 34.1 to 40.5, MindCube from 50.1 to 61.7, and RynnBrain-Spatial from 57.2 to 63.8.The improvements are task-dependent, with several RoboSpatial and EmbSpatial results remaining below RynnBrain-2B.
  • Scale-dependent results: At 9B, RynnBrain 1.1 outperforms RynnBrain-8B on eight of ten shared benchmarks.The gains are described as more systematic at 9B than at 2B.

5.2 Embodied Localization Capability

RynnBrain 1.1 is evaluated on six embodied localization tasks spanning spatial reference, object and region localization, affordance grounding, and trajectory prediction. It improves over previous RynnBrain models at both 2B and 9B scales, with reported gains at 2B and larger gains on 9B relational and motion benchmarks.

  • Evaluation scope: Six tasks assess language-guided spatial reference, object and region localization, affordance grounding, and trajectory prediction.The evaluation includes RefSpatial-Bench, RoboSpatial-Pointing, RynnBrain-Grounding, RynnBrain-Area, RynnBrain-Affordance, and RynnBrain-Trajectory.
  • 2B results: At 2B, RefSpatial-Bench increases from 52.7 to 58.5.This is an improvement over the previous RynnBrain model on the location benchmark.
  • 2B results: At 2B, RynnBrain-Grounding increases from 79.1 to 84.1.This is another improvement over the previous RynnBrain model on a location benchmark.
  • 9B results: At 9B, improvements become larger on relational and motion benchmarks.The supplied passage does not report the corresponding 9B metric values.

5.3 Scaling Analysis

RynnBrain 1.1 and matched-size Qwen3.5 baselines show non-uniform embodied scaling across general cognition, reasoning-intensive cognition, and localization. Embodied pretraining raises performance floors and enables reliable scaling gains, especially where Qwen3.5 degrades.

  • Experimental setup: Scaling analysis compares RynnBrain 1.1 at 2B, 9B, and 122B-A10B with matched-size raw Qwen3.5 baselines across three embodied capability groups.The groups are general embodied cognition, reasoning-intensive cognition, and embodied localization.
  • Scaling regimes: General embodied cognition improves monotonically for both models as scale increases, while their performance gap steadily narrows.This group covers recognition, comprehension, spatial understanding, and object-centric reasoning tasks.
  • Scaling regimes: +38.6% for RynnBrain 1.1 on reasoning-intensive cognition contrasts with −39.2% for Qwen3.5, widening the gap from 18.2 to 50.8.The reasoning-intensive group involves multi-view spatial reasoning and video-based temporal inference.
  • Embodied pretraining: RynnBrain 1.1 consistently establishes a higher performance floor across general cognition, reasoning-intensive cognition, and embodied localization.Figure 4 decomposes scaling into the 2B performance floor and the gain from scaling to 122B-A10B.
  • Embodied pretraining: On reasoning-intensive tasks, embodied pretraining converts degradation under pure VLM scaling into reliable gains with increasing scale.The comparison indicates that embodied data determines whether scaling helps or hurts in this capability category.

5.4 3D Grounding

RynnBrain 1.1 provides native language-conditioned 3D grounding at 2B and 9B scales, with performance improving substantially through model scaling. It localizes objects and predicts their spatial extent, orientation, and oriented 3D bounding boxes across diverse indoor scenes.

  • SUN RGB-D: 34.28 AP@15 from RynnBrain 1.1-2B on SUN RGB-D surpasses Seed1.5-VL (33.5) and Gemini 2.0 Pro (32.5).Scaling to 9B raises performance to 41.12 AP@15, narrowing the gap to Gemini Robotics-ER (48.3).
  • WildDet3D-Bench: 23.44AP3D from RynnBrain 1.1-9B on WildDet3D-Bench surpasses the specialized WildDet3D detector trained with additional in-domain data (22.6).RynnBrain 1.1-2B obtains 17.36 AP3D, showing a scaling benefit alongside explicit 3D supervision.
  • Qualitative Results: RynnBrain 1.1 localizes target objects and estimates spatial extent and orientation from a single image, including beds, tables, and chairs.Qualitative examples demonstrate physically grounded 3D predictions across diverse viewpoints and scene layouts.
  • Qualitative Results: Given an image and target object description, RynnBrain 1.1 predicts an oriented 3D bounding box in camera coordinates across varied scales, shapes, viewpoints, and occlusions.The examples cover indoor environments with different object properties and visibility conditions.

5.5 Contact Point Prediction

RynnBrain 1.1 is evaluated qualitatively for contact-point prediction because no standardized metric reliably captures functional validity. Visualized cases show task-relevant contact locations, appropriate orientations, and target discrimination across varied objects and settings.

  • Evaluation: No standardized metric reliably reflects the functional validity of predicted contact points, motivating qualitative evaluation.Pixel-level distance can penalize equally valid alternatives, while conventional grasp-box metrics do not directly apply to the contact-centered representation.
  • Qualitative Results: RynnBrain 1.1 identifies task-relevant contact locations rather than merely predicting object centers.For a mug, it selects the handle when asked to pick it up and adapts its prediction when the mug is overturned.
  • Qualitative Results: The model produces appropriate contact locations and orientations for thin objects, boxes, and irregularly shaped items.These examples span objects with substantially different shapes and geometries.
  • Qualitative Results: RynnBrain 1.1 distinguishes the instructed target in cluttered or referential settings.The visualized cases demonstrate target selection beyond simple object-center prediction.

5.6 Real-Robot Evaluation

RynnBrain-VLA demonstrates stable real-robot performance across heterogeneous embodiments and control interfaces. It also benefits from embodied-model initialization and joint multi-task, multi-embodiment training, improving both process scores and final success rates.

  • Evaluation Protocol: Evaluation uses 20 randomized trials per task, reporting final success rate and a process score for completed sub-tasks.The process score captures intermediate progress when long-horizon execution does not reach the final goal.
  • Cross-Embodiment Performance: RynnBrain-VLA achieves stable performance across Astribot, Tianji-Wuji, and Unitree G1 platforms despite differing action spaces and control interfaces.The evaluation spans gripper-based whole-body manipulation, dexterous grasping, bimanual coordination, and humanoid whole-body control.
  • Cross-Embodiment Performance: 90% success rate on Pull the Chair exceeds 75% for GR00T N1.7 combined with the same SONIC controller.The task involves coordinated humanoid whole-body motion on Unitree G1.
  • Foundation-Model Initialization: 91.28% average process score and 86.67% average final success rate improve over Qwen-Based-VLA’s 68.33% and 60.00%, respectively.Both policies use the same VLA post-training recipe and action-chunk length, differing in initialization from RynnBrain and Qwen Based Model.
  • Joint Training: 94.14% average process score and 91.67% final success rate for RynnBrain-VLAGeneralist improve over separately fine-tuned policies’ 91.28% and 86.67%.Joint training uses unified action-space masking to combine multiple tasks and embodiments without obvious interference.

6 Conclusion and Future Work

RynnBrain 1.1 establishes a scalable embodied foundation-model family with unified training, 3D grounding, contact-centered interaction grounding, and real-world VLA deployment. Future work targets a unified multimodal embodied agent integrating memory, world modeling, planning, active perception, and closed-loop interaction.

  • Conclusion: RynnBrain 1.1 spans 2B, 9B, and 122B-A10B scales under a unified framework for embodied capability scaling.The framework improves multi-view reasoning, spatial grounding, and robot-oriented understanding.
  • Conclusion: Explicit 3D-grounded supervision enables native language-conditioned 3D grounding for the 2B and 9B models, alongside compact contact-centered interaction grounding.These additions align model outputs more directly with embodied interaction.
  • Conclusion: RynnBrain-VLA substantially outperforms a Qwen-based VLA on long-horizon manipulation tasks and deploys across three robot embodiments.The deployed platforms are Unitree G1, Astribot, and Tianji-Wuji.
  • Conclusion: Unified action spaces with embodiment-specific masking support cross-embodiment deployment, while joint multi-task and multi-embodiment training improves real-robot performance.The results provide initial evidence that heterogeneous robot data contributes complementary supervision.
  • Future Work: Future work will unify perception, understanding, generation, spatial grounding, and action, while integrating long-term memory, world modeling, task planning, active perception, and closed-loop interaction.The goal is to develop RynnBrain into the cognitive core of a general embodied agent.
Loading 2607.17977v2…