Source-linked AI summary
Motubrain: An Advanced World Action Model for Robot Control
Motubrain Team, Chendong Xiang, Fan Bao, Haitian Liu, Hengkai Tan, Hongzhe Bi, James Li, Jiabao Liu, Jingrui Pang, Kiro Jing, Louis Liu, Mengchen Cai, Rongxu Cui, Ruowen Zhao, Runqing Wang, Shuhe Huang, Yao Feng, Yinze Rong, Zeyuan Wang, Jun Zhu
TL;DR
VLA models often lack fine-grained world-dynamics modeling, motivating a unified approach to robot control and prediction. Motubrain jointly models video and action across heterogeneous multimodal data with one world action model supporting multiple inference modes. It achieves strong simulation and world-modeling results, transfers to new humanoid embodiments with 50–100 trajectories, and supports practical real-time deployment.
Problem
VLA models generalize semantically but often lack fine-grained modeling of world dynamics needed for precise robotic control.
Method
Motubrain jointly models video and action with UniDiffuser and a three-stream Mixture-of-Transformers, using heterogeneous multimodal data and unified representations for multiview and cross-embodiment control.
Results
Motubrain achieves strong performance across simulation, world-modeling, and humanoid-transfer evaluations, including 95.8% and 96.1% average success on RoboTwin 2.0 clean and randomized settings.
Takeaways & Limitations
A single pretrained world-action model can serve as a scalable foundation for understanding world evolution and deciding how robots should act.
Takeaways & Limitations
Current adaptation still relies on a modest amount of same-embodiment robot data, and broader testing is needed for longer-horizon mobile manipulation, tactile interaction, and dynamic human-centered environments.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) models generalize semantically well but often lack fine-grained modeling of world dynamics. We present Motubrain, a unified World Action Model that jointly models video and action under a UniDiffuser formulation with a three-stream Mixture-of-Transformers architecture. A single model supports policy learning, world modeling, video generation, inverse dynamics, and joint video-action prediction, while scaling to heterogeneous multimodal data such as video-only, task-agnostic, and cross-embodiment robot data. Building on Motus, Motubrain further introduces unified multiview modeling, an independent text stream for stronger language-action coupling, a shared cross-embodiment action representation, and an efficient post-training and deployment recipe for long-horizon real-world control. Our inference stack combines step reduction, compilation, FP8 quantization, DiT caching, V2A-style action-only inference, and real-time chunked closed-loop execution, achieving over 50x speedup over a naive baseline and up to 11 Hz inference. Experimentally, Motubrain achieves 95.8% and 96.1% average success on RoboTwin 2.0 under clean and randomized settings, respectively, attains the strongest reported EWMScore in our WorldArena comparison, and adapts to new humanoid embodiments with only 50--100 trajectories. These results show that unified world action models can scale in generality, predictive accuracy, and real-world deployability.
1 Introduction
Motubrain addresses the limited temporal understanding of VLA policies by unifying video dynamics and action prediction in one world action model trained on heterogeneous multimodal data. It extends this framework for multiview, cross-embodiment, long-horizon control and reports strong benchmark, world-modeling, and adaptation results.
- Motivation: VLA policies generalize semantically but often neglect fine-grained world dynamics needed for precise robotic control.Their reliance on static image–text pretraining can produce superficial behavior imitation rather than temporal understanding of world physics.
- Motivation: Video generation models provide spatiotemporal priors for world modeling, but VGM+IDM pipelines accumulate video-prediction errors that compromise actions and downstream policy performance.The two-stage approach forecasts future visual trajectories before inferring actions from generated frames.
- Unified world-action modeling: World Action Models jointly predict future visual states and actions, avoiding cascading errors and fragmented functionality while absorbing heterogeneous multimodal data.Relevant data include video-only inputs, aligned robot trajectories across embodiments, and task-agnostic interaction data with missing modalities.
- Motubrain: Motubrain uses UniDiffuser and a three-stream Mixture-of-Transformers to support five inference modes within one model.These modes are policy modeling, world modeling, video generation, inverse dynamics, and joint video-action prediction.
- Motubrain: Motubrain adds arbitrary-layout multiview modeling, an independent text stream, a shared cross-embodiment action representation, and deployment mechanisms for long-horizon control.The deployment recipe includes autoregressive rollout, V2A-style action-only inference, and real-time chunked closed-loop execution.
- Results: 95.8% and 96.1% average success rates are achieved on RoboTwin 2.0 under clean and randomized settings, respectively.Motubrain also attains the strongest reported EWMScore in the WorldArena comparison and adapts to new humanoid embodiments with only 50–100 same-embodiment trajectories.
2 Method
Motubrain unifies video, action, and language modeling in one World Action Model, supporting multiple prediction modes and heterogeneous embodied data. Its architecture and training pipeline combine multimodal integration, cross-embodiment action representation, multiview inputs, staged adaptation, and accelerated action inference.
- Unified formulation: Motubrain uses UniDiffuser to jointly model video and action, supporting policy modeling, world modeling, video generation, inverse dynamics, and joint video-action prediction.
- Model architecture: Its three-stream Mixture-of-Transformers architecture contains dedicated text, video, and action streams with joint cross-modal attention and flow matching for video and action.
- Model architecture: The H-bridge design applies full video-action attention only in the middle 50% of Transformer layers, while decoupling the bottom and top 25% layers.This reduces dense cross-modal attention while preserving intermediate semantic alignment and policy grounding.
- Model architecture: Motubrain supports arbitrary multiview camera layouts by independently encoding views, concatenating tokens, and applying view-dependent spatial 3D RoPE offsets.
- Pre-training: Pre-training uses a four-level data pyramid spanning Internet videos, ego-centric videos, heterogeneous-embodiment data, and specific-embodiment data.Starting from pretrained Vidu weights, stage 1 updates the video branch, while stage 2 updates the action branch with a unified action representation across embodiments.
- Pre-training: Relative end-effector actions represent position, rotation, and gripper state relative to the conditioned frame, improving consistency across embodiments and initial poses.Each action has dimension 10, with quaternion inputs converted to a 6D rotation target and only the gripper dimension normalized to [−1, 1].
- Post-training: Non-autoregressive post-training denoises video and action tokens in one forward pass, producing KSa actions from K future video latents.The action stream operates at fva actions per raw video frame, while each latent frame represents τ consecutive video frames.
- Inference: A short joint denoising prefix followed by an action-only suffix freezes video latents and caches visual-language keys and values for later action updates.Removing repeated video computation enables 11 Hz inference while preserving the V2A-style attention semantics.
3 Evaluations
Motubrain performs strongly across simulation, world-modeling, scaling, and real-world evaluations. It combines high benchmark success with strong dynamics prediction, few-shot embodiment adaptation, and long-horizon execution across diverse tasks.
- Simulation Evaluation: 95.8% clean and 96.1% randomized average success rates make Motubrain the top RoboTwin performer in both settings.It exceeds 90% success on 42 clean tasks and 44 randomized tasks.
- Simulation Evaluation: Motubrain improves especially on multistage manipulation, articulated-object interaction, coordination-heavy tasks, and fine-grained spatial arrangement.Highlighted examples include Handover Block, Open Microwave, Press Stapler, Turn Switch, Blocks Ranking Size, Move Can Pot, Place A2B Left, and Place Can Basket.
- Scaling Analysis: Increasing task diversity improves Motubrain’s average success rate more effectively than merely adding data for a fixed task set in this evaluation setting.Motubrain also scales more strongly than conventional VLA baselines and is more data-efficient than those baselines and Motus.
- World-Model Evaluation: 63.77 EWMScore places Motubrain first on WorldArena, with its lead driven primarily by motion quality.Its predicted motion is substantial, smooth, and concentrated on embodiment-relevant regions, while remaining competitive on other perceptual dimensions.
- World-Model Evaluation: Motubrain’s WorldArena and RoboTwin results jointly indicate dynamics representations that are both perceptually faithful and functionally actionable.This cross-benchmark result is notable because EWMScore correlates only weakly with downstream action-planning success (r=0.36).
- Real-World Evaluation: With 50–100 same-embodiment trajectories, Motubrain adapts to new humanoids and executes long-horizon household tasks, including bimanual and self-correcting behavior.Reported examples include Making Oden at 98.54, Mixing Cocktails at 97.34, Flower Arrangement at 83.30, and closed-loop correction after failed insertions.
4 Conclusion and Future Work
Motubrain jointly models future visual dynamics and action generation, combining large-scale pretraining with lightweight robot adaptation for simulation, world-model evaluation, and humanoid transfer. The authors identify remaining limits in data efficiency, task scope, and open-world robustness.
- Motubrain jointly models future visual dynamics and action generation as a unified robot-control foundation.
- Action learning benefits from predictive world modeling rather than isolated imitation optimization.
- Its representation is more robust to compounding execution errors and more transferable across tasks, embodiments, and environments.
- Future Work: Current adaptation still requires modest same-embodiment robot data, limiting accessibility and deployment speed.The authors propose reducing this requirement further.
- Future Work: Future evaluations should extend to longer-horizon mobile manipulation, richer tactile interaction, and dynamic human-centered environments.The authors also identify stronger uncertainty estimation, explicit memory, and online test-time adaptation as robustness directions.
5 Contributors
The paper identifies contributors by sub-module and leadership role. An asterisk marks core contributors or leaders of each sub-module.
- Data contributors are Chendong Xiang, Louis Liu, Jiabao Liu, James Li, Zeyuan Wang, Hongzhe Bi, and Hengkai Tan.
- Base-model contributors are Zeyuan Wang, Chendong Xiang, Hengkai Tan, Haitian Liu, Yao Feng, Ruowen Zhao, Shuhe Huang, and Hongzhe Bi.
- Post-training contributors include Zeyuan Wang, Chendong Xiang, Haitian Liu, Rongxu Cui, Ruowen Zhao, Hengkai Tan, Jingrui Pang, Yao Feng, Shuhe Huang, Mengchen Cai, and Yinze Rong.
- Evaluation contributors include Rongxu Cui, Zeyuan Wang, Haitian Liu, Chendong Xiang, Hengkai Tan, Mengchen Cai, Ruowen Zhao, Shuhe Huang, Runqing Wang, Kiro Jing, James Li, Yao Feng, and Yinze Rong.
- Hengkai Tan is project lead, while Fan Bao and Jun Zhu are advisors.
- An asterisk denotes core contributors or leaders of each sub-module.