Source-linked AI summary
DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI
En Yu, Haoran Lv, Jianjian Sun, Kangheng Lin, Ruitao Zhang, Yukang Shi, Yuyang Chen, Ze Chen, Ziheng Zhang, Fan Jia, Kaixin Liu, Meng Zhang, Ruitao Hao, Saike Huang, Songhan Xie, Yu Liu, Zhao Wu, Bin Xie, Pengwei Zhang, Qi Yang, Xianchi Deng, Yunfei Wei, Enwen Zhang, Hongyang Peng, Jie Zhao, Kai Liu, Wei Sun, Yajun Wei, Yi Yang, Yunqiao Zhang, Ziwei Yan, Haitao Yang, Hao Liu, Haoqiang Fan, Haowei Zhang, Junwen Huang, Yang Chen, Yunchao Ma, Yunhuan Yang, Zhengyuan Du, Ziming Liu, Jiahui Niu, Yucheng Zhao, Daxin Jiang, Wenbin Tang, Xiangyu Zhang, Zheng Ge, Erjin Zhou, Tiancai Wang
TL;DR
Current VLA systems typically adapt internet-pretrained models to limited embodied data, leaving physical grounding insufficient for dynamic, spatial interaction. DM0 instead trains an Embodied-Native VLA across heterogeneous data through three stages, and achieves state-of-the-art RoboChallenge performance in both Specialist and Generalist Table30 settings.
Problem
Internet-pretrained VLAs typically lack intrinsic physical grounding because static web data does not capture dynamic, continuous, and spatial physical interactions.
Method
DM0 uses a three-stage Embodied-Native pipeline combining heterogeneous pretraining data with a flow-matching action expert, hybrid gradient training, and spatial Chain-of-Thought scaffolding.
Results
DM0 achieves state-of-the-art RoboChallenge performance in both Specialist and Generalist Table30 settings, with 62.0% average Specialist success and 37.3% average Generalist success.
Takeaways & Limitations
DM0 demonstrates that integrating physical grounding from the onset can support unified manipulation and navigation in generalist robot policies.
Takeaways & Limitations
DM0 is currently a lightweight model, with future work requiring larger datasets, larger parameter scales, and additional modalities such as tactile, audio, and depth information.
Abstract
from arXiv · showhide
Moving beyond the traditional paradigm of adapting internet-pretrained models to physical tasks, we present DM0, an Embodied-Native Vision-Language-Action (VLA) framework designed for Physical AI. Unlike approaches that treat physical grounding as a fine-tuning afterthought, DM0 unifies embodied manipulation and navigation by learning from heterogeneous data sources from the onset. Our methodology follows a comprehensive three-stage pipeline: Pretraining, Mid-Training, and Post-Training. First, we conduct large-scale unified pretraining on the Vision-Language Model (VLM) using diverse corpora--seamlessly integrating web text, autonomous driving scenarios, and embodied interaction logs-to jointly acquire semantic knowledge and physical priors. Subsequently, we build a flow-matching action expert atop the VLM. To reconcile high-level reasoning with low-level control, DM0 employs a hybrid training strategy: for embodied data, gradients from the action expert are not backpropagated to the VLM to preserve generalized representations, while the VLM remains trainable on non-embodied data. Furthermore, we introduce an Embodied Spatial Scaffolding strategy to construct spatial Chain-of-Thought (CoT) reasoning, effectively constraining the action solution space. Experiments on the RoboChallenge benchmark demonstrate that DM0 achieves state-of-the-art performance in both Specialist and Generalist settings on Table30.
1. Introduction
DM0 addresses the lack of intrinsic physical grounding in internet-pretrained VLA models by learning semantic and physical representations jointly from heterogeneous data. Its three-stage Embodied-Native framework combines unified pretraining, hybrid gradient training, and spatial reasoning, achieving state-of-the-art RoboChallenge results in Specialist and Generalist settings.
- Motivation: Current VLA systems typically pretrain on static internet data and later fine-tune on limited embodied data, leaving physical grounding underdeveloped.Internet data captures semantic knowledge but not the dynamic, continuous, and spatial nature of physical interaction.
- Motivation: This adaptation paradigm can fragment navigation and manipulation modules or cause catastrophic forgetting of general reasoning abilities.
- Approach: DM0 unifies manipulation and navigation through an Embodied-Native, three-stage pipeline spanning Pretraining, Mid-Training, and Post-Training.
- Approach: Unified pretraining combines vision-language, driving-scene, and embodied-action data so the model acquires physical priors alongside semantic knowledge.
- Approach: A hybrid gradient strategy protects general VLM knowledge during embodied training, while spatial Chain-of-Thought scaffolding constrains the action solution space.
- Results: 62.0% average success on thirty Table30 tasks makes DM0-Specialist state of the art, exceeding GigaBrain-0.1 by more than 10% margins; DM0-Generalist reaches 37.3%.The reported results cover both Specialist and Generalist settings on RoboChallenge.
2. Model Description
DM0 combines a multimodal VLM with a Flow Matching action expert and trains them through structured reasoning and hybrid objectives. The design connects embodied language reasoning to continuous control while limiting degradation of pretrained semantic representations.
- Architecture: DM0 uses a VLM backbone whose embodied reasoning representations condition a Flow Matching action expert that produces continuous robot actions.
- Architecture: At inference, DM0 either predicts continuous actions directly from observations and instructions or generates textual embodied reasoning before action generation.
- Architecture: The joint model factorizes prediction into textual output and a continuous action sequence over a horizon of H time steps.
- Hybrid Training: The hybrid gradient strategy blocks action-expert gradients on embodied data while continuing VLM updates on non-embodied data to preserve semantic knowledge.
- Hybrid Training: The VLM learns embodied reasoning text and discrete action tokens, while the action expert learns continuous action sequences with Flow Matching loss.
- Structured Supervision: Hierarchical supervision progresses from subtask descriptions and goal boxes to end-effector trajectories and discrete actions, progressively constraining the hypothesis space.
3. Training Recipe
DM0 trains from general multimodal representation learning to embodied action prediction and platform-specific visuomotor alignment. Its data mixture combines web, driving, embodied reasoning, and robotic trajectories while retaining general VLM capabilities.
- Training Stages: DM0 uses sequential Pretraining, Mid-Training, and Post-Training stages that progress from multimodal foundations to embodied control and deployment-ready policies.
- Pretraining: Pretraining jointly uses web, document, driving, and embodied data to learn multimodal alignment, fine-grained perception, broad reasoning, and physical priors.
- Pretraining: Driving data contributes depth-aware detection and grounding annotations, while embodied data provides object localization, scene descriptions, and spatial-relation tasks.
- Pretraining: The pretraining recipe processes 1.2T tokens in 370K steps, producing the backbone for action prediction and cross-embodiment control.
- Mid-Training: Mid-training jointly supervises dialogue tokens, discrete action tokens, and continuous actions so language reasoning becomes coupled with executable physical actions.
- Mid-Training: The curated mid-training mixture balances general VLM retention with embodiment-specific learning across vision-language, embodied reasoning, navigation, and single- and dual-arm robotic data.
- Data Scale: Weighted sampling yields 1.13T tokens for pretraining, 200M samples for mid-training, and 50M samples for post-training.
4. Experimental Evaluations
DM0 is evaluated on the real-world RoboChallenge Table30 benchmark using Specialist and Generalist configurations against leading open-source VLA models. It achieves strong performance while retaining multimodal understanding capabilities across embodied and general scenarios.
- Benchmark and setup: RoboChallenge evaluates over 30 long-horizon tabletop manipulation tasks requiring multi-step reasoning, spatial understanding, and continuous control.Tasks include picking, placement, rearrangement, tool usage, and compositional instruction following.
- Benchmark and setup: DM0 is compared with GigaBrain-0.1, Spirit-V1.5, π0.5, and π0 using success rate and, for Generalist models, composite task score.The Specialist analysis reports success rate, while the Generalist analysis reports both metrics because success rates are low across models.
- RoboChallenge results: 37.3/49.08 average success rate/task score is achieved by DM0-Generalist, compared with 17.67/31.27 for π0.5-Generalist and 9.0/20.22 for π0-Generalist.The advantage is reported across nearly all robot platforms and task categories, especially precise manipulation and long-horizon reasoning.
- Multimodal understanding: DM0 retains scene understanding, visual grounding, attribute recognition, and OCR across embodied and lifestyle VQA domains.The mid-training checkpoint also predicts subtasks, detects target objects, forecasts trajectories and actions, and shows potential for mobile-agent contexts.
5. Discussion
DM0 integrates physical grounding from training onset through heterogeneous data and a three-stage VLA pipeline. The authors report state-of-the-art Table30 performance, while identifying lightweight scale and extremely long-horizon reasoning as areas for future work.
- Discussion: DM0 unifies web corpora, autonomous driving logs, and embodied trajectories through Pre-, Mid-, and Post-Training.The framework uses hybrid gradient training and Embodied Spatial Scaffolding with spatial Chain-of-Thought reasoning.
- Discussion: State-of-the-art Table30 performance is reported for DM0 in both Specialist and Generalist settings against strong baselines including Spirit-v1.5 and π0.5.The conclusion presents these results as support for using intrinsic, multi-source physical priors in VLA models.
- Future work: DM0 currently operates as a lightweight model, motivating future scaling with larger datasets and parameter sizes such as 7B or 30B.The proposed data expansion includes simulation data combined with real-world logs.
- Future work: Extremely long-horizon tasks remain a challenge despite Spatial Scaffolding, motivating integration of World Model capabilities for longer-horizon planning.The intended capability is to mentally simulate action consequences before execution.
- Future work: Future multimodal extensions are planned to incorporate tactile feedback, audio, and depth information into unified pretraining.These modalities are intended for finer manipulation in occluded or dynamic environments.
6. Author List
The paper lists authors alphabetically by first name and identifies contributors across training, data, evaluation, infrastructure, sponsorship, and project leadership.
- Author list: Authors are listed alphabetically by their first names.
- Author roles: Contributors are grouped by Pretraining and Mid-training and Post-training, Supervised Fine-tuning, Reinforcement Learning, Data Collection and Process, and Evaluation.
- Author roles: Additional roles include Infrastructure, Sponsors, and Project Lead.