Source-linked AI summary
InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy
Xinyi Chen, Yilun Chen, Yanwei Fu, Ning Gao, Jiaya Jia, Weiyang Jin, Hao Li, Yao Mu, Jiangmiao Pang, Yu Qiao, Yang Tian, Bin Wang, Bolun Wang, Fangjing Wang, Hanqing Wang, Tai Wang, Ziqin Wang, Xueyuan Wei, Chao Wu, Shuai Yang, Jinhui Ye, Junqiu Yu, Jia Zeng, Jingjing Zhang, Jinyu Zhang, Shi Zhang, Feng Zheng, Bowen Zhou, Yangkun Zhu
TL;DR
Robots must connect language with spatially grounded action because VLM training data rarely represents continuous embodied interactions. InternVLA-M1 addresses this with spatially guided two-stage training, achieving stronger instruction following and generalization across simulation and real-world evaluations.
Problem
VLMs capture spatial cues only indirectly, while robot actions require continuous embodied interactions that are scarcely represented in their training data.
Method
InternVLA-M1 combines a spatial-prior VLM planner and action expert through spatial grounding pre-training followed by spatially guided action post-training.
Results
InternVLA-M1 surpasses existing VLA models and specialized systems across instruction following, long-horizon manipulation, and multimodal grounding in simulation and real-world settings.
Takeaways & Limitations
Spatial reasoning can serve as a unifying substrate for scalable and reliable generalist robots within the evaluated settings.
Abstract
from arXiv · showhide
We introduce InternVLA-M1, a unified framework for spatial grounding and robot control that advances instruction-following robots toward scalable, general-purpose intelligence. Its core idea is spatially guided vision-language-action training, where spatial grounding serves as the critical link between instructions and robot actions. InternVLA-M1 employs a two-stage pipeline: (i) spatial grounding pre-training on over 2.3M spatial reasoning data to determine ``where to act'' by aligning instructions with visual, embodiment-agnostic positions, and (ii) spatially guided action post-training to decide ``how to act'' by generating embodiment-aware actions through plug-and-play spatial prompting. This spatially guided training recipe yields consistent gains: InternVLA-M1 outperforms its variant without spatial guidance by +14.6% on SimplerEnv Google Robot, +17% on WidowX, and +4.3% on LIBERO Franka, while demonstrating stronger spatial reasoning capability in box, point, and trace prediction. To further scale instruction following, we built a simulation engine to collect 244K generalizable pick-and-place episodes, enabling a 6.2% average improvement across 200 tasks and 3K+ objects. In real-world clustered pick-and-place, InternVLA-M1 improved by 7.3%, and with synthetic co-training, achieved +20.6% on unseen objects and novel configurations. Moreover, in long-horizon reasoning-intensive scenarios, it surpassed existing works by over 10%. These results highlight spatially guided training as a unifying principle for scalable and resilient generalist robots. Code and models are available at https://github.com/InternRobotics/InternVLA-M1.
1. Introduction
InternVLA-M1 addresses the gap between linguistic instruction understanding and grounded 3D action by separating transferable spatial priors from embodiment-specific control. Its spatially guided framework improves instruction following and generalization across simulated and real-world settings.
- Textual abstractions encode spatial cues indirectly, while real-world actions require continuous embodied interactions that VLM training data scarcely represents.
- Prior hierarchical systems often depend on rule-based decomposition and manually designed planning heuristics, limiting scalability to complex and diverse tasks.
- InternVLA-M1 combines a spatial-prior VLM planner with an action expert that translates grounded representations into executable motor commands.
- 2.3M spatial grounding samples and 0.7M multimodal understanding samples support spatial pre-training and spatially guided action post-training.
- 6.2% average improvement over prior works was achieved across 200 generalizable pick-and-place tabletop scenarios under few-demonstration fine-tuning.
- +20.6% success rate on unseen objects and novel setups was achieved in real-world clustered pick-and-place tasks, alongside robust long-horizon performance under perturbations.
2. InternVLA-M1
InternVLA-M1 is a dual-system end-to-end VLA framework that couples a spatially reasoning language head with an embodiment-specific action head. Its two-stage training explicitly activates spatial priors and transfers them into robotic control.
- The language head performs instruction-to-visual grounding, while the action head conditions on learned spatial priors to generate embodiment-specific motor commands.
- The architecture uses a Qwen2.5-VL-3B-Instruct multimodal encoder as System 2 and an 86M-parameter diffusion policy as the fast System 1 Action Expert.
- Dual supervision jointly processes multimodal and action batches, aggregating both losses in one update so perception and control co-adapt.
- Stage 1 trains only the VLM on multimodal spatial grounding data, covering bounding boxes, trajectories, affordances, and related robotic reasoning tasks.
- Stage 2 jointly optimizes the VLM and Action Expert on demonstrations, keeping semantic understanding and motion generation integrated.
- Spatial prompting appends explicit cues about object relationships and task constraints before action prediction to improve spatial awareness and manipulation generalization.
3. Data
The data program combines broad multimodal understanding, spatial grounding annotations, and synthetic manipulation experience to connect VLM pre-training with robotic execution. Its simulation pipeline emphasizes diversity and efficient verification.
- The multimodal dataset contains over 3M samples spanning General QA, Box QA, Trajectory QA, and Point QA.
- More than 2.3M samples are dedicated to spatial reasoning, supporting adaptation to embodied tabletop tasks.
- General QA contributes approximately 637K samples covering captioning, VQA, OCR, knowledge grounding, and creative writing.
- Trajectory QA combines A0 ManiSkill, InternData-M1 waypoint, and MolmoAct data for precise end-effector trajectory prediction.
- A post-pre-training phase uses large-scale simulated data to initialize the action head and learn action representations after VLM pre-training.
- The simulation pipeline generates diverse manipulation trajectories from randomized object layouts and lighting, then verifies candidate grasps through one physics execution.
- Decoupling planning from rendering allows structured trajectories to be replayed under randomized lighting, materials, and viewpoints.
4. Experiments
Experiments evaluate InternVLA-M1 across public simulation benchmarks, large-scale simulated pick-and-place, real-world clutter, and long-horizon manipulation. Across these settings, spatially guided training improves manipulation, spatial grounding, generalization, and adaptation to changing instructions or environments.
- Public benchmarks: InternVLA-M1 improves over the Vanilla VLA by 14.6% on Google Robot Visual Matching, 12.4% on Visual Aggregation, and 17.0% on WidowX.These comparisons use the Qwen2.5-VL-3B-Instruct-based Vanilla VLA.
- Ablation studies: Spatially guided action post-training accelerates manipulation convergence while improving spatial grounding accuracy.Removing spatial data and guided prompting causes rapid degradation in grounding and slower manipulation convergence.
- LIBERO benchmark: InternVLA-M1 reaches 98.0% success on LIBERO-Spatial and 92.6% on LIBERO-Long, with 99.0% for object placement.The results particularly improve spatial and long-horizon manipulation tracks relative to strong baselines.
- Large-scale simulated pick-and-place: Mid-training yields a +6.2% average gain over GR00T N1.5 across in-distribution, unseen-object, new-background, and unseen-instruction settings.Both InternVLA-M1 variants outperform π0, while the mid-trained variant consistently surpasses GR00T N1.5.
- Real-world cluttered pick-and-place: Synthetic co-training improves performance on novel objects and spatial configurations, complementing limited real-world demonstrations.Simulation expands object-position and orientation coverage beyond what real-world collection can exhaustively provide.
- Long-horizon and reasoning manipulation: Additional post-training notably improves long-horizon and reasoning-intensive performance, underscoring its role in high-level task planning.This result is reported despite GPT-5's strong reasoning capability.
- Long-horizon and reasoning manipulation: InternVLA-M1 maintains minimal performance degradation under physical interference and changing instructions by revising its subtask sequence during execution.The model tracks moved containers and incorporates additional instructions while continuing correct actions.
5. Related work
Related work develops intermediate representations and architectural paradigms to connect high-level reasoning with robot actions, but these approaches vary in modality, modularity, and computational cost.
- Hierarchical robot system: Intermediate representations connect high-level instructions to low-level actions, ranging from symbolic structures and learned embeddings to textual plans and physically grounded outputs.Examples include textual plans, bounding boxes, 3D grasp points, and dense correspondence fields.
- Embodied reasoning and planning in VLA: Textual, visual, and spatial reasoning steps can improve complex-task handling, but generating intermediate steps during inference adds computational overhead.The cited approaches include explicit plans, adaptive thinking and execution, reasoning guides, subtask prediction, and graph-based spatial representations.
- Generalist robot policy: Generalist robot policies follow monolithic, unified, or world-model paradigms that differ in how they divide cognition, action, and planning.Monolithic VLAs directly map multimodal inputs to tokenized actions, unified systems decouple cognition from action, and world models support predictive planning.
6. Discussion and conclusion
InternVLA-M1 uses spatial grounding priors to connect multimodal reasoning with robotic execution. Across simulation and real-world evaluations, it shows strong generalization and surpasses existing models and specialized systems in instruction following, long-horizon manipulation, and multimodal grounding.
- Discussion and conclusion: InternVLA-M1 bridges high-level multimodal reasoning and low-level robotic execution through spatial grounding priors.The framework combines large-scale multimodal pre-training with spatially guided post-training.
- Discussion and conclusion: InternVLA-M1 generalizes to unseen objects, instructions, and environments across simulation and real-world settings.The conclusion reports extensive evaluations spanning instruction following, long-horizon manipulation, and multimodal grounding.
- Discussion and conclusion: Spatial reasoning is presented as a unifying substrate for scalable and reliable generalist robots.This conclusion links spatially grounded understanding with embodied control and broad generalization.
A. Author contributions
The authors are listed alphabetically by last name.
- Author contributions: All contributors are listed in alphabetical order by their last names.
- Author contributions: The contribution-listing convention is alphabetical ordering by surname.
A.1. Core Contributors
The core contributors are listed as a sequence of named researchers.
- Core Contributors: The core contributor list names Yilun Chen, Ning Gao, Jiangmiao Pang, Bolun Wang, and Fangjing Wang.
- Core Contributors: The list continues with Jinhui Ye, Junqiu Yu, Jinyu Zhang, and Yangkun Zhu.
- Core Contributors: Nine core contributors are named in total.
A.2. Contributors
This section lists the contributors to the paper.
- The contributors are Xinyi Chen, Yanwei Fu, Jiaya Jia, Weiyang Jin, Hao Li, Yao Mu, Yu Qiao, Yang Tian, Bin Wang, Hanqing Wang, Tai Wang, Ziqin Wang, Xueyuan Wei, Chao Wu, Shuai Yang, Jia Zeng, Jingjing Zhang, Shi Zhang, Feng Zheng, and Bowen Zhou.