Source-linked AI summary
MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots
Ting Huang, Dongjian Li, Rui Yang, Zeyu Zhang, Zida Yang, Hao Tang
TL;DR
MobileVLA-R1 addresses the challenge of grounding language in quadruped motor control by generating structured reasoning and continuous actions. Its two-stage training combines CoT supervision with GRPO, and the authors report gains across navigation and control benchmarks alongside real-world deployment.
Problem
Current approaches struggle to bridge semantic reasoning and motor control, and latent representations can obscure semantic logic and limit compositional reasoning and error traceability.
Source: §1 · p. 1Method
MobileVLA-R1 uses structured CoT plans and an action decoder for continuous control, trained first with supervised CoT alignment and then with GRPO reinforcement learning.
Source: §1 · p. 2Results
On VLN-CE, MobileVLA-R1 achieves a 5% average improvement in success rate over strong baselines, outperforms recent models across QUARD’s six control tasks, and performs robustly in real-world conditions.
Source: §1 · p. 2Takeaways & Limitations
Decoupling structured chain-of-thought reasoning from continuous motor execution yields interpretable decisions and robust control across diverse environments.
Source: §6 · p. 8The current framework mainly targets navigation and basic manipulation, using predefined behaviors that limit fine-grained control and adaptation to dynamic or novel situations.
Source: §11 · p. 13
Abstract
from arXiv · showhide
Grounding natural-language instructions into continuous control for quadruped robots remains a fundamental challenge in vision language action. Existing methods struggle to bridge high-level semantic reasoning and low-level actuation, leading to unstable grounding and weak generalization in the real world. To address these issues, we present MobileVLA-R1, a unified vision-language-action framework that enables explicit reasoning and continuous control for quadruped robots. We construct MobileVLA-CoT, a large-scale dataset of multi-granularity chain-of-thought (CoT) for embodied trajectories, providing structured reasoning supervision for alignment. Built upon this foundation, we introduce a two-stage training paradigm that combines supervised CoT alignment with GRPO reinforcement learning to enhance reasoning consistency, control stability, and long-horizon execution. Extensive evaluations on VLN and VLA tasks demonstrate superior performance over strong baselines, with approximately a 5% improvement. Real-world deployment on a quadruped robot validates robust performance in complex environments. Code: https://github.com/AIGeeksGroup/MobileVLA-R1. Website: https://aigeeksgroup.github.io/MobileVLA-R1.
1. Introduction
MobileVLA-R1 addresses the gap between semantic reasoning and motor control with structured chain-of-thought plans and continuous quadruped actions. The paper reports gains on navigation and control benchmarks and situates this approach among work on language-guided robotics, reasoning, and reinforcement learning.
1. Introduction
- Existing approaches struggle to bridge semantic reasoning and motor control, while latent intermediate representations can obscure semantic logic and hinder compositional reasoning and error traceability.Source: §1 · p. 1
- MobileVLA-R1 generates structured CoT plans from multimodal observations, decodes them into continuous quadruped actions, and trains with supervised CoT alignment followed by GRPO.The paper also introduces MobileVLA-CoT, a multi-granularity dataset for embodied trajectories.Source: §1 · p. 2
- On VLN-CE, MobileVLA-R1 achieves a 5% average improvement in success rate over strong baselines and outperforms recent VLA models across QUARD’s six control tasks.Real-world deployment on Unitree Go2 showed robust performance under cluttered and partially observable conditions.Source: §1 · p. 2
3. Datasets
The paper combines public navigation and quadruped-control datasets to synthesize MobileVLA-CoT, a multi-granularity reasoning corpus. A CoT data engine generates and filters traces aligned with observations and control targets.
3.1. Public Datasets
- R2R supplies instruction–trajectory pairs, RxR adds multilingual supervision and dense temporal grounding, and QUARD provides quadruped locomotion and manipulation episodes.Together, the datasets provide supervision for language grounding, spatial reasoning, and low-level action execution.Source: §3.1 · p. 2
3.2. Synthetic Dataset
- MobileVLA-CoT records reasoning that connects instructions to actions at episode, step, and navigation levels.It contains 18K episode-level samples, 78K step-level samples, and 38K annotations in MobileVLA-CoT-Nav.Source: §3.2 · p. 3
3.3. CoT Data Engine
- The CoT data engine uses paired RGB–Depth observations, instructions, and state–action histories to generate reasoning followed by control commands.A semi-automatic pipeline combines rule-based filtering and manual inspection to remove malformed or unsafe annotations before training.Source: §3.3 · p. 3
4. The Proposed Method
MobileVLA-R1 connects multimodal reasoning with quadruped control through CoT planning and a staged training process. Its method combines supervised alignment with GRPO-based refinement.
4.1. Overview
- MobileVLA-R1 first aligns CoT reasoning with multimodal observations, then uses GRPO to refine action grounding and continuous control.Its policy integrates RGB, depth, and point-cloud inputs and maps them to continuous velocities and a discrete high-level action; the CoT data engine generates multi-granularity annotations from RGB–Depth inputs and instructions.Source: §4.1 · pp. 3–4, Figure 3 · p. 4
4.2. Cold Start Stage
- The cold-start stage uses supervised fine-tuning to initialize reasoning and output format before reinforcement learning.Training first uses Episode and Nav data for structured reasoning, then a 10K-sample Step subset with executable velocity and action commands.Source: §4.2 · p. 4
4.3. Reinforcement Learning
- GRPO optimizes groups of sampled responses using normalized rewards and a clipped objective with a KL-divergence term to limit over-updating.The method evaluates each response with reward functions for accurate, executable, and well-formatted outputs.Source: §4.3 · pp. 4–5, Figure 4 · p. 5
- Movement reward scores cosine similarity between predicted and ground-truth velocity vectors, while action reward checks whether the discrete action matches its label..Source: §4.3 · p. 5
- Format reward assigns a binary score for following the <think>...</think><answer>...</answer> output template.The template enforces consistent mapping from reasoning traces to executable control commands.Source: §4.3 · p. 5
- MobileVLA-R1 outperforms VLN-CE methods that do not use simulator-pretrained waypoint predictors, including methods with additional inputs.The comparison is on the Val-Unseen split.Source: Table 2 · p. 6
5. Experiments
MobileVLA-R1 is evaluated on continuous language-guided navigation, quadruped control, and real-world instruction following, with reward and modality ablations examining components of its training and perception. The reported results favor the full model across benchmark tasks and real-world settings.
5. Experiments
- Evaluation covers continuous VLN-CE navigation and six QUARD control tasks, with supervised alignment followed by GRPO refinement.VLN-CE uses val-unseen splits; QUARD success rates average 25 episodes per task, and training uses approximately 20K supervised steps followed by 1K GRPO steps.Source: §5 · pp. 6–7
5.1. Main Results
- MobileVLA-R1 outperforms prior methods on R2R-CE and RxR-CE, and achieves the best overall QUARD performance across locomotion and manipulation tasks.Table 3 reports average success rates over 25 evaluation episodes across six tasks grouped by difficulty.Source: §5.1 · p. 7, Table 3 · p. 7
5.2. Real World Evaluation
- Across Workspace, Corridor, and Outdoor, MobileVLA-R1 outperforms NaVILA and GPT-4o in success rate and navigation error under simple and complex instructions.The real-world evaluation uses a Unitree Go2 and defines complex tasks as multi-step reasoning and long-horizon spatial planning.Source: §5.2 · p. 7, Table 4 · p. 7
5.3. Ablation Study
- Removing any movement, action, or format reward degrades navigation success and trajectory efficiency; using all three yields the highest success rate and SPL.The reported effects include unstable, oscillatory locomotion without movement reward, weaker decision consistency without action reward, and malformed outputs without format reward.Source: §5.3 · p. 8, Table 5 · p. 8
- The study also conducts incremental modality-encoder ablations to assess perception-stream contributions.These additional ablations are reported in the supplementary material.Source: §5.3 · p. 8
6. Conclusion
MobileVLA-R1 combines structured chain-of-thought reasoning with continuous motor execution for quadruped robots. Its benchmark evaluations and Unitree Go2 deployments show performance and adaptability over existing methods.
6. Conclusion
- MobileVLA-R1 bridges high-level reasoning and low-level quadruped control by decoupling structured chain-of-thought reasoning from continuous motor execution.The conclusion characterizes this design as supporting interpretable decision-making and robust control across diverse environments.Source: §6 · p. 8
- Its two-stage training combines supervised CoT alignment with GRPO-based reinforcement learning to enhance reasoning consistency, control stability, and long-horizon execution.Source: §6 · p. 8
- VLN-CE and QUARD experiments, alongside Unitree Go2 deployments, demonstrate superior performance and adaptability compared with existing methods.Source: §6 · p. 8
7. More Ablation Study
Incremental modality ablations show that adding depth and point-cloud encoders improves navigation success and trajectory efficiency on both R2R-CE and RxR-CE.
7. More Ablation Study
- Adding depth and point-cloud encoders incrementally improves navigation success and trajectory efficiency on both R2R-CE and RxR-CE.Depth cues enhance local geometry and obstacle awareness; point-cloud features further strengthen 3D spatial reasoning and path planning, while the full configuration performs best.Source: §7 · p. 12, Table 6 · p. 12
8. CoT Data Generation and Quality Control
CoT annotations are generated only for official training splits, and a semi-automatic pipeline filters and reviews annotation quality.
8. CoT Data Generation and Quality Control
- CoT annotations are limited to official R2R, RxR, and QUARD training splits, with evaluations using original benchmarks, human instructions, and protocols.The teacher model is not queried with ground-truth answers for evaluated trajectories.Source: §8 · p. 12
- Rule-based checks filter malformed or invalid annotations before sampled examples receive independent author review for reasoning consistency, action feasibility, and safety.Failed samples are removed or corrected, and long-horizon and cluttered episodes receive additional spot-checks.Source: §8 · p. 12
9. Test-Time Efficiency and Practical Considerations
MobileVLA-R1’s 8B vision-language backbone and remote-server setup impose substantial latency, which suits the reported quasi-static experiments but not highly dynamic control. The section discusses short-horizon execution and possible system optimizations, while leaving systematic exploration of these choices to future work.
9. Test-Time Efficiency and Practical Considerations
- About 10 seconds for inference and roughly 15 seconds per end-to-end control step limit the current setup to cautious, quasi-static experiments, not highly dynamic scenarios.The reported experiments focus on indoor navigation and short-range manipulation, where scenes change slowly.Source: §9 · p. 12
- The decoder issues continuous commands over a short horizon, suggesting low-frequency CoT planning paired with higher-frequency lightweight onboard tracking.The passage presents this as a possible operating regime, not an evaluated deployment change.Source: §9 · p. 12
- Smaller backbones, distillation, or quantization could trade modest accuracy for reduced latency and memory, while caching perception features and CoT prefixes could amortize long-horizon reasoning.The paper says a systematic exploration of these choices is beyond its scope.Source: §9 · pp. 12–13
- The results show CoT-based quadruped reasoning is possible despite relatively high latency, with model compression and hierarchical control identified as promising deployment directions.Source: §9 · p. 13
10. Visualization
The visualizations illustrate MobileVLA-R1’s instruction-conditioned reasoning and continuous actions in simulator validation scenes and physical environments. They portray spatially grounded, stable behavior across unseen or varied scenes, including long-horizon commands outdoors and indoors.
10. Visualization
- Simulator validation trajectories show step-by-step reasoning and continuous actions on R2R-CE and RxR-CE, including unseen scenes and linguistic variations.The reported examples are described as showing coherent perception, spatial reasoning, action grounding, cross-lingual understanding, and spatial consistency.Source: §10 · p. 13
- Unitree Go2 deployments depict long-horizon commands executed with stable motion and continuous spatial grounding in complex indoor and outdoor environments.The paper presents these examples as evidence of generalization from simulated training to physical environments.Source: §10 · p. 13
11. Limitation and Future Work
The paper identifies limits in action-skill flexibility, training coverage, and test-time speed, and outlines learning and system directions to address them. Its visual examples span simulator navigation and indoor and outdoor quadruped deployments.
11. Limitation and Future Work
- The fixed set of predefined behaviors for navigation and basic manipulation limits fine-grained adaptation to highly dynamic or novel situations and constrains skill diversity and compositionality.Source: §11 · p. 13
- Reliance on supervised datasets and semi-scripted training environments constrains generalization to open-ended, long-horizon tasks and truly unstructured settings.Source: §11 · p. 13
- Remote inference with an 8B vision-language backbone takes roughly 10 seconds, with about 15 seconds of end-to-end delay per control step, restricting use to slow, quasi-static scenarios.Source: §11 · p. 13
- Future work proposes expanding the action space and decoding granularity to support more diverse, context-dependent skills, alongside self-supervised and lifelong learning for adaptation without extensive human annotation.Source: §11 · p. 13
- For latency, proposed directions include compression, smaller backbones, and hierarchical control with low-frequency CoT planning and compact onboard low-level policies.Source: §11 · p. 13
- Other proposed directions include richer multimodal sensing, large-scale pretraining for cross-domain transfer, and extensions to dexterous manipulation and collaborative tasks.Source: §11 · p. 13
- The visual examples cover English and multilingual simulator navigation, indoor object interaction and obstacle avoidance, and outdoor obstacle avoidance and target-aware reasoning.The outdoor examples are described as spanning diverse scenes; these captions do not report quantitative outcomes.Source: Figure 6 · p. 14, Figure 7 · p. 15, Figure 8 · p. 16