Source-linked AI summary
MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulation
Zhenyu Wu, Yuheng Zhou, Xiuwei Xu, Ziwei Wang, Haibin Yan
TL;DR
Mobile manipulation needs generalization across tasks and environments, but conventional methods require costly demonstrations while VLA models primarily support fixed-base manipulation. MoManipVLA transfers fixed-base VLA policies through waypoint-guided, bi-level whole-body trajectory optimization. It reports higher OVMM success than state-of-the-art methods and real-world deployment after 50 samples.
Problem
Mobile manipulation methods have limited cross-task and cross-environment generalization because expert demonstrations are costly, while VLA models remain focused on fixed-base tasks.
Method
MoManipVLA transfers pre-trained fixed-base VLA policies by generating end-effector waypoints and optimizing base-arm trajectories for reachability, smoothness, and collision avoidance.
Results
4.2% overall success rate gain and 11.2% partial success rate gain are reported on OVMM, while real-world deployment achieves 40% success after 50 samples.
Takeaways & Limitations
The framework provides an efficient way to adapt fixed-base VLA generalization to mobile manipulation across simulation and real-world tasks.
Takeaways & Limitations
The conclusion states that the method has two limitations, but the supplied passage does not specify them.
Abstract
from arXiv · showhide
Mobile manipulation is the fundamental challenge for robotics to assist humans with diverse tasks and environments in everyday life. However, conventional mobile manipulation approaches often struggle to generalize across different tasks and environments because of the lack of large-scale training. In contrast, recent advances in vision-language-action (VLA) models have shown impressive generalization capabilities, but these foundation models are developed for fixed-base manipulation tasks. Therefore, we propose an efficient policy adaptation framework named MoManipVLA to transfer pre-trained VLA models of fix-base manipulation to mobile manipulation, so that high generalization ability across tasks and environments can be achieved in mobile manipulation policy. Specifically, we utilize pre-trained VLA models to generate waypoints of the end-effector with high generalization ability. We design motion planning objectives for the mobile base and the robot arm, which aim at maximizing the physical feasibility of the trajectory. Finally, we present an efficient bi-level objective optimization framework for trajectory generation, where the upper-level optimization predicts waypoints for base movement to enhance the manipulator policy space, and the lower-level optimization selects the optimal end-effector trajectory to complete the manipulation task. In this way, MoManipVLA can adjust the position of the robot base in a zero-shot manner, thus making the waypoints predicted from the fixed-base VLA models feasible. Extensive experimental results on OVMM and the real world demonstrate that MoManipVLA achieves a 4.2% higher success rate than the state-of-the-art mobile manipulation, and only requires 50 training cost for real world deployment due to the strong generalization ability in the pre-trained VLA models.
1. Introduction
Mobile manipulation must coordinate the base and arm across diverse tasks and unstructured environments, but existing approaches face generalization and data-scaling limitations. MoManipVLA transfers fixed-base VLA capabilities through waypoint-guided, physically feasible trajectory optimization.
- Mobile manipulation requires whole-body control to perform diverse tasks across large spaces and unstructured environments.
- Conventional modular methods suffer compounding errors, while end-to-end methods face costly demonstration collection and limited dataset scale.
- VLA models generalize across manipulation tasks, but their fixed-base focus prevents cooperative base-arm action generation for mobile manipulation.
- MoManipVLA transfers fixed-base VLA policies by predicting end-effector waypoints and optimizing reachability, smoothness, collision avoidance, and whole-body trajectories.
- The framework combines policy adaptation with bi-level trajectory optimization to improve generalization across tasks and environments.
2. Related Work
Prior work includes end-to-end and modular mobile-manipulation systems, increasingly capable VLA models, and trajectory-optimization methods. These approaches address unified control, manipulation generalization, embodiment diversity, or physical trajectory constraints through different learning and optimization strategies.
- End-to-end mobile-manipulation methods use imitation learning to predict actions from visual observations, while related work learns unified scene representations for navigation and manipulation.
- Current VLA models directly predict end-effector 7-DoF actions from RGB observations, with prior work exploring autoregressive actions, reasoning, and adaptive waypoint planning.
- Foundation-model-based VLA research improves action decoding and generalization across viewpoints, objects, and higher-degree-of-freedom embodiments.
- Trajectory-optimization research addresses accuracy, latency, and dynamic constraints, while handcrafted objectives can restrict conventional methods to specific tasks.
3. Approach
MoManipVLA transfers fixed-base VLA waypoint predictions to mobile manipulation by jointly planning base and arm trajectories. Its objectives encode reachability, smoothness, and collision avoidance, and bi-level optimization searches for physically feasible motion.
- Overall framework: The framework uses visual observations, language instructions, and robot proprioception to generate next-step mobile-base and arm states.
- Problem formulation: MoManipVLA addresses costly expert-data collection by transferring pre-trained VLA policies to jointly generate base and arm trajectories.
- Policy transfer: Pre-trained fixed-base VLA models generate end-effector waypoints, while whole-body motion planning adapts them to physically feasible mobile manipulation.
- Bi-level optimization: The upper-level optimization predicts base waypoints, and the lower-level optimization selects end-effector trajectories that satisfy the transferred manipulation policy.
- Trajectory generation: The objective function evaluates physical feasibility across planned base and end-effector poses while preserving waypoint-consistent trajectory endpoints.
- Motion planning objectives: Reachability penalizes trajectories requiring excessive inverse-kinematics iterations, indicating poses that are unreachable or near joint limits.
- Motion planning objectives: Smoothness penalizes abrupt arm-joint and base-motion changes, using IK-solved joint angles to better reflect arm configuration changes.
- Motion planning objectives: Collision cost uses ESDF-based object surfaces and sampled robot query points to penalize insufficient safety margins.
4. Experiment
Experiments on OVMM and real-world tasks evaluate MoManipVLA against baselines, ablate its trajectory-optimization components, and examine failure cases and qualitative behavior. The results show gains in benchmark success, the importance of reachability and bi-level search, and real-world performance under challenging visual and physical conditions.
- Comparison with State-of-the-art Methods: 4.2% overall success rate and 11.2% partial success rate gains are achieved on OVMM over state-of-the-art methods.
- Comparison with State-of-the-art Methods: 12.4% higher Pick success rates are obtained than state-of-the-art methods, while inference latency remains close to the RL-based method.
- Ablation Experiment: Reachability, smoothness, and collision objectives each improve overall success rate, with reachability contributing most significantly.Reachability is especially important because base location can limit interaction with targets across large regions.
- Ablation Experiment: Bi-level objective optimization achieves higher success rate and lower latency than ordinary Dual Annealing without search-space decomposition.The comparison indicates that the proposed search strategy reduces search deficiency.
- Ablation Experiment: Groundtruth object masks outperform vision-foundation-model masks because cluttered household scenes substantially degrade visual-perception quality.
- Real World Experiment: 40% success rate is achieved on the real-world mobile manipulation task after fine-tuning with 50 samples, while drawer opening reaches 10% success rate.Drawer opening is constrained by hinge-object motion, physical trajectory requirements, and collision avoidance.
5. Conclusion
MoManipVLA transfers fixed-base VLA models to mobile manipulation by predicting end-effector waypoints and jointly optimizing physically feasible base and arm trajectories. The method demonstrates generalization in simulation and real-world tasks, while remaining limited by VLA performance, constrained movement spaces, and the absence of long-horizon task planning.
- MoManipVLA predicts end-effector waypoints with fixed-base VLA models and generates physically feasible trajectories for the mobile base and robot arm.Its objectives consider end-effector reachability, trajectory smoothness, and collision avoidance within a bi-level optimization framework.
- The bi-level optimization jointly searches for optimal base and arm trajectory poses to complete mobile manipulation tasks.The framework uses trajectory objectives to optimize the mobile base and robot arm together.
- Extensive experiments demonstrate MoManipVLA’s generalization in both simulation and real-world tasks.
- MoManipVLA is limited by the performance of its VLA models and can only be deployed in highly constrained movement spaces.
- The system cannot generate effective motions for long-horizon tasks because it lacks task planning modules.The authors identify more efficient trajectory generation and integration of task planning from foundation models as future work.