Source-linked AI summary
TONAV: Task-Oriented Navigation and Action-Velocity Chunk Learning for Articulated Object Quadrupedal Mobile Manipulation
Haoran Lin, Mingyu Yang, Pengfei Qi, Kehan Chen, Qiang Diao, Liangji Zeng, Wenrui Chen, Yaonan Wang, Kailun Yang
TL;DR
Quadrupedal mobile manipulation must both reach manipulation-ready configurations and maintain stable contact during articulated-object interaction, while existing approaches can stop at coarse target proximity and omit motion dynamics. TONAV combines task-oriented navigation, position-velocity-coupled teleoperation, and action-velocity chunk learning; real-world experiments report 60.00% overall navigation SR and 80.00% success across all three manipulation tasks.
Problem
Existing navigation may stop near targets rather than reaching manipulation-ready configurations, while static criteria and position-only action representations do not capture evolving motion during sustained articulated-object interaction.
Method
TONAV decomposes language instructions into navigation subgoals, progressively refines the base, collects demonstrations with position-velocity-coupled teleoperation, and jointly models future joint positions and velocities.
Results
60.00% overall navigation SR and 80.00% success across all three manipulation tasks were achieved in real-world experiments.
Takeaways & Limitations
TONAV establishes a unified transition from task-driven target approach to smooth, stable continuous-contact manipulation of articulated objects.
Abstract
from arXiv · showhide
Quadruped mobile manipulation requires two tightly coupled capabilities: reaching manipulation-ready configurations and maintaining stable contact throughout articulated-object interaction. However, existing methods often terminate navigation near the target, leaving a gap between reachability and manipulation readiness, while tracking lag, motion jitter, and contact instability limit continuous interaction. To address these challenges, we present TONAV, a unified framework integrating task-oriented navigation with action-velocity chunk learning. First, we introduce a position-velocity-coupled teleoperation framework that explicitly captures motion dynamics to improve master-follower consistency and collect smooth, temporally consistent demonstrations. Next, task-oriented navigation leverages vision-language reasoning to decompose high-level instructions into executable subgoals and adaptively refine the robot base toward a manipulation-ready configuration. Finally, action-velocity chunk learning jointly models joint positions and their temporal transitions under velocity supervision, enabling smooth and stable sustained-contact manipulation. Real-world experiments across diverse articulated-object tasks demonstrate that TONAV achieves higher success rates in both task-oriented navigation and complete mobile manipulation, mitigating the navigation-manipulation gap and improving continuous-contact interaction. The project page is at https://haochen611.github.io/TONAV.
I. INTRODUCTION
TONAV addresses two coupled challenges in quadrupedal mobile manipulation: reaching manipulation-ready configurations and maintaining smooth, stable contact during articulated-object interaction. It combines task-oriented navigation, position-velocity-coupled teleoperation, and action-velocity chunk learning.
- Motivation: Existing navigation often stops within 1–2 meters of a target, which can leave the robot misaligned, out of reach, or obstructed for manipulation.The resulting mismatch motivates navigation that progressively adjusts the base according to task-specific spatial and kinematic constraints.
- Motivation: Articulated-object manipulation requires continuous contact along constrained object trajectories, so static reachability, orientation, or clearance criteria are insufficient to characterize the evolving motion state.Examples include closing drawers, operating cabinets, and manipulating toilet lids.
- Framework: TONAV decomposes high-level instructions into navigation subgoals and progressively refines the robot base toward a manipulation-ready configuration.The framework uses PP-CoT reasoning and an Adaptive Approach Module for last-mile navigation.
- Results: 60.00% overall navigation SR and 80.00% success across all three manipulation tasks were achieved in real-world experiments.Ablations and trajectory analysis further evaluated PP-CoT, position-velocity-coupled teleoperation, and action-velocity chunk learning.
- Framework: Position-velocity-coupled teleoperation reduces master-follower mismatch and latency while collecting smoother, more temporally consistent demonstrations.It integrates joint-velocity references into impedance-based tracking.
- Framework: Action-velocity chunk learning jointly predicts future joint-position and joint-velocity sequences, modeling both where to move and how to move.The method is intended to improve smoothness, responsiveness, and stability during sustained-contact manipulation.
II. RELATED WORK
Existing navigation methods span coarse target-directed navigation and fine-grained last-mile navigation. Target-directed methods guide robots toward a target’s vicinity, while vision-language and open-vocabulary methods support semantic or language-based goals in unseen environments.
- Navigation Taxonomy: Navigation approaches are broadly divided into coarse-grained target-directed navigation and fine-grained last-mile navigation.The former addresses longer-distance movement toward target vicinity, while the latter handles finer approach behavior.
- Navigation Taxonomy: Recent vision-language and open-vocabulary navigation methods enable legged robots to follow semantic goals or language instructions in unseen environments.
- Navigation Taxonomy: Target-directed navigation guides the robot over relatively long distances toward the vicinity of a target.
B. Mobile Manipulation
TONAV bridges coarse target-directed navigation and fine-grained manipulation by combining task-oriented base-pose refinement with explicit position-velocity modeling. Its pipeline uses language, visual observations, and proprioceptive states to coordinate navigation and contact-rich manipulation.
- Mobile Manipulation Framework: TONAV navigates quadrupedal robots toward manipulation-ready base poses rather than stopping at coarse target proximity.The framework uses language instructions, visual observations, and proprioceptive states.
- Mobile Manipulation Framework: The method overview culminates in action-velocity chunk learning for explicit joint position-velocity modeling in contact-rich manipulation.The supplied method overview introduces this final stage after teleoperation and navigation.
- Mobile Manipulation Framework: Position-velocity-coupled teleoperation incorporates joint-velocity references into impedance-based master-follower tracking to collect smoother demonstrations.
- Mobile Manipulation Framework: PP-CoT decomposes high-level instructions into executable navigation subgoals, while AAM progressively refines the base pose through last-mile navigation.
A. Action-Velocity Teleoperation Framework
The Action-Velocity Teleoperation Framework augments position-based master-follower tracking with velocity references and separates control from recording to produce smoother, temporally consistent demonstrations.
- Motivation: Position-only teleoperation does not explicitly encode motion direction and rate, which can amplify response lag, discontinuities, and contact instability.These effects are especially detrimental to articulated-object manipulation requiring sustained contact along constrained trajectories.
- Velocity Reference Construction: Joint velocities are estimated from consecutive master-arm configurations using a backward finite difference and bounded to suppress noise- or timing-induced peaks.The bounded references use the velocity limit of each joint.
- Coupled Tracking: The teleoperation command couples joint-position and joint-velocity references for follower-arm tracking.The position component specifies configuration, while the velocity component conveys the master arm’s local motion trend.
- Coupled Tracking: The impedance controller combines position stiffness, velocity damping, and feedforward torque to improve follower responsiveness and reduce temporal and trajectory mismatch.The velocity term conveys the local motion trend of the master arm.
- Demonstration Recording: Control runs in an independent high-frequency thread while visual observations and follower feedback are recorded separately, preventing recording from interrupting robot control.Recorded actions include master positions, follower proprioception, and velocities estimated from consecutive master actions.
B. Perception-Planning Chain-of-Thought
Perception-Planning Chain-of-Thought progressively converts high-level instructions into executable navigation sub-instructions through task-conditioned scene perception and perception-guided route planning.
- Motivation: PP-CoT addresses grounding and planning errors by separating scene understanding from route generation instead of performing both in one inference.The framework progressively converts high-level instructions into executable navigation sub-instructions.
- Pipeline: Given a global observation and task instruction, the model extracts a task-relevant scene representation P and then generates navigation sub-instructions Y.The MLLM uses separate perception and planning prompts.
- Task-Conditioned Scene Perception: Task-conditioned perception identifies the task target, candidate landmarks, and traversable regions while filtering task-irrelevant objects.The perception prompt explicitly asks for the target, landmarks, and local surroundings.
- Perception-Guided Route Planning: Perception-guided planning reasons about robot-centric spatial relationships, plans a visible obstacle-avoiding route, and associates directional changes with landmarks.The resulting landmarks support route-based navigation sub-instructions.
C. Adaptive Approach Module
The Adaptive Approach Module refines coarse navigation into a manipulation-ready configuration using delayed-visual-state estimation, target alignment, and workspace-aware stand-off control.
- Motivation: AAM bridges conventional near-target navigation and manipulation-ready base positioning through target-aligned near-field refinement.It is designed to address the lack of guaranteed manipulation-ready poses in conventional object navigation.
- Target observation: AAM grounds the target with open-vocabulary detection and mask propagation, then uses median mask coordinates and valid depth to form a 3D visual observation.The observation is back-projected relative to the onboard camera using camera intrinsics.
- State estimation: An ego-motion-aware Kalman estimator compensates delayed visual updates and provides a control-rate target state using constant-velocity dynamics and robot odometry.Camera motion between image capture and observation arrival is included when incorporating delayed updates.
- Control: AAM computes horizontal alignment and stand-off errors, selecting the desired distance from the robot’s manipulation workspace rather than approaching as closely as possible.The hybrid controller alternates between target alignment and distance-regulated approach, while safety constraints can trigger a stop.
D. Action-Velocity Chunk Learning for Continuous Contact
Action-velocity chunk learning extends position-only action chunks by jointly predicting joint positions and velocities, with consistency constraints aimed at smoother sustained-contact manipulation.
- Motivation: Position-only action chunking may cause rollback, oscillation, and contact instability during articulated-object manipulation.The section motivates explicit modeling of motion evolution during constrained contact.
- Policy design: The proposed policy jointly predicts future joint-position and joint-velocity sequences after navigation reaches a manipulation-ready configuration.RGB observations and proprioceptive state are encoded within an ACT-style CVAE, whose decoder has parallel position and velocity heads.
- Motion consistency: The motion-consistency constraint couples temporal changes in predicted positions with predicted velocities to promote smoother transitions and stable sustained contact.The constraint regularizes intra-chunk motion evolution rather than modeling positions independently.
- Training objective: Training combines position imitation, velocity supervision, motion consistency, and KL regularization.The corresponding loss weights include λv, λc, and β.
- Analysis: Qualitative comparisons report smoother joint trajectories and reduced oscillation than ACT and DP during the Close Drawer task.The trajectory analysis associates the improvement with velocity supervision and position-velocity consistency.
A. Experiment Setup
The real-world setup separates articulated-object mobile manipulation into task-oriented navigation and manipulation, using a quadruped with two arms and zero-shot vision-language planning.
- Hardware and data: The mobile manipulation setup uses a Unitree Go2-W quadruped and two Piper arms.Data preparation decouples the overall task into navigation and manipulation inputs and training data.
- Navigation: Structured PP-CoT prompts guide zero-shot multimodal reasoning to decompose high-level manipulation instructions into sequential low-level navigation subgoals.This supports coarse target-directed navigation before local refinement near the target.
- Inference: PP-CoT is implemented as a zero-shot two-stage inference framework in which perception runs before planning and passes its output to the planning stage.Both stages use Qwen3.7-Max with fixed inference settings described in the implementation.
B. Real-World Experimental Comparison
Real-world comparisons evaluate TONAV on three articulated-object tasks, showing gains in task-oriented navigation and complete manipulation, while qualitative analyses link the gains to progressive pose refinement and motion-consistent control.
- Navigation results: 60.00% overall navigation SR exceeds StreamVLN’s 13.33% and InternNav’s 26.67% by 46.67 and 33.33 percentage points, respectively.TONAV records 80.00%, 60.00%, and 40.00% on Close Drawer, Turn On Lamp, and Lower Toilet Lid.
- Navigation ablation: PP-CoT raises overall SR from 13.33% to 46.67% for Doubao-Seed-2.1-Pro and from 26.67% to 60.00% for Qwen-3.7-Max.Qwen-3.7-Max consistently outperforms Doubao-Seed-2.1-Pro in the reported ablation.
- Navigation behavior: Progressive navigation combines coarse target-directed subgoals with AAM-based local adjustment of orientation and distance to reach a manipulation-ready configuration.The process is reported to reduce the gap between coarse navigation termination and the base pose required for downstream manipulation.
- Manipulation results: 80.00% complete-manipulation success with position-velocity demonstrations exceeds ACT’s 13.33% and DP’s 13.33% overall success rates.Without position-velocity-coupled teleoperation, TONAV achieves 53.33%; with it, TONAV achieves 80.00% across all three tasks.
- Qualitative comparison: TONAV’s terminal base poses are better conditioned for downstream interaction, while its manipulation exhibits more stable and complete behavior than comparison methods.The qualitative analysis contrasts manipulation-ready refinement with approaches whose terminal poses can leave the manipulator near kinematic limits.
- Trajectory analysis: Velocity supervision, position-velocity consistency, and temporally consistent demonstrations are associated with smoother motion, reduced discontinuities, and improved sustained-contact stability.The paper also reports reduced master-follower tracking lag from position-velocity-coupled teleoperation.
V. CONCLUSION
TONAV unifies progressive task-oriented navigation, position-velocity-coupled teleoperation, and action-velocity chunk learning for articulated-object quadrupedal mobile manipulation. The framework targets manipulation-ready configurations and smooth, stable sustained-contact interaction, while identifying tactile sensing and richer multimodal feedback as future directions.
- TONAV combines progressive task-oriented navigation with position-velocity-coupled teleoperation and action-velocity chunk learning.These components form a unified framework for articulated-object quadrupedal mobile manipulation.
- The framework enables robots to reach manipulation-ready configurations and perform smooth, stable sustained-contact manipulation.
- Real-world experiments demonstrate improved navigation and manipulation performance with reduced motion oscillation and contact instability.
- Future work will explore tactile sensing and richer multimodal feedback for robustness against contact disturbances and generalization across scene layouts and initial configurations.