RoboSpan: Learning Modular Coordination Policies for High-DoF Whole-Body Mobile Manipulation with Teleoperation

I. INTRODUCTION

• Heterogeneous action scales and control periods make module coordination difficult, while long-horizon tasks accumulate errors and existing tactile work leaves cross-module coordination underexplored.The base handles large-range locomotion; arms and hands perform precise manipulation, and their coordination with tactile signals across modules remains a gap.Source: §I · p. 1
• RoboSpan provides a 29-control-DoF whole-body teleoperation and demonstration platform with a foot-IMU interface for synchronous multi-module control and time-aligned data collection.Source: §I · p. 1

2) Full-palm distributed tactile sensing and 2-D representation:

• A full-palm tactile scheme maps distributed contact into 2-D representations, creating a tactile–visual isomorphic feature basis for cross-module tactile empowerment.The introduction identifies full-palm sensing and 2-D representation as a contribution; the tactile–visual isomorphic representation is described as a unified basis for cross-module interaction.Source: 2) Full-palm distributed tactile… · pp. 1–2
• HMF uses VLM-driven stage decomposition to suppress long-sequence error accumulation and COMET inter-modular cross-attention to coordinate heterogeneous modules while preserving per-module control precision.Source: 2) Full-palm distributed tactile… · p. 2

A. Teleoperation and Demonstration Collection for Whole-Body Mobile Manipulation

• Mobile teleoperation platforms extend beyond fixed-base systems, but existing platforms mostly lack high-DoF dexterous hands and distributed touch, with weak coupling between locomotion and upper-limb control.Earlier systems include fixed-base VR bimanual platforms,, mobile ALOHA, and household-task platforms; RoboMatch [12] supports continuous long-horizon demonstrations.Source: A. Teleoperation and Demonstration… · p. 2

B. Distributed Tactile Sensing for Dexterous Manipulation

• Tactile sensing can improve grasp stability and manipulation precision, but most work treats touch as hand-local, leaving its propagation across modules for whole-body decisions underexplored.Contact states can also constrain arm-pose adjustment and base-position calibration.Source: B. Distributed Tactile Sensing for… · pp. 2–3

C. Modular Imitation Learning and Long-Horizon Task Policies

• Unified policies expand action dimensionality, while modular policies lack dynamic information exchange; existing work also separates task decomposition from low-level whole-body coordination.Prior modular methods often target fixed workspaces, while VLM-based decomposition offers an alternative to manual stage partitioning –[12].Source: C. Modular Imitation Learning and… · p. 3

• The platform synchronously controls a mobile base, dual 7-DoF manipulators, dual 6-DoF dexterous hands, and a lifting mechanism across 29 control DoFs.The modules support floor-level grasping, desktop manipulation, and object placement at elevated locations.Source: A. RoboSpan: A Whole-Body… · p. 3
• A master–slave cockpit combines leader arms, hand-control interfaces, and head-mounted visual feedback; foot-IMU posture supplies base-motion and lifting commands.This lets operators use their hands for dexterous manipulation while maintaining whole-body coordination.Source: A. RoboSpan: A Whole-Body… · p. 3
• Vision, tactile feedback, and proprioception share a unified timestamp, producing temporally aligned demonstrations for heterogeneous-module coordination learning.Source: A. RoboSpan: A Whole-Body… · pp. 3–4

• Eleven modular flexible tactile units per hand cover the fingertips, finger pads, and palm to address limited tactile coverage.Source: B. Full-Palm Distributed Tactile… · p. 4
• A 15 × 15 tactile matrix maps each coordinate to a hand-contact region and each pixel to local response intensity, preserving contact locations, pressure patterns, and interaction states.The image format makes tactile information isomorphic with visual features for cross-modal fusion and cross-module interaction.Source: B. Full-Palm Distributed Tactile… · p. 4
• RoboSpan synchronizes tactile images, multi-view images, joint and robot states, and commands at 50 Hz, preserving temporal correspondence for COMET.Source: B. Full-Palm Distributed Tactile… · p. 4

C. HMF: A Hierarchical Modular Coordination Learning Framework

• VLM-driven decomposition divides a long-horizon task into semantically clear stages, with staged execution confining errors to individual sub-tasks.Each stage uses a corresponding COMET policy.Source: C. HMF: A Hierarchical Modular… · pp. 4–5
• The VLM partitions an instructed task into consecutive sub-tasks, and execution switches to the corresponding stage-specific COMET policy as each sub-task completes.Stages can have distinct operational goals and module involvement, with independent policy parameters for each stage.Source: C. HMF: A Hierarchical Modular… · pp. 4, 6
• COMET functionally separates the 29-DoF action space into Hand, Arm, and Mobile modules, each with observations tailored to its control role.Hand observations include tactile images for fine manipulation; Arm observations support end-effector adjustment, while Mobile observations support whole-body spatial configuration.Source: C. HMF: A Hierarchical Modular… · p. 5
• Independent Transformer encoders give each module a specialized feature space, while IMCA uses other modules’ features as keys and values for dynamic information fusion.For example, hand-detected slippage can inform arm pose adjustment and mobile-base correction.Source: C. HMF: A Hierarchical Modular… · pp. 5–6
• After cross-module fusion, independent decoders predict module action chunks, which are concatenated for robot control and trained with a weighted behavior-cloning loss.The loss combines the three modules’ demonstrated-versus-predicted action-chunk errors.Source: C. HMF: A Hierarchical Modular… · p. 6

IV. EXPERIMENT AND ANALYSIS

• The experiments target HMF coordination and long-horizon execution, tactile and cross-module ablations on contact-rich tasks, and platform demonstration stability and data quality.These are three groups of real-world experiments on the physical RoboSpan platform.Source: §IV · p. 6
• All policies are trained on human teleoperation demonstrations.Source: §IV · p. 6

A. Validation of the HMF Framework

• HMF achieves the highest success rate on all three tasks, outperforming ACT and DP U-Net overall.The tasks are trash cleaning, fruit transfer, and desktop organization; COMET yields moderate gains, while HMF adds cross-attention and staged decomposition.Source: A. Validation of the HMF Framework · p. 6
• HMF maintains a relatively high final-stage success rate with a gentler decline, while ACT and DP U-Net drop sharply as tasks proceed.The reported stage-wise comparisons cover desktop organization and trash cleaning.Source: A. Validation of the HMF Framework · p. 6

B. Ablation of Tactile Sensing and Cross-Module Coordination

• The ablation compares four settings across toy transfer, bell pressing, and transparent-bottle transfer.The settings are ACT without touch, unified ACT with touch, modular ACT with touch, and full HMF; Table IV covers distributed tactile sensing and modular coordination.Source: B. Ablation of Tactile Sensing and… · p. 6, Table IV · p. 7
• Adding touch raises unified ACT’s average success rate from 51.3% to 54.7%, but modular ACT with touch underperforms unified ACT with touch.The reported explanation is that full decoupling blocks hand tactile signals from reaching arm and mobile modules.Source: B. Ablation of Tactile Sensing and… · p. 6
• In qualitative real-task comparisons, HMF’s tactile-module reference supports perceptual coordination relative to ACT.Source: Figure 5 · p. 7
• HMF reaches an average success rate of 61.3% while cross-attention propagates tactile information among the three modules.This supports hand manipulation alongside arm-pose adjustment and base-position calibration.Source: B. Ablation of Tactile Sensing and… · pp. 6–7

C. Evaluation of the RoboSpan Teleoperation Platform

• The platform evaluation compares VR and exoskeleton teleoperation on collection stability and policy-training effectiveness.Source: C. Evaluation of the RoboSpan… · p. 7
• RoboSpan has the highest one-shot collection success across task-duration ranges, with a pronounced stability advantage beyond 20 s.The passage attributes continuity to foot-IMU locomotion, which lets operators adjust the base without interrupting hand operations.Source: C. Evaluation of the RoboSpan… · p. 7, Figure 6 · p. 7
• Policies trained on the collected data achieved the highest real-task success, improving by about 15% over the VR scheme and 10% over the exoskeleton scheme, respectively.Source: C. Evaluation of the RoboSpan… · p. 7

V. CONCLUSION

• Over 20% average success improvement over mainstream baselines was verified in real-robot experiments, alongside notable robustness gains in contact-rich tasks.Source: §V · pp. 7–8
• HMF uses VLM-driven stage decomposition to suppress long-horizon error accumulation and COMET with IMCA to coordinate heterogeneous modules while preserving per-module control precision.Full-palm tactile representations support contact-aware whole-body decision-making.Source: §V · p. 7
• Future work will explore smoother subtask transitions, more efficient inter-module information exchange, and policy generalization in open scenes.Source: §V · p. 8