Source-linked AI summary
ForceVLA2: Unleashing Hybrid Force-Position Control with Force Awareness for Contact-Rich Manipulation
Yang Li, Zhaxizhuoma, Hongru Jiang, Junjie Xia, Hongquan Zhang, Jinda Du, Yunsong Zhou, Jia Zeng, Ce Hao, Jieji Ren, Qiaojun Yu, Cewu Lu, Yu Qiao, Jiangmiao Pang
TL;DR
Contact-rich VLAs lack explicit force-aware reasoning and active closed-loop force–position control. ForceVLA2 adds force prompts, Cross-Scale MoE fusion, and a dedicated dataset, achieving higher success and reducing overload and unstable-contact failures across five tasks.
Problem
Existing VLAs lack mechanisms for force-aware reasoning and active force–position interaction in contact-rich manipulation.
Method
ForceVLA2 uses force prompts for staged VLM task concepts and a Cross-Scale MoE to fuse them with real-time forces for hybrid control.
Results
ForceVLA2 achieves 66% average success across five tasks, compared with 48% for π0, 35% for π0.5, and 31% for ForceVLA.
Takeaways & Limitations
ForceVLA2 more reliably handles contact-rich manipulation while reducing failures caused by arm overload and unstable contact.
Takeaways & Limitations
For simple tasks requiring only one position–force switch, extra force tokens can add unnecessary degrees of freedom and slightly reduce success.
Abstract
from arXiv · showhide
Embodied intelligence for contact-rich manipulation has predominantly relied on position control, while explicit awareness and regulation of interaction forces remain under-explored, limiting stability, precision, and robustness in real-world tasks. We propose ForceVLA2, an end-to-end vision-language-action framework that equips robots with hybrid force-position control and explicit force awareness. ForceVLA2 introduces force-based prompts into the VLM expert to construct force-aware task concepts across stages, and employs a Cross-Scale Mixture-of-Experts (MoE) in the action expert to adaptively fuse these concepts with real-time interaction forces for closed-loop hybrid force-position regulation. To support learning and evaluation, we construct ForceVLA2-Dataset, containing 1,000 trajectories over 5 contact-rich tasks, including wiping, pressing, and assembling, with multi-view images, task prompts, proprioceptive state, and force signals. Extensive experiments show that ForceVLA2 substantially improves success rates and reliability in contact-rich manipulation, outperforming pi0 and pi0.5 by 48.0% and 35.0%, respectively, across the 5 tasks, and mitigating common failure modes such as arm overload and unstable contact, thereby actively advancing force-aware interactive physical intelligence in VLAs. The project page is available at https://sites.google.com/view/force-vla2/home.
1. Introduction
VLAs connect perception and language to physical action but still lack force-aware reasoning and active closed-loop force–position control for contact-rich manipulation. ForceVLA2 addresses this gap with force-prompt-driven reasoning, force-aware action fusion, and a dataset supporting evaluation.
- Current VLAs perform semantic grounding and language following but lack physical-dynamics reasoning for fine-grained contact interactions.
- Existing VLAs typically treat force as an auxiliary perceptual input rather than an active signal for adaptive force–position interaction.
- ForceVLA2 combines force prompts in the VLM with a Cross-Scale MoE that fuses task concepts and interaction forces for hybrid control.
- ForceVLA2-Dataset contains 1,000 trajectories across 5 contact-rich tasks with multimodal observations, force prompts, and force-aware action supervision.
- 48.0% and 35.0% success-rate increases over π0 and π0.5, respectively, are reported across the 5 tasks.
2. Related Work
Related VLA research has improved multimodal policy learning, but contact-rich manipulation remains limited by weak force integration and sparse standardized force data. Existing tactile and force-aware datasets address parts of this gap without fully providing active force control.
- Vision-language-action models: VLA approaches map visual observations and language instructions to actions using autoregressive, diffusion-based, or flow-based policy formulations.
- Physical modality integration in robot policies: Force-aware work improves contact-rich manipulation, but force is commonly used as a perceptual cue rather than an active control signal.
- Physical modality integration in robot policies: Tactile methods improve robustness under occlusion but do not replace direct force sensing because tactile-to-force estimation remains indirect and noisy.
- Multimodal embodied datasets: Large embodied datasets rarely provide standardized force modalities, producing sparse and inconsistent coverage for contact modeling.
- Multimodal embodied datasets: RH20T and REASSEMBLE add multimodal force data for general manipulation and assembly, respectively, including synchronized visual, proprioceptive, and force signals.
3. ForceVLA2 Framework
ForceVLA2 integrates visual, linguistic, proprioceptive, and force information across reasoning and reactive-control pathways. Its Cross-Scale MoE routes modalities by manipulation phase, while force-aware flow matching generates hybrid force–position commands and subtask transitions.
- Inputs and multimodal integration: ForceVLA2 jointly encodes multi-view images, task prompts, force prompts, EE pose, and force observations for contact-rich manipulation.
- Inputs and multimodal integration: Force prompts provide subtask context for high-level task sequencing and spatial–semantic reasoning when visual cues are insufficient.
- Proprioceptive-force encoding: A direct force pathway bypasses high-level fusion, preserving transient signals for short-term reactive manipulation alongside multimodal state fusion.
- Adaptive routing and decoding: The Cross-Scale MoE uses visual, state, and force experts with dynamic token-wise routing to emphasize the most informative modality at each phase.
- Adaptive routing and decoding: Flow matching progressively denoises actions conditioned on fused visual–language–force context to produce desired pose changes and contact forces.
- Adaptive routing and decoding: The transition indicator combines orientation, distance, and force conditions; exceeding its threshold advances the controller to the next subtask.
4. ForceVLA2-Dataset
ForceVLA2-Dataset is collected through force-feedback teleoperation and records synchronized visual, pose, and force information for contact-rich manipulation. It contains 1,000 trajectories across five tasks and supports force-aware subtask learning.
- Collection and task design: The dataset uses force-feedback GELLO teleoperation to collect demonstrations with natural motion dynamics and explicitly recorded force-control signals.
- Collection and task design: Demonstrations combine three RGB camera views, synchronized EE poses and joint states, and 6D force/torque measurements recorded at 300 Hz.
- Collection and task design: The dataset covers five tasks: press bottle, clean vase, clean board, retrieve plate, and assemble gears, with force-defined subtask boundaries.
- Dataset statistics: 1,000 trajectories contain approximately 500K synchronized timesteps of RGB images, EE poses, and force signals.
- Dataset statistics: Force prompts and force-integrated actions support subtask-structured policy learning for embodied hybrid force–position interaction.
5. Experiments
Experiments evaluate ForceVLA2 on five contact-rich manipulation tasks, comparing baselines, failure recovery, and architectural components. Results show stronger force-sensitive performance, reactive contact maintenance, and consistent gains from the proposed modules.
- Experiment Setting: The benchmark evaluates five contact-rich tasks using success rate over 20 independent trials per task.Tasks include pressing, wiping, retrieving, and assembling.
- Main Results: ForceVLA2 achieves a 66% average success rate across all tasks and surpasses the second-best model by up to 50% on force-sensitive tasks.The comparison includes ACP, π0, π0.5, ForceVLA, and π0 with force input.
- Main Results: ForceVLA2 actively adjusts interaction forces in force-intensive tasks, avoiding overload where other VLAs often fail.The reported tasks include clean board and bottle pressing.
- Main Results: ForceVLA2 rapidly tracks force changes after sudden perturbations, while other VLAs slowly follow the new end-effector pose and lose stable contact.In bottle pressing, lowering the base changes the contact configuration during execution.
- Main Results: ForceVLA2 maintains contact during misaligned vase wiping and retries failed grasps during object search through force-guided probing.These behaviors contrast with trajectory continuation, overload, and failure reported for other VLAs.
- Ablation Study: Success rates increase consistently as Force Prompt, Cross-Scale MoE, and Multimodal Encoder modules are added, with Cross-Scale MoE providing the largest gain.The ablation progressively adds modules on top of π0 across five tasks.
- Ablation Study: Visual and force inputs contribute roughly equally in Cross-Scale MoE, while an extra force token can slightly reduce performance on Clean Board.The reduction is attributed to simple force requirements where additional degrees of freedom may perturb a stable policy.
6. Conclusion
ForceVLA2 combines force-aware hybrid force–position control with a VLA framework and evaluates it on five real-world tasks. It achieves higher average success and more reliable contact-rich manipulation than the reported VLA baselines.
- Conclusion: ForceVLA2 achieves 66% average success across five tasks, compared with 48% for π0, 35% for π0.5, and 31% for ForceVLA.It also surpasses the second-best method by 50 percentage points on assembling gears.
- Conclusion: ForceVLA2 reduces failures caused by arm overload and unstable contact while force prompts, Cross-Scale MoE, and multimodal fusion improve performance monotonically.The full ForceVLA2 variant achieves the best ablation results.
A. Theoretical Analysis of Force Pathway
The analysis identifies a fundamental limitation of position-only control: interaction force is an environment-dependent output, restricting reachable force–position states. Force-aware hybrid control expands controllability by learning to coordinate position and force commands while providing rapid force feedback.
- A direct reactive force pathway improves observability and controllability, enabling rapid feedback, reduced reliance on past trajectories, and active force-guided interaction.
- Position-only control restricts the reachable task space because force is determined by unknown environment dynamics rather than independently controlled.
- The position-only reachable set forms a 6-dimensional manifold embedded in the 12-dimensional task space.
- ForceVLA2 uses a 12-dimensional hybrid force–position action policy that learns from interaction observations and coordinates both command types.
- The effective controllability index is 6/12 = 0.5 for position-only control, while the force-aware policy approaches a task-dependent maximum as training data and physical constraints permit.
B. Statistical details of the Foca-Dataset
The dataset exhibits task-specific force and torque distributions rather than a single common interaction pattern. Its trajectories are additionally decomposed into short-horizon reactive skills defined by force and position changes.
- Force distributions differ by task: pressing concentrates force along negative z, wiping spreads forces across axes, probing centers near zero, and assembly shows broader variation.
- Torque distributions are near zero for pressing but wider for wiping and probing, while gear assembly has the most diverse z-axis torque distribution.
- Figure 7 summarizes force and torque component frequencies, reactive force-position skills, and skill distributions in the dataset.
- Five short-horizon skills are identified from force and position trajectories using task-specific displacement and force thresholds.
C. Implementation and Experiments
The implementation trains the model with a multimodal encoder and evaluates alternative force-injection locations. Force integration into the multimodal encoder and state fusion improves performance, whereas insertion into the VLM pathway degrades it.
- Foca-VLA training uses 8 A100 GPUs, batch size 32, 30,000 steps, approximately 10 hours, and reaches 15 Hz inference on a 4090 GPU.
- The ablation compares force injection in the VLM pathway, the multimodal encoder, and fusion with the EE 6D pose state.
- Force injection into the VLM pathway substantially degrades performance, while multimodal-encoder and state-fusion integration consistently improve it.
- The final design incorporates force at both the multimodal encoder and state-fusion levels.
D. Discussion on hardware and evaluation.
The system is designed for hardware-agnostic deployment through Jacobian-based action mapping, while evaluation emphasizes real-world experiments because force interactions are difficult to model reliably in simulation.
- Jacobian-based mapping decouples the policy from robot kinematics and supports torque-controlled robots, EE force/torque-sensor robots, and torque-interface actuators.
- Real-world evaluation is used because friction and contact modeling make simulation less reliable for force-aware manipulation.
- The evaluation also addresses the absence of a widely available benchmark directly usable for force-aware VLAs.