PaperlayerView exact revision on arXiv

Source-linked AI summary

FP2: Equipping Robotic Foundation Models with Force Control

Hongjie Fang, Shirun Tang, Junjian Hu, Shidong Zhang, Derek Zhang, Linhao Chen, Dehai Li, Mingyu Mei, Wanxi Liu, Cewu Lu, Shiquan Wang

arXiv:2609.37433v1cs.RO

TL;DR

Contact-rich manipulation remains challenging because task-level motions may not regulate physical interaction, and force is typically absent from RFM pretraining. FP2 keeps task-level action generation in a task-adapted RFM and adds downstream force regulation conditioned on compressed policy context and physical feedback. Across four real-world tasks, it improves performance and force regulation over corresponding foundation policies, with favorable comparisons to force-aware and force-control baselines.

  • Problem

    Reliable physical interaction remains challenging in contact-rich manipulation, while force is typically absent from RFM pretraining and adding it changes the pretrained observation interface.

    Source: §I · p. 1

  • Method

    FP2 keeps task-level action generation in a task-adapted RFM and trains a separate force-control policy on compressed RFM context plus wrench and proprioceptive histories.

    Source: §I · pp. 1–2

  • Results

    Across four real-world tasks, FP2 improves task performance and force regulation over corresponding foundation policies; on Insert Peg it reaches 76% under both tuning settings, versus 36% for TA-VLA and 32% for ForceVLA after full fine-tuning.

    Source: §I · p. 2, B. Results · p. 6

  • Takeaways & Limitations

    Separating task-level action generation from high-frequency interaction regulation is an effective interface for equipping existing RFMs with force control without changing their native sensing or action-generation interface.

    Source: §V · p. 7

    FP2 assumes the foundation policy provides viable task-level motion and cannot fully recover from incorrect action generation or geometric reasoning.

    Source: §V · p. 7

Abstract

from arXiv · show

Robotic foundation models (RFMs) are increasingly capable of general-purpose manipulation, yet reliable physical interaction remains challenging in contact-rich settings. We present FP2, a lightweight downstream interface that equips task-adapted RFMs with explicit force control while preserving their action-generation capability. FP2 adopts an action-regulation decomposition: the task-adapted RFM serves as a foundation policy responsible for task-level action generation, while a high-frequency force control policy focuses solely on interaction regulation. To condition force regulation on the ongoing manipulation, FP2 compresses foundation-policy contextual representations and combines them with wrench and proprioceptive histories to predict structured force-control parameters. We evaluate FP2 with four RFM backbones across four real-world contact-rich manipulation tasks. FP2 consistently improves task performance and force regulation quality over the corresponding foundation policies, while comparing favorably with representative force-aware and force-control baselines. Ablations further show that foundation-policy context and physical feedback are complementary for effective force regulation, while preserving foundation-policy action generation improves both efficiency and novel-object generalization. Project website: http://force-policy.github.io/fp2

I. INTRODUCTION

Contact-rich manipulation remains difficult because plausible task-level motions can fail under physical uncertainty, while force is typically absent from RFM pretraining. FP2 leaves action generation with a task-adapted RFM and adds a separate force-control policy, improving performance and force regulation across evaluated backbones and tasks.

I. INTRODUCTION

  • Contact-rich manipulation can fail under geometry, friction, or compliance uncertainty, making reliable physical interaction a bottleneck despite capable task-level action generation.Such failures include excessive forces, unstable contact, or jamming.Source: §I · p. 1
  • Force is typically absent from large-scale RFM pretraining, so adding it to the observation interface requires downstream learning to align it with existing visuomotor representations.The paper asks whether force control requires modifying the foundation model to incorporate force as an additional sensing modality.Source: §I · p. 1
  • FP2 retains task-level action generation in the task-adapted RFM and assigns interaction regulation to a separate force-control policy conditioned on physical feedback and foundation-policy context.The context lets the force-control policy account for ongoing manipulation behavior.Source: §I · pp. 1–2
  • Across four RFM backbones and four real-world tasks, FP2 improves task performance and generally reduces normalized force error versus corresponding foundation policies, while comparing favorably with force-control and force-aware baselines.Ablations find context and physical feedback complementary; preserving foundation-policy action generation improves efficiency and novel-object generalization over downstream action re-generation.Source: §I · p. 2

II. RELATED WORKS

Robotic foundation models primarily target task understanding and action generation, leaving physical interaction largely implicit. Prior force-aware policies add force sensing to policy prediction, whereas FP2 introduces force regulation downstream without changing the standard RFM adaptation interface.

A. Robotic Foundation Models

  • Modern RFMs, including VLAs and world models, advance general manipulation through scaled visuomotor learning, but their task understanding and action generation leave physical interaction largely implicit.FP2 addresses this complementary problem downstream of task adaptation, without changing the standard RFM interface.Source: A. Robotic Foundation Models · p. 2

B. Force-Aware Policy Learning

  • Prior force-aware approaches incorporate force into manipulation-policy prediction, requiring the foundation policy to adapt to an additional sensing modality; FP2 instead combines physical feedback with compact RFM context downstream.This keeps force feedback outside the RFM.Source: B. Force-Aware Policy Learning · p. 2

C. Force Control for Contact-Rich Manipulation

Prior force-control methods combine learned policies with impedance or hybrid force-position control, and Force Policy uses upper-level task context with local feedback. FP2 adapts structured interaction regulation to foundation policies while retaining their task-level action generation.

C. Force Control for Contact-Rich Manipulation

  • FP2 follows structured force-control formulations, including Force Policy, but assigns task-level action generation to the foundation policy and predicts only interaction-regulation parameters downstream.This contrasts with a concurrent reflex module that also re-generates reference actions; FP2 compresses contextual tokens and preserves foundation-policy action generation.Source: C. Force Control for Contact-Rich… · p. 2

A. Overview and Problem Formulation

  • The task-adapted RFM remains the action-generating foundation policy, while FP2 compresses its contextual tokens and combines them with wrench and proprioceptive histories to predict structured force-control parameters.Training proceeds by adapting the RFM, compressing context with the RFM frozen, then training force control with both RFM and compressor frozen; the predicted interaction frame, selection mask, and desired wrench accompany the foundation action for execution.Source: Figure 2 · p. 3, A. Overview and Problem Formulation · p. 3

B. Contextual Token Compressor

The compressor reduces RFM contextual tokens to a compact representation for high-frequency force control. It is trained through reconstruction, then used with fresh physical-feedback histories during execution.

B. Contextual Token Compressor

  • A learned contextual-token bottleneck compresses RFM representations so they can be efficiently supplied to the lightweight force-control policy.The compressor projects tokens and uses a learned query with cross-attention to produce a compact representation.Source: B. Contextual Token Compressor · p. 3
  • The compressor is trained self-supervisedly by reconstructing the original contextual tokens from the compressed representation, forcing information through the bottleneck.Training combines token-wise MSE reconstruction and cosine-distance losses; the decoder is discarded afterward.Source: B. Contextual Token Compressor · p. 3
  • During execution, the latest compressed token is cached between 2 Hz foundation-policy updates while the 50 Hz force policy refreshes wrench and proprioceptive histories.This supports reactive regulation without repeatedly running the RFM or compressor.Source: B. Contextual Token Compressor · p. 3

C. Force-Control Policy

FP2 assigns task-level action generation to the foundation policy and physical-interaction regulation to a separate high-frequency force-control policy. The force policy predicts structured regulation parameters using compact foundation-policy context and interaction histories.

C. Force-Control Policy

  • Unlike Force Policy, FP2 leaves task-level action generation to the foundation policy and restricts the high-frequency force policy to interaction regulation.This removes downstream action re-generation and wrist-camera input; the force policy instead uses compact foundation context and interaction histories.Source: C. Force-Control Policy · p. 4
  • The force policy predicts an interaction frame, a force-position selection mask, and a desired wrench for the hybrid force-position controller.Together, these structured signals define local force-motion coordinates, controlled directions, and target wrench.Source: C. Force-Control Policy · p. 4
  • The policy learns these signals from demonstration-recovered targets while the foundation policy and contextual-token compressor remain frozen.Force information is used to train the lightweight force-control policy, not to update the foundation policy or compressor.Source: C. Force-Control Policy · p. 4

D. Robot Controller

The controller reconciles slow foundation-policy inference with faster execution by aligning and smoothing action chunks, then synchronizing force regulation with the resulting trajectory.

D. Robot Controller

  • The 2 Hz foundation policy’s 15 Hz action chunks are aligned for inference latency, fitted with a quintic spline, and resampled at 50 Hz.Waypoint dropout selects a starting point consistent with recent motion using dynamic time warping, and the spline yields continuous position, velocity, and acceleration.Source: D. Robot Controller · p. 4
  • At 50 Hz, force-control parameters and the action reference are jointly passed to the hybrid controller, whose low-level control law runs internally at 1 kHz.The multi-rate design combines 2 Hz foundation-policy inference, 50 Hz interaction regulation, and 1 kHz low-level control.Source: D. Robot Controller · p. 4

IV. EXPERIMENTS

The experiments assess FP2 across real-world contact-rich tasks and multiple robotic foundation models, including comparisons with force-control and force-aware baselines. The setup specifies the robot, demonstrations, evaluation metrics, and trial protocol.

IV. EXPERIMENTS

  • The experiments ask whether FP2 improves performance across RFMs and tasks, how it compares with force-feedback integration, and how context, feedback, and design choices affect regulation.The questions also examine which RFM representation is most effective and whether removing wrist input and action re-generation benefits performance, generalization, and efficiency.Source: §IV · pp. 4–5

A. Setup

  • FP2 is evaluated on Flip Box, Insert EV Charger, Insert Peg, and Wipe Curve, spanning contact maintenance, force-direction regulation, and precise insertion.Insert Peg uses a circular hole with 0.02 mm tolerance, making insertion sensitive to alignment and force regulation.Source: A. Setup · p. 5
  • Demonstrations are collected through arm-to-arm teleoperation with force feedback, with 50 per task for three tasks and 200 for Insert Peg.The platform is a Flexiv Rizon 4 arm with a gripper, a flange-mounted 6-DoF force/torque sensor, and global and wrist RGB-D cameras.Source: A. Setup · p. 5
  • The evaluation uses four backbones—GR00T N1.7, LaWAM, π0, and π0.5—and compares FP2 with task-specific force-control policies and force-aware VLAs.TA-VLA, ForceVLA, and FP2 share the π0 backbone, LoRA adaptation, task demonstrations, and evaluation configurations.Source: A. Setup · p. 5
  • Success rate is reported for Flip Box and both insertion tasks, while Wipe Curve uses completion rate defined by the fraction of marked regions wiped clean.Normalized Force Error measures force departures outside the demonstrated mean ± standard-deviation range; lower NFE indicates smaller departures, and it is reported only for successful trials.Source: A. Setup · p. 5
  • Policies use shared pre-generated test configurations and workspace, with 25 trials per task unless otherwise specified.All methods are deployed on a workstation with an NVIDIA RTX 5090 GPU.Source: A. Setup · p. 5

B. Results

FP2 improves task performance and force regulation across four RFM backbones, and comparisons favor it over representative force-control and force-aware baselines. The results also show that its gains are smaller when the foundation policy already performs well, while task-level motion failures remain outside its primary scope.

B. Results

  • FP2 improves average performance across all four evaluated RFM backbones and generally reduces NFE, with rollouts showing more stable contact than the foundation policy alone.FP2 maintains contact, builds sufficient insertion force, and stabilizes sustained interaction.Source: B. Results · p. 5, Figure 4 · p. 6
  • FP2 gains are smaller when the foundation policy already performs well, and failures in task-level motion generation remain outside its primary scope.Maintaining appropriate contact can keep subsequent robot states closer to the demonstrated interaction regime, particularly for Insert Peg and Wipe Curve.Source: B. Results · p. 6
  • With capable π-series policies, FP2 achieves substantially higher average task performance than task-specific force-control baselines, with lower NFE on three tasks and competitive NFE on Wipe Curve.The comparison includes explicit force-control baselines.Source: B. Results · p. 6
  • On Insert Peg, full fine-tuning raises vanilla π0 from 12% to 60%, while TA-VLA and ForceVLA reach 36% and 32%, and FP2 reaches 76% under both LoRA and full fine-tuning.Under the matched π0 setting, FP2 improves all four tasks while TA-VLA and ForceVLA show less consistent gains.Source: B. Results · p. 6
  • The separation lets interaction regulation run independently at 50 Hz, faster than the RFM action loop.The paper notes this is desirable for rapidly evolving contact dynamics [23].Source: B. Results · p. 7

C. Failure Analysis

FP2 shifts failures away from interaction-related causes, leaving position mismatch and geometric alignment as more common residual issues.

C. Failure Analysis

  • Adding FP2 shifts failures away from insufficient force and unstable contact toward position mismatch and geometric alignment, which lie outside the force-control policy’s primary role.FP2 stabilizes execution when the foundation policy supplies viable task-level motion; it does not replace task-level action generation or geometric reasoning.Source: C. Failure Analysis · p. 7

D. Ablations

The ablations find that force regulation benefits from combining RFM context with physical feedback, and that the compressed context representation performs best. Keeping action generation in the foundation policy also supports efficiency and novel-object generalization.

D. Ablations

  • Using compressed RFM context or proprioceptive histories alone performs poorly, while combining them substantially improves seen- and novel-object success.The context supplies manipulation intent, while physical feedback captures realized interaction.Source: D. Ablations · p. 7
  • The learned compressed contextual token performs best with comparable latency, while average-pooled tokens and action-expert latents also provide useful conditioning.The authors suggest the learned bottleneck preserves contextual information more effectively than simple token aggregation [27].Source: D. Ablations · p. 7
  • Adding wrist observations provides no seen-object benefit, reduces novel-object generalization, and increases inference latency.The comparison is on the Flip Box task with π0.5.Source: D. Ablations · p. 7
  • Allowing πF to regenerate actions substantially degrades seen- and novel-object performance and nearly triples inference latency.Joint action and force prediction with wrist vision also remains inferior to FP2.Source: D. Ablations · p. 7

V. CONCLUSION

FP2 separates task-level action generation from interaction regulation and consistently improves task performance and force regulation across multiple foundation policies and contact-rich tasks. It assumes viable task-level motion, while bidirectional force-control interfaces and reusable multi-task regulation remain future directions.

V. CONCLUSION

  • FP2’s action-regulation decomposition consistently improves task performance and force regulation quality across multiple foundation policies and diverse contact-rich tasks.The foundation policy retains task-level action generation while a lightweight, high-frequency force-control policy regulates interaction using foundation-policy context and physical feedback.Source: §V · p. 7
  • FP2 assumes the foundation policy provides viable task-level motion and cannot fully recover from incorrect action generation or geometric reasoning.Source: §V · p. 7
  • A shared multi-task force-control policy and a bidirectional interface for adapting actions from summarized physical feedback are proposed future directions.Source: §V · pp. 7–8
Loading 2609.37433v1…