PaperlayerView exact revision on arXiv

Source-linked AI summary

PEARS: Physical-Prior-Guided Efficient Adaptation via Failure Reasoning and Diffusion Steering for Tactile Manipulation

Kun Song, Yiming Wang, Yilin Chen, Tianyi Ding, Jiaxin Tian, Tianqi Gong, Daolin Ma, Jia Pan

arXiv:2610.08784v1cs.RO

TL;DR

Pretrained policies can falter under deployment shifts, while RL post-training for contact-rich manipulation can require costly interactions. PEARS combines VLM-guided force reasoning with tactile-conditioned steering of a frozen policy; it improves simulation success over per-task baselines and achieves high success in two real-world tasks.

  • Problem

    Pretrained policies can perform poorly under deployment shifts, and RL post-training for tactile manipulation can require costly, potentially damaging environment interactions.

    Source: §I · pp. 1–2

  • Method

    PEARS combines VLM-based failure diagnosis and contact-force-bound adaptation with a controller that enforces bounds and tactile-conditioned latent-noise steering of a frozen policy.

    Source: §I · p. 2, B. PFR-Guided Contact-Force Control · p. 4

  • Results

    Across three simulation tasks, PEARS improved success rates by 12.4–37.4 percentage points over the strongest per-task baselines; real-world success was 19/20 and 18/20 test trials.

    Source: B. Comparison with Baselines (Q1) · p. 6, §VI · p. 8

  • Takeaways & Limitations

    The authors report faster online adaptation and fewer force-related failures across the three simulation tasks.

    Source: §VI · p. 8

Abstract

from arXiv · show

Pretrained robotic policies can suffer substantial performance degradation under out-of-distribution (OOD) conditions encountered during deployment, motivating post-training through real-world interaction. However, reinforcement-learning (RL)-based post-training typically requires substantial environment interactions, a burden that is especially significant in manipulation, where each trial can be slow, costly, or destructive. Therefore, we present PEARS, a physics-prior-guided hybrid RL framework for sample-efficient online adaptation of pretrained policies with tactile feedback. After each episode, its physics-guided force reasoning (PFR) module uses physical priors encoded in a vision-language model (VLM) to diagnose failures from the visual outcome and tactile interaction history and update task-appropriate contact-force bounds. A high-frequency hybrid force-position controller then enforces these bounds during contact. Complementarily, tactile-conditioned diffusion steering reinforcement learning adjusts the latent noise of the frozen flow-matching policy to correct errors in free-space motion and contact timing without updating the base model. In simulation, PEARS improves success rates by 12.4-37.4 percentage points over the strongest per-task baselines. PEARS also reduces the number of interaction episodes required for a certain success threshold by up to 53.2% relative to the fastest baseline. In real-world experiments, PEARS achieves success rates of 95% on Whiteboard Erasing and 90% on Pipette Liquid Aspiration. These results show that combining the PFR module with policy steering can accelerate adaptation while reducing costly interactions. The project website is available at https://song-kun.github.io/pears.

I. INTRODUCTION

PEARS addresses costly adaptation of pretrained policies under deployment shifts by combining VLM-guided force-bound updates with latent-noise steering. Prior work incorporates tactile representations and reactive control, while RL adaptation remains interaction-intensive.

I. INTRODUCTION

  • OOD shifts in scene configuration, object properties, or contact dynamics can cause positioning errors, mistimed contact, or inappropriate forces in pretrained policies.RL post-training offers a route to adaptation, but interaction-intensive trial and error can be costly or damaging.Source: §I · pp. 1–2
  • PEARS combines VLM-based failure reasoning to adapt contact-force bounds with tactile-conditioned latent-noise steering of the frozen policy.A hybrid force–position controller enforces the bounds during contact, while steering addresses positioning and contact-timing errors.Source: §I · p. 2

A. Tactile-Aware Visuomotor Policies

  • Tactile-aware visuomotor methods use tactile encoders, cross-modal alignment, auxiliary supervision, or tactile pretraining to guide action generation.Some approaches pretrain VTLA models on diverse tactile datasets and sensor configurations.Source: A. Tactile-Aware Visuomotor Policies · p. 2

B. Reactive Tactile Policies for Contact-Rich Manipulation

  • Reactive tactile policies use high-frequency feedback for action correction, including hybrid force–position control and tactile-conditioned refinement.Examples include slow-fast architectures, semantic replanning, and corrections to low-frequency action chunks.Source: B. Reactive Tactile Policies for… · p. 2

C. Reinforcement Learning for Tactile Policy Adaptation

Prior tactile-policy RL methods adapt through autonomous interaction, but many require substantial exploration and lack explicit directional guidance. PEARS builds on flow-matching policies by steering their initial noise rather than directly optimizing physical actions.

C. Reinforcement Learning for Tactile Policy Adaptation

  • Many existing RL approaches for tactile-policy adaptation require substantial environment interaction for stochastic exploration and lack explicit directional guidance.Prior approaches include trajectory-level VLA optimization, demonstration- and intervention-based learning, and online tactile residual policies.Source: C. Reinforcement Learning for Tactile… · p. 2

A. Flow Matching Policy for Imitation Learning

  • Flow-matching and diffusion policies learn a conditional action distribution by mapping noise to demonstrated action chunks, then iteratively transform sampled noise into an action chunk.Behavior cloning trains from state–action demonstrations; VTLA states additionally include tactile observations.Source: A. Flow Matching Policy for Imitation… · pp. 2–3

B. Diffusion Steering Reinforcement Learning

  • DSRL uses the initial noise of a frozen behavior-cloning policy as the RL action, changing its distribution to steer generated actions without updating the pretrained policy.The noise policy is conditioned on state and is adapted toward successful, contact-safe behaviors.Source: B. Diffusion Steering Reinforcement… · p. 3

IV. METHODOLOGY

PEARS separates between-episode contact-force goal updates from within-episode policy steering and force control. Its real-world actor conditions latent-noise adaptation on visual, proprioceptive, and tactile information while the base policy remains frozen.

IV. METHODOLOGY

  • PFR updates a task-specific force goal from interaction history between episodes, while DSRL steers the frozen policy for position alignment and contact timing.During contact, a tactile-gated low-level controller adjusts the nominal action according to the force goal.Source: §IV · p. 3
  • The critic receives a context-augmented observation containing the current observation and force goal, while the actor is conditioned on the observation alone.The force goal remains fixed within each episode and is updated by PFR between episodes.Source: §IV · p. 3

A. Real-World Tactile-Informed Latent RL Actor

  • The latent actor reuses frozen VLM visual tokens and combines them with proprioception and fingertip wrenches to condition its noise distribution.The base policy expands the actor's 10-dimensional noise output into the 132-channel input used for each action-chunk position.Source: A. Real-World Tactile-Informed Latent… · p. 3
  • Actor–critic training treats each executed action chunk as one RL transition and updates the actor through reparameterized noise samples.The replay buffer stores chunk-level reward and a terminal flag; the actor objective includes an exploration-entropy term.Source: A. Real-World Tactile-Informed Latent… · pp. 3–4

B. PFR-Guided Contact-Force Control

PEARS regulates task-relevant contact forces by combining PFR-inferred bounds with hybrid force–position control. The regulated channels can include object–environment wrench components and, when needed, grasp force.

B. PFR-Guided Contact-Force Control

  • Task-specific contact control combines PFR-inferred force bounds with hybrid force–position regulation.The regulated force differs by task: examples include normal force for wiping and both gentle grasping and pulling force for fruit harvesting.Source: B. PFR-Guided Contact-Force Control · p. 4
  • The regulated vector contains the end-effector-frame object–environment wrench and can append scalar grasp force when gripper–object interaction must be constrained.The wrench comprises three-dimensional force and torque.Source: B. PFR-Guided Contact-Force Control · p. 4
  • A projects the wrench onto task-relevant regulated channels, which must satisfy componentwise force bounds during contact.VLM estimates and manual correction initialize the regulated axes; PFR infers the task-conditioned minimum and maximum bounds.Source: B. PFR-Guided Contact-Force Control · p. 4

1) Force-Control Axis Estimation

The force-control axes are estimated with VLM input, then manually reviewed and corrected before they are used to construct the control matrix. Figure 2 depicts stage-aware force control during whiteboard wiping.

1) Force-Control Axis Estimation

  • A VLM estimates the task-relevant principal force-control axis in the end-effector frame, and a person reviews and corrects the estimate before training.The resulting axis defines a row of the control matrix A.Source: 1) Force-Control Axis Estimation · p. 4
  • The selected force or torque axis defines a row of A; grasp-force regulation adds a row selecting the grasp-force channel.The passage also describes A as having one channel for object–environment or grasp-force regulation alone, and two when both channels are regulated.Source: 1) Force-Control Axis Estimation · p. 4
  • Figure 2 depicts stage-aware force control during whiteboard wiping.Source: Figure 2 · p. 4

2) PFR Module:

After each episode, PFR diagnoses whether force was insufficient, excessive, or adequate and updates task-specific force bounds when the evidence supports a change. Its estimates also condition on the accumulated interaction history.

2) PFR Module:

  • After each episode, PFR uses the final contact image, measured force trajectory, and executed bounds to distinguish force failures from positioning, trajectory, or force-tracking errors.Insufficient force shifts the interval upward, excessive force shifts it downward, and indeterminate evidence or non-force failures leave it unchanged.Source: 2) PFR Module · p. 4
  • Visible marks after whiteboard wiping lead PFR to increase the admissible normal-force range; a crushed raspberry instead leads it to reduce the grasp-force range.If the fruit remains attached to the stem, PFR increases the pulling-force range.Source: 2) PFR Module · p. 4
  • PFR conditions each new estimate on the latest multimodal evidence and the preceding episodes’ bounds, outcomes, diagnoses, and updates.Source: 2) PFR Module · p. 4

3) Stage-aware Force Control:

The controller activates force regulation at contact stages, applying PFR-inferred bounds on selected axes while leaving complementary motion to the VLA policy. Stage-aware activation and release govern the handoff between controller and policy.

3) Stage-aware Force Control:

  • The low-level force controller runs at 50 Hz and activates at contact, while the policy runs at 30 Hz and retains control during free-space motion.This avoids interference with the policy during free-space motion.Source: 3) Stage-aware Force Control · p. 4
  • The contact-stage indicator activates when both stable bilateral grasp and load conditions are satisfied.Source: 3) Stage-aware Force Control · pp. 4–5
  • Once active, an admittance controller modifies coordinates selected by A while remaining coordinates follow the VLA action.For each regulated channel, the force or torque reference is the midpoint of its PFR-inferred interval.Source: 3) Stage-aware Force Control · p. 5
  • A sufficiently large disagreement between VLA and force-controller commands ends the contact stage and returns those coordinates to VLA control.The rule treats the policy command as evidence that the contact stage has ended, rather than making the controller oppose intended task motion.Source: 3) Stage-aware Force Control · p. 5
  • When contact is active, selected-axis motion follows the force-controller command and complementary motion follows the VLA command.Source: 3) Stage-aware Force Control · p. 5
  • For multiple contact stages, predicted stage selects the corresponding force axes and bounds, allowing grasp-force control during fruit contact and pulling-force control during detachment.The controller maintains a gentle grasp during the detachment stage.Source: 3) Stage-aware Force Control · p. 5

C. The Complete System of PEARS

The complete-system description combines force-axis initialization, online latent reinforcement learning, stage-aware force arbitration, and episode-level force-bound adaptation. The experiments are organized around adaptation efficiency, PFR predictions, and physical deployment.

C. The Complete System of PEARS

  • Algorithm 1 summarizes PEARS as a pipeline of force-axis initialization, online latent RL, stage-aware force arbitration, and episode-level PFR force adaptation.It begins with demonstrations, a pretrained VLA policy, and a VLM, and includes manual review of selected axes and initial force-bound inference.Source: C. The Complete System of PEARS · p. 5, §V · p. 5

V. EXPERIMENTS

  • The experiments ask whether PEARS improves sample-efficient adaptation, whether PFR infers task-appropriate bounds robustly, and whether it works on physical robots.Source: §V · p. 5

C. The Complete System of PEARS

  • The described training loop samples latent noise for policy actions, stores transitions, updates critics and the latent actor, and applies PFR updates after each episode.Source: §V · p. 5

A. Experimental Setup

The evaluation covers three simulation tasks and two real-world manipulation tasks, using fine-tuned GR00T policies and comparisons with a frozen policy and four online adaptation methods. It specifies the training setup, task-specific observations, and shared RL reward protocol.

1) Evaluation Tasks:

  • PEARS is evaluated on three simulation tasks and two real-world tasks with different contact-force requirements.The simulation tasks are Thin Sheet Transfer, Bottle Cap Twisting, and Fragile Fruit Picking; the real-world tasks are Whiteboard Erasing and Pipette Liquid Aspiration.Source: 1) Evaluation Tasks · p. 5
  • Each task uses 50 offline demonstrations to fine-tune a GR00T N1.7 base policy, with tactile inputs included only for Bottle Cap Twisting and Fragile Fruit Picking.Thin Sheet Transfer and the real-world base policies do not use tactile features; the two other simulation policies include 6D fingertip force–torque measurements.Source: 1) Evaluation Tasks · pp. 5–6

2) Training and Evaluation Detail:

  • GR00T policies are trained for 30 epochs on an NVIDIA H100, taking approximately five hours per task; adaptation and baseline training use an RTX 5090.GPT-5.6 Sol is the VLM backbone of PFR for baseline comparisons.Source: 2) Training and Evaluation Detail · p. 6

3) Baselines:

  • Comparisons include a frozen policy, Residual RL, DSRL, AWR, and DPPO; all RL methods use terminal rewards of +10 for success and −10 for failure.Table I covers online adaptation and final-policy performance; simulation outcomes are automatic, while real-world outcomes are labeled manually.Source: 3) Baselines · p. 6, Table I · p. 6

B. Comparison with Baselines (Q1)

PEARS outperformed the strongest per-task baselines across three simulation tasks, while its ablations indicate complementary roles for learned motion adaptation and online force-interval correction. PFR refined force and torque intervals during interaction, and its guidance improved adaptation speed over random interval search.

B. Comparison with Baselines (Q1)

  • PEARS achieved the highest success rates on Thin Sheet Transfer, Bottle Cap Twisting, and Fragile Fruit Picking: 94.7%, 87.7%, and 77.7%, exceeding the strongest per-task baselines by 12.4, 19.7, and 37.4 percentage points.The reported nAUC values were 94.7%, 90.1%, and 73.4%, respectively.Source: B. Comparison with Baselines (Q1) · p. 6
  • PEARS reached E80 at episodes 10, 22, and 25; on Bottle Cap Twisting, it reduced episodes to threshold by 53.2% versus Residual RL.No baseline reached 80% on Fragile Fruit Picking within 100 episodes.Source: B. Comparison with Baselines (Q1) · p. 6
  • Force-related failure rates were 2.0%, 10.3%, and 3.3% for PEARS, versus 9.7%, 20.3%, and 36.3% for the strongest SR baselines.The task order is Thin Sheet Transfer, Bottle Cap Twisting, and Fragile Fruit Picking.Source: B. Comparison with Baselines (Q1) · p. 6

C. Evaluation of the PFR Module (Q2)

  • PFR’s initially inaccurate force and torque intervals stabilized within ground-truth ranges over the first 12 episodes in Thin Sheet Transfer and Bottle Cap Twisting.Figure 4 illustrates the interval refinement from interaction outcomes.Source: C. Evaluation of the PFR Module (Q2) · p. 6, Figure 4 · p. 7
  • Ablations showed task-dependent benefits: combining PFR-guided force control with DSRL reached 94.7% SR on Thin Sheet Transfer and 87.7% on Bottle Cap Twisting.On Fragile Fruit Picking, removing PFR and force control or removing DSRL reduced SR from 77.7% to 40.3% or 61.7%, respectively; fixed intervals yielded 44.0%. Table II compares PFR, DSRL, force-range selection, and VLM-backbone variants.Source: C. Evaluation of the PFR Module (Q2) · pp. 6–7, Table II · p. 7
  • Random search matched DSRL’s 40.3% mean SR on two-stage Fragile Fruit Picking, while PEARS reached E80 at episodes 10 and 22 versus 55 and 83 for Rand Search on the other tasks.A usable grasping interval alone cannot ensure success if pulling remains ineffective.Source: C. Evaluation of the PFR Module (Q2) · p. 7
  • Qwen3 achieved SRs of 93.0%, 86.7%, and 67.0%, trailing PEARS by 1.7, 1.0, and 10.7 percentage points, respectively.Both VLM backbones improved over DSRL alone across tasks.Source: C. Evaluation of the PFR Module (Q2) · p. 7

D. Real-world Experiment (Q3)

PEARS improved results across simulation and real-world tasks, while its force-reasoning guidance remains limited to force-related failures.

D. Real-world Experiment (Q3)

VI. CONCLUSION & FUTURE WORK

  • Across three simulation tasks, PEARS improved success rates by 12.4–37.4 percentage points over each task’s strongest baseline and achieved 19/20 and 18/20 real-world trials.Source: §VI · p. 8
  • PFR currently guides corrections only for inappropriate force; other errors still rely on conventional RL.Source: §VI · p. 8
Loading 2610.08784v1…