PaperlayerView exact revision on arXiv

Source-linked AI summary

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

Kelin Yu, Haode Zhang, Harish Ravichandar, Yunhai Han, Ruohan Gao

arXiv:2607.03723v1cs.ROcs.AI

TL;DR

Contact-rich manipulation remains difficult for visual policies because cameras do not directly measure contact forces and geometry. OmniTacTune adapts pretrained visual policies with tactile residuals learned through two-stage real-world RL, improving success from 5–40% to 85–100% across four tasks within 40–80 minutes.

  • Problem

    Visual policies lack direct measurements of contact and forces, limiting reliable performance in contact-rich manipulation.

  • Method

    OmniTacTune uses autonomous base-policy rollouts to warm-start tactile-aware learning, then learns online tactile residual corrections atop frozen visual policies.

  • Results

    5–40% to 85–100% success: OmniTacTune improves visual base policies across four real-world contact-rich tasks within 40–80 minutes.

  • Takeaways & Limitations

    OmniTacTune generalizes tactile residual adaptation across tasks, visual base policies, and tactile representations without offline tactile demonstrations.

  • Takeaways & Limitations

    OMNITACTUNE still requires manual resets and incurs hardware wear during repeated contact-rich interactions, especially with fragile vision-based tactile sensors.

Abstract

from arXiv · show

Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first bootstraps tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5-40% success to 85-100% within 40-80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune-site/

1 Introduction

OmniTacTune addresses the scale gap between visual and tactile data by adapting pretrained visual policies with tactile residual correction through a two-stage real-world RL pipeline. It enables tactile-aware, contact-rich manipulation without training visual policies from scratch.

  • Motivation: Visual data scale across teleoperated demonstrations and human videos supports general policies, but cameras lack direct local force and contact-geometry signals needed for contact-rich manipulation.Visual scaling alone does not resolve contact-rich manipulation challenges.
  • Motivation: Tactile sensing improves precise contact reasoning, yet task-specific collection, setup dependence, and datasets orders of magnitude smaller than visual datasets limit transfer.This mismatch creates a fundamental scale gap because tactile observations are rarely paired with large-scale visual data.
  • Motivation: OmniTacTune combines visual task-level motion priors with tactile feedback, using touch to refine alignment, resistance handling, and other local execution details through practice.The approach leverages vision for global task structure and touch for reliable execution.
  • Method: The two-stage real-world RL pipeline uses autonomous base-policy rollouts to warm-start replay, bootstrap the critic, adapt tactile representations, and then refine a lightweight residual policy online.This design targets sample-efficient real-world tactile adaptation across visual base policies.
  • Contributions: OmniTacTune formulates tactile adaptation as residual correction on pretrained visual policies, avoiding training from scratch and enabling cross-policy tactile-aware practice.The method is designed as a plug-and-play adaptation framework for existing visual policies.

2 Related Work

Prior work learns general visual manipulation policies from scalable robot and human data, but visual policies lack direct contact feedback. Tactile sensing complements vision by providing local information about contact geometry and slip for contact-rich manipulation.

  • Visual Policy Learning from Robot and Human Data: Visual policies are increasingly learned from robot demonstrations, teleoperation data, and human videos to support general manipulation.Robot-data approaches include imitation-learning backbones such as ACT and Diffusion Policy, along with vision-language-action models.
  • Visual Policy Learning from Robot and Human Data: Visual policies still lack direct contact feedback, motivating frozen visual base policies with learned tactile residual corrections for contact-rich execution.This approach preserves the visual policy while adapting its behavior using tactile feedback.
  • Tactile and Multi-Sensory Robot Manipulation: Tactile sensing provides local contact information that RGB observations struggle to infer, including contact geometry and slip.These signals have supported contact manipulation tasks such as grasping, insertion, in-hand manipulation, and dexterous manipulation.

3 Methodology

OmniTacTune adapts tactile feedback to frozen visual policies through a two-stage real-world RL pipeline. Warm-start bootstraps tactile learning from autonomous base-policy rollouts, while online residual RL learns tactile-informed corrections using policy-independent interfaces and object-centric multi-sensory rewards.

  • Overview: The pipeline targets policy-agnostic residual tactile adaptation across visual base policies, tasks, and tactile representations.It addresses the lack of tactile-aware initialization from offline visual-only demonstrations.
  • Stage I: warm-start: Warm-start rollouts initialize the replay buffer, bootstrap a flow-tactile critic, and fine-tune a pretrained tactile encoder.The encoder uses critic loss and reconstruction regularization on contact transitions, jointly stabilizing subsequent tactile residual learning.
  • Stage I: warm-start: Trajectory-level tactile augmentation synthesizes two contact-rich trajectories per rollout while preserving robot states, actions, and reward labels.The augmented transitions provide additional samples for initializing the tactile representation and flow-tactile critic.
  • Stage II: online tactile residual learning: Online residual RL uses generated keypoint goals and base-policy action chunks as policy-independent inputs, enabling one residual actor to correct diverse architectures.The base policy remains frozen, while a scheduler scales residuals for safe exploration and keypoint conditions support closed-loop subgoal corrections.
  • Object-centric multi-sensory reward: The normalized multi-sensory reward combines object-centric flow and tactile signals to guide near-contact behavior and encourage stable, safe exploration.It assigns terminal success reward 1 and uses reaching and flow rewards to guide the end-effector and sparse subgoal progression.

4 Experiments

Experiments evaluate OmniTacTune’s learning efficiency, compatibility with visual base policies and tactile representations, and performance against visuo-tactile learning baselines across four contact-rich tasks. It improves weak visual policies substantially through tactile residual adaptation and online refinement.

  • Experimental scope: Experiments span four contact-rich tasks and assess learning efficiency, visual-policy compatibility, visuo-tactile baselines, tactile representations, and ablations.The tasks use an xArm7 equipped with a GelSight Mini gripper and an Intel RealSense D435 camera.
  • Real-world learning efficiency: 40%, 10%, 5%, and 5% are the base-policy success rates for Peg-in-Hole, Charger Insertion, Cap Opening, and Box Opening, respectively, over 20 trials.For each task, the base human flow policy is trained from 50 human demonstrations.
  • Real-world learning efficiency: 100%, 100%, 90%, and 85% are OmniTacTune’s final success rates on the four tasks after 40–80 minutes, versus 5–40% for the base policies.OmniTacTune outperforms all baselines with more than 40% success rate; PLD* improves more slowly and unstably, while PLD (visual only) has the worst overall results without tactile bootstrapping.
  • Compatibility with visual base policies: 75–100% is the success range reached by OmniTacTune across five visual base policies after 50 minutes, starting from 15–50%.The evaluated policies include human-flow, teleoperation-based flow, ACT, DP, and fine-tuned π0.5 policies.
  • Comparison with visuo-tactile methods: 20%–30% higher success rates are achieved by OmniTacTune than imitation-learning-based visuo-tactile methods across ACT, DP, and π0.5 backbones, reaching 100% with the human flow policy.The comparison includes feature fusion for ACT, RDP for DP, and additional tactile tokens with supervised fine-tuning for π0.5.
  • Tactile representations: OmniTacTune consistently improves performance with AnyTouch2, Sparsh, T3, and low-dimensional Tactile Markers across Peg-in-Hole and Charger Insertion.AnyTouch2 and low-dimensional tactile markers achieve comparable final performance, while T3 and Sparsh perform worse than those representations.

5 Conclusion and Limitation · Appendix

OMNITACTUNE adapts tactile feedback to pretrained visual policies through residual correction, using autonomous rollouts and multisensory reward shaping without offline tactile demonstrations. Despite efficient adaptation, it retains real-world RL costs including manual resets and hardware wear during repeated contact-rich interactions.

  • 5 Conclusion and Limitation: OMNITACTUNE is a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction.
  • 5 Conclusion and Limitation: The method enables tactile adaptation without offline tactile demonstrations by using visual base policies as motion priors.
  • 5 Conclusion and Limitation: Autonomous rollouts warm-start tactile-aware critics and encoders, while multisensory reward shaping supports the adaptation pipeline.
  • 5 Conclusion and Limitation: Across four challenging real-world contact-rich manipulation tasks, OMNITACTUNE improves pretrained visual policies.The supplied passage truncates before stating the improvement values.
  • 5 Conclusion and Limitation: OMNITACTUNE inherits common real-world RL limitations, including manual resets and hardware wear under repeated contact-rich interactions.
  • 5 Conclusion and Limitation: These costs are especially relevant when training uses fragile vision-based tactile sensors.
  • 5 Conclusion and Limitation: Future work should pursue safer, more automated real-world training to reduce reset and hardware costs.
  • 5 Conclusion and Limitation: Future directions also include improving motion priors and data augmentation, potentially with world models to generate more physically plausible data.The passage truncates after “physically pla”.

A Implementation Details · A.1 Flow Generator · A.2 Base Visual Policies

The implementation combines calibrated Quest-based human and teleoperation data collection with a flow generator and several visual base policies. The flow generator provides object-centric motion guidance, while the base policies use lightweight flow, diffusion, action-chunking, or visual-language architectures.

  • A Implementation Details: Quest-based collection calibrates hand poses across Quest, world, and robot-base frames for human-video demonstrations.The system synchronizes third-view RGB video with Quest 6-DoF hand poses and finger keypoints.
  • A Implementation Details: Precise grasp demonstrations retarget thumb-index fingertip geometry into a compact binary gripper command rather than copying human joint angles.Grasp intent is detected from temporal thumb-index distance changes and absolute aperture, producing a binary grasp signal for grasping and placing.
  • A Implementation Details: Teleoperation records synchronized RGB observations, robot proprioception, gripper state, and executed delta Cartesian actions for robot demonstrations.These demonstrations train robot-data-based base policies including ACT, Diffusion Policy, and π0.5.
  • A.1 Flow Generator: The flow generator uses LoRA adapters while keeping the pretrained backbone frozen, improving task-specific flow generation without overfitting the small demonstration set.Training combines a point-wise flow prediction loss with a temporal smoothness regularizer.
  • A.1 Flow Generator: At test time, the generator predicts a complete object-centric flow, sparsifies it into L subgoals, and uses them to guide the flow base policy and dense object-centric RL reward.Object flow is used as a cross-embodiment manipulation interface with less embodiment gap.
  • A.2 Base Visual Policies: The flow policy converts generated and observed object flows into compact object-centric inputs and predicts action chunks from these states plus robot proprioception.It uses lightweight LayerNorm MLPs, keeping online-RL inference fast and stable while following demonstration-derived motion priors.
  • A.2 Base Visual Policies: Diffusion Policy, ACT, and π0.5 provide alternative visual bases trained or fine-tuned on teleoperation demonstrations.Diffusion Policy predicts future action sequences through conditional denoising diffusion, ACT predicts action chunks with temporal ensembling, and π0.5 uses RGB plus language instructions with LoRA fine-tuning.

A.3 Baselines

The evaluation compares OmniTacTune with residual RL, visuo-tactile RL, and tactile-augmented imitation or VLA baselines. These baselines vary in tactile use, initialization, and adaptation mechanism.

  • Residual RL baselines: PLD* trains a SAC residual actor on top of the frozen visual base policy using visual/flow and tactile observations, but cannot initialize its critic from tactile-free expert demonstrations.During warm start, PLD* uses only autonomous rollouts.
  • Residual RL baselines: PLD (Visual Only) initializes its critic from offline visual demonstrations and trains a residual policy without tactile observations, rewards, or encoder optimization.This baseline isolates residual RL on the visual policy from tactile-feedback adaptation.
  • Visuo-tactile RL baseline: ViTAL learns a visuo-tactile policy and critic from scratch without critic or replay-buffer initialization or encoder optimization, using the same online interaction budget.Its inputs are flow, proprioception, and tactile observations.
  • Imitation baselines: ACT + Tactile concatenates RGB, tactile, and proprioceptive features before an ACT transformer and trains by imitation learning from teleoperated demonstrations with tactile observations.The RGB image uses the original visual encoder, while the tactile image uses AnyTouch2.
  • Imitation and VLA baselines: RDP uses slow-fast diffusion branches to produce visual plans and refine actions with tactile feedback, while π0.5 + Tactile appends tactile tokens to visual-language-action inputs during supervised fine-tuning.Both are trained on visuo-tactile teleoperation data with the same data budget as the other imitation baselines.

A.4 Warm-start Implementation … A.8 Hyperparameters

OmniTacTune first bootstraps tactile representations and a tactile-aware critic from frozen-base-policy rollouts, then learns conservative tactile residual corrections with SAC. Its implementation combines contact-aware augmentation and rewards with asynchronous real-world training under shared control settings and hyperparameters.

  • A.4 Warm-start Implementation: During warm-start, the frozen base policy executes with zero residual action to collect on-policy tactile experience and initialize the replay buffer and critic.This prevents policy forgetting and bootstraps learning under the base policy’s deployment distribution.
  • A.4 Warm-start Implementation: The tactile encoder is initialized from pretrained representations or trained as a two-layer MLP for marker observations, with updates focused on contact transitions and lightweight components.Image-based encoders freeze their backbone, while marker-based encoders are trained from scratch; the critic uses twin three-layer MLPs.
  • A.4 Warm-start Implementation: ControlTac augments scarce warm-start contact trajectories with temporally consistent tactile sequences conditioned on force and contact-pose variations.Trajectory-level generation applies a sampled force perturbation ∆f ∈[−3, 10] consistently across each contact trajectory.
  • A.5 Online Residual Policy Learning: After warm-start, SAC trains the residual tactile policy and critic while keeping the base policy frozen; a contact-aware gate activates tactile corrections when contact is present.The residual policy uses proprioception, flow and tactile features, the base action, and its action chunk as inputs.
  • A.5 Online Residual Policy Learning: The residual scale st is scheduled from 0–0.15, keeping early updates conservative before allowing stronger contact corrections during online learning.Constraining residual magnitude limits unsafe exploration relative to the base policy.
  • A.7 Training and Inference Details: Real-world RL runs asynchronously, with robot execution parallel to optimization; trainable networks are updated every 500 optimization steps and ControlTac augmentation does not block execution.Deployment runs the frozen visual base policy and residual tactile policy on a single NVIDIA RTX 5080 GPU.
  • A.8 Hyperparameters: Across tasks and base policies, actions are normalized to [−1, 1], control runs at 10 Hz, and episodes are capped at 300 control steps.The implementation uses shared hyperparameters unless otherwise specified.

B Experimental Settings

The experiments evaluate tactile residual adaptation on four contact-rich manipulation tasks: peg insertion, charger insertion, cap opening, and box opening. These tasks require precise alignment, contact maintenance, and corrective motions under tight geometric constraints.

  • Peg-in-Hole Task: Peg-in-hole insertion uses a cylindrical peg and tabletop locations, with a 5mm insertion tolerance requiring precise contact-rich alignment.The task tests whether reaching and grasping generalize across spatial variations while maintaining insertion precision.
  • Charger Insertion Task: Charger insertion uses a roughly 7 mm-wide, 1.5 mm-thick prong and socket dimensions of about 10 mm long and 2 mm wide.The narrow clearance requires tactile exploration, lateral-misalignment detection, and small corrective motions; initial poses are randomized within 5 cm and 10◦.
  • Cap Opening Task: Cap opening requires the robot to contact a cap edge, maintain that contact, and apply rotational levering despite dynamically changing contact geometry.Failure modes include slipping, contacting the wrong region, or using an incorrect tool orientation.
  • Box Opening Task: Box opening requires catching and lifting a very thin GelSight box edge measuring about 1.3 mm while maintaining contact during dynamic levering.Small visual or pose errors can cause the opener to miss the edge or slide away.

C Additional Experiments · C.1 Analysis of Collected Data and Base Policies

The analysis evaluates trajectory quality using SPARC and LDLJ smoothness metrics over successful rollouts. Human demonstrations and their policy produce smoother trajectories than teleoperation counterparts, whose contact-stage artifacts complicate tactile refinement.

  • C.1 Analysis of Collected Data and Base Policies: Trajectory quality is evaluated through smoothness of tracked object-keypoint centroids, using the complementary SPARC and LDLJ metrics.All compared data sources share this object-centric representation.
  • C.1 Analysis of Collected Data and Base Policies: SPARC measures normalized speed-spectrum arc length, with larger values closer to zero indicating smoother trajectories and fewer high-frequency components.The metric captures high-frequency oscillations in the trajectory.
  • C.1 Analysis of Collected Data and Base Policies: LDLJ measures time-domain smoothness by penalizing abrupt acceleration changes, and larger LDLJ values indicate smoother motion.LDLJ is dimensionless and penalizes jerky acceleration and deceleration changes.
  • C.1 Analysis of Collected Data and Base Policies: Mean SPARC and LDLJ over successful rollouts are reported as M-SPARC and M-LDLJ for comparing trajectory sources.The comparison includes human demonstrations, teleoperation demonstrations, and policies trained from each demonstration type.
  • C.1 Analysis of Collected Data and Base Policies: −2.889 M-SPARC and −18.03 M-LDLJ are achieved by human demonstrations, compared with −7.320 and −19.70 for teleoperation demonstrations.The lower teleoperation values indicate more oscillations and jerkier motion under the reported smoothness convention.
  • C.1 Analysis of Collected Data and Base Policies: Teleoperation lacks direct contact-force feedback, making precise contact exploration and insertion difficult and producing noisy corrections during contact-rich phases.These artifacts can be amplified by imitation, resulting in unstable contact behavior and more difficult tactile residual refinement.

C.2 Ablation Studies · D Failure Cases · E Additional plots

The ablations identify multi-sensory reward shaping, trajectory-level residual guidance, warm-start optimization, and scheduled, scaled actions as important design choices. Failure analysis highlights insufficient or imprecise contact, missed alignment, and changing object pose, while additional plots visualize training success rates across policies and tactile representations.

  • C.2 Ablation Studies: Removing any multi-sensory reward component slows learning and lowers final performance in Peg-in-Hole.The reaching reward guides grasping, dense flow provides object-centric insertion supervision, and tactile reward captures grasping signals.
  • C.2 Ablation Studies: Trajectory-level keypoint guidance preserves reactivity, whereas per-step keypoints overconstrain residual policies and can cause ACT to fail.Trajectory-level guidance is temporally sparser while retaining a structured motion prior for local residual refinement.
  • C.2 Ablation Studies: Removing tactile-encoder and critic optimization during warm-start produces unstable early learning and weak overall performance.The ablation indicates that optimizing both components is important for stable residual reinforcement learning.
  • C.2 Ablation Studies: Removing ControlTac also causes unstable performance, highlighting the importance of tactile augmentation during warm-start.This finding is reported as part of the warm-start design ablation.
  • C.2 Ablation Studies: Removing the scheduler slows learning and lowers final performance, while removing action scaling significantly hurts performance.Stage-aware scheduling balances exploration with stable refinement, and scaling constrains residual corrections during contact-rich manipulation.
  • D Failure Cases: Failure cases involve insufficient or imprecise contact force causing slipping, object misalignment missing the effective opening region, and tilted object poses during manipulation.These representative failures are analyzed for Cap Opening and Box Opening.
  • E Additional plots: Additional plots visualize training success-rate curves across different visual base policies and tactile representations.The plots provide supplementary results for Sections 4.2 and 4.4.
Loading 2607.03723v1…