PaperlayerView exact revision on arXiv

Source-linked AI summary

TacSL: A Library for Visuotactile Sensor Simulation and Learning

Iretiayo Akinola, Jie Xu, Jan Carius, Dieter Fox, Yashraj Narang

arXiv:2408.06506v2cs.RO

TL;DR

TacSL addresses the difficulty of generating visuotactile data and learning tactile policies by combining GPU-based simulation with a policy-learning toolkit. It introduces AACD and demonstrates tactile-policy learning and zero-shot sim-to-real transfer for contact-rich manipulation.

  • Problem

    Visuotactile sensing is difficult to simulate because it combines contact, deformation, illumination, and imaging, limiting simulation-based data collection for tactile policy learning.

    Source: §I · p. 1

  • Method

    TacSL integrates GPU-based visuotactile simulation with sensor models, contact-intensive environments, distillation methods, and AACD for tactile policy learning.

    Source: §I · p. 2, A. Policy Distillation · p. 5

  • Results

    TacSL supports zero-shot sim-to-real transfer, including 82.7% success on peg insertion over 81 real-world trials without additional fine-tuning.

    Source: C. Real-Robot Results · pp. 11–12

  • Takeaways & Limitations

    TacSL provides a testbed for tactile sensing and policy learning across contact-rich robotic tasks, including simulation-based analysis and real-world transfer.

    Source: §VIII · p. 12

    TacSL uses a linear Kelvin-Voigt soft-contact model, while nonlinear contact models may provide improved accuracy.

    Source: §VII · p. 12

Abstract

from arXiv · show

For both humans and robots, the sense of touch, known as tactile sensing, is critical for performing contact-rich manipulation tasks. Three key challenges in robotic tactile sensing are 1) interpreting sensor signals, 2) generating sensor signals in novel scenarios, and 3) learning sensor-based policies. For visuotactile sensors, interpretation has been facilitated by their close relationship with vision sensors (e.g., RGB cameras). However, generation is still difficult, as visuotactile sensors typically involve contact, deformation, illumination, and imaging, all of which are expensive to simulate; in turn, policy learning has been challenging, as simulation cannot be leveraged for large-scale data collection. We present TacSL (taxel), a library for GPU-based visuotactile sensor simulation and learning. TacSL can be used to simulate visuotactile images and extract contact-force distributions over $200\times$ faster than the prior state-of-the-art, all within the widely-used Isaac Simulator. Furthermore, TacSL provides a learning toolkit containing multiple sensor models, contact-intensive training environments, and online/offline algorithms that can facilitate policy learning for sim-to-real applications. On the algorithmic side, we introduce a novel online reinforcement-learning algorithm called asymmetric actor-critic distillation (AACD), designed to effectively and efficiently learn tactile-based policies in simulation that can transfer to the real world. Finally, we demonstrate the utility of our library and algorithms by evaluating the benefits of distillation and multimodal sensing for contact-rich manipulation tasks, and most critically, performing sim-to-real transfer. Supplementary videos and results are at https://iakinola23.github.io/tacsl/.

I. INTRODUCTION

TacSL addresses bottlenecks in visuotactile sensing by integrating fast GPU-based simulation with tools for tactile policy learning and sim-to-real transfer. Its simulation models contact and transduction while supporting force fields, tactile images, and scalable learning.

I. INTRODUCTION

  • Visuotactile simulation remains difficult because it combines contact, elastomer deformation, illumination, and imaging, limiting large-scale data collection for tactile policy learning.These challenges affect both sensor-signal generation and the use of simulation for policy training.Source: §I · p. 1
  • TacSL integrates a GPU-accelerated visuotactile simulator into Isaac Simulator, producing RGB images and normal and shear force fields for tactile learning.The module is designed for highly parallel simulation within a general-purpose robotics simulator.Source: §I · pp. 1–2
  • TacSL provides two GelSight models, three randomized robotic assembly environments, and offline and online distillation algorithms for tactile-based policy learning.The toolkit is intended to support prototyping and training, including online distillation for teacher-student policies.Source: §I · p. 2
  • AACD uses a pretrained critic to accelerate on-policy learning with high-dimensional tactile images, while the actor uses observations available on the real robot.The asymmetric design gives the critic privileged force information while keeping the deployable actor restricted to real-world-accessible inputs.Source: §I · p. 2
  • The toolkit supports sim-to-real transfer through soft-contact parameter randomization and image augmentation, including zero-shot transfer of visuotactile policies.The paper also analyzes policy learning from privileged state, vision, tactile images, and multimodal inputs.Source: §I · p. 2

A. Tactile Simulation

  • TacSL’s soft-contact model uses rigid bodies with force-proportional interpenetration to drive imaging and force-field computation, achieving a 300× physics speed-up.The approach approximates elastomer deformation while retaining force and velocity information for contact computation.Source: A. Tactile Simulation · pp. 2–3
  • Integrated image rendering avoids external-renderer I/O and achieves over 200× speed-ups relative to prior visuotactile rendering methods.The rendering process converts simulated deformations into visuotactile images within the parallelized physics simulator.Source: A. Tactile Simulation · p. 3

B. Tactile Policy Learning

TacSL combines GPU-accelerated visuotactile simulation with learning tools for tactile-based manipulation, using soft-contact dynamics to support fast policy-learning workflows.

B. Tactile Policy Learning

  • TacSL’s learning toolkit supports tactile-based policy development for tasks including peg insertion and nut-and-bolt fastening through parallelized experience acquisition.The provided approaches rely on pretrained reinforcement-learning experts with low-dimensional actor and critic inputs.Source: B. Tactile Policy Learning · p. 3
  • TacSL provides a general-purpose simulator for tactile images and force fields, enabling online learning of end-to-end tactile-image policies for prehensile contact-rich manipulation.The module is designed to support policy learning and sim-to-real transfer through integrated simulation and learning tools.Source: B. Tactile Policy Learning · p. 3, §III · p. 3

A. Contact Simulation

  • The contact solver uses Gauss-Seidel-style sequential impulses over configurable substeps, with semi-implicit Euler integration updating body positions and velocities.Each constraint computes and applies an impulse before positions are integrated over the substep.Source: A. Contact Simulation · pp. 3–4
  • The simulator models tactile contact with rigid bodies, penalty-based soft constraints, and a Kelvin-Voigt spring-damper model for membrane interaction.Interpenetration replaces strict non-penetration constraints to recover soft-contact behavior.Source: A. Contact Simulation · p. 4, Figure 3 · p. 4
  • Implicit treatment of the soft spring evaluates force at the end-of-timestep contact state, improving numerical stability and avoiding overshoot.The formulation is mathematically equivalent to an implicit Euler step for the constraint.Source: A. Contact Simulation · p. 4

B. Tactile Image Generation

TacSL generates tactile RGB images by rendering depth inside the sensor and mapping the depth image through a calibrated lookup function.

B. Tactile Image Generation

  • TacSL renders a depth image from a simulated camera at the sensor’s effective optical location, then maps it to RGB using a calibrated lookup table.Rendering occurs in parallel within the simulator without I/O to an external renderer.Source: B. Tactile Image Generation · p. 4

C. Normal/Shear Force-Field Computation

TacSL computes tactile normal and shear force fields efficiently on GPUs and supplies learning methods for high-dimensional visuotactile policies.

C. Normal/Shear Force-Field Computation

  • TacSL computes force distributions by sampling tactile points and combining SDF-based interpenetration distances, velocities, normals, and friction terms.SDF queries and gradient calculations are computationally efficient and parallelizable.Source: C. Normal/Shear Force-Field Computation · p. 5
  • GPU-parallel force computation scales with tactile-point density and the number of simulated sensors, while supporting arbitrary object meshes rather than only primitive shapes.This contrasts with prior serial CPU computation and primitive-shape restrictions.Source: C. Normal/Shear Force-Field Computation · p. 5

IV. EFFICIENT VISUOTACTILE POLICY LEARNING

  • The toolkit provides offline and online policy distillation plus asymmetric actor-critic methods for high-dimensional tactile reinforcement learning.AACD uses a pretrained low-dimensional state-based critic to train a high-dimensional image-based agent.Source: §IV · p. 5
  • TacSL distinguishes privileged state information from real-world observations when learning policies that map tactile observations to actions.The state may include object poses and contact forces, whereas observations include camera and tactile images.Source: §IV · p. 5

A. Policy Distillation

TacSL distills privileged state-based expert policies into policies using real-world-available tactile observations, with offline, online, and reinforcement-learning approaches.

A. Policy Distillation

  • A state-based expert trained with PPO uses privileged simulation information, while the student policy receives tactile images and proprioception available in the real world.The student learns to imitate expert actions for contact-rich tasks.Source: A. Policy Distillation · p. 5
  • Offline behavior cloning is computationally cheap and fast, but its performance depends on the size and quality of the fixed simulated dataset.Online distillation addresses this limitation by allowing the student to collect progressively more experience.Source: A. Policy Distillation · p. 5
  • During online distillation, the expert supplies training labels while either expert or student actions advance the simulator, with student usage increasing as β decreases.The expert action is always recorded as the correct label for visited states.Source: A. Policy Distillation · p. 5, Figure 4 · p. 6, B. Reinforcement Learning · p. 6

B. Reinforcement Learning

  • Tactile policy learning also uses PPO for on-policy reinforcement learning across contact-rich tasks and sensing modalities.This approach is enabled by the fast tactile simulation module.Source: B. Reinforcement Learning · p. 6

1) Asymmetric Actor-Critic:

Asymmetric actor-critic learning uses privileged simulation information for the critic while keeping the actor’s inputs deployable in the real world. AACD extends this framework by pretraining a low-dimensional critic before training a high-dimensional tactile policy.

1) Asymmetric Actor-Critic:

  • The critic uses privileged robot, object, and contact-force information, while the actor uses tactile images and real-world-available observations.The lower-dimensional critic trains faster and provides better Q-value estimates for updating the larger policy network.Source: 1) Asymmetric Actor-Critic · p. 6

2) Asymmetric Actor-Critic Distillation (AACD):

  • AACD targets the exploration trade-off between applying sufficient force for contact-rich tasks and limiting excessive forces that could damage parts or the robot.The method uses a pretrained critic from the low-dimensional agent to guide the high-dimensional agent’s reward-maximizing behavior.Source: 2) Asymmetric Actor-Critic… · p. 6
  • AACD first trains a low-dimensional privileged-state actor and critic, then trains a high-dimensional image-based actor while fine-tuning the pretrained critic.This two-stage design leverages prior critic knowledge to guide exploration for high-dimensional policy learning.Source: 2) Asymmetric Actor-Critic… · p. 6, Figure 5 · p. 7
  • The AACD workflow retains reinforcement learning’s ability to acquire high-dimensional policies while transferring guidance from an expert critic.The expert critic is initialized from the first-stage agent and fine-tuned during student-policy learning.Source: Figure 5 · p. 7

V. VISUOTACTILE SIM-TO-REAL TRANSFER

TacSL supports sim-to-real transfer through randomized soft-contact physics, tactile-image augmentation, and two-stage policy learning. These strategies address sensor variability and make transferable tactile policies tractable under high-dimensional randomization.

V. VISUOTACTILE SIM-TO-REAL TRANSFER

  • TacSL combines soft-contact parameter randomization, tactile-image augmentation, and policy distillation to transfer simulated visuotactile policies to the real world.The authors report that these strategies facilitate successful transfer of tactile policies from simulation to reality.Source: §V · p. 7, C. Two-Stage Policy Learning · p. 8

A. Physics Parameter Randomization

  • Randomizing stiffness and damping lets policies adapt to elastomer-compliance variation and uncertainty in the real-world contact model.The parameters control contact softness and velocity decay and are randomized during training.Source: A. Physics Parameter Randomization · p. 7

B. Tactile Image Augmentation

  • Tactile-image augmentation randomizes spatial transforms and color properties to handle imperfect camera calibration and optical variation across sensors.Episode-level transforms represent variation from using a new sensor, while reduced color augmentation is applied per timestep.Source: B. Tactile Image Augmentation · p. 7, Figure 6 · p. 7

C. Two-Stage Policy Learning

  • Two-stage learning trains a low-dimensional expert with physics randomization, then distills it into a tactile policy using on-the-fly image augmentation.This separates physics randomization during reinforcement learning from image-based randomization during the second stage.Source: §V · p. 7, C. Two-Stage Policy Learning · p. 8

VI. EXPERIMENTAL RESULTS

  • High-dimensional physics and image randomization increases sampling demands, while the teacher-student framework improves sample efficiency and exploration.The first stage uses privileged low-dimensional information; the second stage post-processes image observations during policy learning.Source: C. Two-Stage Policy Learning · p. 8, §VI · p. 8

A. Tactile Simulation Speed

TacSL evaluates GPU-parallel simulation for tactile images and force fields using shape-sensing and ball-rolling tasks. Its reported throughput exceeds CPU baselines by more than 200× for tactile images and reaches larger speedups for force fields.

A. Tactile Simulation Speed

  • TacSL evaluates tactile-image generation by pressing a hex nut against a sensor and force-field generation by rolling a sphere across a tactile surface.The benchmark compares tactile-image rendering with Taxim and force-field generation with the prior ball-rolling setup.Source: A. Tactile Simulation Speed · p. 8, Figure 7 · p. 8
  • 1,631 FPS with 512 parallel environments versus 7.28 FPS for Taxim produces a more than 200× tactile-image speedup.The improvement is attributed to GPU parallelization and minimal GPU–CPU data transfer.Source: A. Tactile Simulation Speed · p. 8, Table I · p. 9
  • 1,541,043 FPS at 10 × 10 resolution with 32,768 environments yields a 428× speedup over the CPU force-field baseline.At 100 × 100 resolution, TacSL reaches 103,493 FPS with 4,096 environments, still a 46× speedup.Source: A. Tactile Simulation Speed · pp. 8–9, Table II · p. 9
  • GPU parallelization substantially reduces both tactile-modality simulation time and overall computation time, although tactile computation remains the majority of runtime.The breakdown uses 512 environments for tactile images and 32,768 environments for 10 × 10 force fields.Source: A. Tactile Simulation Speed · p. 9, Table III · p. 9

1) Tasks:

  • The task suite includes Peg Placement, Peg Insertion, and Bolt-on-Nut Alignment for Screwing, with the last retained as a challenge task rather than reported in the main results.Peg Placement requires tactile alignment of a rotated rod, while Peg Insertion starts from random rod position and orientation.Source: 1) Tasks · p. 9

2) Simulation Results:

TacSL evaluates sensing modalities and AACD on contact-rich manipulation tasks, finding benefits from multimodal inputs and pretrained critics. The experiments also report successful real-world peg placement and insertion deployments.

2) Simulation Results:

  • The simulation study compares privileged state, reduced state, tactile images, tactile force fields, wrist-camera images, and combinations across behavior cloning, DAgger, and asymmetric actor-critic reinforcement learning.Table IV reports success rates for Peg Placement and Peg Insertion under these sensing and algorithm combinations.Source: 2) Simulation Results · pp. 9–10, Table IV · p. 10
  • Peg Placement requires upright stabilization, Peg Insertion requires insertion into a fixed socket, and both tasks begin with randomized object configurations.The task suite also includes Bolt-on-Nut Alignment for Screwing as a challenge task.Source: Figure 8 · p. 10
  • Tactile images and wrist-camera inputs both recover high Peg Insertion performance, while combining them yields higher success rates and faster learning.The wrist camera offers broader views and faster socket localization, whereas tactile sensing provides fine in-gripper estimates and robustness to illumination changes.Source: 2) Simulation Results · p. 10, Table IV · p. 10
  • The multimodal comparison uses tactile images, a wrist image, or both with minimal channel-swapping augmentation and a noisy socket-position estimate.These inputs are evaluated through training curves for contact-rich policy learning.Source: 2) Simulation Results · p. 9, Figure 9 · p. 11

3) AACD Policy Learning Results:

  • AACD with a pretrained critic learns the high-dimensional insertion task, whereas conventional AAC fails; frozen critics train fastest and unfrozen critics achieve the highest asymptotic performance.The pretrained-critic advantage becomes more pronounced with stronger image augmentation.Source: 3) AACD Policy Learning Results · p. 11, Figure 10 · p. 11
  • Real-world deployment evaluates Peg Placement with a 16 mm peg and Peg Insertion with a socket providing 5 mm diametral clearance.The insertion socket is a higher-clearance version of an asset from [62].Source: Figure 11 · p. 11

C. Real-Robot Results

TacSL policies transferred zero-shot from simulation to real peg placement and insertion, with tactile representations and augmentation affecting placement success. The authors also identify simulation-model and architecture limitations while positioning TacSL as a toolkit for contact-rich policy learning.

C. Real-Robot Results

  • 91.4% success was achieved by Diff+ColorAug on real peg placement, compared with 87.7% for ColorAug and 27.2% for Vanilla across 81 trials.Diff+ColorAug subtracts the nominal tactile image from the current image, whereas ColorAug uses augmented raw RGB input.Source: C. Real-Robot Results · pp. 11–12
  • 82.7% success was achieved on real peg insertion over 81 trials without additional real-world fine-tuning.The Color-Aug insertion policy was evaluated across three socket locations with randomized robot and peg conditions.Source: C. Real-Robot Results · p. 12
  • Zero-shot real-robot transfer remained reactive and robust to human perturbations of the peg during precise placement and insertion.The experiments also target reflective metallic parts under challenging industrial lighting conditions.Source: C. Real-Robot Results · pp. 11–12

VII. LIMITATIONS AND FUTURE WORK

  • TacSL’s limitations include a linear Kelvin-Voigt contact model, calibrated RGB-to-depth lookup tables, and tactile-image transfer that does not yet cover force-field modalities.The authors also identify transformers and diffusion models as future policy architectures.Source: §VII · p. 12

VIII. CONCLUSION

  • TacSL combines accelerated tactile simulation, contact-rich tasks, and AACD-based learning tools to support zero-shot tactile policy transfer.The conclusion frames the framework as a testbed for tactile sensing in contact-rich robotic tasks.Source: §VIII · p. 12

APPENDIX A DYNAMICS SOLVER DETAILS

TacSL uses a substepped Temporal Gauss-Seidel solver with stabilization to resolve soft-contact dynamics efficiently. Its calibration and training procedures match simulated tactile impressions to real measurements while randomizing environments and using recurrent visual policies.

APPENDIX A DYNAMICS SOLVER DETAILS

  • The Temporal Gauss-Seidel solver divides each frame into N substeps and sequentially processes constraints to improve stability in complex interactions.Each substep uses timestep Δt/N before integrating updated body states.Source: Appendix A · pp. 12–13
  • Constraint impulses use gradients, compliance, and temporal stabilization, while sequential updates promote convergence and limit discrete-timestep error accumulation.The solver applies the resulting velocity changes before integrating positions.Source: Appendix A · p. 13

APPENDIX B CONTACT MODEL CALIBRATION

  • Calibration selected κ = 200 for narrow 110g and 210g contacts and κ = 300 for wider 120g and 220g contacts, with κ = [150 −350] randomized during policy learning.The calibration matched simulated and real tactile-impression surface areas using known weights.Source: Appendix B · p. 13, Figure 13 · p. 13

APPENDIX C TRAINING DETAILS

  • Training randomizes robot, peg, socket, soft-contact, damping, and observation conditions to generate varied initial states and imperfect pose estimates.The randomization bounds are summarized in Table V.Source: Appendix C · p. 13, Table V · p. 13
  • The policy combines a CNN visual encoder, low-dimensional observations, an LSTM, and an MLP, while expert training omits the CNN.PPO, behavioral cloning, and DAgger settings are provided separately.Source: Appendix C · p. 14, Table VI · p. 14, Table VII · p. 14

A. Policy Learning with different assets.

TacSL supports multiple tactile assets and sensors while comparing simulated outputs with real force-field and image measurements. The framework uses sampled tactile points, calibrated sensor parameters, and visual or force-field renderings to broaden simulation coverage.

A. Policy Learning with different assets.

APPENDIX D TACTILE FORCE FIELD EDGE-TEST

  • Simulated and real shear force-fields matched in directionality and correlated well in magnitude during the socket-edge test.The comparison uses normalized tactile flow maps for interactions with four socket edges.Source: Appendix D · p. 14, Figure 14 · p. 14

APPENDIX E TACTILE FORCE-FIELD NOTATION

  • Tactile force-field computation runs as a postprocessing step after each dynamics step on predefined sensor-point grids corresponding to physical markers.Its penetration depth refers to different point categories than the dynamics solver’s contact depth.Source: Appendix E · pp. 14–15

APPENDIX F TACTILE IMAGE VISUALIZATION

  • Careful calibration can maximize simulated-real tactile-image similarity, but sensor degradation and elastomer replacement make training randomization crucial for robustness.Figure 15 provides qualitative simulated and real tactile-reading comparisons.Source: Appendix F · p. 15, Figure 15 · p. 15
  • TacSL supports GelSight R1.5 and Mini sensors through sensor-specific elastomer meshes, soft-contact parameters, camera pose, and camera intrinsics.Depth maps are converted to RGB images using a tensorized calibrated polynomial lookup table.Source: Appendix F · p. 15, Figure 16 · p. 15

APPENDIX G ADDITIONAL TACTILE VISUALIZATIONS

  • Additional R1.5 visualizations show tactile images and force fields while a 30k-face, 26k-vertex bolt mesh applies downward force and rotates.The bolt is evaluated at varying poses.Source: Appendix G · p. 15, Figure 17 · p. 15
Loading 2408.06506v2…