Source-linked AI summary
Dense Tactile Force Distribution Estimation using GelSlim and inverse FEM
Daolin Ma, Elliott Donlon, Siyuan Dong, Alberto Rodriguez
TL;DR
The paper addresses the challenge of reconstructing dense contact-force distributions from deformation in vision-based tactile sensors. It combines GelSlim 2.0 hardware with inverse FEM to infer forces from marker displacements, obtaining physically reasonable distributions and resultant forces close to force-torque measurements. The approach supports high-spatial-density force estimation for contact sensing.
Problem
Force distributions cannot generally be reconstructed reliably from pointwise displacement because gel deformation reflects geometry and internal forces.
Method
The method uses GelSlim 2.0 marker displacements and inverse FEM to reconstruct external loading forces through a mechanical model of the deformable gel.
Results
The reconstructed force distributions are physically realistic, while resultant-force measurements are fairly consistent with force-torque sensor measurements.
Takeaways & Limitations
GelSlim 2.0 provides dense force-distribution sensing by combining improved hardware with numerical reconstruction of contact loading.
Abstract
from arXiv · showhide
In this paper, we present a new version of tactile sensor GelSlim 2.0 with the capability to estimate the contact force distribution in real time. The sensor is vision-based and uses an array of markers to track deformations on a gel pad due to contact. A new hardware design makes the sensor more rugged, parametrically adjustable and improves illumination. Leveraging the sensor's increased functionality, we propose to use inverse Finite Element Method (iFEM), a numerical method to reconstruct the contact force distribution based on marker displacements. The sensor is able to provide force distribution of contact with high spatial density. Experiments and comparison with ground truth show that the reconstructed force distribution is physically reasonable with good accuracy.
I. INTRODUCTION
Vision-based tactile sensors offer high-resolution contact information, but reconstructing force distributions requires modeling how gel deformation depends on geometry and internal stresses. This work combines GelSlim 2.0 with inverse FEM to estimate dense force distributions from measured marker deformation.
- Motivation: Force feedback can improve the robustness, precision, and reliability of robotic manipulation, especially during critical contact transitions.The paper highlights grasping, lifting, reorienting, placing, and releasing as examples where force information is useful.
- Motivation: Vision-based tactile sensors provide high resolution and compliance, but prior work focused more on geometry measurement than force reconstruction.
- Problem: Force and displacement distributions are geometry-dependent, so pointwise proportionality assumptions can produce physically unrealistic force estimates.Marker motion outside the contact patch can result from internal forces in the silicone gel rather than external loading.
- Approach: Inverse FEM estimates force distributions from vision-measured deformation while GelSlim 2.0 provides the corresponding hardware platform.The work reports experiments and validation using the new sensor.
II. RELATED WORK
Prior vision-based tactile sensors inferred force from deformation using calibration, learning, or analytical models, while force-distribution methods often relied on restrictive assumptions. The paper positions iFEM as a global elastostatic alternative to pointwise force estimation.
- Vision-based tactile sensors: Vision-based tactile sensors commonly measure contact location, texture, or geometry, while force information must be inferred from the contact imprint.
- Vision-based tactile sensors: GelSight- and GelForce-style systems use marker motion to infer forces, but reported approaches have noise, texture sensitivity, or restrictive mechanical assumptions.GelForce assumes a semi-infinite elastic surface and makes force at a marker depend only on that marker’s displacement.
- Force transduction arrays: Most force sensor arrays measure only normal-force distributions at roughly 2 mm spatial resolution, while some flexible arrays measure both normal and shear forces.
III. HARDWARE: GELSLIM V2
GelSlim 2.0 augments the earlier sensor with regular displacement markers and optical improvements to support real-time inverse FEM and dense contact-force estimation.
- Markers to track displacement: GelSlim 2.0 adds a regular grid of displacement markers aligned with FEM nodes to distinguish gel shear from object sliding and enable real-time iFEM.The markers are tracked across frames to compile a displacement field.
- Force estimation: The design combines marker-based deformation sensing with gel material properties to estimate a dense contact force field.
- Illumination: Dual-color illumination increases signal contrast relative to the previous white-illumination design.
- Illumination: A curved total-internal-reflection light guide increases brightness and frame rate by reducing mirror reflections in the optical path.
C. More rugged
GelSlim 2.0 uses stronger, pre-compressed components and a fully parameterized CAD design to improve durability and adapt sensor dimensions to different tasks and optical systems.
- More rugged: Stronger components and pre-compression in the optical path reduce wear and help prevent tensile failure during repeated use.The redesign addresses degradation associated with damage to the optical path.
- Parametric design: A fully parameterized CAD design makes GelSlim sensors easier to scale across manipulation tasks and optical systems.The design separates camera, independent, and dependent variables.
- Parametric design: Current sensors can use gel pads as small as 20 mm x 20 mm and finger thicknesses as low as 18 mm.Further thinning is limited by keystone warp and depth-of-field requirements.
IV. NEW CAPABILITY: FORCE RECONSTRUCTION
The paper uses FEM to model the gel’s force–deformation relationship and reconstruct force distributions from marker displacements. Hex-8 elements provide the discretization basis for this model.
- The mesh and FEM model support calculating force distributions from measured marker displacements.
- FEM relates force and deformation through small, simple elements governed by elasticity theory.
- The gel pad is modeled with Hex-8 elements, each containing 8 nodes and 24 degrees of freedom.
B. Force vs. deformation: Stiffness Matrix
The stiffness-matrix formulation discretizes the gel into hexahedral elements and represents nodal forces as a linear function of nodal displacements. Material properties and the mesh determine the stiffness matrix.
- The gel is discretized into m 8-node hexahedron elements with n total nodes covering the visible gel area.For implementation, the mesh can extend beyond the gel bounding box before being cropped to the gel’s actual size.
- External forces are modeled at mesh nodes, and the nodal force vector is linearly related to the displacement vector.
- F = KU, where K is the 3n×3n stiffness matrix obtained using standard FEM theory with 8-node hexahedron elements.
- The stiffness matrix requires the gel mesh, Young’s Modulus, and Poisson’s ratio.Young’s Modulus describes gel stiffness, while Poisson’s ratio measures expansion perpendicular to compression.
- The reported Young’s Modulus and Poisson’s ratio are 147MPa and 0.3223, respectively, and can be sensitive to gel preparation.
C. Displacement measurement
Marker tracking estimates tangential displacements from successive images, then interpolates those measurements onto FEM nodes. A fixed single-layer node model makes all node displacements directly observable.
- The image-processing algorithm detects marker locations in consecutive frames, matches corresponding markers, and compiles their displacement field.It checks nearest-marker correspondences because deformation can cause some markers to be missed.
- Because detected markers may be nonuniformly distributed, node displacement vectors are interpolated from the marker displacement field.
- A single layer of nodes with fixed boundary conditions makes all node displacement vectors directly observable.This avoids solving for unobservable internal nodes.
- The gel’s depth-direction deformation provides the markers’ z-direction displacements.
D. Compensation to projection error
Oblique viewing causes observed marker motion to combine tangential deformation with projection effects from normal deformation. The method models this projection error using the virtual camera and optical path, while ignoring acrylic refraction because its refractive index is close to the gel’s.
- An oblique camera observes marker displacement caused by both tangential and normal gel deformation.
- The method therefore decouples tangential and normal components to recover each marker’s true 3D displacement.
- Projection error is the camera-observed displacement component caused by normal deformation, represented as the difference between measured image coordinates.
- The projection model uses a virtual camera reflected across the mirror and the marker-to-camera vector in the gel plane.
- The projection-error calculation assumes known z-direction marker displacement and uses the silicone gel’s refractive index.
- Acrylic refraction is ignored because the acrylic and silicone gel are designed to have similar refractive indices.
E. The algorithm of force estimation
The force-estimation algorithm precomputes the stiffness matrix for a fixed elastic-skin configuration, enabling real-time force estimation.
- Precomputing the constant stiffness matrix K offline enables real-time force estimation.The matrix is loaded into the system during operation for a particular elastic-skin configuration.
V. EXPERIMENTS AND VALIDATION
Experiments validate iFEM by comparing reconstructed force distributions and resultant forces against known geometry and force-torque measurements. The reconstructed fields are physically plausible, and resultant-force estimates closely match ground truth.
- Validation setup: The validation pushes GelSlim 2.0 against a sphere instrumented with an ATI Gamma force-torque sensor, which supplies ground-truth contact forces.The sensor pushes and slides in different directions, producing both tangential and normal forces.
- Tangential force distribution: iFEM reconstructs tangential forces within the contact patch, unlike displacement vectors that extend beyond it.This reflects that reaction forces occur at contact points while gel deformations spread through the material.
- Tangential force distribution: Tangential force directions can differ from displacement directions, including opposite directions near contact boundaries.Normal deformation is obtained by locating the circular contact patch and computing its 3D geometry from the known sphere radius.
- Normal force distribution: The reconstructed normal force distribution is smooth and consistent with the geometry of spherical contact.The experiment obtains normal deformation through analysis of the circular contact patch.
- Resultant-force validation: The resultant-force standard deviations are (0.244N, 0.201N, 0.322N) in (x,y,z), roughly within 15% of ground-truth measurements.Reconstructed readings remain close to the identity line; remaining errors are attributed to calibration, tracking, and material non-linearity.
VI. CONCLUSION
GelSlim 2.0 combines hardware improvements with inverse FEM to estimate dense contact-force distributions from vision-based gel deformation. Experiments show physically realistic distributions and resultant forces consistent with force measurements.
- GelSlim 2.0 adds dense force-distribution estimation while improving ruggedness, illumination, and parameter adjustability.
- Vision-based tactile sensors provide high-resolution contact imprints, but prior methods generally did not construct force distributions from measured deformation.
- FEM models deformable objects using continuum mechanics, while inverse FEM reconstructs external loading forces from measured deformation.
- iFEM enables GelSlim 2.0 to provide physically realistic contact-force distributions with high spatial density.
- Future work includes optimizing hardware and numerical methods and exploring manipulation control using real-time dense force distributions.