Source-linked AI summary
GelTip: A Finger-shaped Optical Tactile Sensor for Robotic Manipulation
Daniel Fernandes Gomes, Zhonglin Lin, Shan Luo
TL;DR
Finger contacts can occur both outside and inside the grasp closure, yet existing tactile sensors have limited sensing areas. GelTip is a finger-shaped optical sensor designed to capture contacts across its surface. It reports a 5 mm average localization error and under 1 mm in the best cases.
Problem
Existing tactile sensors have limited sensing areas, while contacts around fingers matter both outside and inside the grasp closure.
Source: §I · p. 1, B. Optical tactile sensors · p. 2Method
GelTip uses a finger-shaped optical sensor with a camera at its base to capture contact imprints across the sensor surface.
Source: §I · p. 1Results
5 mm average localization error and under 1 mm in the best cases are reported for GelTip contact localization.
Source: §V · p. 6Takeaways & Limitations
The results support GelTip’s potential for all-round tactile sensing, including capturing detailed textures such as human fingerprints.
Source: §V · p. 6The reported localization error remains significant for tactile sensing; the authors attribute it to the simplistic contact detection algorithm.
Source: §V · p. 6
Abstract
from arXiv · showhide
Sensing contacts throughout the fingers is an essential capability for a robot to perform manipulation tasks in cluttered environments. However, existing tactile sensors either only have a flat sensing surface or a compliant tip with a limited sensing area. In this paper, we propose a novel optical tactile sensor, the GelTip, that is shaped as a finger and can sense contacts on any location of its surface. The sensor captures high-resolution and color-invariant tactile images that can be exploited to extract detailed information about the end-effector's interactions against manipulated objects. Our extensive experiments show that the GelTip sensor can effectively localise the contacts on different locations of its finger-shaped body, with a small localisation error of approximately 5 mm, on average, and under 1 mm in the best cases. The obtained results show the potential of the GelTip sensor in facilitating dynamic manipulation tasks with its all-round tactile sensing capability. The sensor models and further information about the GelTip sensor can be found at http://danfergo.github.io/geltip.
I. INTRODUCTION
Contacts around robot fingers matter both before and during grasping, but existing optical tactile sensors have limited sensing areas. GelTip is a finger-shaped optical sensor designed to detect contacts across its surface, with reported millimetre-scale localization errors.
I. INTRODUCTION
- Finger contacts outside the grasp closure can reveal object properties or collisions, while contacts inside it help sense an object already being grasped.The introduction argues that detecting both kinds of contact calls for sensing across the fingertip surface.Source: §I · p. 1
- Optical tactile sensors use cameras to capture high-resolution images of contact-induced deformation; TacTip tracks membrane markers, while GelSight uses colored illumination and photometric stereo.The passage describes these as two main families of optical tactile sensors.Source: §I · p. 1
- Prior camera-based sensors were described as measuring contact over limited areas on one side of the sensor.This limitation motivates sensing across a finger-shaped surface.Source: §I · p. 1
- GelTip places a camera at the base of a finger-shaped sensor to capture high-resolution contact imprints across its surface, including frontal and side contacts.The reported contact-localization error is 5 mm on average and under 1 mm in the best cases.Source: §I · p. 1
II. RELATED WORK
The related-work section contrasts electronic tactile skins with camera-based optical sensors. Electronic skins accommodate varied shapes but are described as having limited spatial resolution and electronics-related challenges.
II. RELATED WORK
- The section frames tactile sensing as assessing object properties such as geometry and texture through physical interaction, in contrast to remote sensing by cameras.Source: §II · p. 2
A. Electronic tactile skins
- Electronic tactile skins transduce applied pressure through electrical characteristics and can be thin, small, and adaptable to varied body-part shapes.The passage groups them into resistive, capacitive, piezoelectric, optical, and OFET categories.Source: A. Electronic tactile skins · p. 2
- Electronic tactile skins can have limited spatial resolution, complicated electronics, and cross-talk between neighboring sensing elements.The passage gives a similar-sized commercial Weiss WTS sensor with 14x6 taxels as an example.Source: A. Electronic tactile skins · p. 2
B. Optical tactile sensors
Camera-based tactile sensors capture touch information inside an enclosed shell, using marker tracking or raw-image analysis. Earlier designs lacked sensing across both the sides and tip of a robotic finger, motivating GelTip’s all-surface design.
B. Optical tactile sensors
- Camera-based tactile sensors place a camera inside an enclosed shell and use marker tracking or raw-image analysis to extract tactile information.The enclosure helps keep captured images unaffected by external illumination variations.Source: B. Optical tactile sensors · p. 2
- TacTip-family sensors track white pins imprinted on a transparent membrane coated black, while related designs use markers at different depths or fluorescent particles.The passage cites TacTip, TacTip-GR2, TacTip-M2, and TacCylinder as examples.Source: B. Optical tactile sensors · p. 2
- GelSight uses the camera image resolution to support high-accuracy geometry reconstruction, but its sensing area is a small flat surface oriented toward the grasp closure.The passage also notes attempts to adapt its morphology and add markers to its membrane.Source: B. Optical tactile sensors · p. 2
- Earlier tactile sensors used distinct designs and working principles, but none sensed the entire robotic-finger surface, including both sides and tip.The passage says manipulation experiments with prior sensors detected contacts only when an object was inside the grasp closure.Source: B. Optical tactile sensors · p. 2
- GelTip addresses the coverage gap with a finger-shaped sensor designed to detect contact throughout its surface.Source: B. Optical tactile sensors · p. 2
A. Overview
GelTip uses a layered tactile membrane and a camera at the sensor base to capture contact imprints. Its opaque shell isolates the membrane from external light, while guided LED rays illuminate it.
A. Overview
- From inside outward, GelTip has a rigid transparent body, a soft transparent membrane, and opaque elastic paint.Source: A. Overview · p. 3
- A camera at the sensor base views the membrane from inside the tube and captures an image of the imprint formed when contact deforms the elastomer.Source: A. Overview · p. 3
- An opaque shell encloses the camera and leaves the tactile membrane as the external interface; LED light rays are guided through the tube and elastomer.Source: A. Overview · p. 3
B. Surface projection into the tactile image
The section derives a projection model that maps tactile-image pixels to points on GelTip’s curved surface, modeled as a semi-sphere joined to a cylinder.
B. Surface projection into the tactile image
- The model maps image-space pixels to 3D sensor-surface points by constraining each projection ray to intersect the membrane.This mapping is needed to detect contact locations on the 3D sensor surface.Source: B. Surface projection into the tactile… · p. 3
- The membrane is modeled as a semi-sphere and an open cylinder sharing radius r, with the sphere centered on the z-axis at distance d.The camera’s focal point is at the origin and it faces along the z-axis.Source: B. Surface projection into the tactile… · p. 3
- An ideal pinhole-camera model projects world-space points into tactile-image pixels using camera extrinsics and intrinsics.The intrinsic matrix encodes focal length, pixel-to-meter ratios, and image-frame offsets; for square pixels, fk and fl are replaced by α.Source: B. Surface projection into the tactile… · p. 3
- Fig. 5 depicts projection rays through the origin and corresponding points on the spherical and cylindrical regions, and the sensor model is validated and visualized there.The ray-surface intersection constrains the z coordinate used in the mapping.Source: B. Surface projection into the tactile… · pp. 3–4, Figure 5 · p. 4
C. Fabrication process
The fabrication process addresses the challenges of a compact, curved sensor body through a test-tube structure, printed components, and a cast, painted elastomer membrane.
C. Fabrication process
- The curved membrane and compact form are the main fabrication challenges for the finger-shaped sensor.An off-the-shelf transparent plastic test tube supplies the rigid curved layer, though it may contain imperfections.Source: C. Fabrication process · p. 4
- The sensor body uses a test tube, a membrane-securing sleeve, a shell for the camera and LEDs, a supporting base, and a three-part elastomer mold.The rigid parts are 3D-printed using FFF and SLA processes.Source: C. Fabrication process · p. 4, Figure 6 · p. 4
- The membrane mixture uses XP-565 and Slacker at a 1:22:22 ratio, with enough mixture for two membranes.More part-B softens the silicone, while Slacker adds tackiness for capturing high-frequency imprints; excess Slacker makes it sticky.Source: C. Fabrication process · p. 4
- A custom airbrushed coating mixes pigment with XP-565 and silicone solvent to avoid the cracking associated with rigid spray-paint coats.The pigments tested were Silver Cast Magic and 1 µm aluminium powder.Source: C. Fabrication process · p. 4
- Three LED clusters, either red, green, and blue or all white, fit into sleeve pockets and use separately tuned resistors.The resistor values range from 30 Ω to 600 Ω.Source: C. Fabrication process · p. 4, Figure 6 · p. 4
IV. EXPERIMENTS
The experiments aim to demonstrate that GelTip can localize contacts at different locations on its finger-shaped body and help a robot perform manipulation tasks.
IV. EXPERIMENTS
- The experiments test whether GelTip can localize contacts across different locations of its finger-shaped body and help a robot perform manipulation tasks.Source: §IV · p. 4
A. Sensor construction parameters
The experiments tune GelTip’s tube size, lens, paint, and mold surface to balance sensing area and image quality. The selected 15 mm tube requires a 170° lens to capture the finger sides.
A. Sensor construction parameters
- A 15 mm tube offers a compromise among observable side area, compactness, and camera-position flexibility; capturing the finger sides requires a 170° lens rather than the default 70° lens.The 15 mm tube is the smallest tested size that fits the camera lens.Source: A. Sensor construction parameters · p. 5
- On GelTip’s curved surface, aluminium-powder paint produces nonuniform color and a poorly illuminated tip, yielding a poor tactile signal at tip contacts.Fig. 7 compares aluminium-powder paint with off-the-shelf metallic elastic paint.Source: A. Sensor construction parameters · p. 5, Figure 7 · p. 5
- FFF mold printing leaves noticeable texture near the sensor base, which may hinder recognition of high-frequency geometries such as fingerprints.Fig. 8 compares surface textures from FFF and SLA printing.Source: A. Sensor construction parameters · p. 5, Figure 8 · p. 5
B. Contact localisation
GelTip contact localization is evaluated by tapping seven printed objects at positions spanning fingertip rotations and side translations. Errors vary with both object shape and contact position.
B. Contact localisation
- The evaluation uses one run of eight taps on each of seven printed objects, across four fingertip orientations and four side translations.The tested objects are a cone, sphere, irregular prism, cylinder, edge, tube, and slab; orientations range from 0 to π/3, and translations from 0 to 15 mm.Source: B. Contact localisation · p. 5
- The actuator is initially aligned with the cone tip, then positioned using added rotations and translations, with a small adjustment to compensate for finger thickness.The Tool Center Point is set to the sensor tip.Source: B. Contact localisation · p. 5
- Contact detection subtracts a pre-contact reference image, filters the difference, keeps regions within an area threshold, and predicts the weighted-average center of the retained clusters.The image difference is mean-filtered with a 15 × 15 kernel before contour and area filtering.Source: B. Contact localisation · pp. 5–6
- Mapping image contacts to finger positions uses a five-parameter projection model, with camera intrinsics calibrated from an annotated initial contact and a 15 mm translation contact.The translation contact is fitted to Eq. 5 to derive α.Source: B. Contact localisation · p. 6
- Errors range from around 1 mm to over 1 cm; rounded or large tops produce larger errors than sharp tops, while near-tip rotations—especially π/4 and π/6—outperform side translations.The aggregate results are reported in Tables I and II.Source: B. Contact localisation · p. 6, Table I · p. 6, Table II · p. 6
- The authors attribute localization errors to weak imprints, sensor flexing, and imperfections in sensor modeling and calibration.Errors over 1 cm commonly occur when an imperceptible imprint leads the algorithm to predict contact near the sensor tip.Source: B. Contact localisation · p. 6
V. CONCLUSIONS
GelTip provides high-resolution tactile readings across a finger-shaped surface and localizes contacts with errors averaging approximately 5 mm, with best cases under 1 mm. The authors identify the simple detection algorithm as a reason for the errors and propose further sensor and manipulation research.
V. CONCLUSIONS
- GelTip captures high-resolution readings across its finger-shaped surface and localizes contacts with approximately 5 mm average error, with best cases under 1 mm.The authors describe the localization error as significant for tactile sensing.Source: §V · p. 6
- The authors attribute the localization error to the simplistic contact-detection algorithm and expect better quantitative results with improved localization methods.They also expect GelTip to support surface reconstruction and incipient slip detection, as with GelSight.Source: §V · p. 6
- Future work will improve the sensor design, demonstrate all-round tactile sensing in manipulation, and apply GelTip to cluttered grasping and deformable-object manipulation.Source: §V · p. 6