Source-linked AI summary
OmniTact: A Multi-Directional High Resolution Touch Sensor
Akhil Padmanabha, Frederik Ebert, Stephen Tian, Roberto Calandra, Chelsea Finn, Sergey Levine
TL;DR
Robotic manipulation needs compact tactile sensors that combine rich spatial information with sensitivity around curved fingertips. OmniTact addresses this gap with a multi-camera gel-based sensor and evaluates it on contact-angle estimation and connector insertion, where it outperforms the compared sensing configurations in the reported tasks.
Problem
Existing tactile sensors do not simultaneously provide high spatial resolution, curved-surface sensitivity, and a compact form factor needed for robotic manipulation.
Method
OmniTact uses multiple micro-cameras embedded around a gel-coated fingertip, with image-based learning methods for tactile state estimation and control.
Results
OmniTact provides higher contact-angle estimation accuracy than GelSight over 22.5°–60° and 60°–90° ranges, while both cameras achieve the best connector-insertion performance.
Takeaways & Limitations
Combining high-resolution optical sensing with multidirectional coverage supports tactile state estimation over more contact angles and tactile connector insertion.
Takeaways & Limitations
The prototype is costly because its endoscope cameras cost US$600 each, totaling US$3200 for five cameras.
Abstract
from arXiv · showhide
Incorporating touch as a sensing modality for robots can enable finer and more robust manipulation skills. Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. In this paper, we introduce OmniTact, a multi-directional high-resolution tactile sensor. OmniTact is designed to be used as a fingertip for robotic manipulation with robotic hands, and uses multiple micro-cameras to detect multi-directional deformations of a gel-based skin. This provides a rich signal from which a variety of different contact state variables can be inferred using modern image processing and computer vision methods. We evaluate the capabilities of OmniTact on a challenging robotic control task that requires inserting an electrical connector into an outlet, as well as a state estimation problem that is representative of those typically encountered in dexterous robotic manipulation, where the goal is to infer the angle of contact of a curved finger pressing against an object. Both tasks are performed using only touch sensing and deep convolutional neural networks to process images from the sensor's cameras. We compare with a state-of-the-art tactile sensor that is only sensitive on one side, as well as a state-of-the-art multi-directional tactile sensor, and find that OmniTact's combination of high-resolution and multi-directional sensing is crucial for reliably inserting the electrical connector and allows for higher accuracy in the state estimation task. Videos and supplementary material can be found at https://sites.google.com/berkeley.edu/omnitact
I. INTRODUCTION
OmniTact addresses the need for compact, information-rich tactile sensing by combining high-resolution optical measurements with sensitivity around a curved fingertip. The paper demonstrates this design on tactile state estimation and electrical-connector insertion.
- Existing tactile sensors commonly trade high spatial resolution on flat surfaces for curved-surface sensitivity with lower resolution.High-resolution sensing is important for precise contact-state information in manipulation.
- OmniTact uses multiple micro-cameras and a gel skin to capture deformations from multiple directions around a thumb-shaped fingertip.The design directly casts the gel on the cameras, removing the support plate and empty space used in previous GelSight-style designs.
- The sensor produces high-resolution image arrays that can be processed with learning-based computer-vision pipelines.These signals support inference of contact-state variables from tactile images.
- OmniTact is evaluated on contact-angle state estimation and tactile control for inserting an electrical connector into a wall outlet.The insertion task requires sensing connector placement relative to the end-effector and the outlet relative to the robot.
- The paper reports that multidirectional sensing is critical for connector insertion and supports state estimation over a wider range of contact angles than flat sensing.The demonstrations use OmniTact’s curved, multidirectional sensing capability for both tasks.
II. RELATED WORK
Related tactile sensors either provide high-resolution sensing on flat surfaces or support curved surfaces with lower spatial resolution. OmniTact targets both capabilities while remaining compact and suitable for manipulation tasks involving contacts around a fingertip.
- GelSight-style sensors provide rich, high-resolution images but are designed around a flat sensorized surface.Their images can support estimation of object geometry, grasp stability, and hardness.
- Unidirectional tactile sensors can restrict tasks that require both object localization and perception of objects grasped around multiple sides.A fingertip sensitive on all sides is better suited to contacts occurring on multiple sides of the fingertip.
- Integrating multidirectionality into GelSight designs is difficult because robotic grippers provide limited space.OmniTact uses micro-cameras to integrate multidirectional sensing into the fingertip design.
- BioTac provides tip-and-side sensitivity using 19 electrodes, a pressure sensor, and a thermistor, but with lower resolution than camera-based tactile sensors.This comparison motivates combining curved-surface coverage with camera-based spatial resolution.
- Experiments compare OmniTact with flat GelSight and multidirectional OptoForce sensors on state estimation and connector-insertion control.The paper reports improved performance over GelSight for state estimation and over OptoForce for touch-based insertion control.
III. DESIGN AND FABRICATION
The design section organizes the OmniTact construction around sensor design goals and the fabrication methodology for a multidirectional tactile fingertip.
- The section summarizes the design goals for OmniTact before describing its detailed design and fabrication methodology.
A. Design Goals
OmniTact is intended as a compact, general-purpose tactile sensor that provides accurate and comprehensive contact-state information for robotic manipulation.
- The work aims to build a universal tactile sensor that increases the capabilities of robot manipulation.Its target is more accurate and comprehensive information about contact between the robot and its environment.
- Rich tactile signals should support accurate extraction of control-relevant features such as object positions.Camera-based sensors can obtain high resolution through camera and sensor-skin properties.
- The sensor must fit inside robot fingertips because fingertip size restricts which manipulation tasks are practical.The paper gives picking up small or thin objects such as plates or forks as an example.
3) Omni-directional sensing.:
OmniTact uses multiple micro-cameras and a directly cast gel skin to provide high-resolution sensing around a curved fingertip. Its camera arrangement supplies broad multi-directional coverage, supported by color-coded illumination.
- Functional scope: Sensitivity on multiple sides supports contact estimation across a wider range of settings, including grasping and non-prehensile manipulation.The inner surface supports grasping, while other sides can help localize objects or perform non-prehensile manipulation.
- Sensor architecture: Five micro-cameras sense a rounded fingertip from multiple sides, while gel cast directly around the cameras reduces sensor size.This design removes the support plate used in GelSight-style sensors.
- Camera design: Camera selection targets compactness through a 5 mm minimum focus distance, about 90° field of view, and 1.35 x 1.35 mm side dimensions.These endoscope cameras enable a compact arrangement.
- Field of view: The camera arrangement provides 270° sensitivity vertically and 360° sensitivity horizontally, excluding small blind spots.The blind spots could be reduced with lenses having larger fields of view.
- Illumination: Three red, green, and blue LEDs are placed beside each camera to illuminate the gel from different directions.The LEDs are equally spaced from the cameras, producing an even distribution of light in each camera image.
C. Sensor Fabrication
The sensor is fabricated by assembling micro-cameras, wrapping a flexible LED PCB around their mount, molding silicone rubber around the assembly, and applying a reflective coating.
- Illumination assembly: A custom flexible PCB mounts the LEDs around the cameras.The PCB is wrapped around the camera mount during assembly.
- Assembly process: The micro-cameras are inserted and glued into the camera mount before the flexible PCB is wrapped and glued around it.The assembly uses E6000 silicone-based adhesive to secure camera cables in their channels.
- Assembly process: After the adhesive sets, the camera mount is placed in a mold and filled with silicone rubber to form the sensor body.The cured silicone rubber finger is removed from the mold before coating.
- Sensor coating: The sensor surface is coated with 1 µm aluminum powder mixed with silicone rubber and diluted with solvent.The coating mixture is poured over the sensor surface after reducing its viscosity.
IV. EXPERIMENTAL EVALUATION
The evaluation examines OmniTact in angle-of-contact state estimation and electrical-connector insertion, comparing its state-estimation performance with a flat GelSight sensor. The paper reports that multidirectional sensing is critical for the insertion task.
- Experimental evaluation: The experiments evaluate robotic-arm control, angle-of-contact estimation, and grasping followed by electrical-connector insertion.The angle-estimation comparison uses a standard GelSight sensor sensitive on one flat surface.
- Experimental evaluation: OmniTact’s multidirectional sensing capability is reported as critical to solving the electrical-connector insertion task.The same evaluation demonstrates high spatial resolution and multidirectional sensing through angle-of-contact estimation.
A. Tactile State Estimation - Estimating the Angle of Contact
The angle-estimation experiment evaluates tactile sensing across three contact-angle ranges using randomized rotary-actuator trials. OmniTact outperforms GelSight in the two higher-angle ranges.
- Experimental setup: The sensor is mounted on a CNC-machine end-effector and pressed against a surface to estimate contact angle.The setup simulates a fingertip contacting a surface at different angles.
- Data collection: Three angle ranges—0° to 22.5°, 22.5° to 60°, and 60° to 90°—are tested with 1000 randomized samples per range.The rotary actuator drives the sensor to a random angle within each specified range.
- Results: OmniTact achieves better angle-estimation accuracy than GelSight from 22.5° to 60° and from 60° to 90°.The table reports medians with interquartile ranges, and the flat GelSight surface does not cleanly contact the surface at these angles.
- Results: The results indicate that a curved finger sensorized on multiple sides enables state estimation across a wider range of angles.GelSight still performs better than random by using deformations in its plastic sensor housing.
B. Tactile Control - Electrical Connector Insertion
The electrical connector insertion task evaluates tactile-only control under randomized grasp and end-effector offsets. OmniTact’s two camera views outperform single-view and OptoForce inputs, with each view supporting a different localization need.
- Control policy: The policy predicts the desired end-effector position for insertion from tactile images and is trained on 100 keyboard-controlled demonstrations.The robot begins near the outlet with the fingertip contacting a textured floor plate.
- Results: Both OmniTact cameras produce the best insertion performance, while the top camera alone outperforms the side camera alone.Success is measured over 30 trials, counting only full plug insertions.
- Results: OptoForce achieves the lowest performance, with only 5 of 30 trials successful.The comparison uses tactile feedback alone for connector insertion.
- Role of camera views: The top camera helps localize the end-effector relative to the plug, whereas the side camera helps localize the connector within the gripper.Single-view failures reflect difficulty estimating either plug position in the gripper or lateral insertion position.
V. DISCUSSION AND FUTURE WORK
OmniTact combines high-resolution tactile sensing with sensitivity around a curved fingertip through multiple micro-cameras. The paper reports successful contact-angle estimation and touch-only connector insertion, while identifying camera cost as the main current limitation.
- Design and capabilities: Multiple micro-cameras detect gel-coated fingertip deformations, enabling high-resolution sensing on curved surfaces.The design uses a multi-directional camera arrangement rather than treating high resolution and curved sensing as conflicting goals.
- Demonstrated tasks: A convolutional neural network estimates contact angle and controls electrical connector insertion using touch sensing.The demonstrated tasks are contact-angle estimation and insertion into an outlet.
- Experimental outcome: OmniTact shows higher sensitivity across angles than GelSight and higher touch-only connector-insertion success rates.These are the paper’s reported experimental comparisons.
- Limitation: The prototype costs US$3200, including five endoscope cameras priced at US$600 each.The authors identify camera price as the principal limitation and suggest scale or alternative cameras could reduce cost.
APPENDIX A: EXPERIMENTAL DETAILS FOR ELECTRICAL CONNECTOR INSERTION TASK
The appendix defines the connector-insertion benchmark using randomized initial plug and gripper positions across 30 trials. Hardware includes a modified 3-axis CNC arena and an OptoForce sensor mounted for comparison.
- Hardware setup: The insertion benchmark uses a modified 3-axis CNC with 3D-printed arena pieces and a mounted OptoForce sensor.The hardware setup supports the electrical connector insertion comparison.
- Success criterion: A successful insertion places the connector prongs in the outlet with less than 2 mm remaining between the outlet and connector surfaces.This criterion defines success for every trial.
- Trial protocol: Each experiment runs 30 trials across five predetermined plug positions, with six trials per position.The initial positions span 5.5 mm, while gripper perturbations are sampled over 8 mm in both x and y.