PaperlayerView exact revision on arXiv

Source-linked AI summary

GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger

Shaoxiong Wang, Yu She, Branden Romero, Edward Adelson

arXiv:2106.08851v1cs.ROcs.CV

TL;DR

GelSight Wedge addresses the difficulty of measuring high-resolution 3D contact geometry with a compact robot finger under visual occlusion. It combines compact sensor design with photometric-stereo reconstruction, evaluates reduced-light configurations, and demonstrates human-finger-scale fabrication and ICP-based pose tracking.

  • Problem

    Compact robot fingers must satisfy thin mechanical constraints while retaining the optical conditions needed for high-resolution 3D contact-geometry reconstruction.

  • Method

    The paper presents GelSight Wedge, designs its optical and mechanical components for compact sensing, and uses learned missing-gradient estimation with fast Poisson integration for reduced-light reconstruction.

  • Results

    The sensor maintains high-resolution 3D reconstruction in a compact form, supports reduced-light evaluation, and produces reconstructed point clouds used for pose tracking.

  • Takeaways & Limitations

    The design can be shrunk to human-finger size for narrow environments and fine manipulation, while reconstructed 3D geometry shows potential for 3D pose tracking.

Abstract

from arXiv · show

Vision-based tactile sensors have the potential to provide important contact geometry to localize the objective with visual occlusion. However, it is challenging to measure high-resolution 3D contact geometry for a compact robot finger, to simultaneously meet optical and mechanical constraints. In this work, we present the GelSight Wedge sensor, which is optimized to have a compact shape for robot fingers, while achieving high-resolution 3D reconstruction. We evaluate the 3D reconstruction under different lighting configurations, and extend the method from 3 lights to 1 or 2 lights. We demonstrate the flexibility of the design by shrinking the sensor to the size of a human finger for fine manipulation tasks. We also show the effectiveness and potential of the reconstructed 3D geometry for pose tracking in the 3D space.

I. INTRODUCTION

The paper targets compact tactile robot fingers that retain high-resolution 3D contact geometry despite visual occlusion and competing optical and mechanical constraints. It presents GelSight Wedge and applies reconstructed geometry to 3D pose tracking.

  • Visual occlusion during manipulation makes vision-only control challenging, motivating tactile sensing for contact geometry and forces.
  • The design goal is a compact robot finger that provides high-resolution 3D contact geometry for localization and object-pose estimation under visual occlusion.
  • Earlier GelSight sensors provide high-resolution geometry but are too bulky, whereas GelSlim is compact but not optically designed for high-resolution 3D reconstruction.
  • The central challenge is satisfying photometric-stereo optical requirements while maintaining a thin, slim finger-like form with sensing at the tip.
  • GelSight Wedge meets compact-finger mechanical requirements while maintaining high-resolution 3D reconstruction and is combined with ICP for 3D pose tracking.

II. RELATED WORK

The paper situates GelSight Wedge among vision-based tactile sensors that recover contact geometry and forces, then addresses the trade-off between compactness and high-resolution reconstruction. Its approach preserves photometric-stereo reconstruction while reducing calibration demands.

  • Vision-based tactile sensors observe soft-gel deformation to obtain high-resolution contact information, while GelSight uses reflective paint and directional lights for photometric stereo.
  • Contact geometry supports pose estimation, material perception, grasp adjustment, contour following, and ball manipulation, while geometry and forces support slip and manipulation tasks.
  • GelSight Wedge follows GelSlim’s compact-finger direction while retaining high-resolution 3D contact geometry for more accurate robotic applications.
  • The reconstruction pipeline maps colors to surface gradients with a lookup table and integrates them with a fast Poisson solver to obtain depth.
  • The sensor design targets compact robotic form while preserving the optical conditions needed for photometric stereo and achieves higher-fidelity reconstruction with 30 calibration images.

III. DESIGN AND FABRICATION

The Wedge design combines a compact optical-mechanical structure with component choices that improve illumination, gradient estimation, durability, and task-specific contact behavior. Its reconstruction workflow converts tactile images into depth through gradient integration.

  • The design goal is a compact robotic shape while maintaining high-resolution 3D reconstruction, requiring trade-offs in optical properties.
  • Three-sided lensless LED arrays preserve a thin sensor while directional lighting and filters compensate for missing or weakened gradient information.
  • Gray filters increase contrast by reducing internal reflections, while diffusers spread illumination across boundary regions to preserve gradient information.
  • Medical tape preserves more contact detail with lower friction, whereas fabric provides higher friction and ruggedness but reduces sensitivity.
  • Curved gels provide more uniform illumination, while flat gels better pick up small objects at the tip and curved gels tolerate misalignment with flat or concave surfaces.
  • The reconstruction workflow unwarps the imprint, subtracts a blank background, maps image differences to surface gradients, and integrates them with a fast Poisson solver.

IV. MEASURING 3D GEOMETRY

The evaluation examines photometric-stereo reconstruction under multiple lighting configurations, including extensions from three lights to one or two lights, and considers disturbance and pose-tracking capabilities.

  • The evaluation covers three-light reconstruction, one- and two-light extensions, disturbance effects, and potential capabilities.
  • Three-light reconstruction is assessed first, followed by comparisons among reduced-light configurations and additional capability demonstrations.
  • The evaluation uses reconstructed 3D geometry as the basis for examining broader sensor capabilities beyond image-to-depth reconstruction.

A. 3D reconstruction with 3 lights

The GelSight Wedge reconstructs 3D contact geometry by mapping image color and position to surface gradients, then integrating those gradients into depth with a fast Poisson solver. Qualitative tests show faithful reconstructions across several object shapes.

  • A. 3D reconstruction with 3 lights: 3D reconstruction maps color and position to horizontal and vertical surface gradients, then integrates them with a fast Poisson solver to obtain depth.The solver uses Gx, Gy, and zero boundary conditions.
  • A. 3D reconstruction with 3 lights: The calibration procedure presses a known-radius ball, estimates its contact circle and pixel scale, and gathers color-position-gradient correspondences.The calibration ball radius is 2.4 mm.
  • A. 3D reconstruction with 3 lights: A small three-hidden-layer MLP learns continuous RGBXY-to-gradient mappings while incorporating position-dependent color changes.The network has architecture 5-32-32-32-2 with tanh activation.
  • A. 3D reconstruction with 3 lights: The training set contains 32 images, the test set contains 8 images, and 5% non-contact pixels are added to balance zero-gradient data.
  • A. 3D reconstruction with 3 lights: Reconstructions cover ball arrays, Lego blocks, rubber tyres, screw caps, and M2 screws, with more surface fidelity than previous GelSlim results.

B. 3D reconstruction with 1 or 2 lights

The method extends 3D reconstruction to sensors with one or two lights by estimating missing gradients with neural networks and integrating them with a Poisson solver. Perpendicular two-light configurations perform best among reduced-light designs, while one-light reconstruction remains artifact-prone.

  • B. 3D reconstruction with 1 or 2 lights: Perpendicular two-light configurations provide gradients for both axes, allowing the fast Poisson solver to produce results comparable to three-light reconstruction.
  • B. 3D reconstruction with 1 or 2 lights: When one axis lacks light, zero-filling the missing gradient flattens reconstruction along that axis and produces negative depth.
  • B. 3D reconstruction with 1 or 2 lights: A neural network estimates missing Gx from Gy using 10k synthesized depth images, after which the inferred gradient and Gy enter the fast Poisson solver.
  • B. 3D reconstruction with 1 or 2 lights: Neural estimation significantly improves reconstruction with missing gradients without additional real data.
  • B. 3D reconstruction with 1 or 2 lights: One-light reconstruction tends to generate small bumps between contact regions and boundaries because too many gradients are missing.
  • B. 3D reconstruction with 1 or 2 lights: The reported preference order is RGB (3 lights) > RG (2 perpendicular lights) > RB (2 opposing lights) > R (1 light).

C. Influence of disturbances

Markers, gel deformation, and shadows disturb reconstructed 3D geometry. Marker gradients can be interpolated for real-time feedback, but deformation and shadows especially affect sharp surfaces and require further processing.

  • C. Influence of disturbances: Markers introduce small 3D bumps; zero-gradient filling removes them but creates flat patches, while gradient interpolation produces more reasonable results.
  • C. Influence of disturbances: Nearest-neighbor interpolation takes 10 ms at 200x150 resolution, compared with 60/70 ms for linear/cubic interpolation.
  • C. Influence of disturbances: Gel deformation and shadows influence reconstruction especially for sharp surfaces and make object-surface extraction challenging.

D. Potential capabilities

The design can be shrunk to human-finger size while retaining high-resolution 3D reconstruction, and its reconstructed point clouds support cube-pose tracking across orientations.

  • D. Potential capabilities: Shrinking the sensor to human-finger size retains high-resolution 3D reconstruction for fine manipulation and future multi-finger hands.
  • D. Potential capabilities: ICP tracks cube poses across various orientations at about 10 Hz when the initial pose is given.Detailed pose-error analysis and complex geometry are outside the paper’s scope.

V. CONCLUSION

GelSight Wedge combines a compact robot-finger form with high-resolution 3D contact geometry, extending the design to human-finger scale and demonstrating pose tracking from reconstructed touch point clouds.

  • GelSight Wedge measures high-resolution 3D contact geometry in a compact shape.
  • The sensor can be shrunk to human-finger size for narrow environments and fine manipulation tasks.
  • Reconstructed 3D point clouds support pose tracking demonstrations using touch alone.The paper combines reconstructed point clouds with ICP to track cube pose by corners.
  • Future work targets reducing marker influence on reconstruction and decreasing fingertip thickness with smaller LED arrays.The acrylic is currently 4.5mm thick, determined by LED size.
  • The authors plan to combine the sensor with robotic grippers and multi-finger hands for more reactive and robust control.
Loading 2106.08851v1…