PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Grasp Stability with Rotation Measurement from Tactile Sensing

Raj Kolamuri, Zilin Si, Yufan Zhang, Arpit Agarwal, Wenzhen Yuan

arXiv:2108.00301v1cs.RO

TL;DR

Rotation from grasping far from an object’s center of gravity is a common but under-studied grasp failure, and increasing gripping force does little to mitigate it. A model-based method analyzes GelSight marker rotations to detect torque-caused grasp rotation, measure it, and guide closed-loop regrasping toward a stable location. In 105 of 109 experiments, the algorithm avoided rotations above 5° (96.3% success rate); the overall average error in rotation angle measurement was 3.96°.

Abstract

from arXiv · show

Rotational displacement about the grasping point is a common grasp failure when an object is grasped at a location away from its center of gravity. Tactile sensors with soft surfaces, such as GelSight sensors, can detect the rotation patterns on the contacting surfaces when the object rotates. In this work, we propose a model-based algorithm that detects those rotational patterns and measures rotational displacement using the GelSight sensor. We also integrate the rotation detection feedback into a closed-loop regrasping framework, which detects the rotational failure of grasp in an early stage and drives the robot to a stable grasp pose. We validate our proposed rotation detection algorithm and grasp-regrasp system on self-collected dataset and online experiments to show how our approach accurately detects the rotation and increases grasp stability.

I. INTRODUCTION

Grasp location depends on physical properties such as mass distribution, which vision-based grasping methods may not capture. This work uses tactile rotation feedback to guide regrasping toward a stable location near an object’s center of gravity.

A. Grasping and Regrasping

  • Vision-based grasping methods use object shape but may not account for mass, mass distribution, friction, or rigidity, which matter to physical interaction.The paper initializes grasps using RGB-D geometry, then adjusts grasp locations using tactile feedback.Source: §I · p. 1, A. Grasping and Regrasping · p. 2

I. INTRODUCTION

  • Rotation from grasping far from an object’s center of gravity is an under-studied failure mode that increased gripping force does little to mitigate.The introduction contrasts this problem with tactile-sensing work focused on detecting and measuring slip.Source: §I · p. 1
  • The proposed model-based tactile method detects torque-caused rotation early and uses feedback to guide the robot toward a stable grasp near the center of gravity.On GelSight, torque twists the soft elastomer and produces rotational marker patterns that are used to estimate torque.Source: §I · p. 1

A. Grasping and Regrasping

  • Prior tactile work addresses slip and other grasp characteristics, while this work targets failures from incorrect grasp locations using tactile feedback.Related regrasping approaches use simulated tactile images, pressure plus force/torque data, or wrist moments; this work uses tactile images for smaller household objects.Source: A. Grasping and Regrasping · p. 2

B. Slip Detection based on Tactile Sensing

Prior tactile slip-detection work classifies or measures slip, while rotational failure from an incorrect grasp location has received less attention. The paper analyzes GelSight marker patterns to detect and measure object rotation at the contact.

B. Slip Detection based on Tactile Sensing

  • Earlier tactile slip research includes model-based and learning-based methods for detecting, classifying, and measuring slip.Other cited work distinguishes translational and rotational slip from contact area or sensor data, or estimates spin using remote force/torque readings.Source: B. Slip Detection based on Tactile… · p. 2
  • GelSight marker tracking has supported detailed study of local slip and rotation, but cited prior work is limited to numerically identifying slip; this paper analyzes rotational patterns and measures angle and orientation using vision-based tactile sensing alone.The authors describe this as, to their knowledge, the first such attempt.Source: B. Slip Detection based on Tactile… · p. 2

III. METHOD

  • The method analyzes GelSight marker patterns to determine the center of rotation and calculate rotation angle, then uses the result to guide the robot toward a stable grasp pose.It measures rotation in real time from tactile images.Source: §III · p. 2

A. Contact Rotation Measurement Overview

The rotation-measurement pipeline detects contact, identifies rotational onset from marker motion, and then estimates rotation about the center of rotation. It accommodates changing contact regions by identifying a stable contact state before extracting the contact area.

A. Contact Rotation Measurement Overview

  • The overview has three stages: detect object contact, detect rotational onset by tracking marker patterns, then calculate rotation angle and orientation about the center of rotation.The output is a rotation measure relative to the calculated center of rotation.Source: A. Contact Rotation Measurement Overview · p. 2

B. Object Contact Detection

  • The detected contact area is the filtered region derived from illumination differences, which the figure shows mostly in the bottom part of the image.Source: Figure 2 · p. 3, B. Object Contact Detection · p. 3
  • Contact is treated as stable either when marker-motion change falls below a threshold after 10 frames or, when no stable stage appears, at the frame after 30 frames (about 1 second).The contact region is then extracted from the maximum illumination change across the RGB channels and filtered.Source: B. Object Contact Detection · p. 3

C. Detection of Rotation Onset

  • Rotation onset is declared when relative marker-motion or angular change exceeds a fixed threshold; the angle between the two motion vectors is one onset cue.The figure also depicts the center of rotation as the intersection of normals to rotating motion vectors.Source: C. Detection of Rotation Onset · p. 3, Figure 3 · p. 3

D. Rotation Angle Measurement

The method estimates a center of rotation (COR) from marker motions, then measures rotation angle and orientation. A majority vote helps distinguish real rotations from marker noise and static-contact false positives.

D. Rotation Angle Measurement

  • Marker-motion normals ideally intersect at the COR; a least-squares solution estimates it despite noise.The method formulates the estimate using motion vectors and a normal equation.Source: D. Rotation Angle Measurement · p. 3
  • Rotation angle is measured between the initial and current marker vectors from the estimated COR, while moment sign determines clockwise or counter-clockwise orientation.The method uses a majority vote across marker moments because some markers may move in the opposite direction; similarly sized orientation groups can indicate static contact rather than real rotation.Source: D. Rotation Angle Measurement · p. 4
  • A grasp is treated as stable when there is no rotation or when rotation is at most 5°.The figure describes three contact-rotation cases measurable from black-marker patterns.Source: D. Rotation Angle Measurement · p. 4, Figure 4 · p. 4

1) Translational Displacement:

The method distinguishes translation from rotation using marker-motion patterns. Translational displacement can arise during lifting when friction and gravity are imbalanced and normal contact force is insufficient.

1) Translational Displacement:

  • Insufficient normal contact force can allow translation during lifting when friction and gravity are imbalanced.The passage notes that increasing normal force is a natural way to avoid this displacement.Source: 1) Translational Displacement · p. 4
  • Marker motions are mostly similar in direction for translation and dissimilar for rotation, providing a cue to distinguish the two.SVD is applied to the marker motions; one dominant direction indicates translation.Source: 1) Translational Displacement · p. 4
  • For irregular-area contact, the illustrated wrench case tracks the contact contour’s long axis.The figure shows the wrench, the GelSight image, and the tracked contour axis.Source: Figure 5 · p. 4

2) Small Contact Area:

For small, irregular contact areas, the method estimates rotation from the contact contour rather than tracking markers within the area.

2) Small Contact Area:

  • Small, irregular contact areas have rich geometry and make markers difficult to track.The described contact is small and non-flat.Source: 2) Small Contact Area · p. 4
  • Rotation is approximated by tracking the relative angular change of the contact contour’s principal axis.The principal-axis change serves as a proxy for rotation of the area.Source: 2) Small Contact Area · p. 4
  • The method filters and smooths the HSV Value channel to extract the main contact area from its intense color gradients.The RGB image is converted to HSV before processing.Source: 2) Small Contact Area · p. 4

F. Closed-loop control based on rotation measurement

The closed-loop framework uses GelSight rotation feedback to adjust grasp locations toward a stable pose. It estimates object length from RGB-D data to set the adjustment scale, and evaluates the approach in offline and closed-loop experiments.

1) Object Size Estimation:

  • Object length is estimated by projecting the segmented point cloud onto the table plane and finding its longest principal axis.The plane and object are segmented with RANSAC; the 95% longest distance from the center estimates half-length to tolerate outliers.Source: 1) Object Size Estimation · p. 4, Figure 6 · p. 5

F. Closed-loop control based on rotation measurement

  • After rotation is detected, the robot releases the object and shifts the grasp along its principal axis, using coarse-to-fine steps to locate a stable pose near the center of gravity.The initial step is 0.4L, later steps are 1/6L, and a direction change brackets the center of gravity and triggers a finer adjustment.Source: F. Closed-loop control based on… · p. 4, 2) Closed-loop Control · p. 5

IV. EXPERIMENTS

  • The experiments comprise offline rotation-measurement tests against ground truth and closed-loop regrasp tests using GelSight feedback.The setup uses a UR5e arm, parallel-jaw gripper, GelSight sensor, Azure Kinect, and a side-view camera for rotation ground truth.Source: §IV · p. 5

A. Contact Rotation Measurement on Offline Data

Offline experiments evaluate GelSight rotation measurement against camera-based ground truth and classify grasps as stable or rotational. The measurements show small average angle error and high classification accuracy, with most failures associated with contact areas at the image margin.

A. Contact Rotation Measurement on Offline Data

  • Across 142 grasps on 14 objects, including irregular and soft objects, GelSight and external USB cameras recorded images at 30 fps during lifting; external-camera ArUco-marker orientation changes on their plane provided rotation ground truth.The trials included 98 rotational cases and 44 stable grasps, including translational slip cases.Source: A. Contact Rotation Measurement on… · p. 5, Figure 7 · p. 6, Figure 8 · p. 6
  • 3.96° average angle error overall, falling to 2.69° for rotations under 10°; large-angle readings are smaller than ground truth.The authors attribute large-angle discrepancies to rotational slip and adhesion between the gel and object.Source: A. Contact Rotation Measurement on… · p. 5
  • 4.22 frames (about 0.14 seconds) was the overall error in detecting rotation onset.The onset delay is measured as the time difference between GelSight detection and ground truth.Source: A. Contact Rotation Measurement on… · p. 6
  • 134 of 142 grasps were classified correctly (94.37%), including 93.18% of stable cases and 94.90% of rotational cases.The evaluation included 44 stable cases and 98 rotational cases.Source: A. Contact Rotation Measurement on… · p. 6
  • Most failures involved contact areas at the image margin, where partial visibility or disappearance made markers and contours difficult to track.Source: A. Contact Rotation Measurement on… · p. 6

B. Closed-Loop Regrasping Experiment

The closed-loop experiment uses GelSight rotation detection to adjust grasp locations and seek stable poses. In 105 of 109 trials, the system drove the robot toward the object’s centroid while avoiding rotations above 5°; it averaged one regrasp across objects.

B. Closed-Loop Regrasping Experiment

  • When rotation exceeded 5°, the robot returned the object to the table and updated its grasp step; absent a signal, it checked after a 5 cm lift and then confirmed stability with a further lift and hold.The rotation detector also analyzed GelSight frames during these steps.Source: B. Closed-Loop Regrasping Experiment · p. 6
  • The 109 regrasp experiments covered 18 objects, and the rotation detector processed each frame in about 0.045 seconds during manipulation.Source: B. Closed-Loop Regrasping Experiment · p. 7
  • 96.3% of closed-loop trials (105/109) reached a centroid-directed grasp without rotation above the 5° threshold.Three rotations were misclassified as stable and one stable pose as rotation; the primary failure reason was a small contact area yielding few detected markers.Source: B. Closed-Loop Regrasping Experiment · p. 7
  • Across objects, the robot averaged one regrasp to reach the center of gravity, with at most three regrasps to reach a stable pose.Fig. 13 reports the average number of grasps for each object.Source: B. Closed-Loop Regrasping Experiment · p. 7, Figure 13 · p. 7

V. CONCLUSIONS

The paper measures rotational grasp failure from tactile-surface rotation and uses that feedback in a closed-loop regrasp system. The authors identify broader object handling and higher-speed sensing as directions for future work.

V. CONCLUSIONS

  • The method measures soft tactile-surface rotation, which the authors state directly correlates with torque at the grasp surface.Source: §V · p. 7
  • Rotation feedback guides repeated regrasps until the robot reaches a stable grasp location, preventing large rotation from happening.Source: §V · p. 8
  • Future work includes learning-based rotation measurement, external vision for varied object dimensions and shapes, multi-fingered arms, and higher camera frame rates with lower latency.Source: §V · p. 8
Loading 2108.00301v1…