Source-linked AI summary
Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation
Alex Alspach, Kunimatsu Hashimoto, Naveen Kuppuswamy, Russ Tedrake
TL;DR
Robots working in uncertain, cluttered environments need tactile sensing and mechanical compliance to make safe contact and manipulate objects. Soft-bubble combines an inflated latex membrane with an internal depth sensor, and demonstrations include object classification, pose estimation and tracking, and nonprehensile manipulation. The discussion also reports that image resolution affects classification accuracy, especially for subtle shape or surface differences.
Problem
Robots in cluttered, uncertain environments face occlusions, variable lighting, and unfamiliar objects, while compliant tactile sensors still lack high-resolution contact sensing.
Source: §I · p. 1Method
Soft-bubble measures deformation of an inflated latex membrane with an off-the-shelf depth sensor in a lightweight, compliant tactile sensor and end effector.
Source: §I · p. 2, §III · p. 2Results
Takeaways & Limitations
The integrated sorting demonstration shows that the highly compliant Soft-bubble end effector can both sense and manipulate objects.
Source: §V · p. 7The current sensor measures geometry only; extracting contact forces requires additional modeling and analysis.
Source: §V · p. 7
Abstract
from arXiv · showhide
Incorporating effective tactile sensing and mechanical compliance is key towards enabling robust and safe operation of robots in unknown, uncertain and cluttered environments. Towards realizing this goal, we present a lightweight, easy-to-build, highly compliant dense geometry sensor and end effector that comprises an inflated latex membrane with a depth sensor behind it. We present the motivations and the hardware design for this Soft-bubble and demonstrate its capabilities through example tasks including tactile-object classification, pose estimation and tracking, and nonprehensile object manipulation. We also present initial experiments to show the importance of high-resolution geometry sensing for tactile tasks and discuss applications in robust manipulation.
I. INTRODUCTION
Robots in unstructured environments need to handle uncertain contacts safely, while tactile sensing and mechanical compliance offer complementary ways to address those demands. Soft-bubble combines a compliant structure with dense contact geometry sensing and demonstrates classification, pose estimation and tracking, and nonprehensile manipulation.
I. INTRODUCTION
- Unstructured domestic environments present clutter, occlusions, variable lighting, and unfamiliar objects, motivating robots that can compensate for uncertainty through touch.The introduction also identifies cost, weight, and manufacturability as considerations for commercially viable home robots.Source: §I · p. 1
- Elastomers and fluids provide passive compliance that can reduce collision impulse and spread force across a larger contact area, but constrained home manipulation also requires precision.The introduction notes that compliance alone can provide stable grasps where precision is not required.Source: §I · p. 1
- Tactile contact geometry can support object classification and shape inference, pose refinement, and manipulation when vision is occluded or lacks salient features.The paper demonstrates shape and texture classification, object sorting by nonprehensile manipulation, and pose estimation and tracking.Source: §I · p. 2
- Soft-bubble combines a highly compliant elastomeric structure with detailed contact-geometry sensing in a tactile sensor and end effector.It captures deformation of an air-filled membrane with an off-the-shelf depth sensor.Source: §I · p. 2
II. RELATED WORK
Prior tactile work spans geometry and force sensing, but questions remain about the resolution needed for manipulation and the relative importance of geometry versus force. Soft-bubble draws on camera-based tactile sensing while using an air-filled membrane and depth camera to capture deformation.
II. RELATED WORK
- Open questions include how much spatial resolution tactile sensing requires and whether geometry or force sensing matters more for manipulation.The related-work discussion notes that tactile sensing has grown substantially, while its use to improve manipulation remains an open question.Source: §II · p. 2
- Camera-based tactile sensors gather substantial data over small contact areas; GelSight generates geometry height maps using internal lighting and photometric stereo.The resulting 3D information has been used in model-based frameworks, particle filters, and contact-manifold sampling.Source: §II · p. 2
- Soft-bubble uses an off-the-shelf depth camera and opaque membrane, which deforms more freely than gel-based sensors and can expose more of an object's geometry.Its air-based construction is described as lightweight and simple to fabricate, with a compliant, high-friction surface suited to contact-heavy manipulation.Source: §II · p. 2
III. HARDWARE DESIGN
The Soft-bubble hardware uses an elastic membrane, an airtight pressurizable hull, and an internal depth sensor to measure contact-induced membrane deformation. Inflation height and membrane thickness involve stiffness, sensitivity, and durability trade-offs.
III. HARDWARE DESIGN
- The sensor has three functional components: an elastic sensing membrane, an airtight pressurizable hull, and an internal depth sensor aimed at the membrane.The depth sensor measures deformation induced by contact.Source: §III · p. 2
- Inflation forms a compliant spherical cap 20–75 mm high, and increasing height or internal pressure increases bubble stiffness.The experiments used a 50 mm membrane height and roughly 0.25 psi internal pressure.Source: §III · p. 2
- The sensing membrane is laser-cut from 0.4 mm latex sheet, with thinner or thicker material trading sensitivity against durability.Source: §III · p. 2
A. Internal depth camera
The internal depth camera is a PMD CamBoard pico flexx time-of-flight sensor selected for its small size and short specified minimum range. Its near-minimum-range use inside the confined membrane enclosure presents illumination and field-of-view overlap problems.
A. Internal depth camera
- The PMD CamBoard pico flexx is a 68×17×7.35 mm time-of-flight camera with USB connectivity and a specified 100 mm minimum sensing range.It has 224×171-pixel resolution, up to 45 fps, weighs 8 grams, and has a 62°×45° field of view.Source: A. Internal depth camera · pp. 2–3
- At 50 mm inflation, the membrane's spatial sensing resolution is roughly 2 pixels/mm2.Source: A. Internal depth camera · p. 3
- Near the camera's minimum range, its bright IR emitter can saturate membrane regions, while emitter-imager field-of-view mismatch creates a dark region with inaccurate depth readings.The camera is designed for 0.1–4 m, and its optics and calibration work best near the middle of that range.Source: A. Internal depth camera · p. 3
B. Airtight hull
The airtight hull houses and mounts the depth sensor, membrane, USB passthrough, and air valve. Close-range imaging can produce glare and depth errors when illumination is excessive or uneven.
B. Airtight hull
- The hull provides mounting points for the depth sensor, elastic membrane, sealed USB 3.0 passthrough, and air valve, and for attaching the tactile sensor assembly to a robot.At the depth sensor’s minimum sensing distance of 100mm, the field of view covers 99.5cm2.Source: B. Airtight hull · p. 3
- Close-range emitter and imager issues create bright IR spots and absent point-cloud points, or insufficiently illuminated points represented farther away than they are.The emitter can saturate the membrane image, while the emitter-imager field-of-view offset causes poorly illuminated regions.Source: Figure 3 · p. 3, Figure 4 · p. 3
- The membrane is glued flat to a printed frame, then bolted to the hull flange with an O-ring sealing the membrane to the hull.The hull’s internal surfaces, passthrough holes, and O-ring channel receive two or more coats of flexible rubber sealant to address FDM porosity.Source: B. Airtight hull · pp. 3–4
C. Parallel gripper concepts
The parallel-gripper concept uses a compliant, high-friction latex bubble for robust grasps, while its prototype informs specifications for compact, short-range depth sensors.
C. Parallel gripper concepts
- The sensorless bubble-based gripper prototype grasps robustly through its compliant, high-friction surface and informs specifications for small, short-range depth sensors.Sensorizing smaller bubbles is ongoing; the benefits of highly compliant grasps are described as immediate.Source: Figure 5 · p. 4, C. Parallel gripper concepts · p. 4
- The pictured prototype is a sensorless bubble-based gripper finger designed to provide insight into small, short-range depth-sensor specifications.Its robust grasps are attributed to the compliant, high-friction surface.Source: Figure 5 · p. 4
IV. EXPERIMENTS
Initial experiments demonstrate the Soft-bubble’s capabilities across tactile classification, object sorting, pose estimation and tracking, and nonprehensile manipulation.
IV. EXPERIMENTS
- Experiments covered two tactile object-classification tasks, object sorting, tactile pose estimation and tracking, and nonprehensile object manipulation.The authors report that the experiments show the utility of high-resolution contact geometry sensing and highly compliant manipulation.Source: §IV · p. 4
A. Classification
The classification experiments use Soft-bubble depth images with a ResNet18 classifier to test object recognition and the effect of image resolution. The six-object task reaches 98.14% average top-1 validation accuracy, while lower resolution particularly harms classification of cubes with subtle surface features.
A. Classification
- The first Soft-bubble classification experiment used data output by the Soft-bubble to distinguish six distinctly shaped blocks and a no-touch class when the Soft-bubble was free from contact.The objects are illustrated in Fig. 6; the cube shown for scale is 40×40×40mm.Source: A. Classification · p. 4, Figure 6 · p. 4
- 98.14% average top-1 accuracy was the classifier’s best validation performance on the seven-class task, including six objects and no-touch.The ResNet18 was trained from scratch on depth images; the objects were pressed against the membrane at varied locations, orientations, and forces, including partial views.Source: A. Classification · pp. 4–5, Figure 8 · p. 5
- The resolution experiment used the previously described six-object dataset and a three-cube dataset, training the same network separately from scratch and repeating training six times per dataset across varying image resolutions.Source: A. Classification · p. 5
- Average top-1 accuracy drops as input image resolution decreases, with a significant drop for the three-cube dataset at N = 2.The cubes share size but have distinct, subtle surface features; Fig. 10 reports average top-1 accuracy across the six-object and three-cube datasets at varying resolutions.Source: A. Classification · p. 5, Figure 10 · p. 5, Figure 9 · p. 5
B. Pose Estimation and Tracking
The Soft-bubble uses ICP on sensed point clouds to estimate and track poses of objects with known geometry. The paper demonstrates pose estimation for a simple object and smooth tracking during contact, while leaving pose accuracy and more complex geometries unassessed.
B. Pose Estimation and Tracking
- Pose estimation uses ICP with a dense object model, multiple initial orientations, and cropping of up to 25% of the model to match visible surfaces.The approach relies on a reasonable local prior for pose, with bounds dependent on the ICP variant and reference geometry.Source: B. Pose Estimation and Tracking · p. 6
- ICP estimates the pose of a contacting object with simple geometry from the Soft-bubble point cloud in about 0.5 seconds.The reported example is a pyramidal frustum; the estimated object model is superimposed on the measured cloud.Source: B. Pose Estimation and Tracking · p. 6, Figure 13 · p. 6
- The basic tracker runs at 1–2 Hz and produces relatively smooth pose tracking as a contacted object moves or rotates on the sensor.The tracker initializes ICP from two fixed rotations and the last successful pose; Fig. 14 depicts a glass triangular prism.Source: B. Pose Estimation and Tracking · p. 6, Figure 14 · p. 6
- The study does not assess pose accuracy, complicated geometries, or the effect of sensor resolution.The pose-estimation setup samples 10 approach angles over a 15° cone and presses the frustum 4 cm into the bubble.Source: B. Pose Estimation and Tracking · p. 6
C. Robot Object Sorting
The Soft-bubble was integrated with a robot and a six-object classifier to demonstrate tactile sorting through pushing. The robot’s motion primitives were pre-scripted, with classification performed between pressing and sliding the object.
C. Robot Object Sorting
- The integrated system touched and classified each of six objects, then pushed it toward a designated table area using pre-scripted motions.Classification ran continuously, but sorting decisions were made between the robot’s pressing and sliding motions.Source: C. Robot Object Sorting · p. 7
- The sorting demonstration shows the Soft-bubble functioning as both a tactile sensor and a compliant, robust end effector.The Soft-bubble was mounted on a KUKA iiwa arm.Source: C. Robot Object Sorting · p. 7
- The robot’s individual motions—pressing down, sliding objects, and returning to its initial pose—were pre-scripted.The demonstration combined those motions with tactile object classification to sort objects.Source: C. Robot Object Sorting · p. 7
V. DISCUSSION AND OUTLOOK
The discussion identifies sensor size and geometry-only sensing as constraints, while outlining resolution needs and future applications. The authors report that classification accuracy declines at lower image resolutions, especially for objects with subtle differences, and describe a robot capable of both sensing and manipulation.
V. DISCUSSION AND OUTLOOK
- The end effector’s large form factor depends on the depth sensor; hand-scale grippers require a smaller, shorter-range sensor with a wide field of view.The authors also envision larger bubbles using multiple synchronized depth sensors to produce a fused point cloud.Source: §V · p. 7
- Classification accuracy drops as input resolution decreases, especially when object-shape or surface-feature differences are subtle relative to sensor resolution.Understanding task-specific tactile resolution is needed to guide specifications for smaller, short-range depth sensors.Source: §V · p. 7
- The Soft-bubble currently senses geometry only; estimating contact forces requires additional modeling and analysis.For convex objects, membrane tension can make bridging between contacts appear to be part of the object surface.Source: §V · p. 7
- Calibration methods and approaches to quantify depth-measurement error and sensor noise remain under development.The authors also identify static and dynamic modeling of the soft mechanics as future work for estimating contact location and pressure.Source: §V · p. 7
- The integrated sorting demonstration shows the compliant Soft-bubble end effector sensing and manipulating objects.Future directions include pose refinement, contact-patch and pressure estimation, dexterous manipulation, and contact-rich planning and control.Source: §V · p. 7