Source-linked AI summary
Cable Manipulation with a Tactile-Reactive Gripper
Yu She, Shaoxiong Wang, Siyuan Dong, Neha Sunil, Alberto Rodriguez, Edward Adelson
TL;DR
Freely moving cables are difficult to manipulate because their shapes and friction change dynamically. This paper combines GelSight tactile perception with reactive grip and pose controllers for cable following without mechanical fixtures, achieving broad cable generalization and headphone-jack insertion.
Problem
Cable following is challenging because cable shape changes dynamically during sliding amid kinks, variable friction, and external forces.
Method
The framework uses GelSight tactile sensing with separate grip and pose controllers to regulate friction and maintain cable alignment during sliding.
Results
98.2% of total cable length was followed across cables with different physical parameters, and the system also demonstrated headphone-cable jack insertion.
Takeaways & Limitations
The results demonstrate that simple models and controllers can manipulate deformable objects across cable types and following velocities.
Abstract
from arXiv · showhide
Cables are complex, high dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and incremental deformations, or various mechanical fixtures such as clamps, pins or rings. We are interested in manipulating freely moving cables, in real time, with a pair of robotic grippers, and with no added mechanical constraints. The main contribution of this paper is a perception and control framework that moves in that direction, and uses real-time tactile feedback to accomplish the task of following a dangling cable. The approach relies on a vision-based tactile sensor, GelSight, that estimates the pose of the cable in the grip, and the friction forces during cable sliding. We achieve the behavior by combining two tactile-based controllers: 1) Cable grip controller, where a PD controller combined with a leaky integrator regulates the gripping force to maintain the frictional sliding forces close to a suitable value; and 2) Cable pose controller, where an LQR controller based on a learned linear model of the cable sliding dynamics keeps the cable centered and aligned on the fingertips to prevent the cable from falling from the grip. This behavior is possible by a reactive gripper fitted with GelSight-based high-resolution tactile sensors. The robot can follow one meter of cable in random configurations within 2-3 hand regrasps, adapting to cables of different materials and thicknesses. We demonstrate a robot grasping a headphone cable, sliding the fingers to the jack connector, and inserting it. To the best of our knowledge, this is the first implementation of real-time cable following without the aid of mechanical fixtures.
I. INTRODUCTION
The paper targets real-time cable following in free space, using tactile feedback and reactive grippers instead of mechanical constraints or vision. Its framework combines tactile perception with separate grip and pose controllers.
- Cable following is difficult because sliding dynamically changes cable shape amid kinks, variable friction, and external forces.
- The goal is to follow a freely moving cable between two grippers in real time without added mechanical constraints.
- A lightweight, fast-reacting gripper with high-resolution tactile sensors enables real-time cable following in free space.
- The system relies on tactile sensing alone, providing precise local cable localization despite occlusion that commonly affects vision.
- Cable grip control regulates gripping force and friction, while cable pose control keeps the cable centered and aligned with the fingers.
- The tactile-reactive gripper provides force and position control with a 60Hz grip bandwidth, while perception estimates cable pose, friction, and imprint quality.
II. RELATED WORK
Prior cable-manipulation methods commonly constrain motion or use slow incremental deformations, while this work exploits controlled sliding with tactile feedback.
- Rigid-object contour-following techniques do not directly transfer because deformable objects undergo dynamic shape changes that are difficult to model in real time.
- Hellman et al. used constant grasping force and discrete slow actions, reaching 0.5 cm/s versus 6.5 cm/s in this work.
- Deformable linear-object manipulation is often simplified with tables, extra grippers, pegs, or long sequences of small deformations.
- Earlier sliding systems may lose sensing and control at the sliding end, whereas this work controls a parallel-jaw gripper during cable sliding.
- The parallel-jaw gripper also supports headphone-jack insertion, unlike specialized passive end-effectors limited mainly to cable sliding.
III. TASKS
The paper evaluates cable following and cable insertion using tactile sensing, reactive grasp-force control, and reactive arm motion coordinated around a fixed gripper.
- Cable following grips the cable’s beginning, follows its contour to the tail, and permits regrasps by returning to the fixed gripper.
- Cable insertion finds and inserts a headphone cable’s cylindrical jack connector by sliding the fingers along the cable.
- The robot system requires tactile measurement of cable pose, orientation, and sliding friction, plus fast reactive gripper and arm control.
IV. METHOD
The method uses tactile control to maintain cable grip and pose while a compliant, fast gripper senses cable configuration and friction during sliding.
- Control architecture: The tactile controller divides the task into grip control for smooth sliding and pose control for keeping the cable centered and aligned.
- Hardware: The gripper combines GelSight sensors, compliant parallel guidance, and servo-actuated slide-crank linkages to support real-time feedback control.
- Mechanism design: The slider-string-spring system drives a crank linkage and parallelogram mechanism to produce lateral finger displacement.
- Mechanism design: The compliant mechanism replaces a 28-piece rigid parallelogram with a single part while preserving the intended kinematic functionality.
- Mechanism dimensions: The gripper targets a 100 mm maximum opening, with candidate dimensions including r1 = 15 mm, r2 = 50 mm, r3 = 30 mm, and r4 = 20 mm.
- Tactile perception: GelSight tactile images provide cable pose estimates from contact-region principal axes and friction-force magnitude and direction from marker displacements.
B. Perception
The tactile perception system extracts cable pose, friction force, and grasp quality from GelSight images to support cable-following control.
- Perception overview: Figure 4 summarizes extraction of cable pose, cable force, and grasp quality from tactile images.
- Cable pose estimation: GelSight depth images are thresholded to isolate the cable contact region, whose principal component gives the cable imprint’s pose axis.Depth is computed with a Fast Poisson Solver before contact-region extraction and PCA.
- Cable friction force estimation: Marker tracking between tactile frames estimates friction force from the mean marker displacement field.Blob detection locates black markers, and regularized matching maintains smooth displacement flow.
- Cable grasp quality: Grasp quality is based on whether the contact-region area exceeds a threshold, with larger gripping force increasing tactile-imprint quality.Poor-quality imprints produce noisy and uncertain pose estimates.
C. Control
The control system regulates cable sliding through tactile grip-force control and LQR pose control based on a learned cable–gripper model.
- Cable Grip Controller: The grip controller combines PD control with a leaky integrator to keep friction suitable for sliding while maintaining tactile signal quality.Too little friction can release the cable, while too much can prevent sliding; poor signal quality raises the PD target.
- Model of Cable-Gripper Dynamics: The cable–gripper state is [y θ α], describing cable position, cable orientation, and inclination relative to the grippers.The control input φ is the pulling direction relative to α.
- Model of Cable-Gripper Dynamics: A data-driven linear model ẋ = Ax+Bu represents cable–gripper dynamics because the deformable gel surface has complex friction dynamics.A and B are fitted linear coefficients, using approximately 2000 data points with one cable and linear regression.
- Baseline controller: The proportional baseline controls robot TCP velocity along y with uniform random noise while keeping x velocity constant.Its stated intuition is that opposite cable pulling drags an initially off-center cable toward the gripper center.
- Cable Pose Controller: The LQR pose controller regulates cable position, orientation, and inclination around zero using feedback from the learned model.The design weights position and orientation more heavily than inclination because preventing cable loss is prioritized.
A. Experimental Setup
The experiments use a UR5 with two reactive GelSight-equipped grippers to evaluate cable following across cable types, velocities, and controller baselines.
- Hardware setup: The setup includes a 6-DOF UR5 arm, two reactive grippers, and two pairs of revised GelSight fingertip sensors.One gripper is fixed to the table and the other is robot-mounted; control loops run at 125 Hz and 60 Hz.
- Test objects: Five cables and ropes vary in surface material and thickness, including USB, HDMI, and nylon specimens.
- Experimental process: The moving gripper follows the cable from the fixed gripper toward its tail while modulating grip force and cable pose, with regrasps permitted.Regrasping is performed by feeding the held section back to the fixed gripper.
- Metrics: Performance is measured by followed-length ratio, normalized distance traveled per regrasp, and normalized velocity.Each metric has a maximum and ideal value of 1.
- Evaluation protocol: The LQR controller is compared with open-loop, emergency-regrasp, and proportional baselines using averaged five-trial experiments.Generalization tests combine the LQR robot controller with the PD gripper controller across three velocities and five cables.
C. Cable following and insertion experiment
The experiments show that tactile feedback and LQR control support real-time cable following, including adjustment of the cable pose during motion and headphone-plug insertion.
- Controller comparison: 36% of the total cable length is followed by the open-loop controller, which loses the cable when it curves.Adding emergency regrasps lets open-loop control finish, but it requires more regrasps and is slower than P and LQR control.
- Controller comparison: The LQR controller uses the least number of regrasps among the tested robot controllers.Its velocity metric does not improve substantially because it travels farther while correcting cable deviations.
- Cable following: The LQR controller adjusts the gripper’s moving direction in real time to achieve the desired pulling angle while following the cable.Figure 8 combines tactile pose estimates, the desired in-hand alignment, end-effector trajectory, and LQR velocity output.
- LQR behavior: The LQR controller’s state includes α, the angle between the in-hand cable center-to-end direction and the fixed gripper orientation.This state term makes the controller move toward the trajectory center and correct the cable pose even when the cable is already centered.
B. Generalization to different velocities
The system was tested across three following velocities and five cables with differing physical properties, maintaining strong cable-following performance while faster motion increased regrasps.
- Different velocities: 0.025, 0.045, and 0.065 m/s are the tested following velocities for the LQR controller.The cable-gripper dynamics model is fit using data collected at 0.025 m/s, then evaluated at all three velocities.
- Different velocities: Performance does not degrade at higher velocities, except that regrasps per unit distance increase.At higher speeds, the controller has less time to react to sudden pose changes and triggers more regrasps.
- Different cables: 98.2% of the total cable length is followed across five cables with different diameters, materials, and stiffnesses.The experiments use the LQR robot controller with the PD gripper controller and average five trials per condition.
- Different cables: Cable 4, a thin and light nylon rope, requires the most regrasps because its un-followed portion moves with the gripper.Cable 5, a thin and stiff rubber USB cable, performs best because it is locally straight most of the time.
D. Cable following and insertion
The paper integrates tactile perception, reactive gripping, and two controllers for cable following, then demonstrates finding and inserting a headphone connector. The authors identify sensor, hardware, and modeling improvements for future work.
- Headphone cable insertion: After sliding to the cable end, GelSight localizes and aligns the headphone plug before insertion into the jack.In a separate insertion-only test with 20 trials, the system achieves an 85% insertion success rate.
- Tactile perception: GelSight tactile perception tracks cable pose and force in real time, providing local information that vision often cannot capture during continuous manipulation.The paper emphasizes tactile sensing because cables are commonly occluded and difficult to localize accurately with vision.
- Tactile gripper: The reactive gripper uses compliant joints and modulates grasping force at 60 Hz for rapid tactile-feedback control.The gripper design also helps protect the tactile sensor from unexpected collisions.
- Tactile control: The framework divides cable manipulation into grip control for friction regulation and pose control for keeping the cable centered and aligned.The grip controller combines a PD controller with a leaky integrator, while the pose controller uses an LQR model of cable-gripper dynamics.
- Conclusion: The authors conclude that simple models and controllers can manipulate deformable objects while generalizing across cables and following velocities.The headphone-cable demonstration illustrates a practical cable-manipulation application.
- Limitations and improvements: The system’s tactile signal and gripper control frequencies could increase from 30 Hz and 60 Hz to 90 Hz and 200 Hz, respectively.The authors also report difficulty pulling the cable back at the finger edge because of the convex GelSight surface.