Source-linked AI summary
ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact
Samarth Brahmbhatt, Ankur Handa, James Hays, Dieter Fox
TL;DR
Functional grasp synthesis for high-degree-of-freedom anthropomorphic hands is difficult when using object shape alone, especially for post-grasp actions. ContactGrasp combines object geometry with object-surface contact, then synthesizes and ranks grasps across diverse hands; experiments cover three hand models, 19 objects, and two functional intents.
Problem
Functional grasp synthesis from object shape alone is challenging for anthropomorphic hands, although useful grasps must support post-grasp actions.
Method
ContactGrasp uses a multi-point, object-centric contact representation and sample-and-rank synthesis procedure that can operate across hand models.
Results
ContactGrasp synthesizes functional grasps for three diverse hand models, 19 household objects, and two functional intents, with contact-based rankings outperforming geometry-based rankings.
Takeaways & Limitations
Object-surface contact demonstrations can modulate grasps for using objects and handing them off across kinematically diverse hands.
Abstract
from arXiv · showhide
Grasping and manipulating objects is an important human skill. Since most objects are designed to be manipulated by human hands, anthropomorphic hands can enable richer human-robot interaction. Desirable grasps are not only stable, but also functional: they enable post-grasp actions with the object. However, functional grasp synthesis for high degree-of-freedom anthropomorphic hands from object shape alone is challenging because of the large optimization space. We present ContactGrasp, a framework for functional grasp synthesis from object shape and contact on the object surface. Contact can be manually specified or obtained through demonstrations. Our contact representation is object-centric and allows functional grasp synthesis even for hand models different than the one used for demonstration. Using a dataset of contact demonstrations from humans grasping diverse household objects, we synthesize functional grasps for three hand models and two functional intents. The project webpage is https://contactdb.cc.gatech.edu/contactgrasp.html.
I. INTRODUCTION
ContactGrasp addresses functional grasp synthesis for anthropomorphic hands by combining object geometry with object-surface contact demonstrations. Its object-centric, sample-and-rank framework supports multiple hand models and post-grasp intents.
- Limitations of prior work: Random-grasp data collection commonly uses parallel-jaw or suction-cup end effectors, which are mostly suited to pick-and-place rather than post-grasp actions.The limitation is linked to the simplicity of the end effectors that make large-scale data collection feasible.
- Motivation: Functional grasps support post-grasp actions such as handing off, clicking, or switching on an object, beyond merely achieving stability.The motivation includes natural collaboration and object designs that encode task-specific grasps.
- Limitations of prior work: Geometry-based grasp synthesis often produces stable grasps but lacks functionality and can differ substantially from human grasps.The section contrasts stability-oriented synthesis with functional, human-like manipulation.
- Proposed approach: ContactGrasp uses an object-centric representation that reproduces demonstrated contact and transfers across hand models rather than relying on hand-pose retargeting.Contact may be manually specified or obtained from human demonstrations.
- Contributions: The framework develops a multi-point contact representation and a sample-and-rank procedure for functional grasp synthesis across multiple hand models.The ranked output addresses cases where multiple hand configurations realize the same contact pattern.
II. RELATED WORK
Prior grasp-synthesis research spans analytic, data-driven, hybrid, and demonstration-based approaches, but existing methods face restrictive assumptions, data demands, simple effectors, or hand-specific representations.
- Analytic and data-driven methods: Analytic methods synthesize grasps from object shape under assumptions such as perfect models, rigid hands, Coulomb friction, and simplified contacts.These assumptions can make purely analytic algorithms difficult to deploy in real-world settings.
- Analytic and data-driven methods: Data-driven methods learn object representations for grasp prediction, but newer deep-learning approaches require large, expensive datasets and typically use simple parallel-jaw effectors.The passage contrasts representation learning with the collection and labeling burden of training data.
- Hybrid methods: Hybrid approaches sample grasps analytically and then use demonstrations or task labels to filter, evaluate, or learn their suitability.Examples include robot success labels and models of post-grasp tasks with discretized hand poses.
- Grasp synthesis from contact: Earlier contact-based methods use hand-pose demonstrations, dataglove-derived grasp spaces, or manually specified contacts, whereas ContactGrasp emphasizes contact for multiple hand models.The comparison identifies hand-pose and manual-contact requirements in related approaches.
III. CONTACT MODEL AND HUMAN DEMONSTRATIONS
ContactGrasp represents human grasping as attractive and repulsive points sampled on an object surface, enabling multi-point, object-centric contact constraints from manual inputs or ContactDB demonstrations.
- Contact representation: Multi-point contact support reflects that humans contact objects with fingertips, palms, and non-tip finger areas.The contact model is designed to capture these varied contact regions.
- Contact representation: The contact map samples N object-surface points and assigns each an attractive value +1 or repulsive value −1.Contacted demonstration points are attractive; other points are repulsive, providing negative information.
- Human demonstrations: Contact maps can be specified manually or obtained from ContactDB’s thermal after-prints of hand-object contact.ContactDB textures object meshes with high-resolution contact maps and includes handoff and use intents.
- Human demonstrations: For the flashlight example, randomly sampled surface points are visualized as green attractive or red repulsive contacts.The color encoding illustrates the contact-map construction from a human demonstration.
IV. HAND MODELS
ContactGrasp optimizes full articulated hand poses against object-surface contact using kinematically diverse hand models. The pose includes global rigid motion and joint articulation, with signed distance fields supporting optimization.
- Hand models: The experiments use three kinematically diverse hand models to test whether the object-centric representation transfers across hand designs.The models are shown with their joint axes and a special thumb contact point.
- Hand models: Each hand model is a kinematic tree whose parts are represented by rigid meshes, with articulation degrees of freedom and a 6-DOF global transform.The global transform specifies the hand’s overall position in 3D space.
- Pose optimization: Signed distance fields are attached to hand parts in local coordinates to support hand-pose optimization.The SDF representation supplies geometric distances for the articulated hand segments.
- Pose optimization: Grasp synthesis estimates the full hand configuration through an initialized nonlinear optimization over palm pose and joint values.The algorithm’s overview is presented in Figure 5.
A. Grasp Optimization
ContactGrasp optimizes hand pose by encouraging contact at attractive points, discouraging contact at repulsive points, and avoiding problematic intersections. Its repulsive-point activation uses surface normals to avoid penalizing nearby hand parts that are not directly above those points.
- 1) Grasp Term:: The grasp term attracts the closest hand segment to attractive contact points and repels it from repulsive points.The term uses signed distances and separate strengths for attractive and repulsive points.
- 1) Grasp Term:: Repulsive points are activated only when the nearest hand-part direction aligns sufficiently with the object-surface normal.This modification avoids penalizing a nearby hand part that is not directly above the repulsive point.
- 1) Grasp Term:: Figure 5 depicts the pipeline from geometry-based grasp sampling to contact-map-based refinement and ranking.The sampled grasps are synthesized into functional grasps by agreement with a human-demonstrated contact map.
2) Thumb Contact Term:
The thumb contact term encourages a designated thumb point to contact the object, adding thumb involvement to the grasp objective. The intersection term separately discourages hand-object and hand self-intersections.
- 2) Thumb Contact Term:: A designated point on the thumb is encouraged to contact the object through a signed-distance penalty.The point may be on the thumb or the hand part most closely resembling it.
- 2) Thumb Contact Term:: The intersection term discourages both hand-object intersection and self-intersection among hand segments.Its strength is controlled by the hyperparameter λi.
3) Intersection Term:
The full grasp objective combines grasp, thumb-contact, and intersection terms, and DART is used to minimize it and obtain an optimized hand pose. Figure 7 illustrates the factors involved and the optimized result.
- 4) Optimization:: The full objective function combines grasp, thumb-contact, and intersection terms.These terms jointly define the optimization objective for the hand pose.
- 4) Optimization:: DART minimizes the objective function to produce the optimized hand pose.The implementation modifies DART's Contact Prior mechanism to support repulsive points and removes its depthmap observation term.
- 4) Optimization:: Figure 7 shows the factors involved in grasp optimization above the optimized result below.
B. Initializing the Grasp Optimization
Because local optimization in a high-dimensional hand-pose space can reach local minima, ContactGrasp initializes it with diverse geometry-based grasps. These candidates are then refined and ranked by their agreement with the contact map.
- B. Initializing the Grasp Optimization: Local Levenberg–Marquardt optimization makes good initialization important in the high-dimensional hand-pose space.The optimization is local and may encounter many local minima.
- B. Initializing the Grasp Optimization: GraspIt! samples diverse full grasps from object geometry without using the contact map.The resulting grasps are later ranked according to the residual after contact-aware optimization.
- B. Initializing the Grasp Optimization: The planner samples approach points, roll angles, and hand distances before simulated annealing refines each coarse grasp.Approach points are sampled over the object surface; the planner runs for 45K iterations per seed and retains the top two grasps.
- B. Initializing the Grasp Optimization: Candidates are refined and ranked by negative optimization residual, placing contact-compatible poses among the top-ranked grasps.Most geometry-only candidates have large residuals because they may lie outside the local optimizer's convergence basin.
- B. Initializing the Grasp Optimization: The process recovers a full hand pose consistent with a contact map without manually annotating the full high-dimensional pose.
VI. RESULTS
ContactGrasp synthesizes functional grasps from human contact demonstrations for diverse hand models, objects, and post-grasp intents. The results include functional behaviors across three hand models and demonstrate contact-aware alternatives to geometry-only grasps.
- Evaluation setup: 19 household objects with demonstrations for using and handing off were selected from ContactDB for evaluation.The study synthesizes grasps for three hand models and two functional intents.
- Synthesized grasps: ContactGrasp synthesizes grasps for 8 objects and 2 functional intents across HumanHand, Allegro, and Barrett hands.Figures 8–10 show results for the three hand models.
- Functional behavior: Functional grasps keep the flashlight button accessible, place fingers on mouse buttons, and hold a knife by its handle.These examples concern the use intent.
B. Quantitative results
Quantitative evaluation shows that ContactGrasp better reproduces human-demonstrated contact than geometry-based ranking and deterministic kinematic retargeting. Its failures arise when sampling misses fine manipulation configurations or when the target hand lacks suitable structure.
- Ranking: The correct grasp has a lower median rank under the DART residual than under GraspIt!’s Contact Energy metric.Table III reports median rank across all objects, where lower is better.
- Contact agreement: ContactGrasp achieves significantly lower Lgrasp disagreement with demonstrated contact than GraspIt! grasps.Lgrasp measures disagreement with the human contact map, with lower values indicating closer agreement.
- Retargeting comparison: Kinematic retargeting produces higher Lgrasp values than ContactGrasp Allegro grasps and human grasps.This indicates that deterministic mapping does not reliably reproduce contact across hand models.
- Failure cases: ContactGrasp can fail when sampling misses fingers-through-holes behaviors or when a hand model lacks geometry for complicated control patterns.These limitations are illustrated in Figure 12.
VII. CONCLUSION
ContactGrasp uses multi-point contact to synthesize functional grasps across kinematically diverse hands. Demonstrations support post-grasp intents such as using objects and handing them off, with results shown for three hand models, 19 objects, and two intents.
- Contribution: The paper develops a multi-point contact model that works with kinematically diverse hand models.The model is intended to plug and play across hand models.
- Hand-model diversity: The wine-glass example shows functional use grasps for HumanHand, Allegro, and Barrett hands.Each hand also has six palm-pose degrees of freedom.
- Comparison: Geometry-only grasps can block functional interfaces, such as flashlight buttons, knife blades, cellphone screens, and wineglass openings.These are examples of top-ranked GraspIt! grasps that are stable but not functional.
- Limitations: Failure cases occur when grasp sampling misses narrow-hole contacts or when an end effector lacks structure for functional grasps.Figure 12 gives fingers-through-holes and resting fingers on mouse buttons as examples.
- Functional intents: Contact demonstrations modulate grasps to support using objects and handing them off.The conclusion identifies both as post-grasp actions.
- Evaluation scope: ContactGrasp is demonstrated on 3 hand models, 19 household objects, and 2 functional intents.These are the reported evaluation dimensions in the conclusion.