Source-linked AI summary
CoToGrasp: Contact-Topology-Conditioned Dexterous Grasp Synthesis via Canonical Workspace Learning
Julien Merand, Boris Meden, Liming Chen, Mathieu Grossard
TL;DR
Dexterous grasp planners often optimize stability rather than task-aligned contact structure, while taxonomy conditioning can require costly object-annotated data. CoToGrasp learns gripper contact manifolds in a canonical workspace using object-agnostic training, achieving superior topology compliance and stability against taxonomy-guided methods and validating diverse grasps on a physical robot.
Problem
Existing planners focus on whether an object can be grasped rather than how it should be grasped, while taxonomy-conditioned synthesis typically requires costly object-annotated datasets.
Method
CoToGrasp projects local geometry into a gripper-centric canonical workspace and learns intrinsic contact capabilities with object-agnostic training.
Results
CoToGrasp demonstrates superior topology compliance and stability against state-of-the-art taxonomy-guided methods in simulation and real-world evaluations.
Takeaways & Limitations
The framework synthesizes diverse, topology-compliant grasps on unseen geometries and demonstrates their physical viability on an Allegro Hand.
Takeaways & Limitations
The object-agnostic formulation decouples grasp semantics from object topology by learning hand capabilities independently of specific objects.
Abstract
from arXiv · showhide
Current dexterous grasp planners primarily optimize for physical stability, focusing on whether an object can be grasped rather than how it should be grasped to support downstream functional tasks. However, conditioning grasp synthesis on specific human grasp taxonomies typically requires prohibitively expensive, object-annotated datasets. To address these limitations, we propose CoToGrasp, a novel generative framework that synthesizes diverse, stable grasps strictly conditioned on specific contact topologies. To bypass the data collection bottleneck, CoToGrasp is trained entirely in an object-agnostic manner. We introduce a feature-based canonical workspace that projects local object features into a unified gripper-centric domain, effectively decoupling the semantic functional intent from the arbitrary object geometry. By learning the intrinsic contact manifold of the gripper within this workspace, our model achieves zero-shot generalization to unseen objects at inference. Extensive evaluations on the large-scale DexGraspNet dataset demonstrate that CoToGrasp achieves state-of-the-art performance, outperforming existing taxonomy-guided planners. Finally, we demonstrate the physical viability and kinematic feasibility of our synthesized contact topologies on a physical robot platform. Code is available on our project website https://cea-list.github.io/cotograspweb/ .
1 Introduction
CoToGrasp addresses the bias of dexterous grasp planners toward physically stable but functionally uniform grasps by conditioning synthesis on structured contact topologies. Its object-agnostic canonical workspace learns gripper contact capabilities independently of object geometry, supporting diverse grasps and evaluation of topology compliance and stability.
- Dexterous manipulation tasks require controllable, distributed multi-point contact topologies beyond the pinch and enveloping grasps of parallel-jaw grippers.
- Existing planners prioritize geometric stability and force closure, producing stable but functionally uniform grasps that often default to enveloping power grasps.
- CoToGrasp conditions grasp synthesis on structured contact topologies derived from human grasp taxonomies rather than arbitrary stable configurations.
- Object-agnostic training learns intrinsic hand capabilities independently of specific objects, decoupling grasp semantics from object topology.
- A gripper-anchored canonical feature workspace projects local geometry into a unified domain, separating functional intent from object identity and enabling zero-shot generalization to unseen geometries.
- CoToGrasp is evaluated in simulation and real-world settings for functional diversity, topology compliance, and physical stability against taxonomy-guided methods.
2 Related Work
Related work shows that data-driven grasp planners commonly prioritize physical stability, suffer mode collapse toward enveloping power grasps, and generalize poorly when grippers are explicitly mapped to objects. Object-agnostic approaches instead learn contact topologies in canonical gripper space to improve generalization to novel geometries.
- Contemporary learning-based grasp planners prioritize physical stability over functional intent across explicit joint and intermediate grasp representations.
- Synthetic force-closure databases contribute to mode collapse toward enveloping power grasps and underuse dexterous hand capabilities.
- Explicitly mapping grippers to specific objects biases grasp planners and hinders shape generalization.
- Object-agnostic methods such as GOAG learn contact topologies in canonical gripper space and demonstrate superior generalization to novel geometries.
3 Method
CoToGrasp represents contact semantics on a gripper-oriented handprint and transfers geometric information through a canonical workspace. Its pipeline combines topology labels, aligned contact geometry, spatial feature aggregation, and attention-based modeling to predict contact maps.
- Grasp formulation: A grasp is represented as a gripper pose (R, t) and internal joint configuration Q, with the objective of generating stable grasps respecting a requested semantic topology.
- Pipeline: The two-phase pipeline trains an intrinsic gripper-centric contact manifold object-independently, then transforms target objects into the canonical frame for filtered grasp synthesis and optimization.
- Object-agnostic learning: Training uses gripper point clouds, while inference uses target-object point clouds; the model learns contact-template masks from local gripper geometry and topology.
- Gripper representation: The gripper handprint H is the spatial embedding of fixed active-surface points under pose and joint configuration, with each point storing position and surface normal.
- Contact topology: The taxonomy contains 21 topology templates, whose active points receive zone labels through a surjective mapping ζ from hand points to physical contact zones.
- Canonical workspace: The method shifts contact reasoning from the unbounded, variable object surface to the fixed, bounded gripper manifold and canonical workspace.
- Feature transfer: Local geometric features are projected onto fixed workspace basis points using weighted kNN aggregation gated by aligned distance, producing geometry-independent spatial tokens.
- Ground-truth templates: Workspace points receive contact labels from nearby aligned hand points within threshold ε; points beyond the threshold are assigned label 0.
4 Experiments
CoToGrasp is evaluated with physical-stability, semantic-compliance, diversity, and real-world validation measures. It produces more balanced, topology-compliant grasps than unconditioned and taxonomy-aware baselines, including executable grasps on a physical Allegro Hand.
- Experimental Setup: The object-agnostic dataset pairs 10,000 valid joint configurations with 21 contact templates, producing 210,000 training data points.The formulation maps 22 active contact zones on the target gripper to the topology templates.
- Evaluation Metrics: Topology Compliance (TC) measures whether generated grasps satisfy the requested functional contact template across evaluated topologies.For each topology, TC is based on effective grasps divided by attempted grasps, averaged across topologies.
- Evaluation Metrics: Normalized entropy measures topology-distribution diversity: values near 1 indicate uniform generation, whereas values near 0 indicate mode collapse.Stability Entropy uses normalized success rates, while Semantic Entropy uses normalized topology-compliance values.
- Overall Performances: Unconditioned baselines collapse toward enveloping power grasps, while CoToGrasp produces a more balanced distribution and the highest semantic entropy.On the MultiDex subset, CoToGrasp reports 36.94% overall SR and 56.18% Top-5 average SR while synthesizing diverse requested topologies.
- Overall Performances: CoToGrasp outperforms taxonomy-aware baselines in physical stability and semantic accuracy, especially for constrained precision grasps.Per-topology analysis reports M2 TC of 30.3% versus 10.5% and identifies inflated baseline scores caused by unverified power-grasp defaults.
- Real-World Validation: Real-world tests successfully planned and executed diverse topology-conditioned grasps on YCB objects using a UR10 manipulator and Allegro Right Hand.Topology M6 was suppressed because the four-fingered Allegro Hand cannot form its five-finger contact template; demonstrated grasps were held statically.
5 Conclusion
CoToGrasp decouples functional intent from object identity to synthesize topology-compliant precision and power grasps on unseen geometries. Its validation pipeline and Allegro Hand deployments support semantic diversity, physical stability, and executable contact topologies.
- CoToGrasp generates contact-topology-conditioned dexterous grasps while decoupling functional intent from object identity.
- The framework synthesizes constrained, topology-compliant precision and power grasps on unseen geometries without costly object-annotated datasets.
- Its validation pipeline filters topologically invalid configurations and yields state-of-the-art semantic diversity and physical stability.
- Successful real-world deployments on the Allegro Hand confirm that generated contact topologies are physically executable.
Supplementary Material
The supplementary material documents taxonomy transfer, handprint and workspace construction, architectural choices, and implementation details. It also describes the model’s conditioning, latent aggregation, and constrained multimodality design.
- A Applying contact topologies to grippers: Handprints are built from forward-kinematic gripper configurations and discretized into 2,048 points representing active grasping surfaces.
- A Applying contact topologies to grippers: Established human grasp taxonomies are mapped onto robotic kinematics using manually defined anatomical zones.
- A Applying contact topologies to grippers: The gripper workspace aggregates 10,000 sampled handprints and uses Farthest Point Sampling to produce uniformly distributed spatial basis points.
- B Architectural Choices and Implementation Details: Persistent hard conditioning concatenates topology embeddings with spatial tokens to preserve local semantic contact information during generation.
- B Architectural Choices and Implementation Details: A Set Transformer aggregates dense fused geometric and semantic features into a global latent descriptor instead of using static pooling.
- B Architectural Choices and Implementation Details: CoToGrasp uses a standard unimodal Gaussian prior because its latent variation is restricted to localized spatial differences within a specified topology.
- B Architectural Choices and Implementation Details: Training uses a variational objective combining reconstruction cross-entropy with KL regularization, and runs on 4 Nvidia A100 GPUs for approximately 35 hours.
C Train-Test Domain Shift and Feature Alignment
The supplementary analysis examines train-test feature alignment and the validation pipeline that converts generated contact templates into feasible hand configurations. It combines topology checking, force-closure assessment, and energy-based optimization with topology-conditioned pose sampling.
- C Train-Test Domain Shift and Feature Alignment: Features from gripper and object domains interleave within shared manifold clusters at task-relevant regions despite macro-structural differences.
- D Validation Pipeline and Energy Optimization Details: The validation pipeline sequentially performs label-consistency checking, force-closure evaluation, and energy-based joint optimization before finalizing hand kinematics.
- D Validation Pipeline and Energy Optimization Details: A tolerance of one in the symmetric-difference check permits at most one missing or hallucinated contact zone while rejecting fundamentally distinct topological failures.
- D Validation Pipeline and Energy Optimization Details: Force-closure is assessed from barycentric contact locations under a Coulomb friction model, while final physical quality is determined by dense kinematic alignment during optimization.
- D Validation Pipeline and Energy Optimization Details: The joint optimizer minimizes a weighted composite energy balancing contact alignment, penetration, self-collision, joint limits, and repulsion from unused hand regions.
- E Topology-Conditioned Pose Sampling Heuristic: Candidate object poses for distal and precision topologies are sampled within restricted active-zone sub-workspaces filtered for reachability and palm clearance.
F Automated Grasp Classification Metric
The automated metric extracts physical contacts, maps them to semantic zones, and compares observed zones with the requested topology using an asymmetric Tversky similarity.
- Contact Extraction and Zone Mapping: Physical contact points are mapped through a surjective zone mapping to obtain observed active zones, while required zones come from the target template excluding label 0.
- Asymmetric Tversky Index: The generated grasp’s topology match is scored with an asymmetric Tversky index comparing observed and required active-zone sets.
- Asymmetric Tversky Index: Extra contact regions receive stronger penalties than missing contacts, using w1 = 2.0 and w2 = 0.5.
- Asymmetric Tversky Index: Grasps with similarity s_m < 0.5 are classified as unknown.
G Standard Evaluation Metrics Protocol
The evaluation protocol measures physical stability, generation efficiency, and generative diversity using established simulation-based metrics.
- Standard Metrics: Success Rate measures the physical stability of synthesized grasps in Isaac Gym simulation.The simulation tests object geometry, gripper kinematics, and actuator torque-saturation limits under external perturbations.
- Standard Metrics: Generation Speed reports the average time required to generate one grasp over 100 attempts.The metric includes network inference and energy-based joint optimization, excluding Isaac Gym physics simulation.
- Standard Metrics: Spatial Diversity quantifies the spatial diversity of generated grasps.
H Detailed Experimental Protocols and Baselines
The experiments separately assess taxonomy-unaware functional bias and taxonomy-aware topology control using established baselines and dataset protocols. CoToGrasp is compared with four unconditioned planners and Dexonomy under filtered, object-topology-compatible evaluation.
- Taxonomy-Unaware Evaluation Protocol: The taxonomy-unaware protocol compares CoToGrasp with DFC, GenDexGrasp, DRO-Grasp, and GOAG on 10 MultiDex test objects.The objects come from ContactDB and YCB, and successfully generated stable grasps are classified for topology compliance, entropy, and spatial diversity.
- Evaluation Outputs: The reported stable grasps are automatically classified to evaluate topology compliance, entropy, and spatial diversity.
- Taxonomy-Unaware Evaluation Protocol: Because the unconditioned baselines lack functional inputs, this experiment exposes their natural generative distributions and power-grasp mode collapse.
- Taxonomy-Aware Evaluation Protocol: The taxonomy-aware protocol compares CoToGrasp with Dexonomy on all 1,126 geometrically diverse objects in the DexGraspNet test set.
- Taxonomy-Aware Evaluation Protocol: An object-topology pair with 0% success across all attempts is excluded as geometrically incompatible before final averaging.
- Taxonomy-Aware Evaluation Protocol: CoToGrasp covers 80.14% of objects per requested topology, closely matching Dexonomy’s 81.05% coverage after geometric filtering.
I Topology Compliance Analysis and Simulation Bottlenecks
Topology compliance is shaped by filtering, physics-based validation, metric-induced misclassification, and object geometry. CoToGrasp retains higher compliance than Dexonomy and shows stronger stability on difficult non-convex objects, while the metric remains an incomplete proxy for functional success.
- Topology Compliance Ablation: Topological filtering rejects geometrically incompatible templates before optimization, improving the semantic quality of successful grasps.It prevents the optimizer from coercing the hand into unnatural configurations that would later fail simulation.
- Topology Compliance Ablation: Physics-based validation lowers raw topology compliance while removing superficially valid but physically unstable configurations and rebalancing semantic entropy.Before Isaac Gym evaluation, geometric optimization produces higher compliance but low semantic entropy; post-physics filtering prunes weak grasps.
- Metric-Induced Misclassification: Kinematic constraints can create incidental phalanx contacts that reclassify intended M2 or M3 precision pinches as M12, despite stable final grasps.Fixed finger lengths and joint limits can prevent repulsive terms from removing adjacent contacts.
- Global Topological Distribution: CoToGrasp achieves TC = 17.18% versus Dexonomy’s TC = 14.28% across the 21 contact topologies.Both methods experience topology shifts during simulation, but CoToGrasp maintains a more balanced and faithful distribution.
- Object-Level Geometric Complexity: CoToGrasp retains 58.72% of physical SR on challenging non-convex objects, compared with 37.42% for Dexonomy and 28.09% for GOAG.
- Object-Level Geometric Complexity: On severe concavities, CoToGrasp reaches 18.30% SR and 19.17% TC, versus Dexonomy’s 12.05% SR and 8.94% TC.These objects represent roughly 4% of the dataset.
J Real-World Experiments
The real-world protocol plans motions in a ROS2 digital twin before executing synthesized grasps on hardware. Results expose limitations of deterministic squeezing and tabletop-only evaluation for preserving intended contact topologies.
- Execution Protocol: Each synthesized grasp is planned and validated collision-free in a ROS2 digital twin using MoveIt before hardware execution.The sequence approaches the object, interpolates to the target pose, and actuates the fingers toward an optimized squeezed configuration.
- Execution Protocol: Physical stability is tested by lifting the object 15 cm, holding it for 3 seconds, and opening the hand to release it.
- Discussion and Limitations: Simple linear interpolation to the squeezed configuration is insufficient because asynchronous contact creates unbalanced moments before force closure.Leading fingers can strike milliseconds early, displacing the object and shifting physical contacts from predicted zones.
- Discussion and Limitations: Tabletop benchmarks can inflate grasp success by favoring objects presented in constrained poses or requiring bimanual setups.The authors identify hybrid force/position control and tactile-reactive policies as future directions for preserving simulated topologies.
K Qualitative Results: CoToGrasp Visualization
Across YCB and DexGraspNet objects, CoToGrasp qualitatively adheres to prescribed contact topologies and recovers intended grasp types across varied surfaces.
- CoToGrasp aligns fingers with semantic contact zones instead of merely targeting an object’s center of mass.
- Grouped by topological identifiers M1-M21, the planner consistently recovers the prescribed contact manifolds.
- The synthesized configurations preserve grasp semantics across delicate fingertip pinches and complex multi-digit wraps.
- These qualitative results span diverse YCB and DexGraspNet objects, including convex and non-convex surfaces.