Source-linked AI summary
InterMASH: A Unified Geometric Representation for Grasp Synthesis
Xuanze Yang, Yumeng Liu, Haiyang Xin, Changhao Li, Haowei Shen, Kai Xu, Ligang Liu, Ruizhen Hu
TL;DR
Grasp synthesis needs representations that can capture hand-object geometry across human and robotic morphologies, but a unified representation is lacking. InterMASH uses shared sphere-fixed anchors and compact spherical-harmonic tokens, with a conditional diffusion model that jointly generates hand geometry and contact. The paper reports improved physical feasibility and contact consistency, cross-embodiment training, and benefits from integrating human grasp data into robotic grasp synthesis.
Problem
Grasp synthesis lacks a unified representation across human and robotic hands, whose morphology and geometry differ.
Source: §1 · p. 2, §4.1 · p. 4Method
InterMASH encodes local hand geometry, object geometry, and contact with spherical-harmonic descriptors at shared anchors, and a conditional diffusion model jointly generates hand geometry and contact.
Source: §1 · p. 2, §4.2 · p. 5Results
Takeaways & Limitations
InterMASH supports integrating MANO-based human grasp data into robotic grasp synthesis and joint training across hand morphologies.
Source: §1 · p. 2, §6 · p. 8With fewer hand-specific inductive biases, InterMASH can be less data-efficient for high-DoF hands in smaller mixed-hand datasets.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Grasp synthesis aims to generate stable and physically plausible hand--object interactions, and has become a fundamental problem in both human hand modeling and robotic manipulation. However, a unified representation across human and robotic hands is still lacking, mainly due to differences in hand morphology and surface modeling. Prior methods typically rely on either contact maps or dense implicit descriptors to represent interaction, but these representations are often incomplete or computationally expensive and redundant. We propose InterMASH, a unified geometric representation that establishes cross-embodiment correspondence using sphere-fixed anchors. At each anchor, low-degree spherical harmonics compactly encode local hand geometry, object geometry, and contact, forming an explicit and interpretable token sequence. Building on this natively tokenized structure, we introduce a conditional Diffusion Transformer that operates directly in the proposed InterMASH representation space and jointly generates hand geometry and contact, improving consistency and physical plausibility. Our method achieves competitive performance with state-of-the-art methods on key physical feasibility metrics in a large-scale ShadowHand benchmark, supports joint training across multiple hands, and shows that cross-embodiment fine-tuning with human grasp data can improve robotic grasp success and diversity. Project page is available at https://inter-mash.github.io/.
1 Introduction
The paper addresses the lack of a compact interaction representation that generalizes across hand morphologies, proposing InterMASH and a conditional generative model built around it. Its experiments report strong physical-feasibility performance and benefits from cross-embodiment training with human grasp data.
1 Introduction
- Contact maps omit hand geometry, while dense pairwise or spatial-field descriptors include redundant information, limiting fine-grained modeling or scalable generation.These shortcomings motivate a representation that is both geometrically expressive and compact.Source: §1 · p. 2
- InterMASH uses sphere-centered fixed anchors and low-degree spherical harmonics to encode hand surface, object surface, and contact in a compact shared token space.The shared indexing supports different hand morphologies while retaining fine-grained local geometry.Source: §1 · p. 2
- InterMASH establishes consistent anchor semantics by ordering MANO reference patches and aligning robotic templates through keypoint-distance signatures.Jointly fitting hand and object coefficients is stabilized with a coarse-to-fine schedule that progressively increases spherical-harmonic degree.Source: §1 · p. 2
- A conditional Diffusion Transformer operates directly on InterMASH tokens and jointly synthesizes hand geometry and contact.The paper states that joint modeling promotes shape-contact consistency and reduces penetration.Source: §1 · p. 2
- The full framework achieves state-of-the-art performance on key physical-feasibility metrics, while cross-embodiment synthesis is strongest on Barrett and competitive on ShadowHand.Joint training on human and robotic hand data benefits robotic grasp generation; integrating human grasp data improves robotic grasp success rate and diversity.Source: §1 · p. 2
- InterMASH enables joint training across hand morphologies, and integrating human grasp data improves robotic grasp success rate and diversity.The paper presents this as a way to transfer human grasp information into robotic grasp generation.Source: §1 · p. 2
2 Related Work
Human and robotic grasp synthesis have developed with distinct data and representations, and transferring human grasp priors to robots remains difficult. Existing interaction representations also trade geometric detail against compactness and computational demands.
2 Related Work
- Human and robotic grasp synthesis use distinct data sources and models, while morphology and kinematic differences require extra retargeting to apply human interactions to robots.Retargeting adds computational overhead at scale; embodiment-specific human-aligned robotic data collection is also difficult to scale.Source: §2 · pp. 2–3
- Human grasp methods range from optimization and retargeting to data-driven synthesis, whereas simulated robotic datasets emphasize physical feasibility and stability over human-like dexterity.Human-to-robot retargeting can require source interactions and semantic object parts, limiting scalability and diversity.Source: §2 · p. 3
- Contact maps ignore hand geometry; interaction-centric alternatives include computationally intensive bisector surfaces, high-dimensional distance descriptors, and memory-intensive skeletal distance fields.The cited methods also face noisy or low-fidelity geometry, loss of fine detail, or incompatibility with continuous hand deformation.Source: §2 · p. 3
- These limitations motivate expressive, compact, explicit geometric representations for grasp synthesis across embodiments.Graph diffusion on robotic links is efficient but limited to rigid-body systems and cannot model continuous hand deformations.Source: §2 · p. 3
3 Preliminary
MASH represents a surface as aligned local patches observed from fixed anchors. It approximates each anchor’s spherical distance function with spherical harmonics and uses visibility-aware masks to keep the local representation compact and stable.
3 Preliminary
- MASH places anchors on a shape, with each anchor acting as a virtual viewpoint that captures a localized surface patch.Rays from the anchor determine a spherical distance function from the first visible surface intersection in each direction.Source: §3 · p. 3
- Each anchor’s spherical distance function is approximated with spherical harmonics to provide a compact, differentiable parameterization.The coefficients are the parameters of the harmonic expansion.Source: §3 · p. 3
- A generalized view-cone mask restricts valid angular support, addressing occlusion-driven visibility changes and allowing low-order harmonics to approximate local patches compactly and stably.The cone has an anisotropic boundary that limits the valid polar angle for each azimuth.Source: §3 · p. 3
- An anchor stores its position, orientation, harmonic coefficients, and vision-mask parameters; the full shape is reconstructed as the union of anchored patches.This explicit patch structure motivates extending MASH to shared-anchor, cross-instance hand-object interactions with contact-aware geometry.Source: §3 · p. 3
4 Methodology
InterMASH represents hand–object interactions as shared anchor-indexed tokens, and a conditional Diffusion Transformer generates grasp geometry and contact before inverse kinematics decodes an articulated pose. Its methodology aligns patch semantics across hand types and incorporates physical guidance during model training and sampling.
4 Methodology
- The pipeline models grasp synthesis as conditional generation from an object and hand template, then decodes generated hand geometry into an articulated pose through inverse kinematics.InterMASH encodes local hand geometry, object geometry, and contact in a unified token space.Source: §4 · pp. 3–4
4.1 InterMASH Representation
- Fixed object-centered sphere anchors and low-degree SH descriptors jointly encode hand and object patches, with MANO-centered alignment giving corresponding regions consistent semantics across embodiments.The correspondence is transferred to grasp instances and used in both InterMASH fitting and IK.Source: §4.1 · pp. 4–5
4.2 Scalable Generative Model Design
- A conditional DiT generates anchor-wise hand geometry and contact from object and template-hand geometry, while physical penalties on reconstructed point clouds guide training and sampling.Sampling evaluates physical plausibility on denoised geometry and steers toward lower penetration, fewer self-collisions, and more stable contact.Source: §4.2 · pp. 5–6
4.3 Final Hand Pose from Inverse Kinematics
- 0.0125 m average total ℓ1 Chamfer error and 99.5% success under Ltotal CD < 0.1 m; decoding averages 0.39 seconds per sample.These patch-wise IK results are for generated ShadowHand samples from the filtered CMapDataset.Source: §4.3 · p. 6
5 Experiments
The experiments assess grasp quality, cross-embodiment training, human-prior transfer, and the effects of InterMASH’s representation and model components. Results include strong physical-feasibility metrics, improved ShadowHand success with mixed-hand training, and benefits from GRAB fine-tuning, alongside a success–diversity trade-off.
5 Experiments
- The experiments are organized to test single-hand quality, joint modeling across morphologies, human-prior transfer, and key representation and model design choices.The setup and evaluation span DexGraspNet and filtered CMapDataset experiments.Source: §5 · p. 6, §5.2 · p. 7, §5.3 · p. 7
5.1 Experimental Setup
- The evaluation uses official train-test splits and reports grasp quality, physical plausibility, and diversity, with separate protocols for single-hand and cross-embodiment tasks.Single-hand metrics include Suc.6, Suc.1, Pen., and Div.; cross-embodiment evaluation reports simulator-based success rate and diversity.Source: §5.1 · pp. 6–7
5.2 Single-Hand Grasp Generation
- InterMASH achieves the best DexGraspNet performance on Suc.1 and Pen., but remains competitive rather than leading on Suc.6 and Div.Suc.1 measures success under at least one disturbance direction, while Suc.6 requires success under all six.Source: §5.2 · p. 7, Table 1 · p. 7
5.3 Cross-Embodiment Robotic Grasp Synthesis
- Adding Barrett training data raises ShadowHand success from 57.62% to 64.15%, while diversity decreases from 0.416 to 0.396.With one jointly trained model, InterMASH achieves the highest success rate on Barrett while remaining competitive on ShadowHand.Source: §5.3 · p. 7, Table 2 · p. 7
- Figure 6 contrasts generated grasp geometries: InterMASH produces more recognizable hand-like geometry and supports arbitrary sampling, unlike fixed-resolution D(R, O) point clouds.Additional cross-embodiment qualitative examples are shown in Figure 9.Source: §5.3 · p. 7, Figure 6 · p. 7
5.4 Integration of Human Priors into Robotic Grasping
- GRAB fine-tuning improves robotic grasp success and diversity, while penetration increases from 14.9 to 18.6 mm.The authors attribute the penetration increase as likely related to higher average penetration in GRAB training data than in DexGRAB.Source: §5.4 · p. 7, Table 3 · p. 7
5.5 Ablation Study
- In representation ablations with matched generation settings, InterMASH achieves the highest success rate across all hand types.The comparison disables physics-guided training, physics-guided sampling, and Neighborhood Enhanced Attention.Source: §5.5 · p. 8, Table 4 · p. 8
- Physics-guided training improves success on Barrett and ShadowHand; physics-guided sampling further improves ShadowHand success, and Neighborhood Enhanced Attention gives the strongest overall performance.The authors report a moderate diversity decrease with these stronger geometric and physical constraints.Source: §5.5 · p. 8, Table 5 · p. 8
6 Conclusion
InterMASH represents hand–object interaction with anchor-indexed patch primitives and uses conditional diffusion to generate hand geometry and object-side contact. Experiments support its use in physical-feasibility and contact-consistency tasks, cross-embodiment training, and integration of human grasp data, while data efficiency for high-DoF hands in smaller mixed-hand datasets remains a limitation.
6 Conclusion
- InterMASH uses anchor-indexed patch-level primitives as a unified geometric representation for hand–object interaction.A conditional diffusion model jointly generates grasping hand geometry and object-side contact, conditioned on object geometry and a hand template.Source: §6 · p. 8
- Experiments show improved physical feasibility and contact consistency, support cross-embodiment robotic hand training, and enable integration of MANO-based human grasp data into robotic grasp synthesis.The authors conclude that intermediate geometric interaction representations are an effective design choice for scalable grasp generation.Source: §6 · p. 8
- InterMASH can be less data-efficient for high-DoF hands in smaller mixed-hand datasets, where stronger built-in correspondence priors may remain advantageous.The authors identify ShadowHand in filtered CMapDataset as an example because it has fewer samples and a higher-dimensional action space than Barrett.Source: §6 · p. 8