PaperlayerView exact revision on arXiv

Source-linked AI summary

Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

arXiv:1903.00425v4cs.RO

TL;DR

High-DOF hands admit many equally effective grasp poses for each object, making consistent single-pose prediction difficult. The method uses an augmented set of candidate grasps with consistency and collision losses; it reports higher accuracy than supervised-learning baselines and demonstrates grasping with a 25-DOF Shadow Hand.

  • Problem

    High-DOF hands have many equally effective grasps per object, making it difficult for one neural network to select a consistent grasp pose.

    Source: §I · p. 1

  • Method

    The method trains direct object-to-pose prediction using candidate grasp poses and consistency loss, then applies collision loss to penalize hand-object penetration.

    Source: §I · p. 1, §VI · p. 7

  • Results

    4× higher accuracy than supervised-learning baselines in grasp-pose representation, measured by distance to groundtruth grasp poses.

    Source: §I · pp. 1–2

  • Takeaways & Limitations

    The method was implemented on a 25-DOF Shadow Hand, computing each grasp in 3–5 seconds from object models that can be reconstructed from multi-view depth images.

    Source: §I · p. 2

    The method requires many effective grasp poses per object, making dataset generation computationally costly for very large object sets.

    Source: §VI · p. 8

Abstract

from arXiv · show

We present a learning-based method for representing grasp poses of a high-DOF hand using neural networks. Due to redundancy in such high-DOF grippers, there exists a large number of equally effective grasp poses for a given target object, making it difficult for the neural network to find consistent grasp poses. We resolve this ambiguity by generating an augmented dataset that covers many possible grasps for each target object and train our neural networks using a consistency loss function to identify a one-to-one mapping from objects to grasp poses. We further enhance the quality of neural-network-predicted grasp poses using a collision loss function to avoid penetrations. We use an object dataset that combines the BigBIRD Database, the KIT Database, the YCB Database, and the Grasp Dataset to show that our method can generate high-DOF grasp poses with higher accuracy than supervised learning baselines. The quality of the grasp poses is on par with the groundtruth poses in the dataset. In addition, our method is robust and can handle noisy object models such as those constructed from multi-view depth images, allowing our method to be implemented on a 25-DOF Shadow Hand hardware platform.

I. INTRODUCTION

High-DOF grasp generation must handle many equally effective poses per object and determine hand joints beyond wrist pose. The method addresses this ambiguity with consistency loss and reports higher accuracy than supervised-learning baselines.

I. INTRODUCTION

  • Earlier learning methods mainly target low-DOF grippers or constrain high-DOF hands to a low-DOF subspace, limiting the grasp poses they represent.Direct prediction is efficient at runtime, but high-DOF pose ambiguity requires additional guidance for selecting the pose to learn.Source: §I · p. 1
  • High-DOF hands have many equally effective grasps for each object, but a single neural network must select one pose consistently.Unlike low-DOF grippers, high-DOF hands have remaining joint angles to determine after the wrist pose is known.Source: §I · p. 1
  • 4× higher accuracy than supervised-learning baselines in representing grasp poses, measured by distance to groundtruth grasp poses.The method was also implemented on a 25-DOF Shadow Hand, with each grasp computed in 3–5 seconds at runtime.Source: §I · pp. 1–2
  • Consistency loss guides pose selection among candidates, while collision loss penalizes hand-object penetrations.The training dataset combines grasp data for 324 objects from four databases.Source: §I · p. 1

II. RELATED WORK

Prior grasp methods often optimize candidate poses at runtime or target low-DOF grippers, while this work predicts high-DOF poses directly. The formulation must address both pose-selection ambiguity and the quality of poses output by the network.

II. RELATED WORK

  • Most learning-based methods estimate grasp success and then optimize a pose; direct pose prediction avoids that runtime search but prior direct prediction does not handle high-DOF ambiguity.The paper assumes complete object models represented as occupancy grids, while reporting robustness to inaccuracies and models reconstructed from multi-view depth images.Source: §II · p. 2

III. PROBLEM FORMULATION

  • The formulation maps an object observation to a hand configuration comprising the wrist transform and remaining joint angles.The assumed observation is a discretized 3D occupancy grid; a signed distance field is derived from the original mesh.Source: §III · p. 2
  • Direct pose prediction is chosen over success scoring because high-DOF optimization is computationally costly and can be ill-posed when many unnatural poses succeed.The alternative success function scores a candidate grasp pose given the object observation, then requires optimization to select a pose.Source: §III · p. 3
  • Because multiple grasps are valid for each object, different choices of groundtruth poses can produce substantially different supervised data losses.The training challenge is to construct a dataset that yields a small data loss after training.Source: §III · p. 3
  • The network must produce high-quality poses directly, including poses without object penetration, because its outputs are sensitive to prediction quality.Unlike methods that optimize a final pose after training, this method relies on the network output itself.Source: §III · p. 3

IV. LEARNING HIGH-DOF GRASP POSES

The architecture maps object observations through a feature representation to a high-DOF grasp pose. It uses a fully connected pose network and an observation-processing network when the input requires one.

IV. LEARNING HIGH-DOF GRASP POSES

  • The section presents the neural-network architecture used for high-DOF grasping.The overall mapping combines the observation and pose networks.Source: §IV · p. 3, A. Neural Networks · p. 3

A. Neural Networks

  • A low-dimensional object feature vector ω is mapped to a high-DOF grasp pose x by the fully connected sub-network NNx.The pose network has three hidden layers with ReLU activation and batch normalization in its first two layers.Source: A. Neural Networks · p. 3
  • When sensor observations need transformation into features, an observation sub-network NNo processes them before NNx predicts the pose.NNo uses three 3D convolutional layers, each with 64 kernels of size 4, followed by batch normalization, ReLU, and max pooling.Source: A. Neural Networks · p. 3

B. Consistency Loss

The consistency loss handles grasp ambiguity by matching each object's prediction to its closest pose among multiple candidates. A collision loss penalizes gripper penetration, using sampled hand points and the object's signed distance field.

B. Consistency Loss

  • For each object, the consistency loss selects the closest of K candidate grasp poses to the network prediction, allowing the network to choose a consistent pose.The candidates are retained as a set rather than choosing one ground-truth pose during dataset construction.Source: B. Consistency Loss · p. 3

C. Collision Loss

  • The collision loss penalizes predicted grasps whose sampled gripper points lie inside the object, using its signed distance field.The loss evaluates points transformed into global coordinates by the gripper's forward kinematics.Source: C. Collision Loss · p. 3
  • The collision-loss sample points are the same contact points used to generate ground-truth grasps, because grasp quality is sensitive to point selection.The ground-truth poses are generated by simulated annealing.Source: C. Collision Loss · p. 4

D. Combined Loss

The combined loss weights consistency against collision avoidance, with the weight changing how predictions relate to ground truth and object surfaces. At runtime, pose refinement searches for a nearby grasp without intersections.

D. Combined Loss

  • Higher β makes grasps more like GraspIt! ground truth, while lower β brings fingers closer to object surfaces and is associated with higher success rates.The combined objective weights consistency by β and collision loss by 1 − β.Source: D. Combined Loss · p. 4

E. Pose Refinement

  • Pose refinement searches at runtime for a grasp pose closest to the network prediction that has no intersections with the object.The search minimizes an objective for a pose x* close to the nominal prediction.Source: E. Pose Refinement · p. 4

V. IMPLEMENTATION AND PERFORMANCE

The implementation generates ground-truth grasps for a multi-dataset object collection, then expands the training data by rotating objects together with their grasp poses.

V. IMPLEMENTATION AND PERFORMANCE

  • The implementation section presents the experimental platform setup, results, and evaluations.Source: §V · p. 4

A. Grasp Training Dataset Generation

1) Grasp Pose Generation:

  • The dataset contains 324 object meshes, with 100 simulated-annealing planner runs from random starts per object to generate grasp poses.The objects come from BigBIRD, KIT Object Models, YCB Benchmarks, and the Grasp Database; the gripper is a 25-DOF Shadow Hand.Source: 1) Grasp Pose Generation · p. 4, Figure 3 · p. 4, Figure 4 · p. 4

2) Data Augmentation:

  • The augmentation rotates each object and its 100 best grasp poses through 27 rotations, making the dataset 27 times larger.Rotations concatenate 60°, 120°, and 180° rotations about the X, Y, and Z axes; the input to NNv is a voxelized occupancy grid.Source: 2) Data Augmentation · pp. 4–5, Figure 5 · p. 5

3) Signed Distance Fields Construction:

The signed distance fields are constructed in a unit-cube coordinate system and shared across augmented versions of each object. The section also specifies the data split, contact-point sampling, and training setup.

3) Signed Distance Fields Construction:

  • Each signed distance field has resolution 128^3 and occupies the unit cube, with a transform mapping the object’s local coordinates into that field.Figure 6 illustrates the field construction and marks the domain of os in green.Source: 3) Signed Distance Fields Construction · p. 5, Figure 6 · p. 5
  • The 324 objects are split into 259 training and 65 test objects, yielding 8,748 voxelized grids after 27-fold augmentation; augmented versions share one signed distance field per object.Source: B. Experimental setup · p. 5
  • The Shadow Hand uses 45 sampled potential contact points, transformed from global coordinates into the signed-distance-field coordinate system.The passage also specifies a transformation for sampling the field at those points.Source: B. Experimental setup · p. 5
  • Training the neural networks takes 40 hours on a desktop with an Intel Xeon W-2123 CPU and NVIDIA Titan Xp GPU.Source: B. Experimental setup · p. 5

C. Results and evaluation

The evaluation shows that single-pose supervised learning is especially inconsistent for the 25-DOF Shadow Hand, while consistency loss better represents grasp poses among many candidates. With K = 100 candidate poses, consistency loss sharply reduces its residual, whereas adding collision loss does not further reduce residual metrics.

1) Challenge of High-DOF Grasp Problems:

  • The Shadow Hand has higher L2 residuals than the Barrett Hand on both training and test sets, indicating more severe inconsistency for the high-DOF gripper.The benchmark trains each network from one grasp pose per object (K = 1).Source: Table I · p. 5, 1) Challenge of High-DOF Grasp Problems · p. 5
  • Consistency loss compares predictions with K groundtruth poses, whereas L2 compares them with a single groundtruth pose.The paper treats these residuals as comparable.Source: 2) Consistency and Collision Loss · p. 6
  • 2.630 for Lconsistency versus 73.61 for L2 with simple supervised training; consistency loss further reduces Lconsistency to 0.043.The comparison uses K = 100 candidate grasp poses; combined loss does not further reduce residual metrics.Source: 2) Consistency and Collision Loss · pp. 5–6, Table II · p. 6

3) Penetration Handling:

Collision loss reduces hand-object penetrations, but some remain after training. Runtime adjustment updates joint angles along the negative collision-loss gradient until penetrations are eliminated.

3) Penetration Handling:

  • With collision loss, mean penetrating sample points fall from 2.563 of 45 to 0.719, and mean penetration depth falls from 0.0553m to 0.0081m.Some small penetrations remain after applying the training-time loss.Source: 3) Penetration Handling · p. 6
  • Runtime adjustment updates joint angles along the negative gradient of Lcollision until no penetrations remain.The method uses interpolation as a second step after collision loss.Source: 3) Penetration Handling · p. 6
  • A neural-network forward pass takes 0.541s, and runtime adjustment takes 0.468s.Source: 3) Penetration Handling · p. 6

4) Multi-View Depth Image as Input:

The method is tested on noisy object meshes reconstructed from multi-view depth images and on real Shadow Hand grasps. On 15 held-out objects, it achieves average grasp quality of 0.102, with 12 successful hardware grasps.

4) Multi-View Depth Image as Input:

5) Comparison with Prior Methods:

The method directly predicts high-DOF grasp poses, using consistency and collision losses to address pose redundancy and penetrations. Its comparisons and limitations show that it still relies on substantial grasp data and object rotations to find feasible poses.

5) Comparison with Prior Methods:

VI. CONCLUSION AND LIMITATIONS

  • Consistency loss improves grasp-pose accuracy over conventional supervised learning, and collision loss resolves penetrations on training and test sets.Source: §VI · pp. 7–8
  • The method requires many effective grasp poses per target object, making dataset generation computationally costly for very large object collections.Source: §VI · p. 8
  • If a predicted pose is unreachable or not collision-free, the target object grid must be rotated and passed through the network again until a feasible pose is found.Source: §VI · p. 8
  • The method has been evaluated on the Shadow Hand; extending it to other high-DOF grippers is a future direction.The paper also identifies high-genus objects as a case requiring a reformulated collision loss.Source: §VI · p. 8
Loading 1903.00425v4…