PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Differentiable Grasp Planner for High-DOF Grippers

Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

arXiv:2002.01530v2cs.RO

TL;DR

Learning high-DOF grasps from unknown objects involves data demands and costly sampling-based optimization. This paper differentiates grasp-quality and collision objectives for training networks to predict grasp poses, reporting higher hardware success and Q1 quality than a supervised baseline.

  • Problem

    Sampling-based optimization of learned grasp metrics is less efficient for high-DOF grippers, and accurate scores across the configuration space require large training datasets.

    Source: §I · p. 1

  • Method

    The method uses a generalized differentiable Q1 metric and differentiable collision losses to train networks to predict grasp poses.

    Source: §I · pp. 1–2

  • Results

    The method achieves a 22% higher success rate on physical hardware and a 0.12 higher Q1 grasp quality metric than supervised learning baseline [29].

    Source: §I · p. 2

  • Takeaways & Limitations

    The approach is presented as both a standalone grasp planner and a neural-network training algorithm, with trained networks performing robustly on unseen objects and hardware platforms.

    Source: §VII · p. 8

    The method requires target objects to be watertight and have non-zero volume; very thin objects can cause tunneling and missed solutions.

    Source: §VII · p. 8

Abstract

from arXiv · show

We present an end-to-end algorithm for training deep neural networks to grasp novel objects. Our algorithm builds all the essential components of a grasping system using a forward-backward automatic differentiation approach, including the forward kinematics of the gripper, the collision between the gripper and the target object, and the metric for grasp poses. In particular, we show that a generalized Q1 grasp metric is defined and differentiable for inexact grasps generated by a neural network, and the derivatives of our generalized Q1 metric can be computed from a sensitivity analysis of the induced optimization problem. We show that the derivatives of the (self-)collision terms can be efficiently computed from a watertight triangle mesh of low-quality. Altogether, our algorithm allows for the computation of grasp poses for high-DOF grippers in an unsupervised mode with no ground truth data, or it improves the results in a supervised mode using a small dataset. Our new learning algorithm significantly simplifies the data preparation for learning-based grasping systems and leads to higher qualities of learned grasps on common 3D shape datasets [7, 49, 26, 25], achieving a 22% higher success rate on physical hardware and a 0.12 higher value on the Q1 grasp quality metric.

I. INTRODUCTION

The paper addresses the data and optimization burdens of learning high-DOF grasps by differentiating grasp-quality and collision objectives. It uses those objectives to train networks that directly predict grasp poses, and reports improved results against a supervised baseline.

I. INTRODUCTION

  • Prior methods learn intermediary metrics or object representations that can improve data efficacy and robustness but complicate training and data preparation [57].Sampling-based metric optimization is less efficient for high-DOF grippers and requires accurate scores across the configuration space, motivating large datasets such as the 6.7 million ground-truth grasps used by [34].Source: §I · p. 1
  • The planner extends continuous grasp optimization with a generalized Q1 metric defined for non-contact poses and differentiable through optimality-condition sensitivity analysis [2].It also provides differentiable collision losses computed from target-object surface meshes.Source: §I · pp. 1–2
  • 22% higher success rate on physical hardware and 0.12 higher Q1 grasp quality metric than supervised learning baseline [29].The network was trained on 400 objects and 40K ground-truth grasps.Source: §I · p. 2

II. RELATED WORK

  • Unlike methods that require a sampling-based optimizer or post-processing, this approach directly predicts robust, usable gripper poses with a smaller training dataset [34] [15] [53] [29].It can also be combined with previous learning-based methods to improve their results.Source: §II · p. 3

III. LEARNING GRASP POSES FOR HIGH-DOF GRIPPERS

The section motivates directly predicting high-DOF grasp poses and introduces a differentiable grasp-planning loss. Its notation defines the contact and wrench quantities used to measure grasp quality.

III. LEARNING GRASP POSES FOR HIGH-DOF GRIPPERS

  • The planner trains a network on multiple depth views to predict a grasp pose, but high-DOF runtime search is costly and valid grasp poses are ambiguous.The predicted pose includes 6D extrinsic parameters and I joint angles; consistent ground-truth poses are needed to train a network that outputs one pose.Source: §III · p. 3

IV. DIFFERENTIABLE GRASP PLANNER

  • The loss combines generalized Q1 quality, a heuristic guide for non-force-closed grasps, and penalties for self-collision and gripper-object collision.Existing grasp metrics have discontinuities and require exact contact, while low-quality meshes can make gradients unreliable.Source: §IV · p. 3, §III · p. 3

A. Notation

  • Q1 measures grasp quality through the radius of an origin-centered 6D sphere in admissible wrench space, constrained by force magnitude and friction.The metric tensor M tunes force and torque weights; the benchmarks set m inversely proportional to the object's average scale.Source: A. Notation · p. 4

B. Generalized Q1 Metric with Inexact Contacts

The generalized Q1 metric extends grasp quality evaluation to inexact contacts. The section presents an inexpensive upper-bound approach and a costlier lower-bound approach that propagates gradients across contact points.

B. Generalized Q1 Metric with Inexact Contacts

  • The generalized Q1 metric handles points away from the object surface by weighting admissible wrenches according to contact distance and alignment of gripper and object normals.The weighting favors points closer to the surface and better-aligned normals, and the generalized metric converges to standard Q1 as α,β →∞.Source: B. Generalized Q1 Metric with Inexact… · p. 4

1) Derivatives of the Q1 Upper Bound:

2) Derivatives of the Q1 Lower Bound

  • The lower-bound method uses SOS optimization, reduced to an SDP, whose gradients are computed by the chain rule and sensitivity analysis.The smooth approximation generally yields nonzero derivatives at all contact points, enabling updates to all fingers, but the SDP solve costs more than the upper-bound computation.Source: 2) Derivatives of the Q1 Lower Bound · pp. 4–5

C. Geometry Related Loss Functions

The section derives geometric quantities and collision losses from triangle meshes. It addresses feature-dependent derivatives and numerical and computational issues in mesh queries.

C. Geometry Related Loss Functions

  • The method computes signed distance and surface normals from a watertight triangle mesh to evaluate Q1 and avoid gripper-object penetration.A bounding volume hierarchy can accelerate mesh queries; triangle meshes also avoid the time and memory cost of conversion to signed distance fields and preserve thin features that low-resolution fields may miss.Source: C. Geometry Related Loss Functions · p. 5
  • The distance gradient is computed from the closest mesh feature, while the normal gradient depends on whether that feature is an edge, vertex, or triangle.For a closest point inside a triangle, the normal derivative is zero.Source: C. Geometry Related Loss Functions · p. 5, Figure 4 · p. 5
  • A bounding volume hierarchy accelerates the search for the nearest triangle by rejecting nodes farther from the query point than the current best distance.The distance computation otherwise loops over all mesh triangles.Source: C. Geometry Related Loss Functions · p. 5
  • Exact rational arithmetic addresses floating-point errors when deciding whether a point near a triangle plane lies inside the triangle mesh.The results are converted back to finite-precision floating-point values after computation.Source: C. Geometry Related Loss Functions · p. 5

D. Self-Collision of the Gripper

The section adds a link-based self-collision penalty and uses guidance and data losses to address degenerate or suboptimal grasp predictions.

D. Self-Collision of the Gripper

  • The self-collision loss penalizes collisions between gripper links, whose shapes are approximated by convex hulls.The loss can be computed from H-representations and accelerated with a bounding volume hierarchy; a larger sample set is used for this loss than for generalized Q1.Source: D. Self-Collision of the Gripper · pp. 5–6

E. Defending Against Degenerate Cases and Local Minima

  • When force closure fails, Q1 and its gradients are zero, so a heuristic term guides grasp points toward the object surface.This supplies guidance in the degenerate case where Q1 alone provides no gradient.Source: E. Defending Against Degenerate Cases… · p. 6
  • A Chamfer data loss guides the network away from suboptimal local minima by matching predicted poses to precomputed ground-truth poses.The distance is measured in the gripper's configuration space.Source: E. Defending Against Degenerate Cases… · p. 6

F. Forward Kinematics

The section describes how predicted gripper configurations pass through differentiable forward kinematics with bounded joint angles, then outlines the dataset and experimental settings used to evaluate the planner and learned network.

F. Forward Kinematics

  • The network predicts a gripper pose, and forward kinematics propagates grasp-point gradients back to it while a sigmoid transform enforces joint limits with non-vanishing gradients.For limits [l,u], the transformation is (u − l)sigmoid(N) + l.Source: F. Forward Kinematics · p. 6

V. EXPERIMENTAL SETUP

  • The dataset contains 500 watertight objects split into 400 training and 100 test objects, with 100 precomputed grasps per object yielding 40K grasps and 400 selected ground-truth grasps.Generating the grasps for the 24-DOF gripper required about 150 CPU hours; samples pair five-view depth images with a mesh and grasp poses, and objects and grippers are augmented across eight symmetric axes.Source: §V · p. 6
  • Experiments use a (6+18)-DOF Shadow Hand on a UR10 arm; the network predicts hand poses, while a runtime motion planner computes arm configurations.The setup labels 45 potential grasp points and samples 15,555 potential contact points for self-collision detection.Source: §V · p. 6
  • A shared ResNet-50 processes multi-view depth images, pools their features, and predicts gripper configurations; training uses ADAM with batch size 16 and an initial learning rate of 1e-4.The learning rate decays by 0.9 every 20 epochs.Source: §V · pp. 6–7

VI. EXPERIMENTAL RESULTS

  • The evaluation considers the method both as a standalone grasp planner and as a way to train grasp-predicting neural networks.The supplied section states these evaluation settings but gives no comparative outcome.Source: §VI · p. 7

A. Grasp Planning without Ground Truth

The experiments test the differentiable planner as a standalone optimizer and as a way to train grasp-pose predictors, culminating in comparisons on simulated and physical hardware. The conclusion highlights mesh-based differentiable losses and limitations involving target geometry, workspace constraints, and grasp type.

A. Grasp Planning without Ground Truth

  • As a standalone planner, the method finds local optima at computational cost comparable to, but can reach local minima without force closure (Q1 = 0).The experiment optimizes a 24-DOF gripper pose vector without data guidance.Source: A. Grasp Planning without Ground Truth · p. 7

B. Learning Grasp Poses with Ground Truth

  • On 100 held-out objects, the trained network achieved a mean Q1 of 0.226 and variance of 0.0045; varied grasps from perturbed object poses can be screened for feasibility.Training pre-trains with data loss and then fine-tunes with the differentiable loss; Figure 7 shows predicted grasps for unseen objects that can be deployed without post-processing.Source: B. Learning Grasp Poses with Ground… · p. 7, Figure 7 · p. 8

C. Comparison

  • Compared with [29], the method improves grasp quality and predicts poses deployable on hardware without post-processing for penetrations or collisions.Figure 9 contrasts the methods and depicts reduced penetrations or collisions with the proposed method.Source: C. Comparison · p. 7, Figure 9 · p. 8

VII. CONCLUSION AND LIMITATIONS

  • The differentiable planner uses generalized grasp-quality and collision losses computed from watertight triangle meshes, and is presented for both standalone planning and network training.The conclusion reports well-defined, efficiently computed sub-gradients and robust performance on unseen objects and hardware platforms.Source: §VII · p. 8

D. Grasping with the Arm on Physical Hardware

VII. CONCLUSION AND LIMITATIONS

  • The implementation requires watertight, non-zero-volume objects, predicts only one grasp per object, and generates precision rather than power or caging grasps.Thin targets can cause tunneling and missed solutions; workspace constraints can also make a predicted grasp infeasible.Source: §VII · p. 8

IX. APPENDIX

The appendix introduces quadratic frictional cones as a more efficient alternative to linearized cones for reducing the Q1 lower-bound optimization problem.

IX. APPENDIX

  • Quadratic frictional cones reduce the problem size of the semidefinite program compared with linearized frictional cones.The appendix describes this as more efficient for computing the Q1 lower bound.Source: §IX · p. 9

1 Q1 Lower Bound Using Quadratic Frictional Cone

This section derives the Q1 lower bound using SOS optimization with quadratic frictional cones, and describes the associated cone construction and computational implications.

1 Q1 Lower Bound Using Quadratic Frictional Cone

  • The lower bound is re-derived using SOS optimization with quadratic frictional cones, following the approach in.The cones are defined for contact points with normals and two tangent directions.Source: §1 · p. 9
  • Equation 7 induces an SDP of the same polynomial order as, but with fewer cones and a smaller linear system for sensitivity analysis.The section also introduces the dual cones associated with the construction.Source: §1 · p. 9

2 SDP Sensitivity Analysis for Lower Bound of Q1

The section sets up sensitivity analysis for the SOS semidefinite program, using its cone structure and optimality conditions to derive parameter sensitivities.

2 SDP Sensitivity Analysis for Lower Bound of Q1

  • The SDP sensitivity setup uses PSD cones, their dual variables, and the coefficient matrix, with the cone count depending on the friction-cone formulation.The passage specifies K = 2 for quadratic frictional cones.Source: §2 · pp. 9–10
  • The coefficient matrix is block-structured because the first PSD cone encodes conditional polynomial positivity, while the other cones encode Lagrangian-multiplier positivity.Different variable groups affect the first cone and the remaining cones.Source: §2 · p. 10
  • Sensitivity is obtained by differentiating the SDP optimality conditions with respect to a parameter.The derivation applies the same cone decomposition to the dual variables.Source: §2 · p. 10
  • The derivation assumes strict complementarity for the specified cones; without it, the SDP problem is not differentiable.Source: §2 · p. 10

Z2 V KN

The sensitivity system is reduced by eliminating a row, making its left-hand side symmetric after removing specified terms; the reduced equation is solved with rank-revealing PLD(PL)^T.

Z2 V KN

  • The reduction retains specified rows and eliminates the 6th row to form a reduced sensitivity equation.Source: Z2 V KN · p. 11
  • System reduction makes the matrix left-hand side symmetric after removing Σ1 and Z1 from the last two rows, respectively.Source: Z2 V KN · p. 11
  • The reduced sensitivity equation is solved using rank-revealing PLD(PL)^T.The passage describes this solver after the system reduction.Source: Z2 V KN · p. 11
Loading 2002.01530v2…