PaperlayerView exact revision on arXiv

Source-linked AI summary

Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

Jeffrey Mahler, Matthew Matl, Xinyu Liu, Albert Li, David Gealy, Ken Goldberg

arXiv:1709.06670v2cs.RO

TL;DR

Suction grasp planning has comparatively little published work, and point-cloud predictions face uncertainty in object state and external wrenches. A compliant model evaluates seal formation and wrench resistance under perturbations, generating robust suction-grasp labels for training a GQ-CNN. A GQ-CNN-based policy trained on Dex-Net 3.0 achieved success rates of 98%, 82%, and 58% on Basic, Typical, and Adversarial datasets, respectively.

  • Problem

    Suction grasp planning has comparatively little published work, and point-cloud predictions face uncertainty in object state and external wrenches.

    Source: §I · p. 1, B. Definitions · p. 3

  • Method

    A compliant model evaluates seal formation and wrench resistance under perturbations, generating robust suction-grasp labels for training a GQ-CNN.

    Source: §I · pp. 1–2

  • Results

    A GQ-CNN-based policy trained on Dex-Net 3.0 achieved success rates of 98%, 82%, and 58% on Basic, Typical, and Adversarial datasets, respectively.

    Source: §I · p. 2

  • Takeaways & Limitations

    With training restricted to Adversarial objects, reported Adversarial success rose to 81%.

    Source: §I · p. 2

Abstract

from arXiv · show

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98$\%$, 82$\%$, and 58$\%$ respectively, improving to 81$\%$ in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net .

I. INTRODUCTION

Suction offers single-point contact and access to narrow spaces, but common point-cloud heuristics can miss useful grasps on non-planar surfaces. The paper introduces a compliant suction model and uses robustness labels to train and evaluate a learned grasping policy.

I. INTRODUCTION

  • Suction provides single-point contact and access to narrow spaces, but centroid and planar-surface heuristics may fail on non-planar objects.Existing analytic models often assume a formed seal and perfectly known object state, unlike point-cloud planning with pose, wrench, and control uncertainty.Source: §I · p. 1
  • The compliant model tests seal formation and suction-contact resistance to external wrenches, then evaluates robustness under pose, material, and wrench perturbations.It combines a quasi-static spring model with a ring-contact wrench basis.Source: §I · p. 1
  • In physical experiments with an ABB YuMi robot and silicone suction-cup tip, a Dex-Net 3.0-trained GQ-CNN policy achieved success rates of 98%, 82%, and 58% on Basic, Typical, and Adversarial datasets, respectively.Source: §I · p. 2

A. Suction Models

  • The model extends prior suction analyses by combining torsional friction and contact moments in a compliant ring-contact model.It uses a quasi-static spring system to estimate the deformation energy needed to maintain a seal.Source: A. Suction Models · p. 2

B. Grasp Planning

The paper frames suction grasp planning as selecting robust target points and approach directions from a single-view point cloud despite uncertainty in object state and external wrenches. It models grasp success probabilistically and trains a GQ-CNN to approximate the most robust feasible grasp.

B. Grasp Planning

  • Grasp planning commonly optimizes wrench-space quality, perturbation robustness, or learned grasp labels; suction planning has also used geometric and point-cloud heuristics.Prior work includes simulated contact labels and learned grasp detectors for parallel-jaw grasping, which this paper adapts to suction.Source: B. Grasp Planning · p. 2

III. PROBLEM STATEMENT

  • The objective is to select a target point and approach direction that maximize the probability of lifting and transporting an isolated object under sensor, control, and wrench perturbations.The task assumes an object resting stably on a planar worksurface and uses a single overhead depth sensor.Source: §III · pp. 2–3

A. Assumptions

  • The stochastic model assumes quasi-static rigid-object physics, non-porous materials, Coulomb friction, a known overhead sensor, and a single linearly elastic disc-shaped cup.Source: A. Assumptions · p. 3

B. Definitions

  • Grasp robustness Q(y, u) is the probability of success conditioned on a point cloud and grasp, accounting for uncertainty in sensing, control, and disturbing wrenches.The robot lacks exact object geometry, inertial and material properties, pose, and external wrench information.Source: B. Definitions · p. 3

C. Objective

  • The GQ-CNN approximates the robustness-maximizing feasible grasp by learning Qθ from point-cloud, suction-grasp, and reward triples with cross-entropy loss.Source: C. Objective · p. 3

IV. COMPLIANT SUCTION CONTACT MODEL

The contact model assigns success only when a suction cup forms a seal and can resist gravity and disturbance wrenches. Seal feasibility is evaluated with a quasi-static spring model projected onto the object surface.

IV. COMPLIANT SUCTION CONTACT MODEL

  • A successful grasp requires both a seal around the cup perimeter and the ability to resist external gravity and disturbance wrenches.Source: §IV · p. 3

A. Seal Formation

  • The cup is modeled as a conical spring system, and its projected contact configuration is evaluated quasi-statically as a proxy for dynamic seal formation.The spring system approximates the cup’s structural and flexion behavior.Source: A. Seal Formation · pp. 3–4
  • Perimeter and cone springs represent cup structure, while flexion springs resist bending; the cup perimeter must lie completely on the mesh to form a seal.Source: A. Seal Formation · p. 4
  • The approach line sets the initial cup configuration, and perimeter springs are projected along the approach direction while preserving their undeformed ring shape.The apex lies on the approach line and is positioned to maintain the specified cup height.Source: A. Seal Formation · p. 4
  • Seal feasibility requires no cone collision or holes within the contact ring and energy below threshold in every spring.Individual spring thresholds account for local geometry that can create air gaps.Source: A. Seal Formation · p. 4

B. Wrench Space Analysis

The analysis represents suction grasps through the object wrenches they can apply, with a compliant ring-contact model that captures elastic torque resistance. Wrench resistance is evaluated against external wrenches subject to friction, vacuum, and material constraints.

1) Wrench Resistance:

2) Suction Contact Model:

  • The ring-contact model hypothesizes that asymmetric pressure from passive elastic restoring forces resists torques in the contact tangent plane.Its wrench basis includes actuated normal force, vacuum force, friction, torsional friction, and elastic restoring torque.Source: 2) Suction Contact Model · p. 4
  • The model approximates friction, elastic-behavior, and vacuum limits with linear constraints, enabling wrench resistance to be computed by quadratic programming.The constraints include a friction limit surface, limits on cup elasticity, and limits on vacuum force.Source: 2) Suction Contact Model · pp. 4–5

C. Robust Wrench Resistance

Robust wrench resistance estimates grasp success under pose, friction, and disturbing-wrench perturbations. The dataset-generation process pairs sampled grasps and noisy rendered point clouds with robustness-based labels.

C. Robust Wrench Resistance

  • Robust wrench resistance is the probability of success under perturbations in object pose, gripper pose, friction, and disturbing wrenches.It is estimated by evaluating binary wrench resistance on samples and taking their mean.Source: C. Robust Wrench Resistance · p. 5

V. DEX-NET 3.0 DATASET

  • The grasp-reward model evaluates gravity-wrench resistance while perturbing gripper pose, and its parameters are tuned using average precision on physical trials with known 3D-printed objects.The observations use multiplicative and Gaussian-process pixel noise.Source: §V · p. 5
  • Training data sample object states, surface-contact candidates, and noisy point clouds, then label grasps positive when a seal forms and robust wrench resistance exceeds a threshold.Candidate points and normals are sampled uniformly from object meshes; grasp labels are projected to image pixels.Source: §V · p. 5

VI. LEARNING A DEEP ROBUST GRASPING POLICY

The GQ-CNN scores candidate suction grasps from a single-view point cloud and target pose. The paper evaluates the learned policy on three object classes using an ABB YuMi suction-grasping setup.

VI. LEARNING A DEEP ROBUST GRASPING POLICY

  • The GQ-CNN estimates grasp success from a single-view point cloud and a candidate target point and approach direction.The policy uses CEM to resample grasps from a Gaussian mixture model fitted to candidates with the highest predicted success probability.Source: §VI · pp. 5–6
  • The GQ-CNN achieved 93.5% classification accuracy on the held-out validation set.Training used an 80-20 image-wise split of Dex-Net 3.0.Source: §VI · p. 5

B. Experimental Protocol

  • Experiments used 55 objects divided into Basic, Typical, and Adversarial classes, with success defined as lifting and transporting the object to the workspace side.Average precision measures performance across success-probability thresholds; success rate is the fraction of successful grasps.Source: A. Object Classes · p. 6, B. Experimental Protocol · p. 6

C. Performance on Known Objects

Robust wrench resistance achieved the highest average precision in known-object trials, while novel-object performance varied with object class and training data. The reported failure mode was attempting seals on surfaces whose geometry prevented seal formation.

C. Performance on Known Objects

D. Performance on Novel Objects

E. Failure Modes

  • The most common failure was targeting surfaces that prevented seal formation, partly because of limited depth-sensor resolution.With geometry known precisely, the seal-formation model detected these surfaces; planarity-centroid performed poorly near non-planar object centroids.Source: E. Failure Modes · p. 7

VIII. FUTURE WORK

  • Future work includes testing sensitivity to training-object distributions, depth-sensor noise and resolution, and suction-hardware variations, as well as extending policies to heap bin-picking and combined suction/jaw grasping.The combined policy is described for a two-armed robot.Source: §VIII · p. 7

APPENDIX I ADDITIONAL EXPERIMENTS

The additional experiments compare grasp metrics and learned policies across known-state and point-cloud settings, while defining how success-attempt tradeoffs are evaluated. They also identify sensing and seal-formation boundaries that limit the system.

C. Performance on Novel Objects

  • Dex-Net 3.0 had the highest AP on Basic and Typical objects, while adversarial-only training outperformed all methods on Adversarial objects.The table covers 125 trials each for Basic and Typical and 100 for Adversarial objects.Source: Table I · p. 8, C. Performance on Novel Objects · p. 9

A. Performance Metrics

  • Success-attempt curves show how raising the confidence threshold trades fewer attempts for higher success, while overly conservative policies handle fewer cases.The section also notes that AP penalizes poor recall and success rate penalizes failures without distinguishing predicted from overconfident failures.Source: A. Performance Metrics · pp. 7–8

B. Performance on Known Objects

  • RWR achieved nearly 100% AP and 92% success on known Adversarial objects; Planarity-Centroid achieved 88% precision and 80% success.The experiments evaluate grasp ranking with object geometry and pose known.Source: B. Performance on Known Objects · p. 8, Table II · p. 8
  • RWR reached 100% success across a large share of test cases, while planarity-and-centroid heuristics fell as low as 67%.Fig. 6 plots success rate against attempt rate over 75 trials per policy.Source: Figure 6 · p. 9

C. Performance on Novel Objects

  • Dex-Net 3.0’s success probabilities were highly correlated with true success on Basic and Typical objects and took about 3 seconds per grasp to plan.The probabilities could help detect failures before execution; adversarial-only training performed better on Adversarial objects but worse on the other classes.Source: C. Performance on Novel Objects · p. 9

D. Classification Performance on Known Objects

  • RWR had the highest average precision and rank correlation with physical-robot successes among the evaluated grasp-quality metrics.The classification experiment used randomly sampled grasps from known Adversarial object geometry and pose.Source: D. Classification Performance on Known… · p. 9

E. Failure Modes

  • Failures arose when surface geometry was imperceptible to the depth sensor or when objects had no sealable surface because of non-porosity, size, or texture.The first category also includes specular or transparent surfaces that prevent geometry sensing.Source: E. Failure Modes · p. 9

APPENDIX II DETAILS OF QUASI-STATIC SPRING SEAL FORMATION

The appendix details how the compliant cup model forms a seal against a mesh and represents suction-contact wrench resistance. It also summarizes policy comparisons and object conditions that remain unhandleable.

APPENDIX II DETAILS OF QUASI-STATIC SPRING SEAL FORMATION

  • The seal-formation procedure determines a final cup configuration C against mesh M from an approach line and a target surface point.The line is parameterized by p, with v pointing toward the target along the approach.Source: APPENDIX II DETAILS OF QUASI-STATIC… · p. 9
  • Dex-Net 3.0 led AP on Basic and Typical objects, while adversarial-object training outperformed other methods on Adversarial objects.Fig. 7 reports near-100% precision for Dex-Net 3.0 on Basic and Typical over a significant portion of attempts; the adversarial-trained model did not perform as well on the other classes.Source: Table III · p. 10, Figure 7 · p. 10
  • The initial cup configuration is a regular n-gon pyramid, aligned so the approach line passes through the apex and is orthogonal to the base.The initial rotation about the approach line is random; with sufficiently large n, the metric is not sensitive to that angle.Source: APPENDIX II DETAILS OF QUASI-STATIC… · pp. 10–11
  • Fig. 8 presents precision-recall curves for classifying successful object lifts and transports using mesh-based grasp-quality metrics.Source: Figure 8 · p. 10
  • Point-cloud policies cannot handle imperceptible objects with undetectable seal-blocking surface variations or impossible objects lacking porosity or a sealable surface.Fig. 9 illustrates both categories.Source: Figure 9 · p. 11
  • Perimeter springs are positioned to preserve their projected locations while lying flush against the mesh along its intersection with a right prism.The prism uses the initial pyramid base as its base and extends through the mesh.Source: APPENDIX II DETAILS OF QUASI-STATIC… · p. 11
  • The apex must lie on the approach line, remain above the mesh surface, and yield average perimeter displacement equal to h; spring stretch uses a 10% length-change threshold.The threshold was used as the spring stretch limit in.Source: APPENDIX II DETAILS OF QUASI-STATIC… · p. 11
  • The suction-ring model uses coupled contact wrenches that may be bounded by material properties, and the paper computes wrench resistance with quadratic programming.RWR combines seal checking, suction-ring wrench analysis, and Monte-Carlo robustness sampling.Source: Table IV · p. 10, APPENDIX III SUCTION CONTACT MODEL · p. 11

A. Computing Wrench Resistance with Quadratic Programming

The section formulates wrench resistance as a constrained optimization problem and derives a compliant suction-ring model whose coupled contact forces and moments yield constraints that can be solved with quadratic programming.

A. Computing Wrench Resistance with Quadratic Programming

  • A grasp resists an external wrench when admissible contact wrenches can cancel it; minimizing the squared residual is a quadratic program when the admissible set has linear constraints.The optimum residual is zero exactly when the grasp resists the wrench.Source: A. Computing Wrench Resistance with… · p. 11

B. Derivation of Suction Ring Contact Model Constraints

  • The compliant ring model assumes quasi-static contact, infinitesimal springs, and linear displacement around a circular contact ring, producing a linear pressure distribution.The model also accounts for a constant vacuum pull and relates normal force to actuation and vacuum forces.Source: B. Derivation of Suction Ring Contact… · pp. 11–12, Figure 10 · p. 12

1) Friction Limit Surface:

  • Tangential forces and torsional moment are jointly bounded by a friction limit surface, approximated by an ellipsoid and linearized with an inscribed rectangular prism.The approximation permits linear constraints for quadratic-program wrench-resistance evaluation.Source: 1) Friction Limit Surface · p. 12

2) Elastic Restoring Torques:

The model bounds torques about the contact x and y axes and characterizes the material limit that constrains elastic behavior.

2) Elastic Restoring Torques:

  • Torques about the contact x and y axes are bounded in the suction-ring model.The material’s elastic limit κ is its yield strength: the stress at which deformation becomes plastic rather than linear.Source: 2) Elastic Restoring Torques · pp. 12–13

3) Vacuum Limits:

The model relates ring actuation and vacuum to normal force, then uses linear constraints for wrench-resistance optimization; the simpler soft-finger model has a specific torque-resistance limitation.

3) Vacuum Limits:

  • The suction ring transmits z-axis forces through motor torques, and its normal force accounts for both actuation and the vacuum pull.The model writes normal force as fN = fz + V.Source: 3) Vacuum Limits · p. 13

4) Constraint Set:

  • The resulting admissible wrench constraints are linear, enabling quadratic-program wrench-resistance evaluation; the paper sets V = 250N and κ = 0.005.The soft-finger contact model uses Coulomb friction and bounds net torsion by normal force, while suction permits fz to be positive or negative.Source: 4) Constraint Set · p. 13, C. Limits of the Soft Finger Suction… · p. 13

C. Limits of the Soft Finger Suction Contact Model

  • A single contact under the soft suction model cannot resist torques about axes in the contact tangent plane.The paper notes that this model limitation contrasts with empirical reports of single-point suction holding and transporting objects in applications such as the Amazon Picking Challenge, [10].Source: C. Limits of the Soft Finger Suction… · p. 13

APPENDIX IV GQ-CNN PERFORMANCE

The appendix reports GQ-CNN validation performance and details the probabilistic sampling and implementation used to generate training data and plan grasps.

APPENDIX IV GQ-CNN PERFORMANCE

  • 93.5% accuracy was achieved by the Dex-Net 3.0 GQ-CNN on a held-out validation set of approximately 552,000 datapoints.The appendix also presents its precision-recall curve and reports that learned first-layer filters may assess surface curvature around the suction contact ring.Source: APPENDIX IV GQ-CNN PERFORMANCE · p. 13, Figure 11 · p. 14

APPENDIX V ENVIRONMENT MODEL

  • Training examples sample object and camera states, candidate grasps, gravity-wrench success, and noisy observations from a joint probabilistic model.The model is illustrated as a graphical model, and the example policy scores candidate targets by predicted success before optimizing a grasp with the Cross Entropy Method.Source: Appendix V · p. 13

A. Details of Distributions

  • The state, grasp, observation, and success distributions specify pose, sensor-noise, and wrench-resistance perturbations for data generation.Distribution parameters were selected by grid search to maximize average precision on grasps attempted with an ABB YuMi on known 3D-printed objects.Source: A. Details of Distributions · pp. 14–15

B. Implementation Details

  • Implementation precomputes grasp-success samples with quadratic programming, renders depth images for stable poses, and generates grasp-centered thumbnails for GQ-CNN training.Success labels threshold the sample mean at τ = 0.5.Source: B. Implementation Details · p. 15

Related papers

This paper cites

Loading 1709.06670v2…