Source-linked AI summary
Graspness Discovery in Clutters for Fast and Accurate Grasp Detection
Chenxi Wang, Hao-Shu Fang, Minghao Gou, Hongjie Fang, Jin Gao, Cewu Lu
TL;DR
Uniform sampling in cluttered 6-DoF grasp detection wastes computation because graspable points are concentrated in small regions. The paper introduces geometry-based graspness, approximates it with a cascaded model, and incorporates it into GSNet. Experiments report large gains over prior methods, including roughly 2x AP for GSNet against the previous state of the art.
Problem
Graspable points are concentrated in small local regions, but prior 6-DoF methods commonly sample scene points uniformly, wasting computation and reducing efficiency.
Method
The paper defines geometry-based graspness using look-ahead search, learns a cascaded model to approximate it, and uses high-graspness sampling in GSNet.
Results
GSNet improves performance by approximately 2x on AP versus the previous state-of-the-art method, with over 140% relative improvement on AP0.4 across all categories.
Takeaways & Limitations
Graspness functions as a transferable module that improves the speed and accuracy of prior methods and supports efficient end-to-end grasp detection.
Takeaways & Limitations
The graspness formulation assumes approach directions are uniformly distributed over a sphere.
Abstract
from arXiv · showhide
Efficient and robust grasp pose detection is vital for robotic manipulation. For general 6 DoF grasping, conventional methods treat all points in a scene equally and usually adopt uniform sampling to select grasp candidates. However, we discover that ignoring where to grasp greatly harms the speed and accuracy of current grasp pose detection methods. In this paper, we propose "graspness", a quality based on geometry cues that distinguishes graspable areas in cluttered scenes. A look-ahead searching method is proposed for measuring the graspness and statistical results justify the rationality of our method. To quickly detect graspness in practice, we develop a neural network named cascaded graspness model to approximate the searching process. Extensive experiments verify the stability, generality and effectiveness of our graspness model, allowing it to be used as a plug-and-play module for different methods. A large improvement in accuracy is witnessed for various previous methods after equipping our graspness model. Moreover, we develop GSNet, an end-to-end network that incorporates our graspness model for early filtering of low-quality predictions. Experiments on a large-scale benchmark, GraspNet-1Billion, show that our method outperforms previous arts by a large margin (30+ AP) and achieves a high inference speed. The library of GSNet has been integrated into AnyGrasp, which is at https://github.com/graspnet/anygrasp_sdk.
1. Introduction
The paper identifies uniform sampling as a major bottleneck because graspable points occupy small regions, then introduces graspness to prioritize promising areas. A cascaded model and GSNet use this principle to improve grasp detection efficiency and accuracy.
- Robust 6-DoF grasp pose detection supports applications including picking, assembling, and home serving, but generality, accuracy, and efficiency remain longstanding goals.
- Uniformly sampling scene points wastes computation because graspable points are concentrated in small local regions.On GraspNet-1Billion, fewer than 10% of sampled points are graspable even with object masks.
- Graspness is a geometry-based quality that distinguishes graspable areas, measured by exhaustively evaluating possible future grasp poses from each point.Statistical analysis finds local geometry differs between high- and low-graspness points.
- The cascaded graspness model predicts point-wise graspness and transfers across objects, viewpoints, scenes, and sensors.Equipping sampling-evaluation methods with the model yields improvements in both speed and accuracy.
- GSNet samples high-graspness points from dense scene clouds and discards the rest before forward propagation.The two-stage design preserves local geometry cues while reducing computation and providing more positive training samples.
- Experiments show graspness-equipped baselines outperform vanilla counterparts in speed and accuracy, while GSNet substantially outperforms previous methods.The library has also been integrated into AnyGrasp.
2. Related Work
Prior cluttered-scene grasping work includes planar and full-DoF approaches, with sampling commonly performed uniformly or through generic downsampling. This paper instead uses local geometric cues and connects graspness to human affordance attention.
- Cluttered-scene grasp detection research divides mainly into planar methods and full-DoF methods.Planar approaches often have restricted applications because of their lower degrees of freedom.
- Common prior sampling strategies include uniform sampling, voxel-grid downsampling, farthest-point sampling, optimization, and learned iterative refinement.These methods generally sample without using geometric cues for graspable points.
- Human visual attention during grasping is biased toward regions with the highest graspability before action preparation.This affordance-attention finding motivates graspness-based grasp sampling.
3. Graspness Discovery
The paper defines point- and view-wise graspness, estimates it through look-ahead search over feasible grasps, and extends the measure to cluttered scenes using collision handling and geometric association.
- 3.1. Preliminary: The method allocates grasp-detection computation first to points and views with high graspness rather than treating all scene points equally.
- 3.1. Preliminary: It defines point-wise and view-wise graspness to represent attention over grasp locations and approach directions.Approach directions are uniformly distributed over a sphere, and view-wise graspness is V-dimensional with values in [0,1].
- 3.2. Graspness Measure: For an object point cloud, higher graspness denotes a larger possibility of successful grasping, based on feasible grasp poses centered at each point and view.
- 3.2. Graspness Measure: Exhaustive look-ahead search is used to ensure that higher graspness values denote higher possibilities of successful grasping.
- 3.2. Graspness Measure: Because an oracle and continuous grasp-pose enumeration are unavailable in practice, the method grid-samples candidate depths and in-plane rotations, scores them analytically, and thresholds unsuccessful grasps.
- 3.2. Graspness Measure: Scene-level graspness accounts for clutter collisions and partial RGB-D observations, then associates scene points with projected object points by nearest-neighbor search and normalizes each scene.Collision checking removes grasps that collide with background or neighboring objects before scene projection.
- 3.2. Graspness Measure: The paper analyzes local features of high- and low-graspness points to test whether their geometry is distinguishable.Points above 0.3 are treated as positive samples, while equal-sized negatives have graspness below 0.1.
4. GSNet Architecture
GSNet uses cascaded graspness estimation to filter point clouds before predicting grasp configurations. It samples graspable seeds, selects views, groups local cylinder regions, and predicts grasp scores and widths.
- Cascaded Graspness Model: GSNet learns point-wise and view-wise graspable landscapes, using the former to filter points before computing the latter.The cascaded design reduces computation because view-wise landscapes are unnecessary for points unlikely to be graspable.
- Backbone Network: The backbone extracts C-dimensional features for N input points and outputs features for graspable sampling and grasp generation.ResUNet14 processes arbitrary-sized point sets with sparse convolutions, although other point-wise networks can replace it.
- Graspable Farthest Point Sampling: Point-wise graspness produces an N × 1 landscape and N × 2 objectness scores; non-object points receive graspness zero.Points above threshold δp are selected, then farthest point sampling produces M seed points with coordinate and feature information.
- Cascaded Graspness Model: View-wise graspness is predicted for M seeds across V Fibonacci-lattice views, after which the best view is selected during inference.During training, probabilistic view selection samples views according to normalized graspness scores before grasp parameters are estimated.
- Grasp Operation Model: Seed point-view pairs define cylinder regions whose grouped points are processed to predict grasp scores, gripper widths, and final grasp configurations.The output contains M × (A × D × 2) values for in-plane rotation and approach-depth combinations, with score and width for each combination.
- Loss Function: GSNet jointly trains objectness, point-wise and view-wise landscapes, grasp scores, and gripper-width regressions with multi-task losses.Classification uses softmax, while regression uses smooth-L1 loss with validity conditions for object points, seed views, and positive-score poses.
5. Experiments
Experiments evaluate graspness across domain shifts, benchmark performance, sampling choices, inference speed, and cluttered-scene grasping. The results support stable transfer, improved detection accuracy, effective graspness-guided sampling, and fast inference.
- 5.2. Performance of Cascaded Graspness Model: GraspNet-1Billion evaluates models across object categories, viewpoints, and cameras, using ranking error to measure graspness prediction quality.The experiments test seen, similar, and novel categories, viewpoint groups, and RealSense/Kinect settings.
- 5.2. Performance of Cascaded Graspness Model: The cascaded graspness model remains stable across object and viewpoint variations, while the RealSense–Kinect difference remains acceptable.Point-wise ranking error differs little between seen and novel categories, and viewpoint variation has low impact.
- 5.3. Comparing with Representative Methods: GSNet outperforms previous methods on both cameras, improves performance by ∼2x over Fang et al. on AP, and exceeds 140% relative improvement on AP0.4 across categories.Top-1 grasp accuracy is reported separately for RealSense and Kinect across the three object categories.
- 5.3. Comparing with Representative Methods: Collision detection improves AP by 1.42/2.31, 1.06/1.79, and 0.33/0.77 on the three categories for RealSense/Kinect, respectively.Colliding grasps are removed using a parallel-jaw gripper model.
- 5.4. Boosting with Cascaded Graspness Model: Adding the cascaded graspness model yields significant gains for both two-step and end-to-end methods while reducing computation from dense sampling.The model selects candidate points from the predicted point-wise graspable landscape before existing post-processing or prediction stages.
- 5.5. Analysis: Graspable landscapes improve sampling results by over 7 AP for both FPS and random sampling, while FPS beats random sampling by at least 4.98 AP.The comparison includes random sampling from the whole cloud, FPS from the whole cloud, and sampling from graspable landscapes.
- 5.5. Analysis: The cascaded graspness model takes approximately 0.1s on RealSense/Kinect data, compared with more than 1s for GPD and PointNetGPD.The model is intended to provide fast, accurate sampling for different grasp detection methods.
- 5.6. Grasping Experiments: Grasping experiments in six cluttered scenes measure success as the ratio of objects cleared to attempts and report effective performance.Each scene contains 6–8 randomly arranged objects from GraspNet-1Billion.
6. Conclusion
The paper introduces graspness as a geometry-based quality for identifying graspable regions and develops both an independent graspness model and the end-to-end GSNet. Experiments report stable, general, robust graspness estimation, improved prior methods, and high accuracy and speed for the final network.
- 6. Conclusion: The paper proposes graspness, a geometrically based quality measured by look-ahead search to distinguish graspable areas.Statistical analysis is used to demonstrate the measure’s effectiveness and rationality.
- 6. Conclusion: An independent model learns graspable landscapes, and GSNet incorporates them into end-to-end grasp pose detection.The graspness model can also be applied to other methods.
- 6. Conclusion: Experiments demonstrate stability, generality, effectiveness, and robustness of the graspness model, with large improvements for previous methods using it.The conclusion also reports high accuracy and speed for the final network.
A. Video Demo and Library
GSNet was demonstrated in real grasping experiments, including on novel objects outside GraspNet-1Billion. Its library has also been integrated into AnyGrasp.
- Video Demo: GSNet was used for real grasping after training on GraspNet-1Billion.The supplementary video demonstrates grasping with GSNet predictions.
- Video Demo: The demo includes novel objects such as chains, mesh bags with marbles, and slippers.These objects were not collected from GraspNet-1Billion, and the model shows robustness on them.
- Library: The GSNet library has been integrated into AnyGrasp to support further robotics research.The paper points readers to the AnyGrasp project website for more details and videos.
B. Grasping Experiment Configuration
The real grasping setup combines a UR-5 arm, a Robotiq two-finger gripper, a RealSense D435 camera, and GraspNet-1Billion object models. GSNet selects a high-scoring grasp and repeatedly directs the arm to remove objects from the table.
- Configuration: The real grasping setup uses a UR-5 robotic arm, Robotiq two-finger gripper, RealSense D435 camera, and GraspNet-1Billion object models.These components are identified as A–D in the experiment configuration.
- Operation: A high-score grasp predicted by GSNet is selected and sent to the robotic arm.The program attempts to grab one object per execution.
- Operation: The execution repeats until all objects are removed from the table.This defines the cleanup procedure used in the real grasping experiments.
C. Robotic Experiments with Baselines
The robotic experiments compare GSNet with baseline methods across repeated object-removal trials. GSNet achieves the highest success rate across all three test sets, while its grasp operation model modifies representation, sampling, scaling, and output selection.
- Experimental Protocol: The real experiments use three object sets of 10 objects each and single-view point clouds as input.The methods remove all objects from the workspace, with four repetitions per object set.
- Robotic Results: GSNet outperforms other methods on all three object sets in real robot experiments.Four repeated experiments were conducted for each set, with success rate defined in Sec. 5.6.
- Grasp Operation Model: GOM replaces repeated depth-bin cylinder groups with one cylinder region whose height is capped at 0.04m.Depth classification is moved to the final output.
- Grasp Operation Model: GOM scales transformed point coordinates by the 0.05m cylinder radius and modifies width prediction accordingly.The scaling addresses small transformed coordinate magnitudes within a shared gripper frame.
- Grasp Operation Model: GOM samples 16 points from approximately 1k seeds and concatenates their coordinates with cascaded grasp-model features.This replaces direct sampling of 64 xyz-only points from approximately 20k input points to reduce sampling overhead.
- Grasp Operation Model: GOM selects each grasp's in-plane rotation–approach-depth combination using the highest grasp score.Grasp scores and gripper widths are output as relative values from 0 to 1.
E. Visualization of Point-wise Graspness
GSNet visualizes point-wise graspness with brighter colors indicating higher scores. The visualizations show that graspness depends on object geometry, scene position, neighboring objects, and object size.
- Scene Effects: Graspness depends on an object's position and can be reduced by collision with the table.Areas with low graspness from table collisions may have higher graspness when the object is isolated.
- Scene Effects: Neighboring objects can disrupt an object's contiguous graspness, as illustrated by a knife lying on a banana.The example shows graspness being influenced by surrounding objects rather than the object alone.
- Visualization: Brighter colors in the visualization indicate regions with higher point-wise graspness.Figure 7 presents the graspness predicted by GSNet.
- Object Effects: Object size limits graspness when a gripper has few viable grasp regions, as for the box lying on the table.The cited example considers a gripper width of up to 0.1m.
F. t-SNE Visualization of Point Features
GSNet point features distinguish high-graspness points from low-graspness points across object, viewpoint, and camera variation settings.
- High-graspness points are distinguishable from other points in GSNet's t-SNE feature visualizations.The visualized points come from GraspNet-1Billion, with high-graspness samples labeled positive and other samples labeled negative.
- The three visualization rows represent object, viewpoint, and camera variation, respectively.
- Orange points denote high graspness, while blue points denote low graspness.