Source-linked AI summary
ACRONYM: A Large-Scale Grasp Dataset Based on Simulation
Clemens Eppner, Arsalan Mousavian, Dieter Fox
TL;DR
Data-driven grasp planning depends on training data, but existing datasets differ in scale, variability, scene coverage, and label realism. ACRONYM provides a large physics-simulation dataset and uses it to retrain two learning-based grasp methods, which generalize better to unseen objects. The paper argues that simulation-based labels are supported by prior real-world transfer evidence, while the dataset’s purely simulated origin remains a quality consideration.
Problem
Data-driven grasping methods critically depend on training data, motivating a more scalable dataset with greater variability and physics-simulation labels.
Method
ACRONYM constructs a large grasp dataset from ShapeNetSem objects, physics-simulation labels, spatial grasps, and procedurally generated cluttered scenes.
Results
Retraining 6-DOF GraspNet and GPD on ACRONYM improves performance on unseen objects from both ACRONYM and the original smaller dataset.
Takeaways & Limitations
ACRONYM provides a large-scale benchmark intended to support new grasp-planning algorithms and comparisons between existing methods.
Takeaways & Limitations
ACRONYM is generated purely in simulation, so its dataset quality depends on how well simulation-based grasps transfer to real-world grasping.
Abstract
from arXiv · showhide
We introduce ACRONYM, a dataset for robot grasp planning based on physics simulation. The dataset contains 17.7M parallel-jaw grasps, spanning 8872 objects from 262 different categories, each labeled with the grasp result obtained from a physics simulator. We show the value of this large and diverse dataset by using it to train two state-of-the-art learning-based grasp planning algorithms. Grasp performance improves significantly when compared to the original smaller dataset. Data and tools can be accessed at https://sites.google.com/nvidia.com/graspdataset.
I. INTRODUCTION
ACRONYM addresses the dependence of data-driven grasping on training data by providing a large, diverse dataset labeled through physics simulation. It combines spatial grasps, dense coverage, physical realism, and cluttered scenes, and improves learning-based grasp planning when used for retraining.
- ACRONYM supports spatial grasping rather than only planar grasp representations, benefiting constrained spaces and semantically important settings.
- Its physics-simulation labels are intended to resemble real-world grasp performance more closely than analytical measures.Simulation is computationally more expensive, but the paper motivates it as a more physically realistic labeling approach.
- The dataset includes cluttered scenarios with multiple objects on a support surface for learning collision-free grasps.
- 17.7M parallel-jaw grasps cover 8872 objects from 262 categories, with labels generated using physics simulation.The dataset provides 2000 grasps per object and uses ShapeNetSem objects.
- Retraining a state-of-the-art learning-based grasp planner on ACRONYM leads to higher performance than training on the original dataset.The paper attributes the improvement to ACRONYM’s greater variability.
II. RELATED WORK
Existing grasp datasets vary in observations, labeling methods, scene variety, and data quantity. ACRONYM combines full 6DOF grasps with many objects, high grasp density, physics-based labels, and support for cluttered scenes.
- Planar vs. Spatial Grasping Datasets: 6DOF grasp poses distinguish ACRONYM from datasets focused on planar grasps in SE(2).
- Synthetic vs. Real Observations: Real RGB-D datasets are limited in scale or planar scope, while synthetic observations provide a more scalable alternative.
- Grasp Labels: Grasp labels across datasets come from robot executions, analytical models, physics simulators, or human annotation.
- Variety of Scenes: Most datasets label single objects, whereas ACRONYM also provides a mechanism for generating multiple-object cluttered scenes.Colliding grasps are ignored when single-object labels are reused in clutter.
- Quantity: 8872 objects with 2000 grasps per object yield 17.744 million grasps, balancing object variability with grasp density.
III. DATASET GENERATION
ACRONYM generates dense parallel-jaw grasp data through physics simulation across semantically meaningful objects, and also supports procedurally generated cluttered scenes.
- Objects and grasp sampling: Antipodal sampling generates diverse 6-DOF pre-grasps for the Panda parallel-jaw gripper.A surface point, a normal-aligned line, its mesh intersection, and a sampled rotation determine each gripper pose.
- Physics simulation: FleX labels grasps by closing simulated rigid-body fingers and applying a shaking motion to test grasp stability.The simulation uses no gravity; shaking was reported to achieve 91% precision and 93% recall relative to successful grasps with gravity.
- Objects and grasp sampling: 17.744 million grasps are simulated for objects sourced from ShapeNetSem, with 2000 grasp proposals generated per object.Objects are made watertight and assigned uniform density and friction; their longest axis is scaled between 6 cm and 35 cm.
- Physics simulation: 59.21% of the 17.744 million simulated grasps succeed, with object-wise success rates summarized in Fig. 3.The mug example in Fig. 4 illustrates the simulated grasp labels.
- Structured clutter: Structured-clutter scenes place multiple objects on support-surface facets and label colliding grasps as failures.Single-object simulation labels are reused, while environmental collisions are treated as failures for the targeted static pre-grasp and finger-closing strategies.
- Observations: The dataset provides rendering code for depth images, segmentation masks, and point clouds rather than supplying images explicitly.The ShapeNet meshes lack textures, although other rendering methods can produce realistic materials.
IV. BASELINE METHODS: GRASPNET AND GPD
The evaluation trains 6-DOF GraspNet and GPD as learning-based grasp planners using ACRONYM data and compares their predicted grasps against ground-truth grasp sets or image-projection features.
- 6-DOF GraspNet: 6-DOF GraspNet combines grasp sampling, grasp evaluation, and iterative refinement to improve predicted grasps.The c-IMLE objective minimizes distance from a sampled prediction to the closest ground-truth grasp.
- GPD: GPD classifies three local image projections of the volume between the gripper fingers with a four-layer CNN.It uses five pre-generated views and 500 grasps per object for training, then samples 100 grasps per point cloud during evaluation.
V. EXPERIMENTS AND RESULTS
Experiments compare identically sized models trained on ACRONYM or a smaller dataset, evaluating generalization to unseen categories and instances with fixed held-out views.
- Evaluation protocol: Evaluation uses uniformly sampled categories, held-out object instances, random viewpoints, and precomputed fixed data for every method.This keeps methods evaluated on the same objects and viewpoints.
- Unseen categories: 2-3 fold improvement occurs across c-VAE, c-IMLE, and GPD models when trained on ACRONYM for unseen-category generalization.The c-IMLE variation consistently outperforms c-VAE and GPD in this evaluation.
- Evaluation protocol: Higher area under the success-coverage curve indicates better grasp performance in Fig. 6.Panel (a) measures generalization to unseen categories, while panel (b) evaluates unseen instances from seen categories.
- Qualitative comparison: Models trained on ACRONYM generate more diverse and robust grasps than models trained on the smaller, less diverse dataset across six example objects.Fig. 7 compares 6-DOF GraspNet c-IMLE and GPD qualitatively, with green markers indicating successful grasps.
- Seen categories, unseen instances: ACRONYM-trained models outperform models trained on [26] even on held-out instances from categories seen during training.The models have identical parameter counts; the ACRONYM model uses more categories and instances, while the comparison model uses fewer categories.
VI. LIMITATIONS
The dataset is generated purely in simulation, raising a quality concern. Prior work nevertheless reports successful sim-to-real transfer for planning, learning, and FleX-based grasping.
- Pure simulation raises a question about the dataset’s quality.
- Prior work has transferred simulation to the real world in both planning and learning contexts.
- FleX-simulated grasps were reproduced on a real robot, with 86% average precision across 2625 grasps.
VII. CONCLUSION
The paper introduces ACRONYM, a large-scale physics-simulation grasp dataset, and retrains two existing learning-based grasp methods with it. The retrained methods perform better on ACRONYM’s unseen objects and generalize better to unseen objects in the original smaller dataset.
- ACRONYM is a new large-scale grasp dataset based on physics simulation.
- The dataset is used to retrain two existing learning-based grasp methods.
- The retrained methods perform better on unseen ACRONYM objects.
- The retrained methods generalize better to unseen objects from the original, much smaller dataset.