Source-linked AI summary
Training-free Suction Grasp Detection for Deformed Aseptic Cartons Using Vision-Language Models and Geometric Surface Scoring
Marin Maletic, Goran Vasiljevic
TL;DR
Deformable cartons are difficult to sort because their geometry challenges conventional vision and suction assumptions, while existing datasets lack relevant depth and grasp annotations. The paper introduces a training-free pipeline combining text-prompted VLM detection, SAM2 segmentation, and geometric suction scoring, achieving 72.6% end-to-end retrieval in clutter and 88.2% pooled single-object grasp success.
Problem
Deformed aseptic cartons remain insufficiently addressed because existing datasets lack depth data, grasp labels, and isolated deformed-carton examples.
Method
The system decouples target identification from grasp selection using VLM detection, SAM2 masks, depth point clouds, and geometric scoring based on flatness and normal alignment.
Results
72.6% end-to-end retrieval was achieved in clutter, while pooled single-object grasp success reached 88.2% for both RANSAC and Sobel.
Takeaways & Limitations
The training-free system supports runtime retargeting through natural-language prompts and demonstrates suction grasping of deformed aseptic cartons.
Takeaways & Limitations
The evaluation assumes a fixed gripper approach vector perpendicular to the conveyor plane and uses datasets that do not provide depth or grasp labels.
Abstract
from arXiv · showhide
Robotic sorting of recyclable waste is challenging due to the deformable and geometrically inconsistent nature of target objects. We present a training-free suction grasping system for sorting deformed aseptic beverage cartons, decoupling target identification from grasp-point selection. An open-vocabulary vision-language model detects cartons from a text prompt, SAM2 refines each detection into an instance mask, and a geometric scoring method selects the suction point by combining surface flatness with normal alignment. Three geometric methods are compared: k-nearest-neighbour PCA, Sobel cross-product, and RANSAC plane fitting. Evaluated on a real robot across three deformation levels and 35 cluttered scenes, single-object grasp success reaches 88.2% and end-to-end retrieval in clutter is 72.6%.
1. INTRODUCTION
Deformed aseptic cartons challenge conventional trained vision systems and suction grasping because their shapes vary and successful sealing depends on local surface geometry. The paper addresses this with a training-free pipeline that separates carton detection from geometric grasp-point selection.
- Deformed cartons are crushed, folded, and partially occluded, differing substantially from shapes used to train conventional vision systems.
- Existing datasets provide 2-D classification or segmentation annotations but lack depth data, grasp labels, and isolated deformed aseptic-carton examples.
- The proposed system uses zero-shot text-prompted VLM detection, SAM2 instance masks, depth-derived point clouds, and geometric ranking of suction points.
- The contributions include training-free open-vocabulary perception, geometric suction scoring, deformed-carton grasping, and comparisons across deformation and clutter conditions.
2. RELATED WORK
Prior work combines learned suction grasping, geometric surface assessment, and open-vocabulary perception, but these strands have largely remained separate for deformable carton sorting. The paper positions its contribution at their intersection.
- Related suction-grasping research is dominated by learned predictors trained on large synthetic or RGB-D datasets.
- Geometric methods retain surface normals and planarity as central suction-candidate criteria, with RANSAC-based plane fitting applied in clutter.
- Vision foundation models support promptable segmentation and open-vocabulary detection, while related language-grounded grasping work primarily targets parallel-jaw grippers.
- No prior work identified by the paper combines vision-language detection, SAM2 segmentation, and geometric suction scoring for deformable beverage cartons.
3. METHODS
The training-free pipeline separates carton identification from suction-point selection, combining open-vocabulary perception with geometric surface scoring and centroid-aware grasp selection. Three geometric methods estimate surface properties for ranking suction candidates under a shared tilt-feasibility rule.
- Pipeline overview: The four-stage pipeline performs open-vocabulary detection and segmentation, surface analysis, 3D grasp selection, and suction execution.A VLM proposes carton boxes, SAM2 refines masks, geometric scoring evaluates grasp suitability, and the robot executes the resulting pose.
- Perception: The VLM detects cartons from a textual prompt, while SAM2 converts each detection into a binary instance mask and labelled object point cloud.Valid masked depth pixels are back-projected using the camera intrinsics, and later geometry is computed independently per object.
- Surface scoring: Each point receives a suction score s(p) formed by multiplying local flatness and approach-normal alignment, penalising surfaces that satisfy only one condition.The approach vector is fixed perpendicular to the conveyor plane, and tilt feasibility decreases linearly across a transition band around the cup’s maximum tilt.
- KNN method: KNN PCA estimates normals from local covariance eigendecomposition and measures flatness through normal consistency within a cup-sized image window.The KNN method uses the smallest-eigenvalue eigenvector as the normal and a normal-variation tolerance to determine unsealable surfaces.
- Sobel and RANSAC methods: Sobel cross-products obtain dense normals through image-space derivatives, making the method faster than KNN but more sensitive to depth-sensor noise.RANSAC instead fits local planes in physical units, uses inlier-supported hypotheses, and applies an edge penalty near mask boundaries because edge grasps rarely seal fully.
- Grasp selection: After cup-footprint score averaging, the system chooses the threshold-clearing point nearest the object centroid to reduce lift torque; otherwise it uses the global score maximum.The selected point and approach-aligned orientation define a 6-DOF grasp pose for the robot arm.
4. EXPERIMENTS AND RESULTS
Three experiments evaluate the training-free carton-sorting system across deformation levels, isolated grasping, and cluttered scenes, including runtime, accuracy, and retargetability.
- Experimental design: The evaluation combines repeated-frame consistency and runtime analysis, isolated-carton grasp execution, and cluttered-scene detection and retrieval tests.The experiments cover three deformation levels and 35 mixed-waste scenes.
- Scoring consistency and runtime: All methods produce millimetre-scale point spreads, but runtime ranges from 134 ms for Sobel to 4477 ms for RANSAC.The VLM API call takes 5321 ms and SAM2 adds 145 ms, making RANSAC the method that roughly doubles pipeline time.
- Single-object grasp success: 88.2% pooled single-object grasp success is achieved by both RANSAC and Sobel, compared with 84.9% for KNN.All methods reach 100% at Level 1, while Level 3 rates range from 69.7% for KNN to 78.8% for RANSAC.
- Cluttered-scene performance: In 35 cluttered scenes, detection recall is 83.4%, conditional grasp success is 87.0%, and end-to-end retrieval is 72.6%.The evaluation contains 175 cartons mixed among distractors.
- Cluttered-scene performance: Detection recall and conditional grasp success decrease with scene density, while missed detections account for 29 of the 48 cartons not retrieved.The reported changes are 86.7% to 82.3% for detection recall and 92.3% to 85.0% for conditional grasp success.
- Prompt-driven target selection: Changing the natural-language prompt changes the detected and segmented target category while leaving geometric scoring unchanged.The demonstrated prompts select all aseptic cartons, heavily deformed cartons, milk cartons, or plastic wrappings.
5. CONCLUSION
The training-free system sorts deformed aseptic beverage cartons using vision-language detection, SAM2 segmentation, and geometric grasp scoring, achieving strong cluttered-scene retrieval. Detection recall remains the primary bottleneck, while implementation resources are publicly available.
- 72.6% end-to-end retrieval was achieved in cluttered scenes containing mixed plastic waste across three deformation levels.Conditional grasp success in clutter was 87.0%.
- 60% of nonretrievals stemmed from VLM failures rather than execution errors, identifying detection recall as the primary bottleneck.Future work includes re-planning after failed grasps, seal-quality feedback, and conveyor-speed throughput.
- All accompanying code and pipeline video are publicly available through the LARICS Lab GitHub.