PaperlayerView exact revision on arXiv

Source-linked AI summary

Picking Bins Empty: A Hierarchical Hybrid Approach with Online Self-Learning of Grasp Points for Reliable Industrial Bin-Picking

Florian Töper, Samarth Kishor Yelvande, Jan Niklas Ewertz, Rudolph Triebel, Peter Ohlhausen

arXiv:2608.28175v1cs.RO

TL;DR

Industrial bin-picking still struggles to empty bins without manual intervention because model-based systems can deadlock and require grasp-point commissioning, while model-free systems trade reliability for flexibility. The paper combines them in a four-tiered hierarchy with online self-learning, achieving full bin clearance while retaining higher grasp success than the model-free baseline.

  • Problem

    Industrial bin-picking remains limited by perception failures, deadlocks, and the manual effort required to configure grasp points for new parts.

  • Method

    A four-tiered hybrid pipeline uses a model-based backbone, model-free fallback and exploration, and online grasp ranking from gripper-stroke feedback with Wilson score intervals.

  • Results

    100% bin clearance was achieved across all 30 bins, compared with 50.9% for the model-based baseline, while average grasp success was 99.1% for the model-based baseline versus 45.6% for the model-free baseline.

  • Takeaways & Limitations

    The hybrid approach autonomously resolves deadlocks without human intervention or manual fine-tuning while retaining high grasp success and repeatability, especially after manual initialization.

Abstract

from arXiv · show

Bin-picking is a cornerstone of modern manufacturing, yet achieving complete bin clearance without manual intervention remains a critical challenge. While model-based methods provide high precision, they frequently suffer from deadlocks when predefined grasps are occluded or perception fails. Labor-intensive fine-tuning of grasp points is commonly required to reach a satisfactory performance for new parts. Model-free algorithms offer a more flexible alternative with "out-of-the-box" versatility but lack the reliability and repeatability required for production. Unlike existing work, which treats the two techniques in isolation, we propose a fourtiered hierarchical hybrid approach to combine the best of both worlds. A model-based pipeline serves as a robust backbone, while a model-free "exploration agent" resolves deadlock situations and discovers new grasp points. This is supported by an online self-learning mechanism that uses gripper-stroke feedback and Wilson score intervals to autonomously rank grasp candidates, reducing manual commissioning effort. Validation on three automotive parts demonstrates that our method significantly outperforms a model-free baseline in grasp success rate while improving the bin clearance rate of the model-based baseline from 50.9% to 100% across all experiments. This transition to full bin clearance marks a significant step towards truly autonomous, intervention-free industrial operation.

I. INTRODUCTION

Industrial bin-picking must reliably clear bins without manual intervention, but model-based and model-free methods each have important limitations. The paper proposes a hierarchical hybrid pipeline combining a model-based backbone, model-free fallback, and online self-learning.

  • Model-based systems provide robustness and repeatability but require labor-intensive manual grasp-point definition and adjustment for new parts.Perception failures can also leave objects in the bin and reduce autonomy and throughput.
  • Model-free methods offer object-agnostic versatility but typically have lower grasp success rates and lack repeatability and oriented placing.These limitations create a trade-off between reliability and flexibility in industrial bin-picking.
  • The proposed four-tiered hierarchy uses a model-based backbone with model-free fallback and exploration levels to address perception errors and missing grasp points.The architecture is designed to achieve full bin clearance despite three potential failure points.
  • Online self-learning uses gripper-stroke feedback and Wilson score intervals to discover, evaluate, rank, and prune grasp candidates during operation.This mechanism is intended to reduce human-in-the-loop parameter tuning and support autonomous commissioning.
  • The approach is validated through real-world benchmarking on three automotive parts against model-based and model-free baselines.The supplied introduction identifies industrial validation as a contribution but does not report its detailed outcomes here.

II. RELATED WORK

Related work shows that model-based bin-picking benefits from geometric precision but remains vulnerable to perception failures and commissioning effort. Automatic grasp synthesis and simulation reduce manual annotation, yet physical interaction and deployment challenges remain.

  • Model-Based Bin-Picking: Model-based pipelines use object detection, 6D pose estimation, and predefined collision-free grasps for precise, repeatable grasping and oriented placing.Their performance is challenged by clutter, occlusion, and reflective objects.
  • Model-Based Bin-Picking: Three failure points are false-negative detection, inaccurate pose estimation, and deadlocks when predefined grasps are unavailable or unreachable.These failures make full clearance difficult, particularly in narrow bin corners, and often require manual intervention.
  • Model-Based Bin-Picking: Integrating new parts requires selecting, prioritizing, and tuning grasp poses in addition to establishing a reliable pose estimator.This commissioning burden limits applicability in High-Mix Low-Volume manufacturing and increases reliance on experts.
  • Model-Based Bin-Picking: Automatic grasp synthesis can use geometric features and clustering to generate and reduce grasp candidates for parallel or suction grippers.Physics simulation can improve grasp success but adds overhead and leaves a sim-to-real gap in cluttered scenes.

B. Model-Free Bin-Picking

Model-free methods generate grasp hypotheses directly from sensor data, reducing dependence on pose estimation and object-specific annotation. However, their industrial deployment remains constrained by lower determinism, limited semantic and geometric knowledge, and training-data requirements.

  • Model-free techniques predict grasp hypotheses directly from sensor data, eliminating pose estimation and object-specific grasp annotation.Deep-learning methods commonly estimate collision-free grasps from depth data, sometimes using unknown-object segmentation.
  • Discriminative methods evaluate sampled grasp candidates, while generative methods synthesize grasp poses and graspability scores.Contact-GraspNet predicts 6-DoF parallel-jaw grasps from 3D point clouds using sampled surface points as potential contacts.
  • Learning-based generalization depends on large, diverse training datasets that closely reflect the target distribution.GraspClutter6D covers 75 real-world objects in dense arrangements and improves CGN performance in cluttered scenes relative to simpler prior datasets.
  • 67.9% is reported for CGN in complex scenes of 15 piled objects, while GraspGen achieves 83.3% in simple cluttered tabletop scenes.These figures come from different methods and scene settings and should not be interpreted as a like-for-like comparison.
  • Model-free approaches remain less deterministic and lack the geometric domain knowledge needed for oriented placing and entanglement detection.They also do not learn from experience in the cited formulation, so repeated mistakes can persist.

C. Hybrid Approaches and Online Self-Learning

Prior hybrid and self-supervised approaches combine complementary modules or collect real-world grasp data, but they face training, setup-sensitivity, or rule-quality constraints. The paper instead emphasizes dynamic, part-specific database expansion during live operation.

  • Hybrid strategies and self-supervision address failed grasps during real operation, including systems that use sensor signals to label grasp outcomes.Prior self-supervised efforts collected more than 50k and over 800k grasps with parallel grippers.
  • Learning grasping policies from scratch requires time-consuming real-world data collection and is sensitive to setup modifications.Prior work therefore limits self-learning to gradually improving an existing baseline.
  • A rule-based and deep-learning hybrid can preserve high grasp success while improving throughput, but its learned model is capped by handcrafted rule quality.The cited approach trains entirely on rule-based outputs rather than extending beyond them.
  • The proposed method expands a part-specific grasp database during live operation, estimates real-world success rates, and prunes unsuccessful candidates in real time.A 6D pose-estimation step is included to support repeatable, high-precision grasping and oriented placing.

A. System Architecture

The system uses a four-level hierarchy that starts with model-based perception and progressively falls back to model-free grasping when pose, reachability, or detection fails.

  • Level I: Level I uses an object detector, 6D pose estimator, and grasp database as the model-based operational backbone.
  • Level II: Level II generates and scores new model-free grasps when estimated poses are reliable but database grasps are inaccessible, then adds discoveries to the database.Gripper-stroke feedback and Wilson score intervals support autonomous outcome evaluation and online learning.
  • Level III: Level III switches to model-free planning when detection succeeds but 6D pose confidence is below the minimum threshold.The detection result narrows the target region in the 3D point cloud.
  • Level IV: Level IV addresses detection errors by applying model-free grasping to remaining bin points when all objects are assumed to be the target part.This level may be skipped when foreign objects or additional packaging are likely.

B. Model-Based Backbone

The model-based backbone combines object detection, 6D pose estimation, and database grasps to execute repeatable, high-precision picks.

  • The backbone is trained separately for each part using fully synthetic data with heavy domain randomization to support fast part integration.
  • Detections produce oriented bounding boxes and confidence scores, while depth refinement and CAD-to-point-cloud matching provide object poses and fitness scores.
  • For sufficiently fit poses, the system checks database grasps for collision-free access and executes the top-ranked feasible grasp across detected parts.The database can be manually initialized or left empty for model-free exploration.

C. Model-Free Fallback and Grasp Exploration

The model-free fallback uses Contact-GraspNet to generate grasp hypotheses from point clouds, reranks them with geometry-aware criteria, and supports exploration when model-based grasps fail.

  • Contact-GraspNet supplies fallback strategies across Levels II–IV and uses contact points to maintain physical interaction with object surfaces.
  • CGN receives 2,048 target points and 20,000 context points, predicting gripper orientation, width, and normalized grasp-success confidence for each target point.Each prediction is duplicated under a 180° z-axis flip because the gripper jaws are symmetric.
  • The target region is selected from projected 6D poses on Level II, oriented bounding boxes on Level III, or remaining load-carrier points on Level IV.
  • Level II extends CGN’s native score with antipodal-contact and center-of-gravity criteria to guide discovery toward reliable grasps.The combined score is Stotal = α · SCGN + β · SAP + γ · SCOG.
  • The reranking step changes the ordering of CGN’s top twenty grasp points for parts A, B, and C in an exemplary scene.
  • SAP evaluates whether the connecting vector between expected contacts lies within friction cones around the two surface normals.The friction-cone half-angle is determined by the friction coefficient.
  • SCOG measures grasp offset from the estimated center of mass because rotational torque during lifting can affect stability.Its calculation uses the offset relative to the object’s bounding-box diagonal.

D. Online Self-Learning and Ranking Logic

Online self-learning evaluates grasp outcomes from gripper feedback and ranks candidates by optimistic confidence while limiting repeated use of weak grasps.

  • Online self-learning evaluates grasp points through real-world experience so high-scoring grasps can be selected more often over time.
  • Gripper width after each attempt indicates whether a part slipped, updating executed attempts n and successful attempts k for that grasp point.
  • The Wilson interval uses z = 1.96 for a 95% confidence interval, with initial bounds pu = 0 and po = 1 when n = 0.
  • Grasps are ranked by the upper confidence bound po, which preserves optimism for untested candidates and those with flawless histories.
  • Unlike standard UCB and Thompson sampling, the formulation prioritizes proven reliability while exploring new candidates when necessary.
  • Candidates with po below 75% are blocked from further Level I execution and similar-grasp learning on Level II.The threshold is intended to avoid immediate blocking from a single statistical outlier while limiting inferior database additions.
  • The long-term goal is to learn enough grasps to empty bins without generating excessive candidates that would lengthen collision checks.New-candidate generation should become less frequent as successful grasps enter the database.

IV. EXPERIMENTS

Experiments compare model-based, model-free, and hierarchical hybrid bin-picking across three automotive parts and multiple initialization strategies. Evaluation uses bin clearance and grasp success while tracking automatic grasp outcomes and a limitation involving entangled parts.

  • Three automotive parts with varying appearance, size, and geometry were tested using a Fanuc cobot, RGB-D camera, and soft-padded parallel gripper.
  • Baseline tests picked from three bins per part, with 50 parts loaded into each carrier.
  • The model-based baseline stopped after perception-induced deadlocks, allowing two camera-scan retries without rearranging bin contents.
  • The model-free condition skipped predefined grasps and used CGN to predict new grasps for every attempt.
  • The hybrid evaluation used all four hierarchy levels, online grasp-point self-learning, Wilson-score ranking, and five bins per part.
  • Manual initialization represented industrial commissioning, requiring an experienced user to annotate grasps through a GUI for 30–45 minutes per part.The experiments also included empty initialization to evaluate Wilson-score ranking; performance metrics included bin clearance and grasp success rates.

V. RESULTS

The hybrid approach achieves complete clearance while retaining high grasp success after manual initialization, and its exploration resolves model-based deadlocks with limited cycle-time impact.

  • 50.9% average bin clearance for the model-based baseline contrasted with 100% for the model-free baseline across the three parts.
  • Deadlocks primarily reflected insufficient collision-free predefined grasps, especially near bin walls for A and C and amid remaining central parts for B.Part B’s complex geometry produced deadlocks even when objects remained in the bin center.
  • 100% bin clearance was maintained for all 30 bins across the experiments, matching the model-free baseline.Deadlocks were overcome through grasp discovery, model-free grasping, and unknown grasping across the hierarchy.
  • 91.9% for A, 83.6% for B, and 89.3% for C were the hybrid grasp success rates after manual initialization.The rates were reported with 95% Wilson confidence intervals over five bins per part.
  • 2.6% (A), 1.2% (B), and 0.0% (C) were the hypothetical ΔGSR penalties from exploring learned grasps while predefined grasps remained available.The initial drop-off was expected to diminish as ranking prioritized robust grasps.
  • 1.01 s for grasp generation and 1.36 s for collision checking the 4,096 predicted grasps were added by Levels II–IV.Extending the databases by 22, 10, and 12 grasps for A, B, and C had negligible impact on Level I cycle time.

VI. CONCLUSION

The hybrid pipeline achieves autonomous full bin clearance while retaining the model-based backbone’s grasping and repeatability benefits. Its broader deployment, especially in zero-shot settings and optimized model-free grasping, remains future work.

  • 100% bin clearance was achieved without human intervention or manual fine-tuning, while maintaining high grasp success and repeatability.The model-based backbone and 6D pose estimation support these latter properties, especially after manual initialization.
  • The Level I approach is independent of the specific model-based backbone, allowing enhancement of existing bin-picking systems.The authors connect this flexibility to improved autonomy and economic value.
  • Reduced manual commissioning effort is particularly relevant to high-mix, low-volume manufacturing, where new parts frequently require configuration.
  • A fully zero-shot plug-and-produce system remains future work because the current implementation uses manual grasp-database initialization.CAD-based initialization is proposed as an alternative.
  • Further optimization of model-free grasp generation and extended experiments are needed to improve Levels II–IV performance and assess long-term learning behavior.The authors also identify parameter selection for the learning mechanism as requiring further analysis.
Loading 2608.28175v1…