Source-linked AI summary
Global Hypothesis Generation for 6D Object Pose Estimation
Frank Michel, Alexander Kirillov, Eric Brachmann, Alexander Krull, Stefan Gumhold, Bogdan Savchynskyy, Carsten Rother
TL;DR
Estimating a known object's 6D pose from an RGB-D image requires generating and selecting pose hypotheses, while existing approaches rely on local rather than global geometric checks at generation. This paper uses a fully connected CRF and an efficient two-step optimization with optimality guarantees. It reports state-of-the-art accuracy on the Occluded Object Dataset.
Problem
Estimating a known object's 6D pose from RGB-D imagery involves generating pose hypotheses, and existing approaches use local rather than global geometric checks at this stage.
Source: §1 · pp. 1–2Method
Results
76.7% average accuracy over all objects, 0.4% better than Hinterstoisser et al..
Source: §5.2 · p. 8Takeaways & Limitations
Against and, which use the same random forest, the comparison shows benefits from global reasoning.
Source: §5.2 · p. 8Infinite pairwise costs can leave QPBO with fewer than three labeled nodes, limiting its partial labeling in this setting.
Source: §4.4 · p. 6
Abstract
from arXiv · showhide
This paper addresses the task of estimating the 6D pose of a known 3D object from a single RGB-D image. Most modern approaches solve this task in three steps: i) Compute local features; ii) Generate a pool of pose-hypotheses; iii) Select and refine a pose from the pool. This work focuses on the second step. While all existing approaches generate the hypotheses pool via local reasoning, e.g. RANSAC or Hough-voting, we are the first to show that global reasoning is beneficial at this stage. In particular, we formulate a novel fully-connected Conditional Random Field (CRF) that outputs a very small number of pose-hypotheses. Despite the potential functions of the CRF being non-Gaussian, we give a new and efficient two-step optimization procedure, with some guarantees for optimality. We utilize our global hypotheses generation procedure to produce results that exceed state-of-the-art for the challenging "Occluded Object Dataset".
1. Introduction
The paper targets 6D pose estimation of a known object in an RGB-D scene and replaces local hypothesis-generation checks with global geometric reasoning. Its fully connected CRF yields a small hypothesis pool, and the approach reports state-of-the-art results on the Occluded Object Dataset.
1. Introduction
- Exhaustive search can evaluate every pose hypothesis with robust ICP scoring, but is computationally infeasible.Most approaches therefore generate a hypothesis pool before selecting and refining a pose.Source: §1 · p. 1
- A fully connected CRF performs a global geometry check and produces between 0–10 hypotheses, unlike prior local checks.The smaller pool permits a complex ICP-like scoring function for selecting the best hypothesis.Source: Figure 1 · p. 1, §1 · p. 2
- The CRF has non-Gaussian pairwise costs, so efficient approximation techniques such as cannot be used; the authors nevertheless solve it efficiently.Source: §1 · p. 2
- A new result shows that partial optimal solutions on subgraphs can yield partial optimal solutions for the whole model, reducing computational complexity.Properly constructed subgraphs enable the reduction.Source: §1 · p. 2
- The approach achieves state-of-the-art results on the Occluded Object Dataset in 1–3 s.The authors report the runtime as reasonable.Source: §1 · p. 2
2. Related Work
Related work includes sampling-based, voting-based, template-based, and graphical-model approaches to object detection and pose estimation. The paper distinguishes its global reasoning from prior local methods and notes a close connection to fully connected graphical models for articulated objects.
2. Related Work
- Sampling-based methods use RANSAC for feature matching or pose hypotheses, while Hough-voting methods aggregate pixel votes and templates search across the image.The authors note that template methods are harmed by clutter and occlusion in their scenario.Source: §2 · p. 3
- Prior graphical-model approaches use layout consistency or fully connected probabilistic models, but the closest related work addresses articulated objects.Bergholdt et al. model geometric connections globally but rely on an exact solution, whereas this paper uses partial optimality.Source: §2 · p. 3
- This method processes every pixel but uses global reasoning to find pose-consistent pixel sets rather than having pixels vote individually.Source: §2 · p. 3
3. Method - Overview
The method predicts object probabilities and coordinates from an RGB-D image, globally reasons over pixel correspondences to generate pose hypotheses, then refines and scores them. The three stages use a random forest, a fully connected graphical model, and an ICP variant.
3. Method - Overview
- The pipeline uses a random forest for dense predictions, global graphical-model reasoning instead of random pose sampling, and a final refinement-and-ranking stage.Source: §3 · p. 3, Figure 2 · p. 4
3.1. Random Forest
- The random forest predicts an object probability and an object coordinate for each pixel; the probabilities provide a soft segmentation mask.The forest uses three trees, and an object coordinate is a 3D point on the object surface.Source: §3.1 · p. 3
3.2. Global Reasoning
- A fully connected graphical model assigns each pixel a correspondence candidate or an outlier label, with pairwise terms encoding geometric checks.Its output is a collection of pose-consistent pixels clustered into sets, each providing one pose hypothesis.Source: §3.2 · p. 4, §3.3 · p. 4, Figure 2 · p. 4
3.3. Refinement and Hypothesis Scoring
- The final stage refines each pose hypothesis with an ICP variant and selects the pose with the lowest score.Only pose-consistent pixels are used for fitting the 3D model to improve robustness to occlusion.Source: §3.3 · p. 4, Figure 2 · p. 4
4. Method - Graphical Model
The method formulates pose estimation as energy minimization over pixel correspondences, then uses a two-stage procedure to reduce the problem and generate pose candidates. The second stage uses induced submodels to obtain partial labelings, with a guarantee that at least one is partially optimal for the full model.
4. Method - Graphical Model
- The method first prunes likely outliers with a sparse graph and TRW-S, then solves the remaining inlier candidates using fully connected submodels.The first stage reduces problem size; the second generates solution candidates and guarantees one submodel solution is partially optimal for the original problem.Source: §4 · p. 4, §4.3 · p. 6, §4.4 · p. 6
4.1. Energy Minimization
- Energy minimization assigns labels to graph nodes using unary and pairwise costs, seeking a labeling with the lowest total cost.The paper notes that this problem is NP-hard in general.Source: §4.1 · p. 4
4.2. Pose Estimation as Energy Minimization
- A fully connected graph checks geometrical consistency across all node pairs, limiting separated inlier islands that could hallucinate objects in different image locations.The model uses 13 labels per node: 12 correspondence proposals from 2 × 2 pixels and one outlier label.Source: §4.2 · p. 5
4.3. Stage One: Problem Size Reduction
- Stage one uses a sparse graph to reduce correspondence candidates, connecting each node to 48 neighbors while excluding the closest eight, and approximately solves the problem with 10 TRW-S iterations.A positive γ penalizes transitions between inlier and outlier labels.Source: §4.3 · p. 6
4.4. Stage Two: Generation of Solution Candidates
- Stage two retains first-stage inliers, gives each node two labels, and uses partial optimal labeling because exact solutions may take minutes or hours.The second-stage solution must contain only correct inliers; three inliers suffice to estimate the pose, and QPBO is used for partial optimality.Source: §4.4 · p. 6
4.5. On Optimality of Subproblem Solutions for Binary Energy Minimization
- Under the stated binary-potential condition, a partial optimal labeling of an induced submodel is also partial optimal for the full model.The proposition applies to binary pairwise graphical models after their costs are transformed to the specified form.Source: §4.5 · p. 7
4.6. Obtaining Candidates for Partial Optimal Labeling
- The candidate submodels are built from first-stage connected inlier components, discarding components smaller than three and usually producing at most 20 submodels.All inlier nodes of an optimal labeling must lie within a circle of diameter D; the construction includes nearby higher-numbered components.Source: §4.6 · p. 7
5. Experiments
On the Occluded Object Dataset, the method averages 76.7% accuracy, narrowly exceeding, with comparisons against and.
5.1. Dataset
- The evaluation uses ’s Occluded Object Dataset, with ground-truth poses for eight objects in 1,214 images spanning varied occlusion levels.The dataset extends.Source: §5.1 · pp. 7–8
- Figure 5 compares the method’s green silhouettes with blue ground truth and red results from [18]; correct predictions overlap the ground truth.Source: Figure 5 · p. 8
- The random forest trained on non-occluded patches handles moderate occlusion but can predict incorrect coordinates and pose under strong occlusion.Training on patches containing occlusions is suggested as a possible mitigation.Source: Figure 6 · p. 8
- Pose accuracy is the percentage of objects correctly estimated, with correctness defined by average point distance below 10% of the object diameter.The distance is measured between points under the estimated and ground-truth poses.Source: §5.1 · p. 8
5.2. Results
- Our method achieves 76.7% average accuracy over all objects, 0.4% better than Hinterstoisser et al.; it performs best on four objects, while performs best on the other four.Results are shown in Table 5.Source: §5.2 · p. 8
- The method improves over by 20.1% and over by 6.4%.The paper notes that both methods use the same random forest as its approach.Source: §5.2 · p. 8
6. Conclusion and Future Work
The paper focuses on pose-hypothesis generation, introducing a global geometry check through a fully connected CRF and an efficient two-step optimization procedure with some optimality guarantees. Future work includes improving object-coordinate regression, for example by replacing random forests with a convolutional neural network.
6. Conclusion and Future Work
- The paper introduces a global geometry check for pose-hypothesis generation using a fully connected CRF.Source: §6 · p. 9
- An efficient two-step optimization procedure addresses the difficulty of directly optimizing the CRF and provides some guarantees on optimality.Source: §6 · p. 9
- Future work could improve object-coordinate regression, for example by replacing random forests with a convolutional neural network.Source: §6 · p. 9