Source-linked AI summary
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, Tae-Kyun Kim
TL;DR
Crowded scenes make 6D object pose estimation difficult because occlusions, clutter, distractors, and viewpoint ambiguity challenge single-shot methods. The paper presents a unified Hough Forest framework with unsupervised patch features, next-best-view prediction, refinement, and realistic datasets, and reports superior results across challenging evaluations.
Problem
Crowded-scene 6D pose estimators using manually designed features struggle with multiple objects, severe occlusions, distractors, clutter, and viewpoint variability.
Method
The framework learns features from depth-invariant patches with Sparse Autoencoders, estimates 6D pose with Hough Forests, predicts next-best views from forest uncertainty, and refines detections through joint optimization.
Results
The framework shows significant improvement over state-of-the-art methods on public datasets and improved next-best-view performance, especially under occlusions.
Takeaways & Limitations
The framework generalizes across varied objects and scenes, including domestic environments and industrial bin-picking scenarios.
Takeaways & Limitations
Experiments sometimes exclude joint registration or active vision for fair comparison, except when specifically indicated.
Abstract
from arXiv · showhide
Object detection and 6D pose estimation in the crowd (scenes with multiple object instances, severe foreground occlusions and background distractors), has become an important problem in many rapidly evolving technological areas such as robotics and augmented reality. Single shot-based 6D pose estimators with manually designed features are still unable to tackle the above challenges, motivating the research towards unsupervised feature learning and next-best-view estimation. In this work, we present a complete framework for both single shot-based 6D object pose estimation and next-best-view prediction based on Hough Forests, the state of the art object pose estimator that performs classification and regression jointly. Rather than using manually designed features we a) propose an unsupervised feature learnt from depth-invariant patches using a Sparse Autoencoder and b) offer an extensive evaluation of various state of the art features. Furthermore, taking advantage of the clustering performed in the leaf nodes of Hough Forests, we learn to estimate the reduction of uncertainty in other views, formulating the problem of selecting the next-best-view. To further improve pose estimation, we propose an improved joint registration and hypotheses verification module as a final refinement step to reject false detections. We provide two additional challenging datasets inspired from realistic scenarios to extensively evaluate the state of the art and our framework. One is related to domestic environments and the other depicts a bin-picking scenario mostly found in industrial settings. We show that our framework significantly outperforms state of the art both on public and on our datasets.
1. Introduction
The paper addresses crowded-scene 6D object pose estimation and next-best-view selection, where occlusion, clutter, distractors, and viewpoint ambiguity challenge single-view methods. It introduces a unified Hough Forest framework combining unsupervised feature learning, pose estimation, active vision, extensive evaluation, and new realistic datasets.
- Crowded scenes make single-view 6D pose estimation difficult because clutter, occlusions, distractors, and viewpoint variability create ambiguous hypotheses.
- Selecting a next-best viewpoint is crucial for increasing accuracy and enabling fast scene understanding when camera movement is possible.
- The unified framework uses Sparse Autoencoders for unsupervised patch features, Hough Forests for joint classification and 6D pose regression, and leaf-node uncertainty for next-best-view prediction.
- The approach includes extensive evaluation of features and detection methods on public datasets, reporting significant improvement over state-of-the-art techniques.
- The authors introduce RGB-D datasets covering domestic environments and industrial bin-picking, including 3D object models and a fully annotated bin-picking dataset.
2. Related Work
Prior work includes unsupervised feature learning, handcrafted and learned features for Hough Forests, hypotheses verification, and active-vision strategies. The paper distinguishes its patch-based approach from methods requiring holistic object images, especially under occlusion.
- Unsupervised feature learning research includes Restricted Boltzmann Machines, multilayer feature extraction, and Sparse Autoencoders trained without labeled data.
- Holistic-image requirements in several prior methods make them prone to occlusion or unsuitable for the paper’s patch-based setting.
- Hypotheses verification has been used to improve true detections, extend verification across multiple views, and assess correspondences between corrupted 3D models.
- Active-vision methods have used boosting, POMDPs, generative convolutional models, and Active Random Forests to plan next-best views.
3. 6 DoF Object Pose & Next-Best-View Estimation Framework
The framework combines unsupervised features from depth-invariant RGB-D patches with Hough Forest classification and 6D voting, then refines hypotheses and selects informative camera views. Next-best-view prediction uses pose-to-leaf mappings and entropy reduction, with occlusion-aware refinement and joint registration for cluttered multi-object scenes.
- Single Shot-based 6D Object Detection: Depth-invariant RGB-D patches are processed by a Sparse Autoencoder, whose learned representation is used to train a Hough Forest for object class and 6D pose recognition.The framework uses features from the Autoencoder before Hough Forest inference.
- Single Shot-based 6D Object Detection: Hough Forest leaves store class histograms and annotation vectors, while test patches generate pose hypotheses from the Hough voting space.The final hypotheses are produced as modes of the voting space.
- Next-Best-View Prediction: Next-best-view selection chooses a viewpoint predicted to reduce hypothesis entropy using an offline mapping from discrete object poses to Hough Forest leaves.The mapping avoids rendering candidate views during prediction and uses the closest mapped viewpoint.
- Next-Best-View Prediction: For multiple objects and clutter, entropy prediction excludes samples expected to be occluded from the candidate viewpoint.The scene point cloud and projected annotated samples are used to estimate which samples will be occluded.
- Hypotheses verification and joint registration: The complete pipeline extracts hypotheses, refines them, moves the camera according to uncertain hypotheses, and re-estimates after the active view change.The stages are illustrated from the input image through verification, camera movement, and re-estimation.
- Hypotheses verification and joint registration: Joint registration and hypotheses verification select the subset of hypotheses that best explains the scene, rejecting false or poorly aligned detections.The verification cost combines normalized geometric, color, conflict, and penalization terms with learned regularizers.
4. Experiments
Experiments evaluate patch and feature choices, pose detection across public and new datasets, and next-best-view selection in crowded scenes. The framework performs strongly, with occlusion refinement improving active-vision view selection and joint optimization improving final performance.
- Patch Size Evaluation: Larger patches significantly improve accuracy, while higher patch resolution offers only slight gains at additional computational cost.Patch size is parameterized by physical size and cell resolution, with maximum size limited to two-thirds of the smallest object dimensions.
- Feature Evaluation using Hough Forests: The evaluation varies Sparse Autoencoder depth and feature-vector lengths of 100, 400, and 800 before Hough Forest classification.Nine configurations use one to three encoder layers, with the final encoder layer providing the Hough Forest feature vector.
- Public-Dataset Evaluation: On the dataset of, the method significantly outperforms state-of-the-art methods, including for visually difficult Camera and structurally varied Joystick objects.The learnt features also perform better on Milk containing distracting objects and remain robust to distractors and occluders without explicit background-class training.
- Own-Dataset Evaluation: On the authors’ dataset, the method outperforms the compared methods even without joint optimization, while joint optimization further improves final performance.The dataset includes six training objects and test images that may contain additional objects; it covers domestic and bin-picking scenarios.
- Active Vision Evaluation: Occlusion refinement improves next-best-view selection: it produces much better entropy estimation in crowded scenes and consistently finds the most appropriate single view in arranged scenes.Without refinement, the selected view was usually the front view, leaving objects behind occluded.
- Computational Complexity: Training takes about 10 minutes for three layers of 800 features with 10^4 patches, while forest training and testing are dominated by tree construction and Hough voting.The forest uses 5·10^6 patches; testing takes about 4–7 seconds for Hough voting and mode extraction, plus 2 seconds when joint optimization is used for six objects.
5. Conclusions
The paper concludes that its framework combines unsupervised feature learning, Hough-Forest-based 6D pose estimation, and next-best-view prediction for crowded scenes. Extensive evaluations, including realistic scenarios, show superior results and generalization across objects and scenes.
- Contributions: The complete framework integrates unsupervised feature learning, 6 DoF pose estimation using Hough Forests, and next-best-view estimation.The conclusion identifies these as the framework’s central components for 6D object detection in crowded scenes.
- Conclusion and Future Work: Extensive evaluation on challenging public datasets and a new realistic dataset supports superior results across varied objects and scenes.The paper proposes combining different patch sizes and exploring convolutional networks as future work.