Source-linked AI summary
HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects
Roman Kaskman, Sergey Zakharov, Ivan Shugurov, Slobodan Ilic
TL;DR
6D pose research needs datasets that go beyond restricted object types, single-object sequences, and real-image training. HomebrewedDB addresses this gap with a 33-object, 13-scene RGB-D dataset, synthetic-model training, and benchmarks for scalability and environmental robustness, with DPOD providing a baseline.
Problem
6D pose datasets are scarce and often restrict object types, sequences, training sources, environmental variation, or benchmark definitions.
Method
HomebrewedDB combines reconstructed 3D models, RGB-D scenes, precise pose annotations, and benchmarks for scalable detection and robustness.
Results
37% higher ADD at the 10% threshold occurs under normal illumination and texture conditions than under altered conditions.
Takeaways & Limitations
The dataset and automated construction pipeline are intended to support comprehensive evaluation and help researchers create extensible 6D pose datasets.
Takeaways & Limitations
Existing real-image training practices can overfit particular datasets, restricting detector applicability and undermining fair comparison.
Abstract
from arXiv · showhide
Among the most important prerequisites for creating and evaluating 6D object pose detectors are datasets with labeled 6D poses. With the advent of deep learning, demand for such datasets is growing continuously. Despite the fact that some of exist, they are scarce and typically have restricted setups, such as a single object per sequence, or they focus on specific object types, such as textureless industrial parts. Besides, two significant components are often ignored: training using only available 3D models instead of real data and scalability, i.e. training one method to detect all objects rather than training one detector per object. Other challenges, such as occlusions, changing light conditions and changes in object appearance, as well precisely defined benchmarks are either not present or are scattered among different datasets. In this paper we present a dataset for 6D pose estimation that covers the above-mentioned challenges, mainly targeting training from 3D models (both textured and textureless), scalability, occlusions, and changes in light conditions and object appearance. The dataset features 33 objects (17 toy, 8 household and 8 industry-relevant objects) over 13 scenes of various difficulty. We also present a set of benchmarks to test various desired detector properties, particularly focusing on scalability with respect to the number of objects and resistance to changing light conditions, occlusions and clutter. We also set a baseline for the presented benchmarks using a state-of-the-art DPOD detector. Considering the difficulty of making such datasets, we plan to release the code allowing other researchers to extend this dataset or make their own datasets in the future.
1. Introduction
HomebrewedDB addresses limitations in existing 6D pose datasets by supporting scalable detection from 3D models across varied scenes, occlusions, illumination, and appearance changes.
- 6D object detection and pose recovery support robotic grasping, manipulation, assembly, and augmented-reality overlay tasks.
- Existing datasets often restrict sequences to one object or specialize in narrow categories such as textureless industrial parts.LineMOD annotates one object per sequence, while T-LESS focuses exclusively on textureless industrial objects.
- Training on real images with similar test poses can overfit datasets, limiting detector applicability and fair comparison.
- Scalability remains neglected because most 6D detectors use one neural network per object rather than one network for all classes.The lack of varied sequences and well-defined benchmarks contributes to reliance on simplistic datasets such as LineMOD.
- HomebrewedDB provides 13 fully annotated RGB-D scenes and 34,830 frames with poses for all objects, spanning simple to heavily cluttered and occluded settings.
- The dataset varies illumination intensity and color and includes altered object appearance, while benchmarking scalable DPOD training using synthetic renderings.DPOD is trained either on all objects or on objects present in the test scene, establishing a scalability baseline.
2. Related Datasets
Earlier 6D pose datasets provide useful but fragmented coverage of objects, scenes, annotations, and environmental variation; HomebrewedDB combines these concerns with scalability-focused benchmarks.
- RGB-D datasets simplify 6D pose annotation because aligned depth and color images make pose estimation more automated.
- LineMOD contains 15 objects but annotates only one object per sequence, with absent or slight occlusions on targets.
- OCCLUSION includes poses for all objects in each frame, but intensive manual labor limited the number of annotated frames.
- T-LESS provides 30 textureless industrial objects across 20 RGB-D scenes with labeled poses, but strong similarity, symmetry, and low texture make it challenging.
- YCB-Video offers 92 video sequences with several household objects and test-scene occlusions, combining many sequences with a LineMOD-like structure.
- HomebrewedDB combines many objects, scenes, and test images with high-quality annotations, severe environment changes, and benchmarks training one network across objects.Its design targets changes in light color and intensity, altered appearance, and scalable RGB-based detection.
3. HomebrewedDB Dataset Creation
HomebrewedDB combines accurately reconstructed 3D models, varied RGB-D sequences, automated 6D pose annotation, and benchmarks targeting complex scene and appearance conditions.
- Dataset and models: 33 reconstructed models cover toys, household objects, and low-textured industrial objects, with diameters from 10.1 to 47.7 cm.The models were produced from multiview scans using an Artec Eva structured-light 3D scanner, followed by mesh cleanup, alignment, artifact removal, and optimization.
- Sequence acquisition: 13 sequences contain 1340 frames each, captured with PrimeSense Carmine 1.09 and Microsoft Kinect 2 RGB-D sensors.Validation used turntable recordings, whereas handheld test sequences introduced variation in camera scale, elevation, and in-plane rotation.
- Scene diversity: Scenes range from three separated objects on plain backgrounds to eight heavily occluded objects with extensive clutter and additional non-dataset objects.The dataset varies object number and size, occlusion, clutter, and surface type across scenes.
- Appearance and lighting variation: A dedicated sequence applies projected light patterns with different colors and intensities, while a domain-adaptation sequence alters object textures with colored chalk.These sequences evaluate robustness to illumination changes and considerable texture changes.
- Accuracy validation: Depth correction uses first-degree models, and rendered-versus-captured depth differences show close alignment, with Carmine absolute mean differences ≤2 mm.The correction models are dc = 1.0391 · d −15.8 for Carmine and dc = 1.0186·d−13.1 for Kinect 2, measured in millimeters.
- Pose annotation: Ground-truth poses are generated by estimating markerboard and camera poses, fusing depth maps into a scene reconstruction, transferring object transformations, and refining them with edge-based ICP.Object poses are estimated in markerboard coordinates and transferred across sensors; refinement selects images where objects are not occluded while keeping camera poses fixed.
4. Benchmarks and Experiments
The paper evaluates DPOD across benchmarks for scalability, changing illumination and texture, and training-data generalization. Results show strong sensitivity to object properties and environmental changes, while synthetic-data training transfers more predictably across sequences.
- Benchmarks: The benchmarks assess scalability, occlusion resistance, illumination changes and object-texture changes, using object detection and 6D pose estimation metrics.The evaluation includes per-scene, domain-adaptation and scalability settings.
- Evaluation Metrics: ADD measures average Euclidean distance between model vertices transformed by ground-truth and predicted poses, with correctness commonly defined below 10% of model diameter.The study also reports 30% and 50% thresholds for broader pose-quality assessment.
- Scalability Benchmark: A single-network scalability benchmark jointly evaluates sequences 1–8, which contain all 33 objects, and reports detection and pose estimation separately for each object.This setup tests whether one detector can handle the complete object set.
- Scalability Benchmark: 17 of 33 objects have ADD scores below 10%, and no object achieves an ADD score above the 50% threshold, indicating limited DPOD scalability.Larger objects with distinct textures and geometry perform best, whereas small, low-textured or glossy industrial objects are challenging.
- Domain Adaptation Benchmark: Altered illumination and texture reduce performance: pose accuracy is 37% lower by ADD at the 10% threshold, while mAP falls 46% and 59% under light and texture variation, respectively.Performance is notably better without added illumination or texture changes.
- Drawbacks of Training on Real Data: Detectors trained on real LineMOD data suffer substantial ADD 10% accuracy drops on HomebrewedDB, whereas synthetic-data detectors show no significant dataset difference and perform better on HomebrewedDB in 5 of 6 cases.Synthetic-data DPOD achieves higher ADD 10% scores for all three tested objects on the HomebrewedDB sequence.
5. Conclusion
The paper presents HomebrewedDB as a challenging 6D object-detection dataset covering scalability and robustness to occlusions, illumination and appearance changes. It also defines corresponding benchmarks and provides an automated dataset-building pipeline intended to support further dataset creation.
- 5. Conclusion: HomebrewedDB contains 33 objects across 13 scenes of varying difficulty and targets scalability plus robustness to occlusions, illumination and appearance changes.These properties are presented as requirements for solid object detectors.
- 5. Conclusion: The paper defines benchmarks for comparing detectors on the targeted properties and develops a comparatively simple, robust and fully automated dataset-construction pipeline.The authors hope the pipeline will help researchers create their own datasets.