Source-linked AI summary
T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects
Tomas Hodan, Pavel Haluza, Stepan Obdrzalek, Jiri Matas, Manolis Lourakis, Xenophon Zabulis
TL;DR
Texture-less objects challenge reliable 6D pose estimation because they lack distinctive local appearance cues. T-LESS introduces a multimodal dataset with varied objects, scenes, models, and accurate pose annotations, and initial evaluation shows substantial room for improvement in current methods, especially under difficult conditions.
Problem
Texture-less objects lack significant texture or discriminative color and reflectance, limiting traditional recognition methods for 6D pose estimation.
Method
T-LESS constructs a dataset of 30 objects using three synchronized sensors, systematically sampled views, two model types, varied test scenes, and accurate 6D pose annotations.
Results
Initial evaluation indicates that state-of-the-art 6D object pose estimation has ample room for improvement; one evaluated method achieves 67.2% mean recall across objects.
Takeaways & Limitations
T-LESS enables systematic evaluation across pose-estimation variants and related detection, segmentation, modality, and reconstruction tasks.
Takeaways & Limitations
Ground-truth pose validation prunes differences exceeding 5 cm, representing around 2.5% of measurements, because of erroneous depth or distractor-induced occlusion.
Abstract
from arXiv · showhide
We introduce T-LESS, a new public dataset for estimating the 6D pose, i.e. translation and rotation, of texture-less rigid objects. The dataset features thirty industry-relevant objects with no significant texture and no discriminative color or reflectance properties. The objects exhibit symmetries and mutual similarities in shape and/or size. Compared to other datasets, a unique property is that some of the objects are parts of others. The dataset includes training and test images that were captured with three synchronized sensors, specifically a structured-light and a time-of-flight RGB-D sensor and a high-resolution RGB camera. There are approximately 39K training and 10K test images from each sensor. Additionally, two types of 3D models are provided for each object, i.e. a manually created CAD model and a semi-automatically reconstructed one. Training images depict individual objects against a black background. Test images originate from twenty test scenes having varying complexity, which increases from simple scenes with several isolated objects to very challenging ones with multiple instances of several objects and with a high amount of clutter and occlusion. The images were captured from a systematically sampled view sphere around the object/scene, and are annotated with accurate ground truth 6D poses of all modeled objects. Initial evaluation results indicate that the state of the art in 6D object pose estimation has ample room for improvement, especially in difficult cases with significant occlusion. The T-LESS dataset is available online at cmp.felk.cvut.cz/t-less.
1. Introduction
T-LESS addresses 6D pose estimation for texture-less rigid objects, which are difficult to recognize from appearance alone. It provides a varied, multimodal benchmark designed to expose limitations of current methods.
- Texture-less objects require full 6D pose estimates for applications including robotic spatial reasoning and augmented-reality assembly guidance.
- Their appearance is dominated by global shape, color, reflectance, and lighting, limiting traditional photometric local-feature recognition and motivating RGB-D approaches.
- T-LESS contains 30 industry-relevant objects with weak texture, nondiscriminative appearance, shape or size similarities, symmetries, and part-to-whole relationships.
- The dataset provides approximately 39K training and 10K test images per sensor from three synchronized sensors, plus two 3D model types and accurate object-pose annotations.
- Its test set spans 20 scenes from isolated objects to multiple instances with substantial clutter and occlusion, while supporting pose, detection, segmentation, modality, and reconstruction evaluation.
- T-LESS was designed with controlled training data, complete accurate ground truth, and graduated complexity so it remains solvable but not solved by current methods.
2. Related Datasets
Existing datasets cover texture, RGB-D sensing, clutter, occlusion, and pose annotation in varied combinations. T-LESS is positioned against these differing scopes and acquisition conditions.
- The Hinterstoisser benchmark contains 15 texture-less objects and approximately 1200 RGB-D images per object, with clutter but only mild occlusion and one instance per image.
- Other RGB-D datasets include mixtures of textured and texture-less objects, multiple instances, clutter, and occlusion, but generally differ in scale or scenario coverage.
- BigBIRD provides 125 mostly textured objects with multiple calibrated sensors but controlled acquisition that excludes occlusion, clutter, lighting changes, and varying object distance.
- Lai et al. provide 300 household objects and 250K segmented RGB-D images, but ground truth is approximate rotation for training and 3D point labeling for testing.
- Depth-only and RGB datasets address smaller object sets or specialized settings, including occluded scenes, cluttered distractors, articulated parts, or categorical pose.
3. The T-LESS Dataset
T-LESS combines systematic multimodal acquisition, calibrated sensors, varied scenes, accurate pose annotation, and paired CAD and reconstructed models. Its preparation includes depth correction and model alignment.
- Image acquisition: The acquisition setup uses a marker-equipped turntable, a removable black-background screen, and three sensors mounted on an adjustable-tilt jig.
- Dataset construction: Test scenes vary in elevation, clutter, occlusion, and object multiplicity, while the dataset preparation covers acquisition, calibration, depth correction, model generation, and pose annotation.
- Calibration and modalities: The synchronized, calibrated sensor triplet produces RGB-D and high-resolution RGB images with registered modalities and systematic scene coverage.
- Image acquisition: Training views densely sample a full view sphere, with elevations from 85° to −85° in 10° steps and complete azimuth in 5° steps.
- Depth correction: A linear depth correction reduces mean absolute error from 12.4 mm to 2.8 mm for Carmine and from 7.0 mm to 3.6 mm for Kinect.
- 3D object models: Each object has a manually created CAD mesh and a semi-automatically reconstructed mesh; both use shared coordinates and vertex normals, while only reconstructed models include surface color.
- 3D object models: The average reconstructed-to-CAD surface distance is 1.01 mm across objects ranging from 58.13 mm to 217.16 mm.
4. Design Validation and Experiments
The evaluation validates ground-truth pose accuracy by comparing captured and rendered depths, then tests 6D localization under object similarity, scene complexity, and occlusion. Results show accurate depth alignment but substantial localization difficulty, especially with limited visibility.
- 4.1. Accuracy of the Ground Truth Poses: Captured depths were compared with rendered model depths at the ground-truth poses, after correcting the captured depth images.Differences exceeding 5 cm, about 2.5% of measurements, were pruned as outliers before computing statistics.
- 4.1. Accuracy of the Ground Truth Poses: Mean absolute depth differences were below 5 mm for Carmine and 9 mm for Kinect, near sensor accuracy and small relative to object size.Kinect showed a positive bias because its RGB and depth images were slightly misregistered.
- 4.2. 6D Localization: The evaluated template-based method estimated poses from Carmine test RGB-D images using training-image templates and CAD-model pose refinement.Correctness used average distance error for objects with indistinguishable views, with a threshold of k · d where k = 0.1.
- 4.2. 6D Localization: Objects similar in shape or size had the lowest recall, and scenes containing similar objects were harder, with scene 20 being the hardest.Scene 20 contains many similar objects and severe occlusions.
- 4.2. 6D Localization: Recall increased proportionally with the visible object-surface fraction, identifying occlusion as one of T-LESS’s main challenges.Figure 6 reports recall per object, per scene, and by visible surface percentage.
- 4.2. 6D Localization: 67.2% mean recall over all objects left a significant margin for improvement, versus 95.4% for the same method on Hinterstoisser et al.’s dataset.That method was close to the state of the art on the latter dataset, where reported recalls were 96.6% and 99.0%.
5. Conclusion
T-LESS is a public dataset for evaluating 6D pose estimation of texture-less objects. Its diverse sensing setup, accurate annotations, and increasingly difficult scenes support systematic comparison, while initial results indicate substantial room for improvement.
- 5. Conclusion: T-LESS provides a public benchmark for 6D pose estimation of texture-less, industry-relevant objects.The dataset is intended to facilitate systematic comparison of pertinent methods.
- 5. Conclusion: The dataset combines many training and test images, accurate 6D ground truth poses, multiple sensing modalities, and scenes with increasing occlusion and clutter.These design choices provide varying levels of evaluation difficulty.
- 5. Conclusion: Initial evaluation results indicate that current 6D object pose estimation methods have ample room for improvement.The conclusion specifically highlights difficulty caused by occlusion and clutter.
- 5. Conclusion: The T-LESS dataset is available online at cmp.felk.cvut.cz/t-less.