Source-linked AI summary
EPOS: Estimating 6D Pose of Objects with Symmetries
Tomas Hodan, Daniel Barath, Jiri Matas
TL;DR
Single-image 6D pose estimation is difficult when object symmetries create ambiguous correspondences. EPOS uses compact surface fragments and multiple predicted 3D locations with robust PnP-RANSAC-based estimation, outperforming competing methods across key BOP benchmarks.
Problem
Object symmetries create many-to-many 2D–3D correspondences, while existing methods may assume one-to-one matches or produce invalid averaged locations.
Method
EPOS represents objects with compact surface fragments, predicts a data-dependent number of precise 3D locations per pixel, and estimates poses using a robust PnP-RANSAC variant.
Results
27% absolute improvement over the second-best RGB method is reported on YCB-V, while EPOS outperforms all RGB methods on T-LESS, YCB-V, and LM-O.
Takeaways & Limitations
The method handles texture-less and globally or partially symmetric objects across multiple instances without requiring pose post-refinement.
Takeaways & Limitations
On T-LESS, accuracy drops with 256 rather than 64 fragments, possibly because fragments become too small and harder to train.
Abstract
from arXiv · showhide
We present a new method for estimating the 6D pose of rigid objects with available 3D models from a single RGB input image. The method is applicable to a broad range of objects, including challenging ones with global or partial symmetries. An object is represented by compact surface fragments which allow handling symmetries in a systematic manner. Correspondences between densely sampled pixels and the fragments are predicted using an encoder-decoder network. At each pixel, the network predicts: (i) the probability of each object's presence, (ii) the probability of the fragments given the object's presence, and (iii) the precise 3D location on each fragment. A data-dependent number of corresponding 3D locations is selected per pixel, and poses of possibly multiple object instances are estimated using a robust and efficient variant of the PnP-RANSAC algorithm. In the BOP Challenge 2019, the method outperforms all RGB and most RGB-D and D methods on the T-LESS and LM-O datasets. On the YCB-V dataset, it is superior to all competitors, with a large margin over the second-best RGB method. Source code is at: cmp.felk.cvut.cz/epos.
1. Introduction
EPOS addresses single-image 6D pose estimation when symmetries make 2D–3D correspondences many-to-many. It represents objects with surface fragments, predicts multiple precise correspondences, and estimates poses robustly, achieving strong BOP Challenge 2019 results.
- Motivation: Symmetric objects can map one image location to multiple model locations, undermining methods that assume one-to-one correspondences.Texture-less objects also challenge local feature methods because detectors and descriptors provide too few reliable, distinctive locations.
- Method: EPOS represents objects with compact surface fragments to handle symmetries systematically and provide uniformly covered candidate 3D locations.The representation uses a controllable number of fragments across object types.
- Method: An encoder-decoder network predicts object presence, conditional fragment probabilities, and precise 3D locations on fragments for densely sampled pixels.Conditional fragment probabilities separate symmetry uncertainty from uncertainty about object presence.
- Method: Many-to-many correspondences support pose estimation for possibly multiple instances using a robust and efficient PnP-RANSAC variant.The estimator uses Progressive-X, GC-RANSAC, and PROSAC to exploit spatial coherence and predicted correspondence probabilities.
- Results: 27% absolute improvement over the second-best RGB method is reported on YCB-V, while EPOS outperforms all RGB methods across T-LESS, YCB-V, and LM-O.On T-LESS and LM-O, it also outperforms most RGB-D and D methods; the reported results use no pose post-refinement.
2. Related Work
Prior work uses local features, CNN correspondences, keypoints, bins, or pose prediction, but symmetries create ambiguous many-to-many matches that these representations often handle poorly. EPOS uses compact fragments and multiple fragment labels, alongside robust model fitting, to address these limitations.
- Existing approaches: Classical and CNN-based methods estimate pose from image–model correspondences, keypoints, templates, or dense object-coordinate predictions.Recent CNN methods primarily target RGB images, while earlier work also used RGB-D and depth inputs.
- Symmetry handling: One-to-one correspondence assumptions fail under symmetry because each image location may support multiple model locations and vice versa.Classification may retain limited support for possible poses, while regression can average incompatible locations into invalid solutions.
- Limitations of related methods: Accurate 2D bounding boxes remain problematic under occlusion for methods that predict multiple poses or distributions to handle symmetry ambiguity.This limitation affects related approaches that otherwise handle global and partial symmetries without a priori symmetry information.
- EPOS representation: EPOS uses near-uniform compact surface fragments and predicts multiple fragment labels per pixel to capture partial symmetries without prior symmetry information.This contrasts with approaches that use semantically defined parts or a single corresponding bin or location.
- Model fitting: Robust model fitting can improve correspondence-based pose estimation over vanilla PnP-RANSAC.EPOS integrates a modern robust estimator rather than relying solely on standard solvePnPRansac implementations.
3. EPOS: The Proposed Method
EPOS represents object surfaces with compact fragments and predicts symmetry-aware, many-to-many pixel-to-3D correspondences for pose fitting. A robust multi-instance estimator selects compatible correspondences and rejects outliers when recovering object poses.
- 3.1. Surface Fragments: 3D object meshes are partitioned into surface fragments using pre-selected centers, with centers obtained by furthest point sampling.Each fragment assigns surface points to the nearest center; the object centroid initializes sampling but is discarded from the final centers.
- 3.2. Prediction of 2D-3D Correspondences: The network separately predicts object presence and conditional fragment probabilities, distinguishing invisibility from symmetry-induced correspondence ambiguity.The conditional formulation models low joint probabilities either as object absence or as multiple plausible fragments at one pixel.
- 3.2. Prediction of 2D-3D Correspondences: Training uses rendered 3D models in ground-truth poses to derive object, fragment, coordinate, and visibility-mask annotations.Pixels outside object visibility masks are treated as background, while fragment distributions are learned from a single training correspondence per pixel under uniformly distributed synthetic poses.
- 3.2. Prediction of 2D-3D Correspondences: Pixels retain correspondences to every fragment passing object and relative fragment-probability thresholds, producing confidence-weighted many-to-many 2D-3D matches.This preserves alternative explanations for symmetric or partially symmetric surfaces while selecting a data-dependent number of 3D locations per pixel.
- 3.3. Robust and Efficient 6D Pose Fitting: Progressive-X combines GC-RANSAC, PEARL, P3P, EPnP, Levenberg-Marquardt, and PROSAC to propose, refine, maintain, and prioritize multi-instance pose hypotheses.At each pixel, hypothesis quality uses only the most accurate correspondence because other matches represent alternative explanations; degenerate or invalid poses are discarded.
4. Experiments
Experiments evaluate EPOS under the BOP19 protocol on three datasets using RGB inputs, then assess accuracy, speed, fragments, coordinate regression, and robust pose fitting. Results show strong benchmark performance and systematic trade-offs between accuracy, computational cost, and correspondence quality.
- Experimental Setup: EPOS is evaluated on T-LESS, YCB-V, and LM-O using the BOP19 task of estimating multiple object-instance poses from single images.The datasets provide color 3D models and RGB-D images with ground-truth poses, but EPOS uses only RGB channels.
- Experimental Setup: The evaluation reports Average Recall across VSD, MSSD, and MSPD pose errors, with AR defined as the mean of the three corresponding average recalls.MSPD additionally measures perceivable deviation and is relevant to RGB pose evaluation and augmented reality.
- Main Results: 27% absolute improvement in both AR and ARMSPD over the second-best RGB method is achieved on YCB-V, while EPOS outperforms all RGB methods across all three datasets.EPOS also outperforms all RGB-D and D methods on YCB-V and obtains the overall best ARMSPD on T-LESS and LM-O.
- Main Results: 0.75 s per image is the average processing time for EPOS with an unoptimized implementation, while faster RGB methods are reported as significantly less accurate.Accuracy-speed trade-offs can be adjusted through fragments, network size, image resolution, correspondence density, and the maximum number of GC-RANSAC iterations.
- Ablation Experiments: 18–33% higher AR and ARMSPD peaks are obtained with 64 or 256 surface fragments than with direct 3D object-coordinate regression.Fragment performance improves across textured, texture-less, and symmetric objects, demonstrating the value of the fragment representation.
- Ablation Experiments: 64 fragments provide a practical speed-accuracy trade-off: more fragments increase correspondences and can reduce fitting iterations, but increase network and fitting costs.Precise 3D-coordinate regression is often more accurate than using fragment centers, while regressed locations remain less than 1 mm from the model surface on average with 64 and 256 fragments.
5. Conclusion
The paper proposes a model-based method for 6D object pose estimation from a single RGB image, using compact surface fragments, multiple 3D correspondences, and robust PnP-RANSAC. It applies to a broad range of objects, including those with symmetries, while leaving object-specific fragment counts for future work.
- The method estimates 6D object poses from a single RGB image using compact surface fragments, multiple corresponding 3D locations, and a robust, efficient PnP-RANSAC variant.
- Its evaluation demonstrates applicability to a broad range of objects, including challenging objects with symmetries.
- Object-specific fragment counts, potentially depending on physical size, shape, or camera distance, remain future work.