Source-linked AI summary
GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting
Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xiangyang Ji, Nassir Navab, Federico Tombari
TL;DR
Category-level pose estimation must predict pose and metric size for unseen instances despite limited object coverage in prior methods and severe intra-class shape variation. GPV-Pose combines confidence-driven rotation, geometry-guided point-wise bounding-box voting, symmetry-aware reconstruction, and geometric consistency terms. It achieves superior public-benchmark performance at 20 FPS, while its point-cloud–pose consistency term is disabled for occluded bounding-box faces.
Problem
Category-level pose estimation seeks pose and 3D metric size for previously unseen instances, but severe intra-class shape variations make the task challenging.
Method
GPV-Pose combines confidence-aware pose prediction, symmetry-aware reconstruction, point-wise bounding-box voting, and PP and PBP geometric consistency streams.
Results
GPV-Pose achieves superior performance on public datasets at a fast inference speed of 20 FPS.
Takeaways & Limitations
The framework provides a geometry-guided category-level pose estimator intended for real-time applications.
Takeaways & Limitations
The point-cloud–pose consistency term is meaningful only for visible bounding-box faces and is turned off for occluded faces.
Abstract
from arXiv · showhide
While 6D object pose estimation has recently made a huge leap forward, most methods can still only handle a single or a handful of different objects, which limits their applications. To circumvent this problem, category-level object pose estimation has recently been revamped, which aims at predicting the 6D pose as well as the 3D metric size for previously unseen instances from a given set of object classes. This is, however, a much more challenging task due to severe intra-class shape variations. To address this issue, we propose GPV-Pose, a novel framework for robust category-level pose estimation, harnessing geometric insights to enhance the learning of category-level pose-sensitive features. First, we introduce a decoupled confidence-driven rotation representation, which allows geometry-aware recovery of the associated rotation matrix. Second, we propose a novel geometry-guided point-wise voting paradigm for robust retrieval of the 3D object bounding box. Finally, leveraging these different output streams, we can enforce several geometric consistency terms, further increasing performance, especially for non-symmetric categories. GPV-Pose produces superior results to state-of-the-art competitors on common public benchmarks, whilst almost achieving real-time inference speed at 20 FPS.
1. Introduction
Category-level pose estimation addresses the limited object coverage and CAD dependence of instance-level methods, but must handle severe intra-class shape variation. GPV-Pose combines three prediction branches with geometric relations and consistency terms for category-level pose estimation.
- Category-level estimation predicts pose and object scale for previously unseen instances from known classes, beyond instance-level methods handling few objects.
- GPV-Pose uses direct pose regression, symmetry-aware reconstruction, and point-wise bounding box voting on a 3D graph-convolutional encoder.
- Confidence-aware rotation prediction represents the rotation matrix through bounding-box plane normals and weights easier-to-predict normals more strongly.
- Two geometric relation streams, Point Cloud–Pose and Point Cloud–Bounding Box–Pose, provide consistency terms for the model’s predictions.
- The point-wise voting mechanism aggregates direction, distance, and confidence predictions using a confidence-weighted least-squares algorithm.
2. Related Works
Prior category-level methods address unseen objects and intra-class variation through canonical spaces, shape priors, or pose consistency. GPV-Pose instead uses geometric information from point clouds and object bounding boxes through PP and PBP streams.
- Instance-level Pose Estimation: Instance-level methods typically handle only one or a few objects and require CAD models for training and testing, limiting practical applications.
- Category-level Pose Estimation: Category-level approaches predict poses for previously unseen objects using canonical spaces, normalized coordinates, learned shape spaces, or tracking.
- Category-level Pose Estimation: Some category-level methods incorporate point-based shape priors or pose consistency to reduce the influence of intra-class shape variation.
- Category-level Pose Estimation: GPV-Pose introduces a geometry-guided framework based on two geometric-information streams derived from sampled point clouds and object bounding boxes.
3. GPV-Pose
GPV-Pose combines direct confidence-aware pose regression, symmetry-aware reconstruction, and point-wise bounding-box voting on sampled RGB-D point clouds. Geometric consistency terms connect these outputs to improve category-level pose estimation under intra-class shape variation.
- Architecture: GPV-Pose samples 1028 backprojected depth points and processes global and per-point features through three parallel branches.The branches perform pose prediction, symmetry-aware reconstruction, and point-wise bounding-box voting.
- Confidence-aware Pose Regression: Confidence-aware rotation regression predicts two bounding-box plane normals and calibrates them to perpendicular normals before recovering the rotation matrix.The confidence weighting favors more reliable normal predictions during geometric calibration.
- Confidence-aware Pose Regression: The pose decoder recovers translation by adding predicted residual translation to the input-cloud mean and size by adding residual size to the category mean size.Rotation, translation, and size are recovered as the pose and metric-size outputs.
- Symmetry-aware Reconstruction: Symmetry-aware reconstruction predicts reflected or rotationally symmetric point clouds for categories such as mugs, laptops, cans, bowls, and bottles.The reconstruction branch uses category-specific reflection or rotational symmetry during supervision.
- Point-wise Voting for Bounding Box: Point-wise voting predicts each point’s direction, distance, and confidence to six bounding-box faces, then estimates face planes with confidence-weighted least squares.The method moves points toward predicted faces and fits plane parameters using confidence as the weight.
- Geometric Consistency: Geometric consistency terms use canonical point-cloud transformation, scale constraints, and bounding-box guidance to enforce pose–shape relations during training.The point-cloud scale term is meaningful only when the corresponding bounding-box face is visible.
4. Experiments
GPV-Pose is evaluated on category-level and instance-level benchmarks, with ablations showing that its confidence-aware rotation and geometric consistency terms improve performance. It achieves strong REAL275 results and near-real-time inference.
- Experimental Setup: GPV-Pose is evaluated on CAMERA25, REAL275, and LineMod using established pose and 3D IoU metrics.REAL275 contains real category-level data, while LineMod evaluates instance-level pose estimation.
- Comparison with State-of-the-Art Methods: GPV-Pose achieves state-of-the-art performance on 5 of 7 REAL275 metrics, including 83.0 versus NOCS at 80.5 for 3D50.For 5°5cm, GPV-Pose reaches 42.9 versus SGPA at 39.6, while Ours(M) reaches 44.2.
- Comparison with State-of-the-Art Methods: GPV-Pose outperforms DualPoseNet especially for 3D rotation of non-symmetric objects and remains accurate under partial detections.The qualitative and per-category comparisons are reported on REAL275.
- Comparison with State-of-the-Art Methods: The full pipeline runs at about 20 FPS with YOLO-V3 and ATSA, while the pose estimator alone exceeds 50 FPS.The reported speeds exclude object detection for the standalone estimator and include detection for the full pipeline.
- Comparison with State-of-the-Art Methods: On LineMod, GPV-Pose matches DualPoseNet at 98.2% ADD(-S), compared with 99.4% for PVN3D, while running almost in real time.This evaluates the method in the instance-level 6D pose estimation setting.
- Ablation Studies: Confidence-aware rotation prediction improves 3D75 from 52.0 to 56.9 and 5°2cm from 19.9 to 22.7.The ablation compares configurations A1 and A2 on REAL275.
5. Conclusion
GPV-Pose combines confidence-driven pose prediction, symmetry-aware reconstruction, point-wise bounding box voting, and two geometric consistency streams. It achieves strong public-benchmark performance at 20 FPS, with future work targeting holistic 3D understanding.
- Conclusion: GPV-Pose jointly predicts confidence-driven pose, symmetry-aware reconstruction, and point-wise bounding box voting.Its geometric consistency streams are Point Cloud–Pose and Point Cloud–Bounding Box–Pose.
- Conclusion: GPV-Pose achieves superior performance on public datasets at 20 FPS, supporting real-time applications.The paper identifies holistic 3D understanding and 3D scene-graph-based object interaction description as future directions.