Source-linked AI summary
PCRNet: Point Cloud Registration Network using PointNet Encoding
Vinit Sarode, Xueqian Li, Hunter Goforth, Yasuhiro Aoki, Rangaprasad Arun Srivatsan, Simon Lucey, Howie Choset
TL;DR
PointNet representations are sensitive to pose misalignment, motivating a learned registration framework for aligning source and template point clouds. PCRNet compares their PointNet features, supports shape-specific or general training, and is evaluated against established methods for accuracy, noise robustness, and computational speed.
Problem
PointNet-based processing requires minimal canonical-frame misalignment, whereas real-world point clouds are seldom aligned to such a frame.
Method
PCRNet uses a Siamese PointNet architecture to encode source and template clouds and data-driven techniques to estimate the aligning pose.
Results
Iterative PCRNet achieves AUC 0.972 when trained and tested within the same object category, compared with AUC 0.862 for ICP and 0.998 for PointNetLK.
Takeaways & Limitations
Iterative PCRNet provides highly accurate, noise-robust registration and can achieve accuracy comparable to global methods while being computationally faster.
Takeaways & Limitations
PCRNet has limited learning capacity for large shape variations, giving PointNetLK better generalization across object categories.
Abstract
from arXiv · showhide
PointNet has recently emerged as a popular representation for unstructured point cloud data, allowing application of deep learning to tasks such as object detection, segmentation and shape completion. However, recent works in literature have shown the sensitivity of the PointNet representation to pose misalignment. This paper presents a novel framework that uses the PointNet representation to align point clouds and perform registration for applications such as tracking, 3D reconstruction and pose estimation. We develop a framework that compares PointNet features of template and source point clouds to find the transformation that aligns them accurately. Depending on the prior information about the shape of the object formed by the point clouds, our framework can produce approaches that are shape specific or general to unseen shapes. The shape specific approach uses a Siamese architecture with fully connected (FC) layers and is robust to noise and initial misalignment in data. We perform extensive simulation and real-world experiments to validate the efficacy of our approach and compare the performance with state-of-art approaches.
1. Introduction
PCRNet addresses PointNet’s sensitivity to pose misalignment by learning registration directly from source and template point clouds. The framework supports shape-specific or general approaches and is evaluated against established registration methods.
- Point clouds lack inherent structure, complicating direct use in deep learning despite their growing use across 3D tasks.
- PCRNet estimates the misalignment between two point clouds using PointNet as an encoding function, without handcrafted features or voxelization.
- The framework adapts to shape knowledge, noise, and computational requirements through shape-specific or general registration approaches.
- A Siamese PointNet architecture encodes source and template shapes as feature vectors, then data-driven techniques estimate the aligning pose.
- The paper introduces two PointNet-based alignment algorithms and validates them against PointNetLK, ICP, and Go-ICP on simulated and real-world data.
2. Related Work
Prior registration methods trade off computational cost, feature requirements, differentiability, and generalization. PointNet-based learned registration reduces reliance on explicit correspondences while PCRNet’s closest baseline is PointNetLK.
- Classical registration: ICP variants rely on explicit closest-point correspondences, which scale poorly with point count and make ICP nondifferentiable.
- Interest point methods: Interest-point methods are computationally favorable but require identifiable features that persist between the point clouds.
- Globally optimal methods: Go-ICP seeks globally optimal solutions through branch-and-bound optimization but has extended computation times that impede real-time use.
- PointNet: PointNet directly processes point clouds and supports tasks including classification, segmentation, detection, and completion through global representations.
- Learned registration: PointNetLK registers arbitrary point clouds by minimizing distances between fixed-length PointNet descriptors and serves as PCRNet’s closest baseline.
3. Method
PCRNet encodes source and template point clouds with shared PointNet-style networks and predicts their rigid transformation. An iterative variant repeatedly transforms the source, while EMD trains the registration and ModelNet40 supplies shape-specific or multi-category data.
- PCRNet representation: A symmetric max-pooling operation produces a permutation-invariant global feature vector from each point cloud.
- PCRNet representation: PCRNet compares source and template PointNet features and estimates the rigid-body transformation T that minimizes their difference.
- PCRNet architecture: The Siamese encoder uses shared MLPs for source and template clouds, concatenates their global features, and feeds them to fully connected layers.
- PCRNet architecture: The network outputs seven transformation parameters: three for translation and four normalized values for the rotation quaternion, enabling single-shot estimation.
- Iterative PCRNet: The iterative variant predicts an initial misalignment, reapplies it to the source, and repeats registration to improve the estimate.
- Iterative PCRNet: After n iterations, the overall transformation is composed as T = T(n) × T(n − 1) × · · · × T(1).
- Loss function: EMD trains registration by finding a bijection that minimizes distances between transformed source and template points.
- Training: ModelNet40 training varies from multiple categories to multiple models or a single model within one category, with iterative training using eight iterations.
4. Results
Experiments evaluate PCRNet across unseen and object-specific categories, Gaussian noise, iteration efficiency, and computation speed. Iterative PCRNet benefits from object-specific information, remains robust to noise, aligns in fewer iterations, and approaches Go-ICP accuracy much faster.
- 4.1. Generalizability versus specificity: On unseen categories without noise, iterative PCRNet reaches AUC 0.682, while PointNetLK reaches 0.998 and ICP reaches 0.802.
- 4.1. Generalizability versus specificity: Retraining with object-specific information raises iterative PCRNet’s AUC from 0.682 to 0.972, while PointNetLK remains at 0.998.
- 4.2. Gaussian noise: Iterative PCRNet has more successful registrations at smaller rotation errors than ICP and PointNetLK in noisy tests, especially when the object category is known.
- 4.2. Gaussian noise: Iterative PCRNet performs best near its trained noise range of 0.02-0.06, whereas PointNetLK drops sharply above 0.02 and PCRNet becomes competitive with ICP beyond that range.
- 4.2. Gaussian noise: Iterative PCRNet takes only 3 iterations to approach convergence, compared with upwards of 15 iterations for the other methods.
- 4.3. Computation speed comparisons: Go-ICP takes three orders of magnitude longer than iterative PCRNet and five orders longer than PCRNet, while iterative PCRNet has similar accuracy to Go-ICP.
5. Model replacement using segmentation
The paper demonstrates model replacement by registering a scene chair to a different ModelNet40 chair and applying the predicted transformation. Iterative PCRNet outperforms ICP and MIP in this cross-shape example.
- Iterative PCRNet registers a scene chair to a different ModelNet40 chair and replaces the original object after applying the predicted transformation.The procedure uses a chair selected from the Stanford S3DIS scene and a chair model from ModelNet40.
- ICP fails to register the chair to the correct pose, while Go-ICP does not improve upon ICP in this example.
- Iterative PCRNet produces a more accurate replacement than ICP and mixed integer programming because those methods require identical source and template objects.Shape variation within a category can cause poor registration for the classical methods.
6. Discussions and future work
The framework uses learned PointNet-based information to support accurate, noise-robust registration while offering different computational trade-offs. The authors identify partial or occluded clouds and learning-capacity limits as future challenges.
- The framework provides accurate registration robust to noise and initial misalignment while being computationally faster than existing methods.Its learned prior captures appearance variation such as noisy data and category-specificity.
- Iterative implementation can achieve accuracy comparable to global registration methods, whereas a non-iterative deeper network offers two to five orders of magnitude speed improvement.
- Future work includes adapting the network to partial and occluded point clouds and integrating it into larger deep neural network systems.
- The learning capacity of the fully connected registration layers may be limited by the size of the data distribution.