Source-linked AI summary
Deep Closest Point: Learning Representations for Point Cloud Registration
Yue Wang, Justin M. Solomon
TL;DR
Point cloud registration must recover rigid alignments despite difficult correspondence and optimization conditions. DCP combines learned point-cloud embeddings, attention-based soft matching, and differentiable SVD, and is reported to outperform classical and learning-based baselines while generalizing features to unseen data.
Problem
Rigid registration must align point clouds under noise, partiality, unknown correspondences, and non-convex optimization that can trap ICP in local optima.
Method
DCP uses point-cloud embeddings, an attention-based module with pointer-style soft matching, and differentiable SVD to predict rigid transformations end-to-end.
Results
DCP is reported to outperform ICP, its extensions, and PointNetLK across several settings, while learned features generalize to unseen data.
Takeaways & Limitations
Learned features and attention can provide correspondences sufficient for a high-quality single-pass rigid alignment, optionally followed by classical ICP polishing.
Takeaways & Limitations
The formulation assumes rigid motion, and the simplest problem statement assumes equal point-cloud sizes with corresponding points ordered for the basic error expression.
Abstract
from arXiv · showhide
Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into another so that they align. Iterative Closest Point (ICP) and its variants provide simple and easily-implemented iterative methods for this task, but these algorithms can converge to spurious local optima. To address local optima and other difficulties in the ICP pipeline, we propose a learning-based method, titled Deep Closest Point (DCP), inspired by recent techniques in computer vision and natural language processing. Our model consists of three parts: a point cloud embedding network, an attention-based module combined with a pointer generation layer, to approximate combinatorial matching, and a differentiable singular value decomposition (SVD) layer to extract the final rigid transformation. We train our model end-to-end on the ModelNet40 dataset and show in several settings that it performs better than ICP, its variants (e.g., Go-ICP, FGR), and the recently-proposed learning-based method PointNetLK. Beyond providing a state-of-the-art registration technique, we evaluate the suitability of our learned features transferred to unseen objects. We also provide preliminary analysis of our learned model to help understand whether domain-specific and/or global features facilitate rigid registration.
1. Introduction
Point cloud registration seeks rigid alignment despite noise, partiality, and unknown correspondences. DCP revisits the ICP pipeline with learned embeddings, attention-based matching, and differentiable SVD to address local optima.
- Geometric registration predicts a rigid motion aligning one shape to another, potentially under noise and partiality.
- ICP alternates correspondence estimation and rigid-motion updates but often stalls in suboptimal local minima because registration is non-convex.
- DCP learns point-cloud embeddings, predicts soft correspondences with attention and a pointer module, and extracts the rigid transformation using differentiable SVD.
- The model is trained and tested end-to-end on ModelNet40 and reported to outperform ICP, its extensions, and PointNetLK across several settings.
- The paper evaluates efficiency, performance, ablations, and whether local or global features are more useful for registration, while releasing code.
2. Related Work
Prior work spans classical registration, geometric deep learning, pointer networks, and non-local architectures. These lines of research address optimization, geometric representation, combinatorial matching, and long-range dependencies relevant to DCP.
- ICP alternates correspondence estimation with least-squares alignment, while variants address noise, partiality, sparsity, and uncertain data.
- Go-ICP searches SE(3) with branch-and-bound and can outperform local ICP for global solutions, but is several orders of magnitude slower.
- Geometric deep learning includes graph and point-cloud methods, with DGCNN applying graph neural-network ideas to point clouds using dynamic edges.
- PointNet primarily encodes global features, whereas PointNet++ and DGCNN incorporate local geometry through neighborhood or graph-based operations.
- Pointer Networks use attention to produce differentiable soft pointers over input positions, making them relevant to the combinatorial matching problem in registration.
3. Problem Statement
Rigid registration estimates a rotation and translation minimizing alignment error between two point clouds. Unknown correspondences create a coupled optimization that ICP solves iteratively but can trap in poor local optima.
- The problem represents two point clouds X and Y in R3 and seeks a rigid transformation, with the simplest exposition assuming equal point counts.
- The transformation consists of RXY in SO(3) and tXY in R3, and the objective minimizes mean-squared error when corresponding points are ordered together.
- With known correspondences, centroids and the cross-covariance matrix lead to a closed-form minimizing alignment obtained through SVD.
- Unknown correspondences require a mapping from each point in X to a corresponding point in Y, changing the alignment objective.
- Transformation and correspondence determine each other, producing a chicken-and-egg problem that ICP addresses by alternating updates but remains prone to local optima.
4. Deep Closest Point
Deep Closest Point maps two point clouds into matching-aware features, predicts soft correspondences, and extracts a rigid motion differentiably. Its pipeline combines point embeddings, attention, pointer generation, and SVD-based transformation estimation, with DGCNN local features reported as important for matching.
- Initial Features: DCP embeds unaligned point clouds into a common space designed to support matching while factoring out rigid motion.The embedding stage evaluates PointNet and DGCNN, retaining per-point representations for correspondence estimation.
- Initial Features: DGCNN incorporates local geometry through k-nearest-neighbor graphs, whereas PointNet primarily extracts pointwise information.DGCNN applies edgewise nonlinearities and vertex-wise aggregation over graph neighborhoods.
- Initial Features: DGCNN’s local features are empirically critical for high-quality matching in the subsequent registration pipeline.The paper motivates the transition from PointNet by jointly using local and global information for rigid alignment.
- Attention: An attention module updates each cloud’s embeddings using information from the other cloud, producing task-specific contextual features.The module uses a residual, asymmetric Transformer-based function so features depend jointly on both point clouds.
- Pointer Generation: Pointer generation produces a differentiable soft map that assigns each point in one cloud a probability vector over points in the other.This avoids non-differentiable hard assignments and approximates matching through soft pointers.
- SVD Module: The final module converts soft correspondences into averaged matched points and uses differentiable SVD to recover the rigid motion.Gradients pass through the SVD during end-to-end training, while the complete model is trained against ground-truth rigid motions on synthetic point-cloud pairs.
5. Experiments
DCP is evaluated against classical and learning-based registration methods on ModelNet40 across unseen clouds, unseen categories, noise, hybrid refinement, and inference time. Across these settings, DCP generally performs strongly, while attention improves category generalization and DCP provides a useful initialization for ICP.
- 5.1. ModelNet40: Full Dataset Train & Test: DCP-v1 outperforms other methods under all performance metrics on unseen ModelNet40 point clouds, while DCP-v2 performs even better.The comparison includes ICP, Go-ICP, FGR, and PointNetLK.
- 5.2. ModelNet40: Category Split: On unseen categories, DCP-v2 has smaller error than all compared methods except FGR, including PointNetLK.DCP-v1 performs much worse than DCP-v2, supporting the attention module.
- 5.3. ModelNet40: Resilience to Noise: Go-ICP, PointNetLK, and DCP remain robust to Gaussian noise, whereas ICP typically converges to a far-away fixed point and FGR is noise-sensitive.The model used here was trained on noise-free data from all ModelNet40.
- 5.4. DCP Followed By ICP: Initializing ICP with DCP’s predicted transformation lets ICP converge to the global optimum after failing at the global alignment task.The authors characterize ICP in this configuration as polishing DCP’s alignment.
- 5.5. Efficiency: DCP-v1 is the fastest method compared, and DCP-v2 is slower only than vanilla ICP in inference time.Inference time is averaged over 100 results on a desktop with an Intel I7-7700 CPU and Nvidia GTX 1070 GPU.
6. Ablation Study
The ablation study tests the feature extractor, transformation layer, and embedding dimension. Results favor DGCNN and differentiable SVD, while increasing the embedding dimension to 1024 helps DCP-v2 only marginally.
- 6.1. PointNet or DGCNN?: Models perform consistently better with DGCNN than with their PointNet counterparts.The comparison uses both DCP-v1 and DCP-v2 on ModelNet40.
- 6.2. MLP or SVD?: Both DCP-v1 and DCP-v2 perform better with the SVD layer than with MLP.The result supports computing the rigid transformation with a custom SVD layer.
- 6.3. Embedding Dimension: Increasing the embedding dimension from 512 to 1024 marginally helps DCP-v2, while DCP-v1 shows small degeneracy.The results are consistent with accuracy improving up to a critical dimensionality threshold, after which differences become insignificant.
7. Conclusion
DCP uses learned features and attention to extract high-quality point correspondences for reliable rigid alignment in a single pass. Its alignment can be further improved through iteration or classical ICP polishing, while future work includes recursive refinement and feature transfer.
- Learned features and attention reliably extract correspondences for rigid motions aligning two input point clouds.The model is designed to produce a high-quality alignment in a single pass.
- DCP produces a high-quality alignment in a single pass that can be improved by iteration or classical ICP polishing.
- DCP is presented as a drop-in replacement for ICP with improved behavior in rigid alignment problems.
- Future directions include transferring learned embeddings to classification and segmentation and training DCP recursively to refine alignments.