Source-linked AI summary
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency
Xuyang Bai, Zixin Luo, Lei Zhou, Hongkai Chen, Lei Li, Zeyu Hu, Hongbo Fu, Chiew-Lan Tai
TL;DR
Feature-based point cloud registration depends on rejecting outlier correspondences, yet learning frameworks have largely omitted the spatial consistency imposed by Euclidean transformations. PointDSC integrates spatially guided feature embedding with differentiable spectral matching and reports improved robustness across diverse datasets, domains, and descriptors.
Problem
Existing learning-based outlier rejection methods omit or insufficiently use the 3D spatial consistency that inlier correspondences should satisfy.
Method
PointDSC uses SCNonlocal for spatially guided feature embedding and neural spectral matching for spatially informed outlier pruning, with seeding for difficult overlap cases.
Results
PointDSC brings remarkable improvement over state-of-the-art methods and generalizes to unseen domains while cooperating with different local descriptors.
Takeaways & Limitations
Explicit spatial consistency can be integrated into deep outlier rejection while retaining applicability across indoor and outdoor scenarios and multiple descriptors.
Takeaways & Limitations
Joint end-to-end training of the feature and outlier-rejection networks did not improve feature-network performance because the current losses may provide inadequate gradients.
Abstract
from arXiv · showhide
Removing outlier correspondences is one of the critical steps for successful feature-based point cloud registration. Despite the increasing popularity of introducing deep learning methods in this field, spatial consistency, which is essentially established by a Euclidean transformation between point clouds, has received almost no individual attention in existing learning frameworks. In this paper, we present PointDSC, a novel deep neural network that explicitly incorporates spatial consistency for pruning outlier correspondences. First, we propose a nonlocal feature aggregation module, weighted by both feature and spatial coherence, for feature embedding of the input correspondences. Second, we formulate a differentiable spectral matching module, supervised by pairwise spatial compatibility, to estimate the inlier confidence of each correspondence from the embedded features. With modest computation cost, our method outperforms the state-of-the-art hand-crafted and learning-based outlier rejection approaches on several real-world datasets by a significant margin. We also show its wide applicability by combining PointDSC with different 3D local descriptors.
1. Introduction
Feature-matching pipelines require robust outlier rejection because correspondences become unreliable, particularly with small scene overlap. PointDSC combines spatial consistency with deep feature learning during embedding and pruning.
- Motivation: Feature-matching pipelines commonly extract and match local features before rejecting outliers for robust point cloud alignment.Matched correspondences remain vulnerable to outliers, especially when scene-fragment overlap is small.
- Motivation: 3D spatial consistency is defined between point pairs by rigid transformations and is a key geometric property of inlier correspondences.This is stronger than the weak point-to-epipolar-line relation available in 2D.
- Limitations of prior methods: Spectral matching uses distance-preserving compatibility graphs and eigen analysis, but length consistency alone is ambiguous and weak under high outlier ratios.High outlier ratios reduce the dominance of inlier clusters, making them difficult to identify spectrally.
- Limitations of prior methods: Learning-based methods predict per-correspondence inlier probabilities but omit essential 3D spatial relations during embedding and pruning.Their individual classification overlooks spatial compatibility between inliers.
- PointDSC: PointDSC explicitly leverages spatial consistency during both feature embedding and outlier pruning in a two-stage deep neural network.The design combines advantages associated with traditional spatial matching and learning-based feature processing.
- PointDSC: SCNonlocal combines length consistency with feature similarity, while NSM uses embedded features and coordinates to alleviate spectral-matching ambiguity.A seeding mechanism is additionally proposed for small-overlap cases.
2. Related Work
Related work extends learning across registration pipelines, but end-to-end methods may remain insufficiently robust in complex scenes because matches contain many outliers.
- Learning-based registration: Recent learning-based methods replace components of classical registration pipelines, including keypoint detection and feature description.End-to-end registration networks have also been proposed.
- Learning-based registration: End-to-end registration robustness and applicability in complex scenes may remain insufficient because matches are highly contaminated by outliers.
3. Methodology
PointDSC is a two-stage network that explicitly uses spatial consistency during geometric feature embedding and outlier pruning. It selects correspondence subsets, applies neural spectral matching, and estimates transformations from confidence-weighted correspondences.
- PointDSC takes putative correspondence coordinates as input and aims to predict inlier/outlier labels while recovering a rigid transformation.
- 3.4. Neural Spectral Matching: Neural spectral matching estimates correspondence inlier probabilities on each subset, and these confidences weight least-squares transformation fitting.The approach is designed to prune potential outliers even when a subset is not entirely composed of inliers.
- 3.2. Geometric Feature Embedding: SCNonlocal embeds correspondences using spatial consistency between inlier correspondences to produce a discriminative geometric feature space.The module combines rigid-transformation length consistency with learned feature similarity.
- 3.2. Geometric Feature Embedding: The spatial consistency term β measures length differences between corresponding segments and assigns zero compatibility when the difference exceeds σd.Compatible correspondence pairs receive larger β values, which regulate feature similarity.
- 3.3. Seed Selection: PointDSC selects highly confident, well-distributed seeds, then retrieves feature-space k-nearest-neighbor subsets expected to have higher inlier ratios.Feature-space neighbors are favored because SCNonlocal makes them more likely to follow transformations similar to the seed.
- 3.6. Loss Formulation: Training combines node-wise confidence supervision with edge-wise spectral matching supervision over pairwise correspondence relations.The edge-wise loss supervises compatibility between correspondence pairs and experiments report that it improves performance.
4. Implementation Details
Training uses fixed-size sampled correspondence batches, while testing processes full correspondence sets with specified spatial and optimization hyperparameters.
- Training: Training samples 1,000 correspondences per point-cloud pair and uses batch size 16.The network is implemented in PyTorch.
- Training: NSM uses neighborhood size k = 40 during training.
- Training: The model is optimized with ADAM for 100 epochs using an initial learning rate of 0.0001 and decay factor 0.99.The experiments use a single RTX2080 Ti graphics card.
- Testing: Testing uses the full correspondence set and applies NMS to select spatially uniform seeds.The NMS radius is specified in the implementation details.
5. Experiments
Experiments evaluate PointDSC for pairwise, outdoor, and multiway registration, and use ablations and visualizations to examine its components and robustness.
- Pairwise Registration: 3DMatch evaluation uses 1,623 partially overlapped fragments from eight scenes, with Registration Recall, Rotation Error, and Translation Error as metrics.The data are voxel-downsampled at 5cm and use a 10cm inlier threshold.
- Pairwise Registration: PointDSC achieves 86.54% Inlier Recall, while SM and TEASER achieve 38.36% and 68.08%, respectively.SM and TEASER have slightly better Inlier Precision, yielding a precision-recall trade-off for PointDSC.
- Pairwise Registration: 78.5% Registration Recall is achieved by PointDSC with FPFH, surpassing the competing methods and showing robustness to different input descriptors.The model trained with FCGF is reused without fine-tuning for FPFH evaluation.
- Generalization to Outdoor Scenes: PointDSC pretrained on 3DMatch remains competitive on KITTI, while retraining from scratch further improves performance over the baselines.KITTI evaluation uses FPFH descriptors and outdoor registration thresholds.
- Multiway Registration: PointDSC achieves the lowest average ATE in three of four tested Augmented ICL-NUIM scene types.Pairwise PointDSC poses are refined with pose graph optimization for multiway registration.
- Ablation Studies: SCNonlocal consistently improves registration results across all tested settings and metrics.Removing its spatial term makes the Nonlocal baseline perform drastically worse.
- Ablation Studies: With SCNonlocal, inlier-pair feature similarity concentrates near 0.8 and generally exceeds similarity for non-inlier pairs.The embedding therefore separates inlier and non-inlier pair similarities more clearly than the baseline methods.
- Robustness: PointDSC remains robust at extremely high outlier ratios in examples from 3DMatch and KITTI.The visualization compares input correspondences, RANSAC-100k, and PointDSC results.
6. Conclusion
The paper concludes that PointDSC combines spatial consistency with deep feature learning for robust outlier rejection and performs across domains and descriptors.
- Conclusion: PointDSC uses SCNonlocal for feature embedding, NSM for outlier pruning, and seeding to improve robustness under high outlier ratios.The network explicitly incorporates spatial consistency established by Euclidean transformations.
- Conclusion: Experiments report improvement over state-of-the-art methods, generalization to unseen domains, and compatibility with different local descriptors.
7. Supplementary Material
The supplementary material details implementation, evaluation, ablation, and failure analyses for PointDSC. It reports strong performance across descriptors and settings, while identifying limitations in joint training, sparse correspondences, and symmetric scenes.
- Implementation: PointDSC’s post-refinement alternates correspondence weighting and transformation estimation, using residuals and a 10cm inlier threshold on 3DMatch.The implementation also uses a 60cm threshold for KITTI.
- Implementation: The compatibility metric multiplies spatial consistency and feature similarity; alternative weighted arithmetic and geometric averages produce only marginally different performance.Power iteration typically converges in fewer than five iterations.
- Additional statistics: PointDSC consistently outperforms state-of-the-art methods on both rotation-error and translation-error AUC metrics on 3DMatch.Table 7 reports exact AUC values at different error thresholds, with higher values preferred.
- Additional experiments: PointDSC consistently outperforms RANSAC with different descriptors and increases registration recall by 16.3% with FCGF and 7.3% with Predator under the 5000-points setting.The method also boosts Predator, a descriptor designed for low-overlap registration.
- Additional experiments: Prioritized RANSAC improves registration recall by more than 30% over classic RANSAC by using predicted inlier probabilities to bias seed sampling.The result indicates that PointDSC’s predicted inlier probabilities can help RANSAC find all-inlier subsets earlier.
- Limitations and additional results: Joint training did not improve the feature network, while fewer confident keypoints improved D3Feat + PointDSC within a reasonable range.The authors attribute the keypoint improvement to removing nonsalient regions associated with failures from large symmetric objects.
- Qualitative results: Large symmetric objects can produce dominant outlier clusters and incorrect hypotheses, with unsuccessful registrations often showing rotation errors around 90° or 180°.These failures arise when many outlier correspondences satisfy an incorrect transformation.