Source-linked AI summary
PREDATOR: Registration of 3D Point Clouds with Low Overlap
Shengyu Huang, Zan Gojcic, Mikhail Usvyatsov, Andreas Wieser, Konrad Schindler
TL;DR
Point-cloud registration methods and benchmarks have largely focused on pairs with sufficient overlap, leaving low-overlap registration under-addressed. PREDATOR introduces an overlap-attention architecture that exchanges information between two scans early, conditions per-point predictions on the other scan, and focuses sampling on overlapping regions. It substantially improves low-overlap registration and achieves the reported benchmark results on 3DMatch.
Problem
Existing benchmarks and methods largely focus on point-cloud pairs with ≥30% overlap, while registration performance deteriorates rapidly in the low-overlap regime.
Method
PREDATOR uses an overlap-attention module for early information exchange between latent encodings, conditioning decoded per-point features and scores on the other point cloud.
Results
PREDATOR increases low-overlap registration recall by >15 percentage points and reaches >90% registration recall on 3DMatch.
Takeaways & Limitations
Focusing feature-point sampling on predicted overlap regions provides a practical strategy for registering point-cloud pairs with low overlap.
Takeaways & Limitations
PREDATOR is tightly coupled to fully convolutional encoders, requires a reasonable number of bottleneck superpoints, and cannot solve all low-overlap cases.
Abstract
from arXiv · showhide
We introduce PREDATOR, a model for pairwise point-cloud registration with deep attention to the overlap region. Different from previous work, our model is specifically designed to handle (also) point-cloud pairs with low overlap. Its key novelty is an overlap-attention block for early information exchange between the latent encodings of the two point clouds. In this way the subsequent decoding of the latent representations into per-point features is conditioned on the respective other point cloud, and thus can predict which points are not only salient, but also lie in the overlap region between the two point clouds. The ability to focus on points that are relevant for matching greatly improves performance: PREDATOR raises the rate of successful registrations by more than 20% in the low-overlap scenario, and also sets a new state of the art for the 3DMatch benchmark with 89% registration recall.
1. Introduction
Existing registration benchmarks largely exclude point-cloud pairs below 30% overlap, masking a practical failure regime. PREDATOR addresses this gap by learning to identify and sample salient points in the overlap region, improving low-overlap registration.
- Motivation: Benchmarks commonly evaluate only point-cloud pairs with ≥30% overlap, while practical scans may contain substantially less overlap.The paper argues that this lenient protocol makes registration appear more solved than it is.
- Motivation: Registration performance deteriorates rapidly below 30% overlap, despite human operators remaining able to register such pairs.
- Contribution: PREDATOR learns to detect the overlap region between unregistered scans and focus feature-point sampling there.
- Contribution: Its contributions include overlap attention, cross-cloud-conditioned descriptors, and a loss for training matchability scores that support better interest-point sampling.
- Results: PREDATOR increases low-overlap registration recall by >15 percentage points and reaches >90% registration recall on 3DMatch.
2. Related work
Prior work developed local and fully convolutional descriptors, interest-point detectors, and contextual matching methods, but these approaches generally process point clouds without the proposed overlap-focused cross-cloud conditioning. PREDATOR combines encoder-decoder processing with early contextual exchange between the two clouds.
- Local descriptors: Early point-cloud descriptors characterized local geometry but were vulnerable to clutter and occlusions and limited by patch-based computation.
- Learned descriptors: 3DMatch uses a Siamese 3D CNN, while PPFNet and FoldNet combine point-pair features with PointNet-style architectures for descriptor extraction.
- Fully convolutional features: Fully convolutional architectures compute dense features efficiently; FCGF achieves performance comparable to strong patch-based descriptors while being orders of magnitude faster.
- PREDATOR: Its overlap-attention module exchanges information between the two point clouds at the superpoint bottleneck before decoding per-point descriptors and overlap or matchability scores.
- Contextual matching: Contextual methods such as D2D-Net and SuperGlue condition learned representations on information from another image through attention and message passing.
3. Method
PREDATOR uses a two-stream encoder-decoder that exchanges information between point clouds at a bottleneck overlap-attention module, then predicts conditioned per-point descriptors, overlap, and matchability scores.
- Architecture: PREDATOR encodes both point clouds into superpoints with shared weights, exchanges co-contextual information, and decodes mutually conditioned representations.The architecture comprises shared encoding, bottleneck overlap attention, and decoding into point-wise outputs.
- Problem setting: Point-cloud registration is posed as recovering a rigid transformation aligning P to Q, with sufficient overlap required for determination.The method targets low-overlap pairs with an overlap threshold above 0.1 rather than the typical threshold above 0.3.
- Encoder: The encoder aggregates raw points into superpoints and associated latent features using voxel-grid downsampling, strided convolutions, and graph-based contextual processing.Superpoints correspond to fixed receptive fields, so their counts can differ between input clouds.
- Overlap attention module: The cross-attention block connects every superpoint pair in a bipartite graph and uses learned queries, keys, and values to exchange information in both directions.A temperature parameter controls soft assignments, which converge to hard nearest-neighbour assignment as t→0.
- Decoder: Overlap attention produces overlap probabilities for superpoints and their soft correspondences, while the decoder outputs per-point descriptors plus overlap and matchability scores.Matchability is conditional saliency based on the other point cloud; overlap and matchability remain separate to distinguish location from descriptor ambiguity.
- Training: PREDATOR is trained end-to-end with circle, overlap, and matchability losses supervised using ground-truth correspondences or labels derived during training.The overlap loss treats overlap estimation as binary classification, while the circle loss supervises point-wise descriptors.
4. Experiments
Experiments evaluate PREDATOR on indoor, synthetic object-centric, and outdoor point-cloud benchmarks, including low-overlap settings. Results show that overlap-aware filtering and sampling improve registration, while cross-attention remains applicable to large outdoor scans despite its quadratic memory cost.
- Benchmarks and metrics: PREDATOR is evaluated on 3DMatch, 3DLoMatch, ModelNet40, ModelLoNet, and odometryKITTI using registration, matching, geometric-error, and overlap-based metrics.3DLoMatch contains scan pairs with 10–30% overlap, complementing official 3DMatch pairs with more than 30% overlap.
- Overlap filtering: 71% of low-overlap 3DLoMatch pairs exceed 30% overlap after removing points with predicted overlap scores below 0.5.Discarding low-score points increases average overlap by 133% in 3DLoMatch and by more than 50% in 3DMatch.
- Interest-point sampling: Informed interest-point sampling increases inlier ratios and registration recall, with low-overlap inlier ratios more than tripling for samples of up to 1000 points.Top-k (om) finds many inliers but can cluster them; probability-based (om) sampling is more robust for estimating transformations and yields the best registration recall.
- 3DMatch and 3DLoMatch: PREDATOR improves low-overlap registration recall by 15.5–19.7 percentage points over the closest competing feature-based method.The supplied passages identify this comparison on 3DLoMatch and note that PREDATOR does not solve every case.
- ModelNet40: On ModelNet, overlap-aware sampling produces approximately 8° lower RRE at completeness 0.4 than random sampling.PREDATOR also outperforms methods tuned for ModelNet in low-overlap RRE and reduces the number of pairs below 70% overlap by more than 40 percentage points.
- odometryKITTI: On odometryKITTI, PREDATOR performs on par with state-of-the-art methods, while superpoint-level cross-attention enables processing large outdoor scans on a single GPU.The cross-attention module has O(n^2) memory complexity and is the model’s memory bottleneck.
5. Conclusion
PREDATOR is designed for pairwise registration of low-overlap point clouds, using overlap attention to exchange information early and infer points likely to lie in the overlap region. The authors identify encoder coupling, superpoint density, and unresolved difficult cases as boundaries for extension.
- PREDATOR uses an overlap attention module to exchange information between point-cloud latent encodings and infer likely overlap points.This module is the core of the model’s approach to low-overlap registration.
- The model is tightly coupled to fully convolutional point-cloud encoders and requires a reasonable number of bottleneck superpoints.The authors identify uneven point density as a scenario where this design may be limited.
- PREDATOR does not solve all registration cases in the low-overlap regime.
- The released D3Feat code fails to reproduce the paper’s results, possibly because of hyper-parameter changes.
A. Appendix
The appendix documents evaluation metrics, preprocessing, architectures, training, additional results, ablations, runtime, and visualizations.
- The supplementary material defines evaluation metrics and describes data preprocessing, network architectures, and dataset-specific training.
- It also reports additional results, ablation studies, runtime analysis, and visualizations on 3DLoMatch and ModelLoNet.
A.1. Evaluation metrics
The appendix defines metrics for correspondence quality, registration accuracy, error distributions, and synthetic-data registration quality. Registration recall is presented as the most reliable end-to-end measure.
- Correspondence metrics: Inlier ratio measures the fraction of reciprocal feature-space correspondences that lie within τ1 = 10 cm under the ground-truth transformation.The Iverson bracket indicates whether each correspondence satisfies the threshold condition.
- Correspondence metrics: Feature Match Recall measures the fraction of pairs whose inlier ratio exceeds τ2 = 0.05 and may support robust transformation recovery.It does not verify that the correspondence geometry is sufficient to determine the transformation.
- Registration metrics: Registration recall measures end-to-end registration performance using ground-truth correspondences after applying the estimated transformation.It computes correspondence RMSE and checks the fraction with RMSE < 0.2.
- Evaluation protocol: The appendix notes that immediately adjacent point clouds are excluded because their overlap is very high by construction.
- Synthetic-data metric: Chamfer distance measures registration quality on synthetic data using a modified metric.The raw and input source and target point clouds are distinguished in the metric setup.
- Pose errors: Relative translation and rotation errors measure deviations from the ground-truth pose using the estimated rotation matrix R and translation vector t.
- Distribution metric: ECDF measures the distribution of a value set, with x ranging from its minimum to maximum; here the values are overlap ratios.
A.2. Dataset preprocessing
The dataset section describes 3DMatch’s multi-scene benchmark composition and synthetic partial-cloud generation, while the appendix also lists dataset-specific hyperparameters.
- 3DMatch: 3DMatch contains 62 scenes drawn from five earlier datasets and uses 54 training scenes plus 8 testing scenes.
- 3DMatch: The benchmark spans varied indoor environments and depth sensors, including Kinect, Structure Sensor, Asus Xtion Pro Live, and Intel RealSense.
- Configuration: A table provides hyperparameter configurations for different datasets.
- Synthetic preprocessing: Synthetic partial point clouds are created by splitting the unit sphere with a shifted plane, then applying random rotations, translations, and Gaussian jitter.The rotation angle is sampled below 45°, and translation lies in [-0.5, 0.5].
A.3. Implementation and training
PREDATOR is trained with dataset-specific schedules and hyperparameters, using stochastic gradient descent and batch size 1 throughout.
- Training uses stochastic gradient descent for 30, 200, and 150 epochs on 3DMatch, ModelNet, and KITTI, respectively.Initial learning rates are 0.005, 0.01, and 0.05; momentum is 0.98 and weight decay is 10^-6.
A.4. Network architecture
PREDATOR uses a KPConv-based encoder–decoder with overlap attention shared across datasets, while adapting configurations to their substantially different point-cloud densities and scales.
- A.4. Network architecture: PREDATOR builds its detailed architecture on the KPConv implementation from D3Feat, adding instance normalization and Leaky ReLU to each KPConv layer.Strided convolutions operate on voxel-downsampled point clouds, while decoder upsampling queries the closest feature from the previous layer.
- A.4. Network architecture: 3DMatch contains approximately 20k points after voxel downsampling, whereas ModelNet40 contains only 717 points and uses unit-sphere-normalized clouds.3DMatch also has bounding boxes up to 3×3×3 m^3, unlike ModelNet40's normalized unit-sphere extent.
- A.4. Network architecture: The encoder and decoder are adapted per dataset to account for these density and spatial-scale differences, while the overlap attention model remains unchanged.The differing network hyperparameters are listed in Table 7.
A.5. Additional results
Detailed scene-level evaluation reports registration recall alongside error metrics, with PREDATOR achieving strong results across nearly all scenes.
- A.5. Additional results: Registration Recall, Relative Rotation Error, and Relative Translation Error are reported per scene for 3DMatch and 3DLoMatch.The detailed results are presented in Table 8.
- A.5. Additional results: RRE and RTE are averaged only over successfully registered pairs, preventing complete registration failures from dominating these error metrics.This averaging condition applies separately for each scene.
- A.5. Additional results: PREDATOR obtains the highest RR and the lowest or second-lowest RTE and RRE for almost all scenes.The passage attributes these results to the overlap attention module together with probabilistic sampling.
A.6. Additional ablation studies
Additional experiments examine sampling choices, backbone augmentation, runtime, and qualitative behavior, including results on low-overlap benchmarks.
- Sampling ablations: Probabilistic sampling based on the product of overlap and matchability scores achieves the highest registration recall among the tested sampling strategies.The comparison includes random sampling and sampling based on overlap, matchability, or their combination; Table 9 summarizes the combinations.
- Backbone ablation: Registration recall almost doubles on 3DLoMatch when FCGF uses the overlap attention module and samples only 250 points.The experiment is described as using a sparse convolution backbone, with FCGF + OA denoting the augmented model.
- Runtime: PREDATOR is marginally faster than D3Feat despite its additional overlap attention module, while FCGF uses a more efficient sparse-convolution data loader.Runtime is measured per fragment pair on 1,623 3DMatch test pairs; Table 11 reports the measurements.
- Qualitative results: Qualitative examples are provided for 3DLoMatch and ModelLoNet, with point clouds rotated and translated to visualize overlap and matchability scores.The examples appear in Figures 12 and 13.
- Network details: The cross-attention module reshapes four attention heads, computes a score matrix from queries and keys, updates messages from values, and reshapes the output.This architecture is illustrated in Figure 11 for 3DMatch and ModelNet.