Source-linked AI summary
Geometric Transformer for Fast and Robust Point Cloud Registration
Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, Kai Xu
TL;DR
Point cloud registration needs accurate correspondences despite difficult keypoint detection and sparse, loose superpoint matching in low-overlap scenes. GeoTransformer uses pair-wise distances, triplet-wise angles, and geometric attention to learn transformation-invariant features, yielding robust RANSAC-free registration with reported speed and benchmark gains.
Problem
Keypoint detection and sparse superpoint matching make accurate correspondence extraction difficult, especially for low-overlap point clouds.
Method
GeoTransformer combines geometric self-attention and feature-based cross-attention while encoding pair-wise distances and triplet-wise angles among superpoints.
Results
GeoTransformer improves inlier ratio by 17∼30 percentage points and registration recall by over 7 points on 3DLoMatch, with registration over 100 times faster than RANSAC.
Takeaways & Limitations
Reliable correspondences enable accurate registration without RANSAC and support a fast coarse-to-fine registration pipeline.
Takeaways & Limitations
Uniformly downsampled superpoints can create high memory and computational costs for large-area point clouds, requiring a performance-efficiency trade-off in the downsampling rate.
Abstract
from arXiv · showhide
We study the problem of extracting accurate correspondences for point cloud registration. Recent keypoint-free methods bypass the detection of repeatable keypoints which is difficult in low-overlap scenarios, showing great potential in registration. They seek correspondences over downsampled superpoints, which are then propagated to dense points. Superpoints are matched based on whether their neighboring patches overlap. Such sparse and loose matching requires contextual features capturing the geometric structure of the point clouds. We propose Geometric Transformer to learn geometric feature for robust superpoint matching. It encodes pair-wise distances and triplet-wise angles, making it robust in low-overlap cases and invariant to rigid transformation. The simplistic design attains surprisingly high matching accuracy such that no RANSAC is required in the estimation of alignment transformation, leading to $100$ times acceleration. Our method improves the inlier ratio by $17{\sim}30$ percentage points and the registration recall by over $7$ points on the challenging 3DLoMatch benchmark. Our code and models are available at https://github.com/qinzheng93/GeoTransformer.
1. Introduction
Point cloud registration estimates a rigid alignment between partially overlapping 3D point clouds, but keypoint detection and sparse superpoint matching remain difficult in low-overlap cases. GeoTransformer encodes transformation-invariant geometric structure to improve correspondence quality and enable fast RANSAC-free registration.
- Point cloud registration estimates a rigid transformation aligning two partially overlapping 3D point clouds.
- Keypoint detection is challenging with small overlap, often producing correspondences with low inlier ratios.
- Superpoint matching relaxes repeatability requirements through patch overlap, but its sparse and loose nature requires globally contextual geometric features.
- GeoTransformer encodes pair-wise distances and triplet-wise angles to learn transformation-invariant geometric features for robust superpoint matching.
- 100 times faster than RANSAC, the registration stage runs in 0.01s for two point clouds with 5K correspondences without relying on RANSAC.
- 17∼30 percentage points higher inlier ratio and over 7 points higher registration recall are reported on the 3DLoMatch benchmark.
2. Related Work
Related work comprises correspondence-based registration, including keypoint-based and keypoint-free correspondence extraction, alongside emerging direct registration methods.
- Correspondence-based methods extract point correspondences and recover transformations with robust pose estimators such as RANSAC.
- Keypoint-based methods seek repeatable keypoints and powerful descriptors, while keypoint-free methods consider matches without keypoint detection.
- GeoTransformer follows detection-free methods and improves correspondence accuracy by leveraging geometric information.
- Direct registration methods estimate transformations with neural networks in an end-to-end manner.
3. Method
The method uses a hierarchical coarse-to-fine pipeline that matches superpoints with GeoTransformer before propagating correspondences to dense points and estimating alignment. GeoTransformer combines transformation-invariant geometric self-attention with cross-attention to encode intra-cloud structure and inter-cloud consistency.
- Method overview: The pipeline downsamples point clouds into superpoints, matches overlapping local patches, propagates matches to dense points, and recovers the transformation without RANSAC.KPConv-FPN provides multi-level features; the final alignment is recovered from dense correspondences.
- Geometric Transformer: GeoTransformer alternates geometric self-attention and feature-based cross-attention to learn hybrid features for reliable superpoint matching.Self-attention models each cloud's geometric structure, while cross-attention models consistency across the two clouds.
- Geometric self-attention: Geometric self-attention computes weighted sums of projected features, with attention scores incorporating geometric structure embeddings.Weights are obtained by row-wise softmax, and the geometric embedding aggregates distance and angle embeddings using max pooling.
- Geometric structure encoding: GeoTransformer encodes pair-wise distances and triplet-wise angles, whose rigid-transformation invariance supports robust matching in low-overlap cases.Distance embeddings use a sinusoidal function, while angular embeddings use nearest-neighbor triplets.
- Registration: The method achieves accurate low-overlap registration with fewer correspondences and a RANSAC-free registration stage.The registration stage is reported to run 0.01s for two point clouds with 5K correspondences, 100 times faster than RANSAC.
4. Experiments
Experiments on indoor and outdoor benchmarks show that GeoTransformer improves correspondence quality and supports accurate, fast registration without RANSAC. Ablations and visualizations attribute these gains to geometric self-attention and overlap-aware supervision.
- Indoor benchmark results: GeoTransformer improves Inlier Ratio by 7–33 percentage points on 3DMatch and 17–31 percentage points on 3DLoMatch over prior methods.The gain is larger with fewer correspondences, indicating more reliable correspondence extraction.
- Indoor benchmark results: At least 5 percentage points of Feature Matching Recall improvement on 3DLoMatch demonstrates effectiveness in low-overlap cases.
- RANSAC-free registration: 86.5% Registration Recall on 3DMatch and 59.9% on 3DLoMatch with weighted SVD is close to Predator with RANSAC.Without RANSAC, high inlier ratio is necessary for successful registration.
- RANSAC-free registration: 91.5% Registration Recall on 3DMatch and 74.0% on 3DLoMatch with LGR surpasses all RANSAC-based baselines by a large margin.LGR also gains over 100 times acceleration in pose time compared with RANSAC.
- Ablation and visualization: Injecting geometric information generally improves performance, while coordinate-based embeddings provide limited gains due to transformation sensitivity.The ablation compares graph, self-attention, coordinate, point-pair, distance, and geometric-structure embeddings.
- Ablation and visualization: Circle loss outperforms cross-entropy variants for supervising superpoint matching, supporting overlap-aware circle loss.
- Ablation and visualization: Geometric self-attention learns consistent attention patterns across matching anchors, including across disjoint overlap areas.Visualizations show geometric relationships help recover wall matches and reject similar flat or corner outliers.
- Outdoor benchmark results: On KITTI, GeoTransformer performs on par with RANSAC-based methods, outperforms RANSAC-free baselines by a large margin, and with LGR beats all RANSAC-based methods.
5. Conclusion
The paper presents Geometric Transformer for robust coarse-to-fine point cloud correspondence and fast, accurate RANSAC-free registration.
- Geometric Transformer encodes pair-wise distances and triplet-wise angles among superpoints to capture transformation-invariant cross-cloud geometric consistency.
- Reliable correspondences enable fast and accurate registration without relying on RANSAC.
A. Network Architecture Details
The architecture embeds geometric structure in superpoint features, interleaves geometric self-attention with feature-based cross-attention, and propagates matches through optimal transport to dense correspondences.
- Geometric structure embedding: Geometric structure embedding represents superpoint-pair distances and superpoint-triplet angles using sinusoidal functions.Temperature parameters control sensitivity to distance and angular variations.
- Point matching: An optimal transport layer produces soft local point assignments, with Sinkhorn iterations and mutual top-k selection extracting confident correspondences.Matches below confidence 0.05 are ignored before combining local matches into global dense correspondences.
- Backbone and implementation: The implementation uses KPConv-FPN with voxel subsampling, four backbone stages for 3DMatch, and five for KITTI.
- Superpoint matching module: The matching module interleaves geometric self-attention and feature-based cross-attention three times to extract hybrid superpoint features.
- Registration: Local-to-global registration uses superpoint matches with at least three local correspondences and iteratively recomputes transformations five times.
B. Metrics
The evaluation uses correspondence, registration, and error metrics tailored to indoor and outdoor point-cloud registration benchmarks.
- 3DMatch and 3DLoMatch: Inlier Ratio is the fraction of putative point matches whose transformed residual is below 10cm.
- 3DMatch and 3DLoMatch: Feature Matching Recall is the fraction of point-cloud pairs with Inlier Ratio above 0.05, measuring potential registration success.
- 3DMatch and 3DLoMatch: Registration Recall counts point-cloud pairs whose transformation error is below 0.2m, with error computed from transformed ground-truth correspondences.
- 3DMatch and 3DLoMatch: Patch Inlier Ratio measures the fraction of superpoint matches with actual overlap under the ground-truth transformation.
- KITTI: On KITTI, Relative Rotation Error is angular rotation distance, Relative Translation Error is Euclidean translation distance, and Registration Recall requires RRE<5° and RTE<2m.
C. Analysis of Cross-Entropy Loss
The analysis shows that cross-entropy suppresses high-confidence positive classes in multi-label matching, motivating the overlap-aware circle loss used for superpoint correspondence supervision.
- Cross-Entropy Analysis: In multi-label classification, each positive class has zero-gradient confidence gi/Σgj, which is below one when multiple positives exist.A positive class is increased below this point and reduced above it.
- Cross-Entropy Analysis: Cross-entropy therefore suppresses positive classes with higher confidence scores during training.
- Superpoint Matching: Superpoint patches commonly overlap multiple patches, making their matching a multi-class problem rather than a single-label classification task.Ground-truth correspondences are determined by neighboring patch overlap.
- Superpoint Matching: CoFiNet’s cross-entropy supervision can hinder reliable superpoint correspondences, even with gradient reweighting.GeoTransformer instead uses an overlap-aware circle loss in a metric-learning formulation.
D.1. Detailed Results on 3DMatch
Experiments across overlap conditions, scenes, and robust-estimator comparisons show that GeoTransformer consistently improves registration performance, including on challenging low-overlap data.
- Different Overlaps: GeoTransformer outperforms vanilla self-attention on all metrics across overlap ratios, with larger gains below 30% overlap.This demonstrates stronger robustness in low-overlap scenarios.
- Scene-wise Results: GeoTransformer surpasses baselines by a large margin in registration recall across all 3DLoMatch scenes.It also performs especially well on difficult 3DMatch scenes such as Home 2 and Lab.
- Scene-wise Results: GeoTransformer achieves consistently superior rotation and translation errors in the scene-wise comparisons.
- Deep Robust Estimators: GeoTransformer outperforms deep robust estimators on both 3DMatch and KITTI.Its parameter-free LGR registers in 0.013s versus 0.08s for PointDSC in the reported experiments.
D.3. Additional Ablation Studies
Ablations find that geometric self-attention is transformation-invariant, while selected design choices provide modest accuracy gains or speed–accuracy trade-offs in registration.
- Transformation Invariance: Geometric self-attention remains strongly invariant to arbitrary rigid transformations, unlike absolute coordinate embeddings whose performance degrades under enlarged rotations.Relative coordinate embeddings remain stable but contribute little when masked during testing.
- Geometric Structure Embedding: Increasing angular-embedding neighbors slightly improves registration recall but increases computation, leading to the choice k=3.
- Geometric Structure Embedding: Max pooling slightly outperforms average pooling when aggregating triplet-wise angular embeddings, although their results are very close.
- Dual-Normalization: Dual-normalization slightly improves correspondence accuracy in low-overlap scenarios by mitigating outlier matches from less discriminative patches.
- Pose Refinement: Registration recall consistently improves with more pose-refinement iterations and saturates after about five iterations.The experiments therefore use five iterations to balance accuracy and speed.
E. Limitations
GeoTransformer’s uniformly downsampled-superpoint hierarchy can become costly for large-area point clouds and is inflexible for patch sampling.
- Computational Scope: Large-area point clouds may produce many uniformly downsampled superpoints, causing substantial memory use and computational cost.The downsampling rate must be selected carefully to balance performance and efficiency.
- Sampling Scope: Uniform superpoint sampling is inflexible when one object is split across multiple patches.The paper identifies integrating registration with semantic scene understanding as future work.
F. Qualitative Results
GeoTransformer performs well in low-overlap qualitative cases, including distinguishing similar objects at different positions. Failures occur when overlap is small and geometrically indiscriminative or when nonoverlapping regions are complicated.
- Qualitative Results: GeoTransformer performs well on low-overlap 3DLoMatch cases in qualitative comparisons with Predator and CoFiNet.Predator and CoFiNet use RANSAC-50k, whereas GeoTransformer uses LGR.
- Qualitative Results: GeoTransformer distinguishes similar objects at different positions in some low-overlap cases.The paper attributes this to transformation-invariant features.
- Qualitative Results: Registration can fail when the overlap is small and geometrically indiscriminative, such as wall, ceiling, or floor regions.These regions do not provide adequate geometric cues to detect overlap and extract reliable correspondences.
- Qualitative Results: Registration can also fail when the nonoverlapping region is relatively complicated.The paper identifies combining information from multiple point clouds as a possible future solution.