Source-linked AI summary
GeoTransformer: Fast and Robust Point Cloud Registration with Geometric Transformer
Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, Slobodan Ilic, Dewen Hu, Kai Xu
TL;DR
Point-cloud registration needs accurate correspondences despite difficult keypoint detection and sparse, loose superpoint matching in low-overlap cases. GeoTransformer learns transformation-invariant geometric features from pair-wise distances and triplet-wise angles, yielding robust matching and fast registration. Across indoor, outdoor, synthetic, multiway, and non-rigid benchmarks, it improves challenging-case performance and enables RANSAC-free estimation.
Problem
Accurate correspondence extraction is difficult because keypoint detection is challenging at low overlap, while superpoint matching requires features that capture geometric structure.
Method
GeoTransformer combines geometric self-attention and feature-based cross-attention while encoding pair-wise distances and triplet-wise angles for transformation-invariant superpoint matching.
Results
Across indoor, outdoor, synthetic, multiway, and non-rigid benchmarks, GeoTransformer improves the inlier ratio by 18∼31 percentage points and registration recall by over 7 points on 3DLoMatch.
Takeaways & Limitations
Reliable correspondences support accurate registration without RANSAC and reduce registration computation time by more than 100 times.
Takeaways & Limitations
Uniformly downsampled superpoints can create large memory and computational costs and may inflexibly split a single object across multiple patches.
Abstract
from arXiv · showhide
We study the problem of extracting accurate correspondences for point cloud registration. Recent keypoint-free methods have shown great potential through bypassing the detection of repeatable keypoints which is difficult to do especially in low-overlap scenarios. They seek correspondences over downsampled superpoints, which are then propagated to dense points. Superpoints are matched based on whether their neighboring patches overlap. Such sparse and loose matching requires contextual features capturing the geometric structure of the point clouds. We propose Geometric Transformer, or GeoTransformer for short, to learn geometric feature for robust superpoint matching. It encodes pair-wise distances and triplet-wise angles, making it invariant to rigid transformation and robust in low-overlap cases. The simplistic design attains surprisingly high matching accuracy such that no RANSAC is required in the estimation of alignment transformation, leading to $100$ times acceleration. Extensive experiments on rich benchmarks encompassing indoor, outdoor, synthetic, multiway and non-rigid demonstrate the efficacy of GeoTransformer. Notably, our method improves the inlier ratio by $18{\sim}31$ percentage points and the registration recall by over $7$ points on the challenging 3DLoMatch benchmark. Our code and models are available at \url{https://github.com/qinzheng93/GeoTransformer}.
1 INTRODUCTION
GeoTransformer addresses unreliable correspondence extraction in low-overlap point-cloud registration by learning transformation-invariant geometric features for robust superpoint matching. Its accurate correspondences enable fast, RANSAC-free registration across diverse benchmarks.
- Keypoint detection is difficult across partially overlapping point clouds, especially with small overlap, producing correspondences with low inlier ratios.
- Keypoint-free methods match downsampled superpoints by patch overlap and propagate those matches to dense points, making superpoint accuracy critical.
- Superpoint matching relaxes repeatability requirements but requires contextual features that capture global geometric structure.
- GeoTransformer encodes pair-wise distances and triplet-wise angles to learn transformation-invariant geometric features for robust matching.
- 100 times faster registration is achieved without RANSAC, while inlier ratio improves by 18∼31 percentage points and registration recall by over 7 points on 3DLoMatch.The method uses high-quality superpoint matches, an optimal transport layer for dense correspondences, and RANSAC-free registration.
- The paper contributes a keypoint-free, RANSAC-free method, a geometric transformer architecture, and an overlap-aware circle loss.The loss reweights superpoint matches according to patch overlap ratio.
2 RELATED WORK
Correspondence-based registration methods first extract point correspondences and then estimate an alignment transformation, commonly using robust pose estimators. They include keypoint-based approaches and detection-free methods that consider possible matches without keypoint detection.
- Correspondence-based methods recover the transformation from extracted point correspondences using robust pose estimators such as RANSAC.
- Keypoint-based methods detect repeatable keypoints and learn descriptors, whereas detection-free methods retrieve correspondences without keypoint detection.
3 METHOD
GeoTransformer uses a hierarchical coarse-to-fine pipeline that matches superpoints with transformation-invariant geometric and cross-cloud features, propagates matches to dense points, and estimates alignment locally to globally.
- 3.1 Superpoint Sampling and Feature Extraction: The method downsamples point clouds with KPConv-FPN, treating coarsest points as superpoints and an earlier resolution as dense points.This reduces redundant point-wise matching while retaining the correspondences needed for registration.
- 3.1 Superpoint Sampling and Feature Extraction: Each dense point is assigned to its nearest superpoint, producing Voronoi-like local patches for subsequent correspondence refinement.Superpoints with empty patches are removed.
- 3.2 Superpoint Matching Module: GeoTransformer interleaves geometric self-attention and feature-based cross-attention to encode intra-cloud structure and inter-cloud geometric consistency.The resulting hybrid features support reliable superpoint matching.
- 3.2 Superpoint Matching Module: Geometric structure embeddings combine pair-wise distances and triplet-wise angles, which remain invariant under rigid transformation.Distance embeddings use sinusoidal encoding, while angular embeddings are computed from neighboring superpoints.
- 3.4 RANSAC-free Local-to-Global Registration: The method propagates superpoint matches to dense correspondences and achieves accurate low-overlap registration with fewer correspondences without RANSAC.A local-to-global registration procedure obtains comparable accuracy to RANSAC while reducing computation time by more than 100 times.
4 EXPERIMENTS
Experiments evaluate GeoTransformer across indoor, outdoor, synthetic, multiway, and non-rigid registration benchmarks, using benchmark-specific backbones and shared matching settings.
- Experiments cover indoor 3DMatch and 3DLoMatch, outdoor KITTI, synthetic ModelNet40, multiway Augmented ICL-NUIM, and non-rigid benchmarks.
- GeoTransformer uses four-stage backbones for 3DMatch, ModelNet40, and 4DMatch, and a five-stage backbone for larger KITTI point clouds.
- Geometric and feature-based attention modules are interleaved three times across all benchmarks, with four attention heads per module.
- Training lasts 40 epochs on 3DMatch/4DMatch, 200 on ModelNet40, and 80 on KITTI, using Adam on an RTX 3090 GPU.
4.2 Indoor Benchmark: 3DMatch & 3DLoMatch
On indoor 3DMatch and especially low-overlap 3DLoMatch, GeoTransformer produces more reliable correspondences and accurate registration, including without RANSAC.
- 18–31 pp higher Inlier Ratio on 3DLoMatch and 8–33 pp higher on 3DMatch show stronger correspondence reliability than the baselines.The gain is larger with fewer correspondences.
- 7.9 pp higher Registration Recall on 3DLoMatch and 1.5 pp higher on 3DMatch establish new state-of-the-art results with 50K RANSAC iterations.
- Geometric self-attention recognizes small overlap regions and distinguishes similar objects at different positions in complex 3DLoMatch scenes.The visualized attention patterns are consistent between matching anchor patches, even across disjoint overlap areas.
- Without RANSAC, weighted SVD gives registration recall of 86.7% on 3DMatch and 60.5% on 3DLoMatch.These results are close to Predator with RANSAC, while the baselines degrade severely or fail to achieve reasonable results.
- LGR raises registration recall to 91.8% on 3DMatch and 74.5% on 3DLoMatch, surpassing RANSAC-based baselines while gaining over 100× acceleration in pose time.The lite model produces similar results, reduces overall time by 17%, and runs at 13 fps.
4.3 Outdoor Benchmark: KITTI odometry
On KITTI odometry, GeoTransformer outperforms both RANSAC-free and RANSAC-based methods, with LGR achieving the strongest reported comparison.
- GeoTransformer outperforms FMR, DGR, and HRegNet by a large margin among RANSAC-free methods on KITTI.
- LGR beats all RANSAC-based methods on the KITTI odometry benchmark.The authors identify GeoTransformer as the first RANSAC-free method surpassing RANSAC-based methods on this benchmark.
4.4 Synthetic Benchmark: ModelNet40
On synthetic ModelNet40, GeoTransformer is comparable in easy high-overlap settings and is substantially stronger under low overlap, large rotations, or heavy perturbations.
- On high-overlap ModelNet40 with small rotations, RPM-Net, Predator, and GeoTransformer achieve comparable results as performance tends to saturate.
- GeoTransformer surpasses other methods by a large margin in the low-overlap setting.
- With large rotations, GeoTransformer performs significantly better than baselines on both high- and low-overlap settings.The authors attribute this robustness to geometric self-attention providing more point-cloud structure information.
- Correspondence-based methods perform better than end-to-end methods under large perturbations or heavy occlusion, while robust estimators further improve stability.
4.5 Multiway Benchmark: Augmented ICL-NUIM
On Augmented ICL-NUIM, GeoTransformer uses memory-efficient superpoint processing and achieves the best performance on nearly all testing trajectories.
- Multiway registration: Multiway registration first performs pair-wise GeoTransformer registration, then optimizes poses with global pose graph optimization.
- Multiway registration: GeoTransformer achieves better per-frame ATE on most frames across the four Augmented ICL-NUIM trajectories.
- Multiway registration: GeoTransformer downsamples superpoints and interpolates features before applying the geometric transformer, reducing memory usage without sacrificing performance.
- Multiway registration: GeoTransformer attains the best performance on all testing trajectories except Office1, indicating strong generality to unknown scenes and complex applications.
4.6 Non-rigid Benchmark: 4DMatch & 4DLoMatch
On non-rigid 4DMatch benchmarks, GeoTransformer provides dense, high-inlier-ratio correspondences, with especially strong performance in low-overlap 4DLoMatch cases.
- Benchmark setup: The 4DMatch benchmark evaluates non-rigid registration using Non-rigid Inlier Ratio and Non-rigid Feature Matching Recall.
- Correspondence quality: GeoTransformer extracts denser correspondences, enabling more precise deformation description, and achieves higher inlier ratios despite significant deformations.
- Evaluation results: GeoTransformer surpasses D3Feat and Predator by a large margin on both high- and low-overlap 4DMatch scenarios.
- Evaluation results: GeoTransformer achieves performance close to Lepard on 4DMatch and a significantly better inlier ratio on 4DLoMatch.
- Evaluation results: Higher inlier ratios in low-overlap cases benefit subsequent non-rigid registration algorithms because few effective outlier-rejection methods exist.
4.7 Ablation Studies
Ablations show that geometric information, overlap-aware supervision, robust angular aggregation, and pose refinement each improve GeoTransformer’s registration behavior under challenging conditions.
- Geometric self-attention: Geometric information generally boosts performance, while coordinate-based embeddings provide limited gains because they vary under transformation.
- Geometric self-attention: Geometric self-attention finds inlier matches using relationships to salient regions and rejects outliers between feature-similar but spatially different patches.
- Geometric structure embedding: Geometric structure embedding with both distance and angular embeddings outperforms distance-only embedding by a large margin.
- Overlap-aware circle loss: Overlap-aware circle loss beats vanilla circle loss by a large margin on all evaluated metrics.
- Geometric structure embedding: Increasing angular-embedding neighbors slightly improves performance but increases computation, leading to the default choice k=3.
- Geometric structure embedding: Max pooling outperforms average pooling for angular embedding aggregation because it is more robust to neighbors missing under viewpoint changes.
- Pose refinement: Pose refinement consistently improves registration results and saturates after 5 iterations.
4.8 Comparison with Deep Robust Estimators
GeoTransformer outperforms several deep robust estimators on 3DMatch and KITTI, while its parameter-free LGR registration is substantially faster than PointDSC.
- Robust estimator comparison: GeoTransformer outperforms 3DRegNet, DGR, PointDSC, DHVR, and PCAM by a large margin on both 3DMatch and KITTI benchmarks.
- Registration speed: 0.013s of LGR versus 0.08s of PointDSC demonstrates faster registration in the reported experiments.
- Registration speed: LGR is parameter-free and does not require training a task-specific network.
5 CONCLUSION
GeoTransformer learns robust coarse-to-fine correspondences through geometrically consistent, transformation-invariant features, enabling fast and accurate RANSAC-free registration. Its limitations concern scalability and the inflexibility of uniformly sampled superpoints.
- GeoTransformer encodes pair-wise distances and triplet-wise angles among superpoints to capture geometric consistency with transformation invariance.
- Reliable correspondences enable fast and accurate registration without RANSAC.
- Extensive experiments on five challenging benchmarks demonstrate GeoTransformer’s efficacy.
- Limitations: Numerous superpoints in large-area inputs can cause huge memory footprints and computational costs.The downsampling rate may need careful selection to balance performance and efficiency.
- Limitations: Uniformly sampled superpoints can decompose a single object into multiple patches, motivating integration with semantic scene understanding.