Source-linked AI summary
HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration
Fan Lu, Guang Chen, Yinlong Liu, Lijun Zhang, Sanqing Qu, Shu Liu, Rongqi Gu
TL;DR
Large-scale outdoor LiDAR point cloud registration is challenging, and existing learning-based methods are often unreliable or time-consuming in this setting. HRegNet registers hierarchically extracted keypoints, combining deeper-layer features with shallower-layer position information and consensus-based correspondence features. The authors report that it significantly outperforms existing methods in accuracy and efficiency on KITTI and NuScenes.
Problem
Existing learning-based registration methods are often unreliable or time-consuming for sparse, large-scale, variably distributed outdoor LiDAR point clouds.
Source: §1 · pp. 1–2Method
HRegNet hierarchically registers keypoints and descriptors, combining coarse matching from deeper-layer features with finer positional refinement and consensus-based correspondence features.
Source: §1 · p. 2, §3 · p. 3, §3.2.1 · p. 3Results
HRegNet significantly outperforms existing methods in accuracy and efficiency on the KITTI odometry and NuScenes datasets.
Source: §1 · p. 2Takeaways & Limitations
The hierarchical approach uses a small number of keypoints and is described as highly efficient for applications requiring real-time performance.
Source: §5 · p. 8Fine registration assumes the coarse transformation is correct but not accurate enough, so it refines that estimate in upper layers.
Source: §3.3 · p. 5
Abstract
from arXiv · showhide
Point cloud registration is a fundamental problem in 3D computer vision. Outdoor LiDAR point clouds are typically large-scale and complexly distributed, which makes the registration challenging. In this paper, we propose an efficient hierarchical network named HRegNet for large-scale outdoor LiDAR point cloud registration. Instead of using all points in the point clouds, HRegNet performs registration on hierarchically extracted keypoints and descriptors. The overall framework combines the reliable features in deeper layer and the precise position information in shallower layers to achieve robust and precise registration. We present a correspondence network to generate correct and accurate keypoints correspondences. Moreover, bilateral consensus and neighborhood consensus are introduced for keypoints matching and novel similarity features are designed to incorporate them into the correspondence network, which significantly improves the registration performance. Besides, the whole network is also highly efficient since only a small number of keypoints are used for registration. Extensive experiments are conducted on two large-scale outdoor LiDAR point cloud datasets to demonstrate the high accuracy and efficiency of the proposed HRegNet. The project website is https://ispc-group.github.io/hregnet.
1. Introduction
Outdoor LiDAR registration is difficult because these point clouds are sparse, large, and variable, while existing learning-based approaches are often unreliable or too slow for the setting. HRegNet uses hierarchical keypoints and a correspondence network, and is reported to outperform existing methods in accuracy and efficiency on two outdoor LiDAR datasets.
1. Introduction
- Point cloud registration estimates the optimal rigid transformation between two point clouds and is important in applications including robotics and autonomous driving..Source: §1 · p. 1
- ICP depends strongly on an initial transformation guess and can reach a local minimum, while global-optimal variants are typically time-consuming on large point clouds.The stated local-minimum issue is attributed to the problem's non-convexity.Source: §1 · p. 1
- Outdoor LiDAR registration is challenging because its point clouds are sparse, large-scale, and variably distributed, making existing learning-based methods unreliable or time-consuming.The cited learning-based methods are mostly designed for object-level or indoor point clouds.Source: §1 · pp. 1–2
- HRegNet hierarchically downsamples point clouds into keypoints and descriptors, using deeper-layer features for coarse matching and shallower-layer positions to refine the transformation.The authors state that using only a small number of keypoints makes the network efficient.Source: Figure 1 · p. 1, §1 · p. 2
- A correspondence network uses bilateral and neighborhood consensus through novel similarity features to improve keypoint matching and registration performance.Bilateral consensus requires mutual nearest neighbors in descriptor space; neighborhood consensus requires similar spatial neighborhoods.Source: §1 · p. 2, Figure 2 · p. 2
- HRegNet significantly outperforms existing methods in accuracy and efficiency on the KITTI odometry and NuScenes datasets.The authors describe experiments on both large-scale outdoor LiDAR datasets.Source: §1 · p. 2
2. Related works
Related work covers classical registration and learning-based methods, including approaches for object-level, indoor, and LiDAR point clouds. The cited methods use differing strategies for matching or estimating transformations, with several limitations noted for large-scale outdoor or nonlocal registration.
2. Related works
- Classical approaches include ICP, global-search variants such as Go-ICP, handcrafted features such as FPFH, and RANSAC for robust correspondence matching.ICP can get stuck in a local minimum; Go-ICP uses Branch-and-Bound to search for a global optimum.Source: §2 · p. 2
- Learning-based object-level methods include PointNetLK, DCP, and IDAM, but the cited methods are not applicable to complex large-scale LiDAR point clouds.PointNetLK combines PointNet with Lucas–Kanade; DCP uses a transformer for soft matching and differentiable SVD for transformation estimation.Source: §2 · pp. 2–3
- Indoor learning-based methods include direct transformation regression, feature-metric registration, multiview registration, and DGR, which uses learned features and a 6D convolutional network to score correspondences.The passage reports DGR as state of the art for indoor point cloud registration.Source: §2 · p. 3
- DeepVCP targets LiDAR registration with virtual-point correspondences, but its keypoint matching is local in 3D coordinates, limiting it primarily to local registration..Source: §2 · p. 3
3. Methodology
HRegNet registers point clouds through a three-layer coarse-to-fine hierarchy, combining reliable deeper-layer features with precise positions from upper layers. A learned correspondence network and similarity features support matching, while fine registration refines the transformation using spatial neighborhoods.
3. Methodology
- Three-layer registration first matches bottom-layer keypoints globally, then refines the transformation through local matching in middle and top layers.The coarse transformation is applied to upper-layer keypoints before fine registration produces the final estimate.Source: §3 · p. 3
3.1. Feature extraction
- Each feature extraction module uses weighted farthest-point sampling, neighborhood clusters, and learned keypoint refinement, uncertainty, and descriptor extraction.The module predicts attentive weights to refine candidate keypoint locations; its detailed network structure is in the supplementary material.Source: §3.1 · p. 3
3.2. Coarse registration
- Because descriptor errors can make nearest-neighbor matching produce mismatches, HRegNet uses a learned correspondence network for bottom-layer coarse registration.The coarse stage matches keypoints in the bottom layer.Source: §3.2 · p. 3
3.2.1 Correspondence network
- For each source keypoint, the correspondence network scores a cluster of candidate matches using geometric, descriptor, and similarity features, predicting a corresponding keypoint and confidence.Predicted correspondences and confidence scores are used to calculate the transformation with a weighted Kabsch algorithm.Source: §3.2.1 · pp. 3–4, Figure 3 · p. 4
3.2.2 Similarity features
- Similarity features combine descriptor-based and neighbor-aware similarities to incorporate bilateral and neighborhood consensus into matching.Neighbor-aware descriptors aggregate neighboring keypoint information, allowing their similarities to encode similarities between neighborhoods.Source: §3.2.2 · pp. 4–5, Figure 4 · p. 4
3.3. Fine registration
- Fine registration assumes coarse alignment is approximately correct, searches locally in spatial neighborhoods, and refines the transformation at middle and top layers.Similarity features are dropped in these upper-layer clusters due to their computational complexity with more keypoints.Source: §3.3 · p. 5
3.4. Loss function
- The training loss combines translation and rotation losses, with the rotation term weighted by α.The losses compare estimated and ground-truth transformations.Source: §3.4 · p. 5
4. Experiments
Experiments on KITTI and NuScenes compare HRegNet with classical and learning-based methods, reporting stronger registration performance and efficiency. Ablations examine hierarchical refinement and similarity features on KITTI.
4.1. Experiment settings
- Experiments compare HRegNet with classical and learning-based registration methods on KITTI and NuScenes, using KITTI sequence and NuScenes scene splits and paired point clouds.Inputs are voxelized at 0.3 m; HRegNet samples 16,384 KITTI and 8,192 NuScenes points, while DCP and IDAM use 4,096 points on both datasets. Learning-based baselines are retrained on both datasets.Source: §4.1 · pp. 5–6
4.2. Evaluation
- HRegNet outperforms all baselines on both datasets; on KITTI it runs at almost 1/5 of RANSAC’s runtime and is almost 15× faster than DGR.The reported KITTI RTE is almost 1/3 of DGR’s; average RTE and RRE are calculated only for successful registrations, using thresholds of 2m and 5deg.Source: §4.2 · p. 7, Figure 6 · p. 7, Table 1 · p. 7
4.3. Ablation study
- Layer-by-layer refinement reduces average RTE and RRE, while adding layers beyond the selected three brings no significant registration gain and reduces efficiency.Layer 2 improves rotation error over Layer 3, and Layer 1 improves translation accuracy over Layer 2.Source: §4.3 · p. 8, Figure 7 · p. 8, Table 2 · p. 8
- The full model’s original and neighbor-aware similarity features are complementary, and their combination outperforms the ablated cases on KITTI.Removing both features causes a significant drop in registration recall; neighbor-aware features incorporate neighboring keypoints but may overlook a keypoint’s unique features.Source: §4.3 · p. 8
5. Conclusion
HRegNet combines hierarchical coarse-to-fine registration with a correspondence network incorporating bilateral and neighborhood consensus. The authors report that it is accurate and efficient on two large-scale LiDAR datasets.
5. Conclusion
- Experiments on two large-scale LiDAR datasets demonstrate HRegNet’s high accuracy and efficiency.The conclusion names both datasets collectively, without giving specific metric values.Source: §5 · p. 8
- The hierarchical paradigm uses different keypoint and descriptor characteristics across layers for coarse and fine registration.The conclusion distinguishes deeper and shallower layers but does not specify their individual roles further here.Source: §5 · p. 8
- A correspondence network generates keypoint matches using similarity features that incorporate bilateral and neighborhood consensus.Source: §5 · p. 8
- Ablation studies support the effectiveness of the hierarchical paradigm and similarity features, while using few keypoints contributes to efficiency.Source: §5 · p. 8