Source-linked AI summary
DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration
Weixin Lu, Guowei Wan, Yao Zhou, Xiangyu Fu, Pengfei Yuan, Shiyu Song
TL;DR
LiDAR point cloud registration remains difficult because sparsity, dynamic objects, and scene variety complicate robust correspondence and transformation estimation. DeepICP learns an end-to-end registration pipeline with salient keypoints, generated correspondences, and locally and globally constrained training, achieving comparable or better performance than state-of-the-art geometric methods on KITTI and Apollo-SouthBay.
Problem
LiDAR registration must align sparse, large-scale, noisy point clouds, while a universal robust solution remains unresolved.
Method
DeepICP uses an end-to-end network that learns keypoint selection, generates corresponding points from candidate matching probabilities, and combines local similarity with global geometric constraints.
Results
DeepICP achieves comparable or better performance than state-of-the-art geometry-based methods, validated on KITTI and Apollo-SouthBay.
Takeaways & Limitations
The reported low registration error and high robustness make DeepICP attractive for applications relying on point cloud registration.
Takeaways & Limitations
The method assumes semantic features and weighting can avoid dynamic-object interference and focus on stable, unique registration features.
Abstract
from arXiv · showhide
We present DeepICP - a novel end-to-end learning-based 3D point cloud registration framework that achieves comparable registration accuracy to prior state-of-the-art geometric methods. Different from other keypoint based methods where a RANSAC procedure is usually needed, we implement the use of various deep neural network structures to establish an end-to-end trainable network. Our keypoint detector is trained through this end-to-end structure and enables the system to avoid the inference of dynamic objects, leverages the help of sufficiently salient features on stationary objects, and as a result, achieves high robustness. Rather than searching the corresponding points among existing points, the key contribution is that we innovatively generate them based on learned matching probabilities among a group of candidates, which can boost the registration accuracy. Our loss function incorporates both the local similarity and the global geometric constraints to ensure all above network designs can converge towards the right direction. We comprehensively validate the effectiveness of our approach using both the KITTI dataset and the Apollo-SouthBay dataset. Results demonstrate that our method achieves comparable or better performance than the state-of-the-art geometry-based methods. Detailed ablation and visualization analysis are included to further illustrate the behavior and insights of our network. The low registration error and high robustness of our method makes it attractive for substantial applications relying on the point cloud registration task.
1. Introduction
Point cloud registration aligns LiDAR scans, but sparsity, scale, dynamic objects, and scene variety make universal solutions difficult. DeepICP addresses these challenges with an end-to-end learned pipeline that selects keypoints, generates correspondences, and estimates transformations.
- Point cloud registration estimates relative transformations between LiDAR point clouds for applications including SLAM, reconstruction, mapping, positioning, and localization.
- Local sparsity, large data volume, dynamic-object noise, and scene variety make robust and efficient LiDAR registration challenging.
- Traditional registration pipelines combine keypoint detection, descriptor extraction, feature matching, outlier rejection, and transformation estimation, yet universal solutions remain unresolved.
- DeepICP is an end-to-end learning-based method whose inference uses one iteration while training is similar to ICP.
- The framework extracts semantic features, learns registration-relevant keypoint weights, generates corresponding points, and combines local and global geometric constraints in its loss.
- DeepICP is evaluated on KITTI and Apollo-SouthBay, with reported registration accuracy comparable to state-of-the-art geometric methods and detailed ablations.
2. Related Work
Prior learning-based work often replaces individual registration components or targets recognition and relocalization rather than accurate local matching. DeepICP instead processes unordered point clouds directly as a general registration solution.
- Earlier learning-based approaches replace individual components of the classic point cloud registration pipeline, including keypoint detection and representation.
- Learned 3D descriptors have targeted scene recognition and relocalization, where accurate local matching is not the primary goal.
- ICP remains necessary for registration refinement in approaches that do not directly obtain accurate local matching results.
- Some methods encode LiDAR data into specialized 2D representations and infer 6-DOF poses, whereas DeepICP processes the original unordered point cloud directly.
3. Method
DeepICP is an end-to-end registration network that learns salient source keypoints, generates target correspondences from candidate probabilities, and jointly optimizes local and global geometric objectives.
- 3.1. Deep Feature Extraction: PointNet++ extracts local features, and a learned weighting layer selects source keypoints intended to emphasize stable, distinctive features while avoiding dynamic objects.The weighting layer assigns learned saliency weights and retains the most significant N points.
- 3.3. Deep Feature Embedding: A mini-PointNet deep feature embedding layer describes each keypoint neighborhood using local geometry, reflectance, and extracted point features.Neighborhood points are gathered within radius d and aggregated into a 32-dimensional descriptor.
- 3.4. Corresponding Point Generation: Instead of selecting existing target points, DeepICP generates correspondences by applying learned similarity probabilities to candidate points arranged around transformed source keypoints.The candidates are voxel centers, and the generated correspondence is their probability-weighted sum.
- 3.4. Corresponding Point Generation: The network estimates transformation from corresponding keypoint pairs with one SVD optimization iteration, avoiding the iterative matching and RANSAC steps used by conventional alternatives.The single-iteration design remains trainable because SVD backpropagation is supported.
- 3.5. Loss: DeepICP combines a local correspondence loss with a global geometric loss so matching is constrained both by individual point similarity and the shared transformation.Using only the local loss would make keypoint matching independent and omit global geometric constraints.
4. Implementation Details
The implementation uses a simplified PointNet++ feature extractor, supervised augmentation and training, bidirectional inference matching, and dataset-specific handling for Velodyne HDL64 scans.
- Feature extraction: The feature extractor uses simplified PointNet++ layers with specified subsampling, hierarchical MLP, upsampling, fully connected, and dropout settings.The dropout layer uses a keeping probability of 0.7.
- Training: Training uses batch size 1, learning rate 0.01, decay rate 0.7, and uniformly sampled pose and coordinate noise for supervised augmentation.The injected noise spans [0.0 ∼ 1.0]m in x, y, z and [0 ∼ 1.0]° in roll, yaw, and pitch.
- Training: The dataset is split into training and validation sets at a 4:1 ratio, with training stopped at 200 epochs when performance no longer improves.The stopping rule is based on the absence of performance gain.
- Inference: Bidirectional matching is used during inference by treating each point cloud as source and target, although it is omitted during training because it does not improve overall performance.This strategy is intended to improve registration accuracy at inference time.
- Dataset scope: The settings target KITTI and Apollo-SouthBay scans collected with Velodyne HDL64, whose narrow z-distribution reduces precision in z, roll, and pitch.The limitation arises because points constraining z differ from those constraining the other directions.
5. Experiments
Experiments on KITTI and Apollo-SouthBay compare DeepICP with geometric and learning-based baselines, then examine runtime, ablations, and visualizations. DeepICP achieves competitive accuracy and robustness, while its learned designs explain keypoint selection, matching, and loss behavior.
- Datasets and setup: KITTI and Apollo-SouthBay provide separate training and testing evaluations, with no overlap between training and testing data.KITTI uses sequences 00–07 for training and 08–10 for testing; Apollo-SouthBay follows a similar split.
- Evaluation: DeepICP is compared with ICP-family, NDT-P2D, CPD, and 3DFeat-Net methods using angular and translational transformation errors.Angular error uses rotation-matrix chordal distance, while translational error is Euclidean distance.
- Performance: DeepICP achieves comparable accuracy to AA-ICP and NDT-P2D on KITTI but performs slightly worse than G-ICP and ICP, especially for angular error.Lower maximum angular and translational errors indicate robustness and stability in the reported evaluation.
- Performance: On Apollo-SouthBay, the duplication design achieves the second-best mean translational accuracy, comparable angular accuracy, and the lowest maximum translational error.The authors associate broad performance improvement with the dataset’s more accurate ground-truth poses.
- Runtime: 2 seconds is DeepICP’s reported end-to-end inference time for registering a frame pair with the duplication design.The network is faster than 3DFeat-Net because it extracts 64 rather than 1024 keypoints and does not use RANSAC.
- Ablations: The combined loss is necessary because α values of 0.0 and 1.0 produce larger mean angular and translational errors, whereas α between 0.1 and 0.9 yields similar accuracy.The experiments use α = 0.6.
- Visualizations: The point weighting layer places front-network keypoints mainly on salient stationary objects and back-network keypoints mostly on the ground, avoiding dynamic objects.Visualizations report no detected keypoints on cars or buses in scenes containing many such objects.
- Visualizations: The CPG layer visualizes matching similarity probabilities between detected source keypoints and generated target candidates before and after registration.The visualization uses x and y dimensions with nine fixed z values, distinguishing source and generated target points.
6. Conclusion
The paper presents an end-to-end point cloud registration framework combining learned keypoint detection, generated correspondences, and locally and globally constrained training. These designs achieve registration accuracy comparable to state-of-the-art geometric methods.
- The framework detects keypoints through a point weighting deep neural network.
- Corresponding points are generated from matching probabilities among candidates rather than directly selected from existing points.
- The loss function combines local similarity with global geometric constraints.
- The resulting learning-based system achieves registration accuracy comparable to state-of-the-art geometric methods.