Source-linked AI summary
Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences
Xiaoshui Huang, Guofeng Mei, Jian Zhang
TL;DR
Point cloud registration is challenged by correspondence dependence and the need for extensive labels. The paper proposes feature-metric optimization without correspondences, trained semi-supervisedly or unsupervisedly, and reports robust, accurate, and fast performance across challenging conditions. A stated limitation is that conventional direct Jacobian computation does not transfer to unordered 3D point clouds because they lack grid structure.
Problem
Existing registration methods rely on geometric projection error and point correspondences, which are impaired by noise, outliers, density differences, and costly label requirements.
Method
The framework minimizes feature-metric projection error without correspondence, using an encoder, an unsupervised encoder-decoder branch, and a feature-metric registration branch.
Results
Experiments report better accuracy than classical, feature-learning, and deep-learning registration methods, with robustness to noise, density difference, and partial overlap.
Takeaways & Limitations
The framework supports robust, accurate, and fast registration trained in an unsupervised manner, including challenging same-source and cross-source settings.
Takeaways & Limitations
Direct Jacobian computation is not applicable to unordered 3D point clouds because they lack grid structure for computing spatial gradients.
Abstract
from arXiv · showhide
We present a fast feature-metric point cloud registration framework, which enforces the optimisation of registration by minimising a feature-metric projection error without correspondences. The advantage of the feature-metric projection error is robust to noise, outliers and density difference in contrast to the geometric projection error. Besides, minimising the feature-metric projection error does not need to search the correspondences so that the optimisation speed is fast. The principle behind the proposed method is that the feature difference is smallest if point clouds are aligned very well. We train the proposed method in a semi-supervised or unsupervised approach, which requires limited or no registration label data. Experiments demonstrate our method obtains higher accuracy and robustness than the state-of-the-art methods. Besides, experimental results show that the proposed method can handle significant noise and density difference, and solve both same-source and cross-source point cloud registration.
1. Introduction
Point cloud registration aligns scans into one coordinate system, but correspondence-dependent geometric optimization is vulnerable to missing matches, noise, outliers, and density differences. The proposed framework replaces correspondence search with feature-metric registration and supports semi-supervised or unsupervised training.
- Point cloud registration transforms scans of the same 3D scene or object into one coordinate system for tasks including robotics vision and augmented reality.
- Existing methods alternate correspondence searching and transformation estimation while minimizing geometric projection error.
- Missing point-point correspondences can prevent the best geometric alignment, while soft correspondence requires difficult weight selection.
- Noise, outliers, and density differences impair point-point correspondences and make geometric registration more challenging.
- A semi-supervised approach trains distinctive registration features with limited or no registration-label data.
- The proposed framework minimizes feature-space projection error without correspondence, using an encoder, unsupervised encoder-decoder branch, and feature-metric registration branch.The feature difference is intended to be smallest when the point clouds are well aligned.
2. Related works
Prior registration methods use optimization, learned descriptors, or end-to-end regression, but the proposed work combines classical nonlinear optimization with deep learning to estimate transformations without correspondences.
- Classical registration methods typically use two stages: correspondence searching followed by transformation estimation.
- Descriptor-based methods learn features, select point correspondences, and use RANSAC to reject outliers before solving registration.
- End-to-end learning methods treat registration as regression by learning point-cloud features and predicting transformation parameters.
- The proposed framework combines classical nonlinear optimization with deep learning rather than replacing registration with a black-box neural network.
- Unlike conventional methods that minimize geometric projection error, the approach minimizes feature-metric projection error.
3. Feature-metric Point Cloud Registration
This section introduces the problem formulation and framework overview, then describes the encoder, multi-task learning streams, and loss functions.
- The section first describes the registration problem formulation and framework overview.
- It then explains the Encoder module in detail.
- The section presents how multi-task streams learn distinctive features and solve registration.
- Finally, it details the loss functions.
3.1. Problem Formulation
Registration seeks a rigid transformation that aligns one point cloud to another by minimizing feature-space error, avoiding correspondence search. The framework combines learned features with iterative inverse-compositional optimization, while computing Jacobians through finite gradients for unordered 3D points.
- Registration finds rotation R and translation t that best align point cloud Q to point cloud P.
- The feature-metric projection error compares F(P) with F(RQ+t), where F is learned by the encoder and has dimension K=1024 in the experiments.
- The framework extracts rotation-attentive features, then estimates transformation parameters by minimizing the projection error through a multi-task module.
- For unordered 3D point clouds, conventional image-gradient Jacobian computation is unavailable because point clouds lack grid structure.
- A finite-gradient approach computes the Jacobian efficiently instead of using a stochastic-gradient approach.
- Six perturbation parameters update rotation and translation during iterative optimization; after 10 iterations, the method outputs the estimated transformation and residual error.
3.2. Encoder
The encoder learns a distinctive, rotation-attentive feature extraction function so feature differences reflect rotation differences during transformation estimation.
- The encoder learns feature extraction function F to generate distinctive features for input point clouds.
- Its rotation-attentive design makes extracted features reflect rotation differences during transformation estimation.
- Following PointNet, the encoder uses two MLP layers and a max-pool layer, while discarding input-transform and feature-transform layers.
3.3. Multi-task Branches
The multi-task registration branch directly minimizes feature-metric error without correspondence search, using feature learning and iterative transformation estimation. As alignment improves, feature-map differences decrease toward near zero.
- Feature-metric Registration Branch: Transformation estimation is finalized by directly minimizing feature-metric projection error without searching correspondences.
- Encoder-decoder Branch: An encoder-decoder branch reconstructs features into 3D point clouds and can be trained unsupervised to make features aware of rotation differences.
- Encoder-decoder Branch: The decoder uses four fully connected layers with LeakyReLU and outputs the same dimension as the input point cloud.
- Feature-metric Registration Branch: The registration branch estimates transformation parameters with inverse-compositional nonlinear optimization to minimize feature-metric projection error.
- Feature-metric Registration Branch: The error uses global features F(P) and F(g · Q), with g containing rotation R and translation t.
- Feature-metric Registration Branch: Feature-map differences become smaller from the first through the fifth and final tenth iterations as alignment becomes more accurate.
3.4. Loss functions
Training uses feature-learning losses to support semi-supervised registration, with an encoder-decoder loss enabling unsupervised learning and a geometric loss comparing estimated and ground-truth transformations. The unsupervised variant retains only the feature-learning loss.
- The encoder is trained with two loss functions in a semi-supervised framework, which can become unsupervised by omitting the supervised geometric loss.
- Chamfer loss: The encoder-decoder branch uses Chamfer distance loss and can be trained without registration labels.
- Geometric loss: The geometric loss minimizes differences between estimated transformation gest and ground-truth transformation ggt using 3D point error.
- Geometric loss: The point-cloud geometric error is defined over a point cloud P containing M points.
- The final semi-supervised objective combines the training losses, whereas unsupervised training uses only losscf.
4. Experiments
Experiments evaluate the method on ModelNet40, 7Scene, outdoor KITTI scans, partial-overlap data, and cross-source point clouds. Across these settings, it achieves strong accuracy and robustness under rotation, density differences, noise, and varied training regimes while reducing runtime.
- ModelNet40: On ModelNet40, the method achieves the overall best performance across initial rotation angles, with registration error below 0.02 before 60°.It outperforms classic optimization, feature-learning, and deep-learning baselines; its advantage over PointNetLK is especially apparent at 30°−80°.
- ModelNet40: With 90% of source points removed, creating a ten-times density difference, the method achieves better registration and aligns point clouds accurately.3DSmoothNet and PointNetLK struggle in the high-density-difference setting.
- ModelNet40: Under Gaussian noise, the method obtains the lowest root-mean-square error of transformation-matrix difference and remains accurate when noise is large.Visual comparisons report accurate alignment while the compared methods fail under challenging noise.
- Training strategy: Unsupervised training obtains better performance than semi-supervised training on ModelNet40.The results indicate that the framework can be trained without registration labels while improving robustness and accuracy.
- Real-world and cross-domain evaluation: On 7Scene, the method outperforms the classic optimization, feature-extraction, and recent deep-learning registration methods.The authors attribute this result to the unsupervised branch and feature-metric registration framework.
5. Conclusion
The feature-metric framework improves point cloud registration accuracy and robustness while supporting semi-supervised or unsupervised training. Experiments show robustness to noise, density differences, and partial overlap, including fast unsupervised registration.
- The framework achieves better accuracy than classical, feature-learning, and deep-learning registration methods.
- Ablation studies show the method handles significant noise, density difference, and partial overlap.
- Semi-supervised or unsupervised training enables robust, accurate, and fast registration without relying exclusively on registration labels.
- The framework is presented as having potential for academic and industrial applications.