Source-linked AI summary
RPM-Net: Robust Point Matching using Learned Features
Zi Jian Yew, Gim Hee Lee
TL;DR
Rigid point cloud registration is difficult because ICP’s spatially based hard correspondences are sensitive to initialization and noise. RPM-Net learns hybrid spatial-geometric features, uses Sinkhorn-based soft assignments with adaptive annealing, and reports state-of-the-art performance across varied conditions. It also accommodates missing correspondences and partial visibility within the stated formulation.
Problem
ICP often converges to wrong local minima because hard spatial correspondences are sensitive to initialization and noisy or outlier points.
Method
RPM-Net uses learned hybrid features, differentiable Sinkhorn soft assignments, annealing, and a secondary network predicting annealing parameters.
Results
RPM-Net achieves state-of-the-art performance on ModelNet40 and on evaluations spanning clean, noisy, and partially visible datasets.
Takeaways & Limitations
RPM-Net provides a registration approach designed to reduce initialization sensitivity and explicitly handle outliers, missing correspondences, and partial visibility.
Takeaways & Limitations
The formulation assumes point normals can be easily computed from the input points.
Abstract
from arXiv · showhide
Iterative Closest Point (ICP) solves the rigid point cloud registration problem iteratively in two steps: (1) make hard assignments of spatially closest point correspondences, and then (2) find the least-squares rigid transformation. The hard assignments of closest point correspondences based on spatial distances are sensitive to the initial rigid transformation and noisy/outlier points, which often cause ICP to converge to wrong local minima. In this paper, we propose the RPM-Net -- a less sensitive to initialization and more robust deep learning-based approach for rigid point cloud registration. To this end, our network uses the differentiable Sinkhorn layer and annealing to get soft assignments of point correspondences from hybrid features learned from both spatial coordinates and local geometry. To further improve registration performance, we introduce a secondary network to predict optimal annealing parameters. Unlike some existing methods, our RPM-Net handles missing correspondences and point clouds with partial visibility. Experimental results show that our RPM-Net achieves state-of-the-art performance compared to existing non-deep learning and recent deep learning methods. Our source code is available at the project website https://github.com/yewzijian/RPMNet .
1. Introduction
Rigid point cloud registration must jointly recover unknown correspondences and a rigid transformation, while ICP is vulnerable to initialization and noise. RPM-Net addresses these issues with learned hybrid features, soft assignments, adaptive annealing, and reported state-of-the-art performance across clean, noisy, and partially visible data.
- Motivation: Rigid point cloud registration jointly estimates point correspondences and the rigid transformation aligning two clouds.Knowing either correspondences or the transformation makes the problem trivial.
- Limitations of Existing Methods: ICP alternates nearest-neighbor correspondence assignment with least-squares transformation estimation, but is sensitive to initialization and noisy or outlier points.These weaknesses limit ICP in noisy scans and global registration.
- RPM-Net: RPM-Net replaces spatial distances with learned hybrid feature distances combining point coordinates and local geometry.A differentiable Sinkhorn layer and annealing produce soft correspondences.
- RPM-Net: A secondary network predicts optimal annealing parameters at each iteration instead of using a fixed schedule.The predicted parameters include α and β.
- Contributions: RPM-Net is designed to handle noisy or outlier points, missing correspondences, and partial visibility while reducing sensitivity to initialization.The paper also introduces a modified Chamfer distance for symmetry or partial visibility.
- Results: RPM-Net achieves state-of-the-art performance on experimental evaluations covering clean, noisy, and partially visible datasets.The conclusion also reports state-of-the-art performance on ModelNet40 over various evaluation criteria.
2. Related Work
Prior registration methods use handcrafted correspondence strategies, iterative alignment, or learned features, each with limitations involving initialization, outliers, partial overlap, or geometric distinctiveness.
- Feature-Based Methods: Feature-based methods match descriptors of distinctive keypoints before computing the rigid transformation, but require well-designed descriptors and distinctive geometry.Handcrafted 3D feature descriptors commonly accumulate measurements into histograms.
- Handcrafted Registration Methods: ICP variants alternate correspondence estimation and transformation fitting, yet most still require good initialization to avoid bad local minima.Variants modify point selection or correspondence weighting.
- Soft Assignment Methods: Soft-assignment methods widen ICP’s convergence basin, while RPM gradually hardens assignments through deterministic annealing.IGSP combines spatial coordinates with handcrafted BSC features but does not learn feature weighting.
- Learned Registration Methods: Learned methods replace or augment iterative registration with global descriptors, deep networks, or pointwise soft matching to improve robustness or initialization.The related methods include PointNetLK, PCRNet, and Deep Closest Point.
3. Problem Formulation
The problem is to recover the unknown rotation and translation aligning source and reference point clouds without assuming one-to-one correspondence or equal size and extent.
- Problem Definition: RPM-Net recovers a rotation R and translation t aligning source cloud X with reference cloud Y.Point normals are assumed readily computable from the points.
- Problem Definition: The formulation permits different point counts and differing spatial extents, so it does not assume one-to-one point correspondence.This supports point clouds with missing correspondences or partial overlap.
4. Background: Robust Point Matching
Robust Point Matching relaxes hard permutation assignments into soft doubly stochastic assignments, alternates soft correspondence estimation with rigid alignment, and uses slack variables for unequal or outlier-corrupted clouds.
- Match Matrix: RPM represents point correspondences with a binary match matrix M, whose permutation constraints describe one-to-one assignments in the square case.The matrix indicates whether each source point corresponds to each reference point.
- Soft Assignment: RPM relaxes binary assignments to values in [0,1] and alternates softassign correspondence estimation with rigid transformation estimation.This replaces the strict permutation constraint with a doubly stochastic one.
- Deterministic Annealing: Deterministic annealing increases β from small values, moving assignments from soft toward hard while alternating row and column normalizations enforce doubly stochastic constraints.Small initial β values help avoid local minima.
- Transformation Estimation: The rigid transformation is computed from estimated correspondences using SVD in this paper.This follows the cited prior approach.
- Outlier Handling: When point counts differ or nonmatching outliers exist, slack variables convert inequality constraints into an equality-constrained formulation for Sinkhorn normalization.The implementation adds an extra row and column of ones.
5. Our RPM-Net
RPM-Net replaces spatial-distance matching with learned hybrid features and uses differentiable soft assignment, learned annealing parameters, and weighted transformation estimation for robust iterative registration.
- Network overview: RPM-Net transforms the source using the previous estimate, extracts hybrid features, predicts annealing parameters, and computes soft correspondences at each iteration.The network is recurrent, with Sinkhorn normalization unrolled for five steps and transformation gradients stopped between iterations.
- Feature extraction: Learned feature distances replace spatial distances so correspondence estimation can use local geometric characteristics and avoid wrong local minima.The features combine spatial coordinates with local geometry.
- Feature extraction: Each hybrid feature uses a local neighborhood, centroid-relative coordinates, rotation-invariant point pair features, and the centroid’s absolute position.A PointNet-based network pools an arbitrary number of neighboring points into one descriptor.
- Parameter prediction: A secondary network predicts annealing parameters from the current alignment instead of relying on manually tuned, fixed dataset-specific schedules.This design makes parameter selection depend on alignment state rather than iteration count.
- Transformation and loss: RPM-Net estimates rigid transformations with SVD while weighting each source point by its total match probability to accommodate missing correspondences.The loss combines transformation error with a match-matrix loss that encourages inliers, with later iterations receiving higher weights.
6. Experiments
The experiments evaluate registration using isotropic transformation errors, anisotropic absolute errors, and a modified Chamfer distance designed for symmetry and partial visibility.
- Evaluation metrics: Registration is evaluated with mean isotropic rotation and translation errors against the ground-truth and estimated transformations.The rotation error is the angle of the relative rotation matrix, reported in degrees.
- Evaluation metrics: Mean absolute errors over Euler angles and translation vectors are also reported for consistency with previous work.These measures are anisotropic.
- Evaluation metrics: The modified Chamfer distance compares transformed source and reference clouds with clean, complete versions to avoid unfair penalties for symmetric alternatives.This metric is intended for alignment evaluation when symmetry or partial visibility makes direct transformation errors misleading.
6.3. Baseline Algorithms
RPM-Net is compared with handcrafted registration methods and recent learned approaches, using retrained author implementations where pretrained models were unavailable for the required settings.
- Baseline algorithms: The baseline set includes ICP, FGR, RPM, PointNetLK, and Deep Closest Point DCP-v2.ICP and FGR use Intel Open3D implementations, RPM uses the authors’ implementation, and learned baselines use author-provided implementations retrained for the experiments.
- Baseline algorithms: Table 1 reports clean-data performance, marking the best and second-best measures and noting that DCP-v2 uses a trained model.The caption also gives DCP-v2’s previously reported anisotropic errors as 2.007° rotation and 0.0037 translation.
6.4. Clean Data
On clean data, RPM-Net ranks first or second across all reported measures and outperforms the learned and handcrafted methods except FGR; noisy, independently sampled clouds test missing-correspondence handling.
- Clean Data: RPM-Net achieves very accurate clean-data registration, ranking first or second in all measures and outperforming every learned and handcrafted method except FGR.The evaluation samples identical 1,024-point subsets for source and reference clouds, preserving exact point correspondences.
- Gaussian Noise: The Gaussian-noise experiment independently samples 1,024 points, adds clipped N(0, 0.01) jitter, and creates non-one-to-one correspondences.All learned models are trained on noisy data except PointNetLK, which reuses the previous model.
- Gaussian Noise: RPM-Net outperforms all handcrafted and learned methods on noisy data because it explicitly handles points with no correspondences.DCP requires every point to have a correspondence and performs poorly when that condition is violated.
6.6. Partial Visibility
The partial-visibility evaluation tests registration when point clouds do not fully overlap, including noisy data, and examines which components and iteration counts matter.
- Partial visibility: Partial-visibility experiments retain approximately 70% of each point cloud and downsample to 717 points to maintain comparable point density.The study trains DCP-v2 and RPM-Net on partially visible data.
- Ablation studies: The ablation study evaluates isotropic and Chamfer distance metrics on the partial-visibility setting.
- Ablation studies: Absolute positions, local coordinates, and PPF features are all required for highest performance, while removing absolute positions causes a significant drop.The table denotes absolute centroid coordinates by x and neighboring-point local coordinates by ∆x.
- Ablation studies: The learned annealing schedule from the parameter prediction network improves registration performance over shared learnable α and β values.
- Iteration analysis: Most performance gains occur in the first two iterations, and registration mostly converges after 5 iterations.The experiments use 5 iterations.
6.8. Computational Efficiency
RPM-Net is faster than RPM in inference but slower than ICP and non-iterative DCP-v2 under the reported hardware and iteration settings.
- Inference time: RPM-Net is significantly faster than RPM, which requires many iterations, but slower than ICP and non-iterative DCP-v2.RPM-Net timings use 5 iterations; ICP and FGR run on CPU, while the remaining algorithms run on a GPU.
7. Conclusion
RPM-Net presents a deep learning-based RPM approach for rigid point cloud registration, combining learned fusion features with explicit outlier handling. Experiments report state-of-the-art performance on ModelNet40 across varied evaluation criteria.
- Conclusion: RPM-Net desensitizes initialization and improves convergence behavior using learned fusion features instead of spatial distances alone.
- Conclusion: Differentiable Sinkhorn normalization with slack variables enables explicit handling of outliers and partial correspondences.
- Conclusion: A secondary network predicts optimal annealing parameters to further improve performance.
- Conclusion: Experimental results show state-of-the-art performance on ModelNet40 over various evaluation criteria.