Source-linked AI summary
Robust Point Cloud Registration Framework Based on Deep Graph Matching
Kexue Fu, Shaolei Liu, Xiaoyuan Luo, Manning Wang
TL;DR
Point cloud registration remains difficult with outliers, especially for correspondence-based learning methods. RGM uses deep graph matching, AIS, and transformer-generated edges to improve correspondences, achieving state-of-the-art performance across clean, noisy, partial-to-partial, and unseen-category datasets.
Problem
Outliers make point correspondences unreliable in point cloud registration, while existing methods also face initialization or time constraints.
Method
RGM constructs graphs from point clouds, integrates structural information into node features, predicts soft correspondences with AIS, and uses transformer-generated edges.
Results
RGM achieves state-of-the-art performance on clean, noisy, partial-to-partial, and unseen-category point-cloud datasets.
Takeaways & Limitations
Graph structure and transformer-based edge generation provide the framework's basis for establishing more accurate correspondences under outlier and partial-overlap conditions.
Abstract
from arXiv · showhide
3D point cloud registration is a fundamental problem in computer vision and robotics. There has been extensive research in this area, but existing methods meet great challenges in situations with a large proportion of outliers and time constraints, but without good transformation initialization. Recently, a series of learning-based algorithms have been introduced and show advantages in speed. Many of them are based on correspondences between the two point clouds, so they do not rely on transformation initialization. However, these learning-based methods are sensitive to outliers, which lead to more incorrect correspondences. In this paper, we propose a novel deep graph matchingbased framework for point cloud registration. Specifically, we first transform point clouds into graphs and extract deep features for each point. Then, we develop a module based on deep graph matching to calculate a soft correspondence matrix. By using graph matching, not only the local geometry of each point but also its structure and topology in a larger range are considered in establishing correspondences, so that more correct correspondences are found. We train the network with a loss directly defined on the correspondences, and in the test stage the soft correspondences are transformed into hard one-to-one correspondences so that registration can be performed by singular value decomposition. Furthermore, we introduce a transformer-based method to generate edges for graph construction, which further improves the quality of the correspondences. Extensive experiments on registering clean, noisy, partial-to-partial and unseen category point clouds show that the proposed method achieves state-of-the-art performance. The code will be made publicly available at https://github.com/fukexue/RGM.
1. Introduction
Point cloud registration must jointly estimate transformations and point correspondences, yet outliers, initialization dependence, and computational cost challenge existing methods. RGM addresses these issues with deep graph matching, an AIS correspondence module, and transformer-generated graph edges.
- Point cloud registration aligns two point clouds by estimating a rigid transformation and correspondences between their points.
- Outliers from sensor imperfections or incomplete overlap make jointly solving correspondence and transformation especially difficult.
- ICP depends on initialization and may converge locally, while branch-and-bound methods reduce initialization dependence but are time-consuming.
- RGM constructs graphs and uses deep graph matching to incorporate neighboring-node features and graph structure when establishing correspondences.
- The AIS module analyzes node affinity globally with Sinkhorn normalization to reduce incorrect correspondences.
- A transformer generates soft graph edges, improving correspondences for overlapping regions in partial-to-partial registration.
2. Related Work
Prior registration methods trade off initialization robustness, speed, accuracy, and handling of partial data. RGM differs from learning-based approaches by incorporating graph structural information into point representations for matching.
- Traditional Registration Method: Traditional methods such as ICP require a good initial transformation and converge only to a local optimum near that initialization.
- Traditional Registration Method: Branch-and-bound methods seek global optima without initialization but are very slow and practical only in limited scenarios.
- Correspondence-Based Registration: Keypoint-matching methods use descriptors and RANSAC-like registration, but their speed and accuracy are sensitive to outliers and repetitive geometry.
- Learning-Based Registration: Learning-based methods extract global or local point-cloud features, yet existing approaches still have room to improve accuracy and robustness on partial-to-partial data.
- Learning-Based Registration: RGM constructs graphs and merges graph structural information into node features, distinguishing it from existing learning-based registration methods.
- Graph Matching: Learning-based graph matching has been studied broadly, but the paper identifies no prior application of it to point cloud registration.
3. Problem Formulation
The formulation estimates a rigid transformation and a correspondence matrix between source and target point clouds. An end-to-end network predicts correspondences, after which SVD computes the rigid transformation.
- 3D rigid registration estimates rotation R and translation t to align source point cloud X with target point cloud Y.
- The correspondence matrix C is binary, with C_i,j = 1 when x_i and y_j correspond and 0 otherwise.
- Under strict one-to-one matching, source and target point counts are equal and each point participates in exactly one correspondence.
- When one-to-one correspondences do not exist, inequality constraints and slack variables accommodate unmatched points in C.
- RGM predicts C with an end-to-end neural network and obtains the rigid transformation from the predicted correspondences using SVD.
4. RGM
RGM constructs graphs from point clouds, combines local, intra-graph, and cross-graph information, and predicts correspondences for rigid registration. Its AIS module produces soft correspondences that are converted into hard one-to-one matches for SVD-based transformation estimation.
- 4.1. Local Feature Extractor: RGM extracts shared high-dimensional local features from each point cloud and uses them as initial graph node features.The local descriptor uses point coordinates and K-nearest neighbors before nonlinear mapping to the final feature space.
- 4.2. Edge Generator Based on Transformer: A transformer-based edge generator builds source and target graphs so graph features can be effectively aggregated.The graphs represent original points as nodes and their connections through adjacency matrices.
- 4.3. Graph Feature Extractor and AIS Module: Graph feature extraction aggregates node information along within-graph edges to capture self-correlation, then uses cross-graph edges weighted by soft correspondences.The cross-graph convolution gives greater weight to more similar node pairs between the two graphs.
- 4.3. Graph Feature Extractor and AIS Module: The AIS module computes affinities between graph nodes, applies instance normalization and Sinkhorn, and produces a soft correspondence matrix.Its global matrix analysis is intended to reduce incorrect correspondences compared with processing rows or columns independently.
- 4.4. LAP-SVD: At test time, RGM converts the soft matrix into a binary one-to-one correspondence matrix with a Hungarian LAP solver and estimates the transformation by SVD.Training uses cross-entropy directly between the soft and ground-truth correspondence matrices, so later registration calculations need not be differentiable.
5. Experiments
Experiments evaluate RGM on clean, noisy, partial-to-partial, and unseen-category point clouds using transformation, correspondence-related, and recall metrics. RGM achieves strong performance across these settings, while ablations show that AIS and transformer-generated edges improve registration accuracy.
- Experimental Setup: Experiments use ModelNet40 and report transformation errors, clip chamfer distance, and recall under multiple registration conditions.The metrics include MIE, MAE, CCD, and recall thresholds based on MAE(R) and MAE(t).
- Clean Point Clouds: RGM achieves the best clean-data performance, with 100% success and most error metrics close to 0.It greatly outperforms the strongest learning-based method; DeepGMR also reaches 100% success but has larger errors.
- Gaussian Noise: On Gaussian-noise data, RGM is much more accurate than learning-based and traditional methods, with recall close to 100%.Noisy correspondences are rebuilt by repeatedly selecting point pairs within the 0.1 distance threshold from remaining points.
- Partial-to-Partial: On partial-to-partial data, RGM exceeds 90% success and has errors roughly half those of the second-best RPM-Net.The evaluation retains 70% of points after independently generated plane-based cropping.
- Unseen Categories: For unseen categories, RGM generalizes better than the compared learning-based methods, while traditional methods change little and RPM-Net also generalizes well.Training and validation use the first 20 categories, while testing uses the last 20 categories.
- Ablation Studies: Ablations show that replacing AIS or transformer-generated edges reduces registration performance relative to the full RGM model.The AIS variant generates fewer correct matches, and the full-connection edge variant is inferior to transformer-based edge generation.
6. Conclusion
RGM applies deep graph matching to point cloud registration and reports state-of-the-art performance. Its AIS module improves node correspondences, while a transformer-based edge generator offers an alternative graph-construction strategy.
- RGM introduces deep graph matching for point cloud registration and reports state-of-the-art performance.
- The AIS module establishes accurate correspondences between graph nodes to improve registration performance.
- A transformer-based edge generator provides an alternative to full connection, nearest-neighbor connection, and Delaunay triangulation for building graph edges.
- The framework may extend to other registration problems, including 2D-3D and deformable registration.