Source-linked AI summary
R3PM-Net: Real-time, Robust, Real-world Point Matching Network
Yasaman Kashefbahrami, Erkut Akdag, Panagiotis Meletis, Evgeniya Balmashnova, Dip Goswami, Egor Bondarau
TL;DR
Point cloud registration methods are often developed and evaluated on clean synthetic data, leaving their performance on imperfect industrial scans insufficiently represented. R3PM-Net addresses this gap with a lightweight, global-aware network and two real-world-oriented datasets, achieving competitive accuracy with high speed.
Problem
Deep-learning PCR methods are mostly trained and evaluated on synthetic datasets that omit real-world noise, occlusions, sparsity, and incomplete coverage.
Method
R3PM-Net uses a lightweight global-aware feature extraction architecture with an expanded receptive field, supported by Sioux-Cranfield and Sioux-Scans datasets.
Results
R3PM-Net matches state-of-the-art performance on synthetic benchmarks and consistently outperforms more complex models on realistic data while maintaining real-time latency.
Takeaways & Limitations
R3PM-Net provides a high-speed registration approach for sparse, noisy, and incomplete industrial point clouds.
Takeaways & Limitations
Improving generalization and accuracy across diverse shapes, densities, structures, and perturbation levels remains challenging in point cloud registration.
Abstract
from arXiv · showhide
Accurate Point Cloud Registration (PCR) is an important task in 3D data processing, involving the estimation of a rigid transformation between two point clouds. While deep-learning methods have addressed key limitations of traditional non-learning approaches, such as sensitivity to noise, outliers, occlusion, and initialization, they are developed and evaluated on clean, dense, synthetic datasets (limiting their generalizability to real-world industrial scenarios). This paper introduces R3PM-Net, a lightweight, global-aware, object-level point matching network designed to bridge this gap by prioritizing both generalizability and real-time efficiency. To support this transition, two datasets, Sioux-Cranfield and Sioux-Scans, are proposed. They provide an evaluation ground for registering imperfect photogrammetric and event-camera scans to digital CAD models, and have been made publicly available. Extensive experiments demonstrate that R3PM-Net achieves competitive accuracy with unmatched speed. On ModelNet40, it reaches a perfect fitness score of $1$ and inlier RMSE of $0.029$ cm in only $0.007$s, approximately 7 times faster than the state-of-the-art method RegTR. This performance carries over to the Sioux-Cranfield dataset, maintaining a fitness of $1$ and inlier RMSE of $0.030$ cm with similarly low latency. Furthermore, on the highly challenging Sioux-Scans dataset, R3PM-Net successfully resolves edge cases in under 50 ms. These results confirm that R3PM-Net offers a robust, high-speed solution for critical industrial applications, where precision and real-time performance are indispensable. The code and datasets are available at https://github.com/YasiiKB/R3PM-Net.
1. Introduction
Point Cloud Registration estimates the rigid transformation aligning two point clouds and supports downstream 3D tasks, but current methods remain limited by synthetic evaluation and local features. R3PM-Net addresses this gap with broader geometric context, lightweight computation, and datasets representing imperfect industrial scans.
- Point Cloud Registration estimates the rotation and translation aligning two point clouds and underpins 3D reconstruction, SLAM, and automated quality inspection.
- Traditional ICP and RANSAC struggle with noise, outliers, incomplete scans, and sensitivity to initial pose estimation.
- Most deep-learning approaches use synthetic datasets that omit real-world noise, occlusions, sparsity, and incomplete coverage, restricting industrial generalization.
- R3PM-Net expands the effective receptive field to capture broader geometric context instead of relying on local neighborhoods or complex backbones.
- The Sioux-Cranfield dataset spans pristine CAD models and noisy photogrammetric reconstructions, enabling evaluation across varying data quality levels.
- The Sioux-Scans dataset targets sparse, occluded event-camera scans registered to CAD models, while experiments report state-of-the-art synthetic performance and faster inference.
2. Related Work
Point cloud registration research has progressed from iterative geometric optimization to learned correspondence and attention-based global reasoning. R3PM-Net responds by reducing dependence on engineered local features and complex architectures through a broader receptive field.
- Traditional PCR methods use iterative geometric optimization, with ICP vulnerable to initialization and local minima and RANSAC constrained by iterative scalability.
- Deep-learning approaches learn features through projection, voxel, or point-based representations, trading geometric detail or computational efficiency in different ways.
- Learned correspondence methods use soft matching, Sinkhorn normalization, overlap prediction, or outlier filtering to identify reliable point relationships.
- Transformer models capture long-range geometric relationships, but global self-attention can create ambiguity in low-overlap scenarios.
- R3PM-Net expands the receptive field to reduce reliance on under-populated local neighborhoods while avoiding engineered features and complex architectures.
3. Method
R3PM-Net is a lightweight, global-aware registration network that estimates robust correspondences and rigid transformations for sparse and imperfect point clouds. Its coarse-to-fine pipeline combines global alignment with GICP refinement and is trained using registration and geometric-alignment losses.
- R3PM-Net processes sparse and imperfect industrial point clouds with a lightweight architecture designed for real-time registration.It adopts RPMNet as a baseline while reconsidering complex hybrid features for noisy real-world settings.
- Feature Extraction: Global-receptive-field feature extraction maps raw 3D coordinates into high-dimensional, globally aware embeddings using a lightweight network.The mapping φ is defined from R^3 to R^D with D=1024.
- Feature Extraction: Shared Siamese weights place source and target features in a common embedding space for direct Euclidean comparison.The same learned transformation is applied to both point clouds.
- Correspondence Estimation: Soft correspondence estimation computes a match matrix from feature distances, while Sinkhorn normalization enforces bistochastic constraints.Each matrix entry represents the probability that a source point corresponds to a target point.
- Correspondence Estimation: Dynamically predicted α and β make matching lenient in early iterations and strict in later iterations, suppressing outlier matches as alignment improves.α acts as an outlier decision boundary, while β controls matching sharpness.
- Transformation Estimation: Weighted target correspondences feed a differentiable weighted SVD module that estimates the optimal rigid transformation and permits gradient backpropagation.The estimated transformation is iteratively applied to the source cloud for refinement.
- Coarse-to-Fine Registration: The unified coarse-to-fine pipeline preprocesses both clouds, obtains a robust global pose with R3PM-Net, and performs final high-precision refinement using GICP.Downsampling, normalization, and centroid alignment improve numerical stability and memory efficiency; the loss combines registration and geometric-alignment terms.
4. Experiments
Experiments evaluate R3PM-Net on synthetic CAD data, imperfect reconstructions, and sparse event-camera scans using registration accuracy, fitness, runtime, and visual success. Across these settings, the results emphasize competitive precision, real-time inference, and improved robustness from global features and selective fine-tuning, while event-camera noise and sparsity remain challenging.
- 4.1. Datasets: 13-object Sioux-Cranfield and seven-object Sioux-Scans datasets evaluate robustness on imperfect reconstructions and raw event-camera scans against CAD targets.Sioux-Cranfield combines photogrammetric, synthetic, and pristine geometries; Sioux-Scans pairs CAD targets with event-camera scans.
- 4.2. Evaluation Metrics: R3PM-Net is evaluated with rotation error, translation error, Chamfer distance, fitness, inlier RMSE, inference time, and visual inspection.The paper states that these metrics should be considered jointly, with visual inspection particularly important for Sioux-Scans.
- 4.4. Experimental Results: 0.007 s inference time makes R3PM-Net 6.5× faster than RegTR on ModelNet40 while achieving a perfect Fitness score of 1.000 and highly competitive precision.Each evaluation is repeated over seven independent runs with different random seeds.
- 4.4. Experimental Results: 1.000 Fitness and inference speed over 6.5× faster than RegTR show competitive accuracy and real-time efficiency on Sioux-Cranfield.R3PM-Net outperforms Predator, GeoTransformer, and LoGDesc across nearly all reported metrics while remaining comparable to RegTR in RRE and RTE.
- 4.4. Experimental Results: 41 ms average runtime on Sioux-Scans is comparable to the fastest baselines, while R3PM-Net succeeds on more difficult geometries and aligns teeth and cube cases under 50 ms.Complete success remains challenging because event-camera scans contain noise, outliers, occlusions, and insufficient overlap for feature-sparse objects such as Lego.
- 4.5. Ablation Studies: Fine-tuning reduces rotation error by over 50% on ModelNet40 and Sioux-Cranfield and raises high-sparsity Sioux-Scans success rate from 28.6% to 42.9%.Fine-tuning on diverse geometric structures performs best, whereas full-dataset fine-tuning overfits and reduces generalization and overall performance.
5. Conclusion
R3PM-Net targets the gap between synthetic benchmarks and real-world industrial point-cloud registration. It combines robust registration of sparse, noisy, and occluded clouds with substantially lower computational cost than sophisticated alternatives.
- R3PM-Net bridges synthetic benchmarks and real-world object-level industrial data.
- Expanded receptive fields support efficient registration of sparse, noisy, and occluded point clouds.
- Real-world registration across diverse shapes, densities, structures, and perturbation levels remains challenging.
A.1. Datasets
The datasets combine synthetic CAD models with reconstructed and real-world scan data to evaluate registration under varying data quality. Sioux-Cranfield contains 13 objects, while Sioux-Scans uses reconstructed and synthetic sources for target clouds.
- ModelNet40 contains synthetic CAD models from 40 object categories, generated by randomly sampling 2,000 surface points per model.
- Sioux-Cranfield contains 13 objects for evaluating robustness across varying data qualities.
- Sioux-Cranfield includes reconstructed and synthetic Sioux models used to produce Sioux-Scans target point clouds.
A.2. Evaluation Metrics
The evaluation uses complementary metrics for geometric accuracy, overlap, and alignment quality. Ground-truth-based errors are supplemented by ground-truth-free metrics suited to real-world data, whose limitations require contextual interpretation.
- RRE measures overall rotational misalignment relative to ground-truth rotation matrices.
- RTE measures the Euclidean distance between ground-truth and estimated translations.
- Chamfer Distance averages nearest-neighbor distances between resulting and target point clouds without requiring exact correspondences.
- Fitness measures overlapping areas as the ratio of inlier correspondences to target-cloud points, with 1 indicating perfect alignment.
- Inlier RMSE measures average Euclidean error across inlier correspondences, but zero inliers can misleadingly produce zero RMSE.
- Chamfer Distance, fitness, and inlier RMSE are ground-truth-free but scale-dependent, so qualitative analysis remains necessary.
A.3. Results
Real-life industrial evaluation averages results across repeated trials and uses visual inspection to determine registration success. A method is successful only when accurate alignment occurs in at least four of seven runs.
- Results are averaged across seven independent runs for each of seven test objects.
- A case is successful only when accurate registration occurs in at least four of seven runs.
- Registration success is determined by visual inspection.