Source-linked AI summary
3DRegNet: A Deep Neural Network for 3D Point Registration
G. Dias Pais, Srikumar Ramalingam, Venu Madhav Govindu, Jacinto C. Nascimento, Rama Chellappa, Pedro Miraldo
TL;DR
3DRegNet addresses 3D scan registration from noisy point correspondences, where identifying outliers is central to accurate alignment. It jointly classifies correspondences and estimates pose using DNN or Procrustes registration, with refinement improving transformation estimation. The method is reported as efficient, significantly faster than baselines, and state-of-the-art across challenging evaluations.
Problem
3D registration must estimate scan motion from noisy correspondences while efficiently handling outliers that affect registration accuracy.
Method
3DRegNet jointly predicts correspondence inlier confidences and scan pose, using either DNN registration or differentiable Procrustes with an optional refinement network.
Results
3DRegNet performs as well as current baselines while being significantly faster, with refinement improving transformation estimation and L1 performing best across evaluation criteria.
Takeaways & Limitations
The approach provides an efficient correspondence-based solution for pairwise 3D scan registration that combines outlier rejection with pose estimation.
Takeaways & Limitations
Overall performance depends on selecting an appropriate rotation parameterization, which the paper evaluates against over-parameterized alternatives.
Abstract
from arXiv · showhide
We present 3DRegNet, a novel deep learning architecture for the registration of 3D scans. Given a set of 3D point correspondences, we build a deep neural network to address the following two challenges: (i) classification of the point correspondences into inliers/outliers, and (ii) regression of the motion parameters that align the scans into a common reference frame. With regard to regression, we present two alternative approaches: (i) a Deep Neural Network (DNN) registration and (ii) a Procrustes approach using SVD to estimate the transformation. Our correspondence-based approach achieves a higher speedup compared to competing baselines. We further propose the use of a refinement network, which consists of a smaller 3DRegNet as a refinement to improve the accuracy of the registration. Extensive experiments on two challenging datasets demonstrate that we outperform other methods and achieve state-of-the-art results. The code is available.
1. Introduction
3D registration estimates six-degree-of-freedom motion from noisy point correspondences, with outlier handling as a central challenge. 3DRegNet jointly classifies correspondences and computes scan-alignment transformations, offering state-of-the-art performance with significant speedup.
- Motivation: 3D registration computes 6 DoF motion parameters between scans from noisy point correspondences.Classical pipelines commonly use minimal solvers within RANSAC, followed by refinement such as ICP.
- Proposed approach: 3DRegNet jointly classifies point correspondences as inliers or outliers and computes the rotation and translation aligning two scans.The architecture uses classification and registration blocks for these tasks.
- Motivation: Outlier handling is the main challenge for maintaining registration accuracy when correspondences are noisy.The paper frames pose estimation as less difficult than efficiently handling outliers.
- Proposed approach: The first architecture combines classification and registration blocks, while the second replaces its registration block with differential Procrustes.Both architectures share the classification block.
- Contributions: A refinement network can fine-tune registration results while preserving significant speedup and state-of-the-art registration performance.The refinement network is a smaller 3DRegNet used after the initial network.
2. Related Work
Prior work addresses 3D registration through classical optimization, point-cloud networks, learned correspondence generation, and related pose-estimation tasks. 3DRegNet focuses specifically on pairwise registration from 3D point correspondences.
- Classical registration: Classical 3D registration methods include ICP, robust extensions, nonrigid formulations, and optimal least-squares solvers.ICP is widely regarded as a gold-standard approach but can become trapped in local minima.
- Deep registration: Deep networks such as PointNet and PointNetLK process unordered point clouds and estimate alignment using learned features or iterative schemes.PointNet uses pointwise MLPs followed by a symmetric function, while PointNetLK uses a DNN loop.
- Related deep methods: Other methods learn correspondences, matched probabilities, or point-cloud features for registration, detection, and pose estimation.The cited approaches span learned matching, convolutional features, spherical CNNs, and 3D bounding-box pose estimation.
- Correspondence classification: Learned correspondence classification has also been studied for 2D geometry, with separate or eigendecomposition-free estimation of the Essential Matrix.Later work aggregates local correspondences hierarchically and handles correspondence-order insensitivity.
- Scope: This paper addresses pairwise registration of 3D scans rather than rotation averaging across many scans or three-view feature matching.The scope is explicitly distinguished from large-scale reconstruction and multi-view matching.
3. Problem Statement
The problem is to estimate a rigid transformation between corresponding 3D points while also identifying which correspondences are reliable. The network outputs correspondence confidences alongside rotation and translation parameters.
- Transformation estimation: Given N 3D correspondences {(p_i, q_i)}, the goal is to estimate rotation R ∈ SO(3) and translation t ∈ R^3.The points belong to the first and second scans, respectively.
- Network outputs: The network outputs N correspondence weights plus M rotation parameters and three translation parameters.The total output dimensionality is N + M + 3 variables.
- Outlier classification: Each weight w_i ∈ [0, 1) represents confidence that correspondence i is an inlier, with w_i ≥ T defining the inlier classification.Correspondences below the threshold are treated as outliers.
- Rotation representation: Rotation parameterization is a design choice because different representations can substantially affect overall performance.The paper evaluates parameterizations beyond the exact three degrees of freedom of a 3D rotation, including prior over-parameterized representations.
4. 3DRegNet
3DRegNet uses a shared correspondence-classification network followed by either a DNN registration block or differentiable Procrustes. It trains these components with classification and registration losses.
- Architecture: 3DRegNet offers two registration blocks: a DNN-based regressor and a differentiable Procrustes method.The choice of registration block does not change the loss functions.
- Classification: The classification block processes each 6-tuple correspondence with shared fully connected layers and C deep ResNets.It outputs weights in [0, 1) indicating correspondence confidence.
- DNN registration: The DNN registration block pools features from C + 1 classification stages and predicts M rotation variables plus three translation variables.Max-pooling performed best among the tested pooling choices.
- Procrustes registration: Differentiable Procrustes filters outliers, centers the inlier point clouds, and estimates rotation and translation from the remaining correspondences.The inlier set is obtained from the classification block.
- Loss functions: Training combines classification and registration losses, with cross-entropy penalizing incorrect correspondence labels.The registration loss measures distances between second-scan points and transformed first-scan points; tested metrics include L1, weighted least squares, L2, and Geman-McClure.
5. 3DRegNet Refinement
The refinement architecture chains two 3DRegNet blocks: the first estimates a coarse transformation, while a smaller second network estimates residual corrections. The second network uses cumulative regression and correspondence updates to refine the registration.
- Architecture: The first network handles larger rotation and translation values, while the smaller second network estimates smaller transformations.This staged design follows rough registration with a refinement strategy.
- Architecture: Two 3DRegNet blocks are arranged sequentially, with the first producing coarse registration and the second performing refinement.The architecture uses regression parameters and classification weights for each block.
- Training and updates: The second network’s loss accounts for the cumulative regression of both 3DRegNets.The overall transformation update depends on estimates from both networks.
- Training and updates: Refinement updates the point correspondences so the second network corrects residual transformation after the first block.The second stage is constrained to estimate smaller corrections.
- Training and updates: Classification and registration losses are computed at each step and averaged into the total loss.The procedure applies the same loss computations at both refinement stages.
6. Datasets and 3DRegNet Training
The study evaluates 3DRegNet on synthetic and real connected point-cloud pairs, using FPFH correspondences and held-out testing splits. Training uses staged rotation augmentation through curriculum learning, with separate training configurations for the two datasets.
- Datasets: About 3,000 FPFH 3D point correspondences are extracted for each scan pair in both datasets.Correspondences are labeled as inliers or outliers using ground-truth transformations and a distance threshold.
- Datasets: Outliers comprise about 50% of total matches, with 70% of ICL-NUIM pairs used for training and 30% for testing.SUN3D uses 10 scenes for training and three completely unseen scenes for testing.
- Training: The network is trained for 1000 epochs with Adam, a 10^-4 learning rate, cross-validation, and batch size 16.The ICL-NUIM and SUN3D runs use 1092 and 200 steps per epoch, respectively.
- Training: Curriculum-learning augmentation starts with small rotations and progressively increases task difficulty to improve generalization to unseen rotations.Training proceeds to harder tasks after easier ones are completed.
7. Experimental Results
The experiments evaluate loss functions, rotation representations, correspondence counts, augmentation, refinement, and comparisons with established baselines. 3DRegNet’s refinement improves transformation estimation, while performance varies with representation, input density, and rotation range.
- Experimental design: The experiments cover distance metrics, rotation parameterizations, correspondence counts, data augmentation, refinement, baseline comparisons, and unseen sequences.Ablations are conducted on ICL-NUIM, followed by comparisons with previous methods and tests on unseen scenes.
- Distance metrics: L1-norm training gives the best results across all reported evaluation criteria.The remaining evaluations use the L1-norm because weighted L2-norm performs worse despite using classification weights.
- Rotation parameterization: Three-parameter Lie algebra rotation achieves the best results among Lie algebra, quaternion, and nine-parameter linear matrix representations.The authors use Lie algebra in subsequent experiments but caution that this conclusion may not generalize to human pose estimation.
- Correspondence sensitivity: Regression accuracy degrades as correspondences decrease from 100% toward 10%, while inlier/outlier classification remains unaffected.Each scan pair contains around 3000 correspondences, and the authors associate more inliers with better transformation estimates.
- Data augmentation: Data augmentation improves testing on rotations beyond those present in the original dataset, but is unnecessary for datasets containing only small motions.The augmentation perturbs scan pairs with rotations up to θ = 50°; it is omitted from later experiments.
- Refinement: The refinement network improves transformation estimation without significantly compromising runtime, although classification accuracy decreases by 2%.The first network estimates coarse motion and the smaller second network refines it; refined 3DRegNet is used thereafter.
- Baseline comparisons: Against FGR, 3DRegNet performs better for a larger number of cases, while FGR performs better for rotation errors below 2°.FGR has lower median but higher mean rotation error, and 3DRegNet-based initialization improves subsequent ICP results.
8. Discussion
3DRegNet jointly rejects outlier correspondences and estimates scan pose, providing an efficient registration approach that remains competitive with existing baselines.
- 3DRegNet jointly classifies point correspondences as inliers or outliers and computes the alignment pose.
- The approach is reported to perform as well as current baselines while being significantly faster.
(SUPPLEMENTARY MATERIALS)
The supplementary materials identify the paper’s authors.
- The author list includes G. Dias Pais, Srikumar Ramalingam, and Venu Madhav Govindu.
- The author list includes Jacinto C. Nascimento, Rama Chellappa, and Pedro Miraldo.
- The paper credits seven authors in total.
B. Discriminate Results for SUN3D
SUN3D errors vary substantially across sequences, with method rankings depending on sequence difficulty and refinement. Overall, refined 3DRegNet is generally strongest, while ICP and FGR perform well in less challenging scenarios.
- Individual SUN3D errors vary significantly because each sequence has its own characteristics.
- On MIT, ICP performs better than 3DRegNet, whereas 3DRegNet is superior on Harvard with and without refinement.
- On Brown, 3DRegNet beats the state of the art in the mean without refinement, although RANSAC and FGR have slightly better median results.
- The comparison includes 3DRegNet, 3DRegNet + ICP, FGR, and FGR + ICP on 20 scans, without drift removal or transformation averaging.
- With Umeyama or ICP refinement, 3DRegNet is generally the best method, except for a small-margin advantage for FGR + ICP on estimated median and translation.
- All methods were run under the same conditions using only CPU computation.