PaperlayerView exact revision on arXiv

Source-linked AI summary

OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration

Hao Xu, Shuaicheng Liu, Guangfu Wang, Guanghui Liu, Bing Zeng

arXiv:2103.00937v5cs.CV

TL;DR

Partial-to-partial registration is difficult because sparse correspondence methods depend on distinctive geometry, while global features are affected by non-overlapping points. OMNet predicts overlapping masks within an iterative global-feature network and reports state-of-the-art performance, while also changing CAD-model sampling to address over-fitting. Its supported scope includes robustness to noise and partiality, with mask labels relying on a fixed closest-point threshold.

  • Problem

    Correspondence methods rely on distinctive local structures and sparse matches, whereas global-feature methods ignore the effect of non-overlapping points when aggregating inputs.

  • Method

    OMNet is an end-to-end iterative global-feature network that predicts overlapping masks to reject non-overlapping points during transformation estimation.

  • Results

    OMNet achieves state-of-the-art performance compared with traditional and deep learning methods under clean, noisy, and different partial datasets.

  • Takeaways & Limitations

    Learning overlapping masks converts partial-to-partial registration toward registration of the same shape and supports robustness to noise and partiality.

  • Takeaways & Limitations

    The assumed overlapping-mask labels use a fixed 0.1 threshold on closest-point distances between ground-truth-transformed source points and the reference.

Abstract

from arXiv · show

Point cloud registration is a key task in many computational fields. Previous correspondence matching based methods require the inputs to have distinctive geometric structures to fit a 3D rigid transformation according to point-wise sparse feature matches. However, the accuracy of transformation heavily relies on the quality of extracted features, which are prone to errors with respect to partiality and noise. In addition, they can not utilize the geometric knowledge of all the overlapping regions. On the other hand, previous global feature based approaches can utilize the entire point cloud for the registration, however they ignore the negative effect of non-overlapping points when aggregating global features. In this paper, we present OMNet, a global feature based iterative network for partial-to-partial point cloud registration. We learn overlapping masks to reject non-overlapping regions, which converts the partial-to-partial registration to the registration of the same shape. Moreover, the previously used data is sampled only once from the CAD models for each object, resulting in the same point clouds for the source and reference. We propose a more practical manner of data generation where a CAD model is sampled twice for the source and reference, avoiding the previously prevalent over-fitting issue. Experimental results show that our method achieves state-of-the-art performance compared to traditional and deep learning based methods. Code is available at https://github.com/megvii-research/OMNet.

1. Introduction

Point cloud registration aligns partial and noisy inputs, but existing correspondence-based and global-feature methods have complementary limitations. OMNet addresses these issues with iterative overlapping-mask prediction, revised data generation, and reported state-of-the-art performance.

  • Point cloud registration predicts a 3D rigid transformation aligning two point clouds despite partiality and noise.
  • ICP estimates transformations iteratively from closest-point correspondences but is sensitive to local minima, runtime, and initial positions.Go-ICP searches across a larger transformation space but is much slower than ICP.
  • Correspondence-based deep methods require distinctive local geometry and cannot fully exploit the geometric information in overlapping point clouds.
  • Global-feature methods aggregate entire inputs but ignore the negative effect of non-overlapping points during feature aggregation.
  • OMNet iteratively predicts overlapping masks, rejects non-overlapping regions, and estimates transformations coarse-to-fine for robustness to noise and partiality.The authors report state-of-the-art performance against previous algorithms.
  • The proposed data generation samples CAD models separately for source and reference point clouds to address over-fitting and axisymmetrical-category issues.

2. Related Works

Prior registration methods either estimate transformations from sparse correspondences or aggregate global features over all input points. OMNet is presented as an iterative global-feature architecture that emphasizes overlapping regions in partial-to-partial registration.

  • Correspondence Matching based Methods: Correspondence-matching methods alternate establishing source-reference correspondences with least-squares rigid transformation estimation.ICP uses spatial distances, while later variants detect keypoints or weight correspondences.
  • Correspondence Matching based Methods: These methods can become trapped in local minima because correspondence estimation is non-convex.
  • Correspondence Matching based Methods: Deep correspondence methods replace handcrafted descriptors with learned features but still depend on distinctive geometric structures for sparse matches.Limited or poorly distributed matches leave much of the point cloud unused in transformation estimation.
  • Global Feature based Methods: Global-feature methods estimate transformations from complete input point clouds without correspondences, including overlapping and non-overlapping regions.PointNetLK, PCRNet, and FMR are described as examples of this category.
  • Global Feature based Methods: OMNet’s architecture duplicates global features across points before concatenating them with point-wise features, with shared weights indicated by matching background colors.Superscripts indicate iteration counts.

3. Method

OMNet iteratively extracts global features, predicts overlapping masks, and regresses rigid transformations in a coarse-to-fine pipeline. Mask-weighted aggregation suppresses non-overlapping interference, while the model jointly trains mask prediction and transformation regression.

  • Pipeline: OMNet transforms the source using the previous estimate, extracts global features, predicts overlap masks, and regresses the next transformation at each iteration.After N iterations, the estimated transformations are accumulated into the overall transformation.
  • Global Feature Extraction: Mask-weighted max pooling rejects non-overlapping points so global features focus on corresponding overlapping regions.The masks are applied before aggregation, converting partial-to-partial registration toward registration of the same shape.
  • Overlapping Mask Prediction: Hybrid features concatenate point-wise and global features from both inputs before shared processing for overlap segmentation and transformation regression.The fused features combine local point information with geometric information from both point clouds.
  • Rigid Transformation Regression: The transformation regressor outputs a 7D vector containing translation t ∈ R3 and quaternion rotation q ∈ R4.The quaternion is constrained by qTq = 1.

4. Experiments

The experiments evaluate OMNet across synthetic, noisy, unseen-category, real-scan, and varying-partiality settings, using standard registration errors and controlled ablations. OMNet achieves strong or leading performance while the data-generation protocol addresses sampling and symmetry issues in ModelNet40.

  • Datasets and preprocessing: ModelNet40 evaluation uses separate train, validation, and test splits, excluding axisymmetrical categories and generating rotations within [0°, 45°] and translations within [−0.5, 0.5].The test split includes unseen categories, while the first 14 categories provide training and validation data.
  • Implementation: The implementation runs four iterations and delays mask application until after the second iteration to reduce the effect of initially inaccurate masks.Quaternion and translation gradients are stopped at each iteration's beginning to stabilize training.
  • Experimental setup: Experiments use anisotropic rotation and translation RMSE/MAE plus isotropic rotation and translation errors, with perfect alignment producing zero error.Angular metrics are reported in degrees.
  • ModelNet40 results: OMNet ranks first in all metrics on the twice-sampled unseen-shape setting and outperforms all other methods on unseen categories.The unseen-shape result is reported on TS data, while the unseen-category experiment evaluates categories absent during training.
  • Ablation studies: Ablations show that mask prediction, mask loss, fusion layers, and module connections each improve performance, while unsupervised mask prediction provides no baseline improvement.Using only top-k mask-selected points performs worse than using the entire masks because selected points may not correspond or be centrally distributed.

5. Discussion

Discussion experiments examine OMNet’s iterative masks, noise robustness, and tolerance to decreasing overlap ratios. The visualizations show progressively improved alignment and masks, while the method maintains comparable performance across noise levels and performs best under tested overlap conditions.

  • Iterative behavior: Early iterations show large global-feature differences and inaccurate alignment, while later iterations produce nearly identical features, near-perfect alignment, and nearly perfect overlapping masks.Figure 5 visualizes registration, feature differences, and masks at each iteration; red and blue denote overlapping and non-overlapping predictions.
  • Noise robustness: OMNet achieves comparable performance across ModelNet40 noise levels using noise sampled from N(0, σ^2) and clipped to [−0.05, 0.05].The experiment uses the same data as the noisy unseen-shape evaluation.
  • Partiality robustness: As the overlapping ratio decreases from 1.0 to 0.1, OMNet shows the best performance among the tested methods, while DeepGMR fails to obtain sensible results.The evaluation uses the ModelNet40 TS validation set and selects adjacent partial regions for the reference cloud.

6. Conclusion

The paper concludes that OMNet addresses partial-to-partial registration by learning masks that reject non-overlapping points and reports state-of-the-art performance. It also proposes a data-generation procedure intended to avoid ModelNet40 over-fitting and removes axisymmetrical categories.

  • Conclusion: OMNet learns overlapping masks to reject non-overlapping points, making global-feature registration more robust to partiality.The method treats the retained regions as the basis for registration rather than aggregating all input points equally.
  • Conclusion: The paper proposes a practical ModelNet40 data-generation manner to address over-fitting and removes axisymmetrical categories from registration data.The supplementary material documents the alternative data-preparation manners and associated experiments.
  • Conclusion: Experiments are reported to show state-of-the-art performance for OMNet.The supplementary material provides additional experimental details and results for the data-preparation protocols.

B.1. Data Generation Manner of PRNet

The PRNet data-generation manner samples two point clouds from a shared pool of once-sampled CAD-model point clouds, applies a rigid transformation, and then partially crops both clouds.

  • Point-cloud sampling: Forty point clouds are sampled once from CAD models, and two are randomly selected as source X and reference Y, each containing 2,048 points.The source and reference are therefore drawn from the same once-sampled pool.
  • Rigid transformation: A rigid transformation is generated with per-axis rotations in [0°, 45°] and translations in [−0.5, 0.5], then applied to Y to obtain X.The passage describes random sampling along each axis.
  • Partiality generation: X and Y are simultaneously made partial by randomly placing a point in space and selecting neighboring points.The supplied passage truncates the remaining details of the neighborhood-selection procedure.

B.2. Data Generation Manner of RPMNet

The RPMNet-style generation samples source and reference point clouds independently from separate half-spaces, retaining approximately 70% of points before downsampling. This produces more decentralized pairs and can create cases with small overlap that are harder to register.

  • Data generation: Source and reference clouds are generated independently by sampling separate half-spaces with random directions and retaining approximately 70% of points.Each cloud is then downsampled to 717 points to maintain a similar point density to previous experiments.
  • Data characteristics: RPMNet-style pairs are more decentralized than PRNet-style pairs.
  • Data characteristics: Changing from once-sampled to twice-sampled data reduces the performance of all evaluated methods.OS denotes once-sampled data, while TS denotes the proposed twice-sampled data.
  • Data characteristics: Some generated cases have small overlap ratios, making registration more difficult.

C. More Experiments on ModelNet40

Additional ModelNet40 experiments examine over-fitting under changed sampling and evaluate registration across unseen shapes and categories. Results show substantial degradation from once-sampled to twice-sampled data, while OMNet remains competitive across settings.

  • Experimental scope: The experiments cover once-sampled versus twice-sampled data, PRNet-style partiality, and RPMNet-style partiality on ModelNet40.
  • Efficiency: Inference speed is profiled across point-cloud pair sizes, with OMNet evaluated using N = 4 iterations.The comparison reports times in milliseconds; Go-ICP is omitted because of its obviously slow speed.
  • C.1. Over-fitting Issue: All partial-to-partial methods perform well on once-sampled data but decrease noticeably when only the sampling manner changes to twice-sampled data.The experiment uses eight axisymmetrical categories and demonstrates over-fitting to the original once-sampled distribution.

C.2. Results on PRNet dataset

On unseen ModelNet40 categories with Gaussian noise and PRNet-style partiality, OMNet achieves accurate registration and ranks first among the evaluated methods.

  • Results: OMNet achieves accurate registration and ranks first on unseen categories with Gaussian noise generated using the PRNet partial-data manner.Noise is sampled from N(0, 0.012), clipped to [−0.05, 0.05], and added on each axis.

C.3. Results on RPMNet dataset

RPMNet-style experiments test OMNet on unseen shapes, unseen categories, Gaussian noise, efficiency, and iterative refinement. OMNet outperforms the compared methods across the reported generalization and noise settings, with most gains occurring in the first two iterations.

  • Experimental scope: The RPMNet-style experiments include unseen shapes with and without Gaussian noise and report results in Tables 7(a)–(d).
  • Unseen shapes: On unseen shapes, traditional and most deep-learning methods perform poorly under large initial-position differences and partiality.OMNet outperforms all traditional and deep-learning methods except on three metrics when compared with RPMNet on once-sampled data.
  • Unseen categories: Deep-learning methods worsen on unseen categories, whereas traditional methods are less affected; OMNet outperforms all compared methods.
  • Gaussian noise: With Gaussian noise, almost all deep-learning methods degrade, while OMNet achieves the best performance among all competitors.Noise is sampled from N(0, 0.012) and clipped to [−0.05, 0.05] on each axis.
  • Efficiency: Inference time is compared across point-cloud pair sizes, with deep-learning models tested on an NVIDIA RTX 2080Ti GPU and other methods on CPUs.OMNet is evaluated with four iterations, and Go-ICP is excluded because of its obviously slow speed.
  • Iterative refinement: Most performance gains occur in the first two registration iterations, leading to the choice of N = 4 as a speed–accuracy trade-off.Anisotropic and isotropic errors are computed after each iteration.
  • Cross-dataset evaluation: The Stanford 3D Scan evaluation uses a ModelNet40-trained model without fine-tuning and displays initial and registered point clouds with anisotropic and isotropic errors.
Loading 2103.00937v5…