PaperlayerView exact revision on arXiv

Source-linked AI summary

SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration

Sheng Ao, Qingyong Hu, Bo Yang, Andrew Markham, Yulan Guo

arXiv:2011.12149v2cs.CVcs.AIcs.LGcs.RO

TL;DR

Robust 3D local descriptors must handle rotation and imperfect scans while generalizing across datasets and sensor modalities. SpinNet uses a Spatial Point Transformer and neural feature extractor to learn compact, rotation-invariant descriptors, achieving strong registration results and unseen-scenario generalization.

  • Problem

    Existing local descriptors are either sensitive to rotations or depend on handcrafted features, limiting robust and general 3D surface matching.

  • Method

    SpinNet transforms local surfaces into cylindrical space with a Spatial Point Transformer and learns descriptors using point-based and 3D cylindrical convolutional neural layers.

  • Results

    92.8% average recall is reported after training only on 3DMatch, with nearly 13% improvement over the state of the art on unseen outdoor ETH feature matching.

  • Takeaways & Limitations

    SpinNet provides rotation-invariant, descriptive local representations with strong generalization across unseen scenarios and sensor modalities for point cloud registration.

Abstract

from arXiv · show

Extracting robust and general 3D local features is key to downstream tasks such as point cloud registration and reconstruction. Existing learning-based local descriptors are either sensitive to rotation transformations, or rely on classical handcrafted features which are neither general nor representative. In this paper, we introduce a new, yet conceptually simple, neural architecture, termed SpinNet, to extract local features which are rotationally invariant whilst sufficiently informative to enable accurate registration. A Spatial Point Transformer is first introduced to map the input local surface into a carefully designed cylindrical space, enabling end-to-end optimization with SO(2) equivariant representation. A Neural Feature Extractor which leverages the powerful point-based and 3D cylindrical convolutional neural layers is then utilized to derive a compact and representative descriptor for matching. Extensive experiments on both indoor and outdoor datasets demonstrate that SpinNet outperforms existing state-of-the-art techniques by a large margin. More critically, it has the best generalization ability across unseen scenarios with different sensor modalities. The code is available at https://github.com/QingyongHu/SpinNet.

1. Introduction

SpinNet targets robust, rotation-invariant local descriptors for matching noisy, incomplete 3D surfaces across differing viewpoints, densities, and sensor scenarios. It combines a Spatial Point Transformer with a neural feature extractor and reports strong indoor performance and unseen outdoor generalization.

  • Motivation: Accurate partial-surface matching supports 3D registration, segmentation, and recognition, but scans vary in viewpoint, completeness, noise, and point density.The goal is to align partially overlapping scans using point correspondences and shared local geometric patterns.
  • Limitations of Existing Methods: Existing handcrafted descriptors struggle to generalize from high-quality point clouds to noisy, large-scale real-world scans.These methods compute low-level geometric features or handcrafted statistical histograms.
  • Limitations of Existing Methods: Many learned descriptors are rotationally variant, while rotation-invariant alternatives rely on handcrafted features or external local reference frames that limit robustness and representation.These limitations reduce performance or generalization on unseen scans with noise and different point densities.
  • SpinNet: SpinNet learns rotation-invariant, descriptive features through end-to-end point transformations and neural layers without handcrafted features.Its design aims to preserve local patterns despite noise, incompleteness, and varying point densities.
  • SpinNet: SpinNet maps local surfaces into cylindrical space with a Spatial Point Transformer, then uses point-based and 3D cylindrical convolutional layers to produce representative descriptors.The transformer drives the representation toward SO(2) equivariance while retaining local point information.
  • Results: 92.8% average recall is achieved when SpinNet is trained only on 3DMatch and evaluated for feature matching across the indoor 3DMatch and outdoor ETH settings.The paper reports the strongest generalization ability across the unseen ETH dataset and nearly 13% improvement over the state of the art there.

2. Related Work

Prior descriptors trade off geometric detail, rotation robustness, and generalization. SpinNet instead learns compact rotation-invariant features end to end without handcrafted features or unstable external reference frames.

  • Handcrafted Descriptors: Handcrafted descriptors are either LRF-free and geometrically sparse or LRF-based and vulnerable to rotation errors, while both can be sensitive to noise and novel scenarios.LRF-free methods exploit properties such as curvatures and normal deviations; LRF-based methods characterize spatial attributes but introduce rotation errors.
  • Learning-based Descriptors: Deep neural descriptors generally provide stronger descriptive ability and robustness by learning local features from large-scale datasets.The related work contrasts these data-driven methods with traditional handcrafted constructions.
  • Rotation-Variant Descriptors: Rotation-variant learning methods are sensitive to rigid transformations in Euclidean space, and augmentation only alleviates the problem while subsequent-task performance remains sub-optimal.This limitation is reported for methods evaluated on 3DMatch and KITTI registration datasets.
  • Rotation-Invariant Descriptors: Rotation-invariant approaches encode surfaces using spherical histograms or point-pair features before learning compact descriptors with neural networks.Examples include oriented spherical histograms, rotation-invariant PPFs, and FoldingNet-based descriptors.
  • SpinNet: SpinNet transforms point clouds into a cylindrical volume and learns compact, descriptive rotation-invariant features end to end without handcrafted features or external LRFs.The paper presents this design as a way to improve generality on unseen 3D surfaces across datasets.

3. SpinNet

SpinNet formulates local 3D surfaces in a cylindrical representation and learns compact features through point-based and cylindrical convolutional layers. Its design combines spatial transformations, SO(2)-equivariant processing, and end-to-end optimization for rotation-invariant matching.

  • 3.1. Problem Statement: Point cloud registration seeks correspondences and a rigid transformation that align partially overlapping fragments into a complete scene.When correspondences are known, registration can be simplified to minimizing an L2 distance.
  • 3.1. Problem Statement: SpinNet maps correctly matching local points to equal descriptors under arbitrary rigid transformations through a Spatial Point Transformer and Neural Feature Extractor.The two modules are connected for end-to-end optimization with contrastive loss.
  • 3.2. Spatial Point Transformer: The Spatial Point Transformer aligns a surface with a reference axis, normalizes translation, voxelizes it spherically, and transforms voxel patterns before cylindrical formulation.The XY-plane transformation removes an additional rotational degree of freedom while preserving voxel patterns.
  • 3.2. Spatial Point Transformer: Cylindrical volume formulation preserves relative voxel positions and larger topological structures while making the representation suitable for 3D cylindrical convolution.The resulting feature maps retain surface patterns for subsequent feature learning.
  • 3.3. Neural Feature Extractor: Shared MLPs and max-pooling learn an initial signature for each voxel, producing cylindrical feature maps F ∈ R^J×K×L×D.The learned voxel features have D dimensions and are stacked into cylindrical maps.
  • 3.3. Neural Feature Extractor: The 3D Cylindrical Convolution Network wraps convolution across the cylinder’s periodic boundary, avoiding explicit padding and yielding SO(2)-equivariant feature maps.Stacked 3DCCN layers followed by max-pooling compress the maps into a compact feature vector.

4. Experiments

Experiments evaluate SpinNet on indoor and outdoor datasets, under varying sampling and error thresholds, and across unseen sensor modalities. SpinNet generally achieves the strongest matching and generalization results, while ablations identify the importance of its transformation and feature-learning components.

  • Experimental setup: Experiments cover 3DMatch, KITTI, and unseen-dataset generalization, using FMR, RTE, RRE, and success rate as evaluation measures.Training samples use overlapping fragment pairs, anchor points, and local support regions; 3DMatch uses FMR, while KITTI uses registration errors and success rate.
  • Indoor 3DMatch: SpinNet achieves the highest average FMR and lowest standard deviation on both original and arbitrarily rotated 3DMatch benchmarks, without rotation-based training augmentation.The comparison uses 5,000 sampled points, τ1=10 cm, and τ2=5%.
  • Indoor 3DMatch: SpinNet retains the best FMR scores when sampled points decrease from 5,000 to 250, indicating robustness to the number of input points.With random point selection, SpinNet also outperforms D3Feat-pred, which includes an explicit keypoint detector.
  • Indoor 3DMatch: 85.7% FMR at τ2 = 0.2 exceeds D3Feat’s 75.8% and FCGF’s 67.4%, showing stronger performance under a stricter inlier-ratio threshold.SpinNet consistently outperforms other methods across the evaluated distance and inlier-ratio thresholds.
  • Outdoor KITTI: On KITTI, SpinNet is on par with D3Feat but marginally below D3Feat-pred, whose joint descriptor and keypoint detector benefit from the dataset’s gravity-aligned scans.KITTI evaluates RTE, RRE, and success rate, with success requiring both RTE < 2 m and RRE < 5°.
  • Generalization and ablation: When trained on 3DMatch and tested on unseen datasets, SpinNet surpasses competing methods, including LMVD by about 13% on all four ETH scenes and reaching 69.19% success on KITTI.Other descriptors show substantial performance drops on ETH, while ablations indicate that the Spatial Point Transformer, learned local features, and 3DCCN each support performance or generalization.

5. Conclusion

The paper presents a neural descriptor for complex 3D surfaces that is rotation invariant, descriptive, and preserves local geometric patterns. Experiments report strong generalization across unseen scenarios and superior 3D point-cloud registration results.

  • SpinNet learns compact representations for complex 3D surfaces.
  • The learned representations are rotation invariant and preserve complex local geometric patterns.
  • Extensive experiments demonstrate remarkable generalization across unseen scenarios.
  • The descriptor achieves superior results for 3D point cloud registration.

B. Theoretical Proof of Equivariance

The theoretical analysis establishes equivariance for the Spatial Point Transformer and 3D cylindrical convolution network under a discrete 2D rotation group. The proof uses cylindrical-coordinate transformations and the periodicity of the azimuth axis.

  • The Spatial Point Transformer is proved equivariant for a discrete 2D rotation group R ⊂ SO(2).
  • The transformer maps a local patch into a cylindrical volume whose entries transform under rotations of the input patch.
  • The cylindrical representation uses periodicity along the azimuth axis when rotated coordinates wrap around.
  • The 3D cylindrical convolution network is proved equivariant for the same discrete 2D rotation group.
  • In cylindrical feature maps, a rotation shifts the azimuth coordinate while preserving the radial and height coordinates.

C. Detailed Network Architecture

The architecture stacks 3D cylindrical convolution layers into a hierarchical network that progressively learns compact local representations. Figure 6 specifies the architecture and its reproducibility-related layer settings, while the channel limit keeps the network lightweight.

  • 3D cylindrical convolution layers are stacked hierarchically to learn descriptive, compact local feature representations.
  • Figure 6 provides kernel size, stride, and filter-count details for reproducing the cylindrical convolution architecture.
  • The cylindrical feature map uses at most 128 channels, compared with 1024 in D3Feat.
  • The smaller channel count makes the network lightweight and less prone to overfitting.

D. Detailed Evaluation Metrics

The evaluation uses Feature Matching Recall for 3DMatch and ETH, and Relative Translational Error, Relative Rotation Error, and Success Rate for KITTI. These metrics are defined from fragment correspondences and estimated-versus-ground-truth transformations.

  • Feature Matching Recall is the main metric for evaluating learned descriptors on 3DMatch and ETH.
  • For 3DMatch, fragment pairs with overlap greater than 30% are aligned using ground-truth rigid transformations.
  • Point correspondences are generated by nearest-neighbor search in feature space.
  • KITTI evaluation reports Relative Translational Error, Relative Rotation Error, and Success Rate.
  • Relative rotation and translation errors compare estimated transformations with ground-truth transformations.
  • Success Rate counts fragment pairs satisfying RREh < 2m and RTEh < 5°.

E. Implementation Details

The section reports dataset-specific hyperparameter settings and quantitative-result table formats for 3DMatch and rotated 3DMatch.

  • E. Implementation Details: SpinNet uses the same parameter settings on unseen datasets except for support radius R and query radius Rv.The radii are adapted to varying point densities using a ratio-based scheme following D3Feat.
  • E. Implementation Details: Table 8 reports average recall (%) on the 3DMatch benchmark using τ1 = 10cm and τ2 = 0.05.Unavailable results are marked '-', while † identifies results reported from [13] that differ from Table 1.
  • E. Implementation Details: Table 9 reports average recall (%) on the rotated 3DMatch benchmark using τ1 = 10cm and τ2 = 0.05.Unavailable results are marked '-', while † identifies results reported from [13] that differ from Table 1.
  • E. Implementation Details: Table 10 lists the hyperparameters selected for SpinNet across different datasets.

F. Additional Results on 3DMatch

The paper additionally reports SpinNet’s quantitative results on both the standard and rotated 3DMatch benchmarks.

  • F. Additional Results on 3DMatch: SpinNet is quantitatively evaluated on the 3DMatch and rotated 3DMatch datasets.
  • F. Additional Results on 3DMatch: The standard 3DMatch results are reported in Table 8.
  • F. Additional Results on 3DMatch: The rotated 3DMatch results are reported in Table 9.

G. Additional qualitative results.

Additional qualitative experiments examine SpinNet’s cross-dataset registration behavior on 3DMatch, KITTI, and ETH, highlighting its generalization across sensor modalities and scene types.

  • Additional qualitative results on the 3DMatch dataset: On 3DMatch, SpinNet consistently registers fragments despite being trained only on sparse outdoor KITTI LiDAR scans.FCGF and D3Feat are prone to mismatching fragments with substantial differences between partial scans.
  • Additional qualitative results on the KITTI dataset: On KITTI, SpinNet achieves satisfactory registration after training only on indoor 3DMatch data.FCGF and D3Feat tend to misalign fragments containing geometrically similar objects such as cars.
  • Additional qualitative results on the ETH dataset: On ETH, the comparison evaluates methods on outdoor scans collected by static terrestrial lasers and dominated by bushes and vegetation.The ETH examples therefore extend qualitative testing beyond the indoor 3DMatch and sparse LiDAR KITTI settings.
  • Additional qualitative results: Figures 7–9 provide additional qualitative comparisons among FCGF, D3Feat, and SpinNet across the three datasets.Red boxes or circles identify failure cases in these figures.
Loading 2011.12149v2…