PaperlayerView exact revision on arXiv

Source-linked AI summary

USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds

Jiaxin Li, Gim Hee Lee

arXiv:1904.00229v1cs.CV

TL;DR

3D keypoint detection lacks robust detectors that are repeatable under arbitrary transformations without unavailable ground-truth labels. USIP learns keypoints from transformed point-cloud pairs with a feature proposal network and probabilistic chamfer loss, and extensive evaluations report significant improvements over existing detectors. Its distinctiveness evaluation is confounded with descriptor performance, limiting that criterion as a detector measure.

  • Problem

    3D keypoint detection lacks ground-truth labels for supervision, while handcrafted detectors have limited robustness to arbitrary transformations and point-cloud disturbances.

  • Method

    USIP uses a feature proposal network trained on randomly transformed point-cloud pairs, with probabilistic chamfer and point-to-point losses for repeatable, accurately localized keypoints.

  • Results

    USIP significantly outperforms existing hand-crafted and deep learning-based 3D keypoint detectors in repeatability, distinctiveness, and computational efficiency across Lidar, RGB-D, and CAD evaluations.

  • Takeaways & Limitations

    USIP provides a fully unsupervised 3D keypoint detector designed for repeatability and accurate localization across varied point-cloud datasets.

  • Takeaways & Limitations

    Distinctiveness is confounded with descriptor performance and is therefore less suitable than repeatability for evaluating keypoint detectors.

Abstract

from arXiv · show

In this paper, we propose the USIP detector: an Unsupervised Stable Interest Point detector that can detect highly repeatable and accurately localized keypoints from 3D point clouds under arbitrary transformations without the need for any ground truth training data. Our USIP detector consists of a feature proposal network that learns stable keypoints from input 3D point clouds and their respective transformed pairs from randomly generated transformations. We provide degeneracy analysis of our USIP detector and suggest solutions to prevent it. We encourage high repeatability and accurate localization of the keypoints with a probabilistic chamfer loss that minimizes the distances between the detected keypoints from the training point cloud pairs. Extensive experimental results of repeatability tests on several simulated and real-world 3D point cloud datasets from Lidar, RGB-D and CAD models show that our USIP detector significantly outperforms existing hand-crafted and deep learning-based 3D keypoint detectors. Our code is available at the project website. https://github.com/lijx10/USIP

1. Introduction

3D keypoint detection seeks stable, well-localized points that remain repeatable under arbitrary transformations, but existing approaches lack robust handcrafted methods and ground-truth supervision. USIP addresses this gap with an unsupervised detector, transformation-based training, localization-aware proposals, and repeatability losses.

  • Motivation: 3D keypoints support registration, SLAM, object recognition, and place recognition by providing correspondences or locations for representative signatures.Reliable keypoints are important because failures can make these downstream tasks fail catastrophically.
  • Motivation: Handcrafted 3D detectors struggle to extract meaningful information from coordinates and to handle noise, density variation, and arbitrary transformations.Their performance deteriorates because they rely on local geometric properties.
  • Motivation: Ground-truth keypoints cannot be identified and labeled reliably, leaving very few deep learning-based 3D keypoint detectors.Unlike descriptor training, registered overlapping point clouds do not provide ground-truth keypoint labels.
  • Contributions: USIP is fully unsupervised and analyzes detector degeneracy to avoid requiring unavailable ground-truth annotations.The paper explicitly presents degeneracy analysis and solutions for preventing degenerate network outputs.
  • Contributions: The feature proposal network estimates keypoint positions instead of selecting existing input points, improving localization.Proposed keypoints need not coincide with points in the input cloud.
  • Contributions: Probabilistic chamfer and point-to-point losses encourage repeatability and accurate localization, while random training transformations support rotation robustness.The losses compare keypoints from transformed training pairs.

2. Related Work

Existing 3D keypoint detectors are predominantly handcrafted and select points using local geometric properties or projected representations. Their performance can deteriorate under noise, density changes, and arbitrary transformations, motivating USIP’s learned alternative.

  • Handcrafted Detectors: Most existing 3D keypoint detectors remain handcrafted, despite recent progress in deep learning-based 3D keypoint descriptors.The reviewed methods include curvature extrema, principal-axis variation, scale-space operators, and Laplace-Beltrami saliency.
  • Handcrafted Detectors: LSP, SI, ISS, KPG, MeshDoG, SP, and LBSS identify salient points from local curvature, principal-axis variation, or scale-space geometric operators.These approaches extend geometric saliency ideas such as SIFT-like local extrema detection to 3D data.
  • Handcrafted Detectors: LORAX projects point sets into depth maps and applies PCA to select keypoints with commonly occurring geometric characteristics.It represents a projection-based alternative to the other handcrafted detectors.
  • Limitations and Learned Alternative: Handcrafted methods rely on local geometry, so their performance deteriorates under noise, density variations, and arbitrary transformations.USIP instead learns from data and is designed to encourage repeatability and accurate localization.
  • Limitations and Learned Alternative: 3DFeatNet was the only cited deep learning-based 3D keypoint detector, but its attention score map is a descriptor-learning by-product and does not ensure strong detection.USIP is presented as fully unsupervised and explicitly optimized for repeatability and localization.

3. Our USIP Detector

USIP trains on transformed point-cloud pairs and combines probabilistic correspondence modeling with point-to-point regularization to learn repeatable, accurately localized keypoints.

  • Random SE(3) transformations create paired training inputs, whose FPN outputs proposal keypoints and saliency uncertainties.The transformed proposals are mapped back before comparison.
  • USIP minimizes a total loss combining probabilistic chamfer loss Lc with weighted point-to-point loss Lp.Lc compares proposal correspondences probabilistically, while Lp keeps estimated keypoints near the input point cloud.
  • The probabilistic chamfer loss assigns higher probability to proposal pairs with shorter distances and learns saliency uncertainties for those proposals.The formulation uses an exponential distribution and nearest-neighbor distances.
  • The loss is asymmetric when the point-cloud order is swapped because nearest-neighbor distances and uncertainties can differ by direction.The joint distribution therefore differs between Q′ and Q.
  • The point-to-point loss permits subpoint proposal locations while penalizing proposals that lie too far from the point cloud.The authors report similar performance for point-to-point and point-to-plane variants, using point-to-point by default.

4. Feature Proposal Network

The FPN builds hierarchical, translation-equivariant features from FPS-based local neighborhoods, then predicts proposal keypoints and their saliency uncertainties.

  • Farthest Point Sampling selects M nodes, and point-to-node grouping assigns neighborhoods of varying sizes to those nodes.Every input point is associated with one node, and grouping adapts to point-density variation.
  • Neighborhood coordinates are normalized by subtracting their associated node positions to make local processing translation equivariant.The normalized clusters are processed by a PointNet-like network.
  • A kNN layer aggregates local feature vectors hierarchically before an MLP predicts M proposal keypoints and positive saliency uncertainties.The predicted offsets are un-normalized by adding the node positions back.
  • The numbers of proposals M and neighborhood neighbors K control receptive-field size and therefore the scale of the learned features.Larger receptive fields produce features salient at larger scales.

5. Degeneracy Analysis

USIP’s equivariance objective admits trivial centroid and principal-axis solutions; the paper analyzes these degeneracies and prevents them by limiting receptive fields.

  • Degeneracy Analysis: A network is degenerate when its outputs transform equivariantly for every rotation and translation without providing nontrivial interest points.The formal condition is f(Y′) ≡ Rf(Y) ⊕ t.
  • Degeneracy Analysis: Outputting the input centroid satisfies the equivariance condition, producing a centroid degeneracy.The centroid transforms directly under rotations and translations.
  • Degeneracy Analysis: Outputs lying on a principal axis also satisfy the equivariance condition under the analyzed translationally equivariant setting.The output points are represented by scalar coefficients along a principal axis.
  • Discussions: Degeneracies require sufficient global semantic information, so the paper prevents them by limiting FPN receptive fields through reasonable M and K values.More regular shapes, such as ModelNet40 objects, exhibit better-defined centroids and principal axes.
  • Degeneracy Analysis: Increasing K causes a progression from no degeneracy to principal-axis and centroid degeneracies when M = 64.The figure associates K = 9, 24, and 64 with these three cases.

6. Experiments

Experiments evaluate USIP on KITTI, Oxford, Redwood, and ModelNet40 for repeatability, robustness, registration, and computational efficiency. USIP generally outperforms competing detectors across these evaluations.

  • Repeatability: Repeatability is measured as the fraction of detected keypoints passing the repeatability test, across four datasets and multiple keypoint counts.The datasets are KITTI, Oxford, Redwood, and ModelNet40; evaluations include 4 through 512 keypoints where feasible.
  • Repeatability: 34%, 23%, 10%, and 60% relative repeatability are achieved by USIP with four keypoints on KITTI, Oxford, Redwood, and ModelNet40, respectively.
  • Robustness: USIP remains robust under Gaussian noise, while other detectors fall to random-sampling performance at specified noise levels on KITTI, Oxford, and Redwood.On ModelNet40, USIP maintains 91% repeatability at σnoise = 0.02 while other methods drop below 8%.
  • Robustness: USIP maintains satisfactory repeatability under 16× downsampling on KITTI, Oxford, and ModelNet40, whereas Redwood remains difficult for nearly all detectors.Redwood contains large flat surfaces, few distinguishable non-occluded structures, and severe downsampling effects.
  • Registration and efficiency: USIP achieves the lowest registration failure rate and highest inlier ratio across four descriptors, and is an order of magnitude faster than other detectors except random sampling.Registration uses six detectors and four descriptors; efficiency is measured for extracting 128 keypoints from KITTI point clouds.

7. Conclusion

The paper presents USIP as an unsupervised 3D keypoint detector that learns repeatable, accurately localized points without ground-truth keypoint labels. Experiments across Lidar, RGB-D, and CAD data report advantages in repeatability, distinctiveness, and computational efficiency.

  • USIP is an unsupervised deep learning-based detector for 3D point clouds that avoids unavailable ground-truth keypoint labels.
  • The detector uses probabilistic chamfer loss to learn highly repeatable keypoints and analyzes network degeneracy with proposed solutions.
  • Evaluations on Lidar scans, RGB-D images, and CAD models report significant improvements over existing detectors in repeatability, distinctiveness, and computational efficiency.

B. More Examples on Degeneracy

The supplementary analysis examines USIP degeneracy by enlarging the feature proposal network’s receptive field. The analysis links degeneracy to receptive-field control through proposal and neighborhood parameters.

  • USIP’s feature proposal network degenerates when its receptive field becomes sufficiently large to acquire global semantic information.
  • The receptive-field size is proportional to the number of KNN neighbors K and inversely proportional to the number of keypoint proposals M.
  • Degeneracy is visualized by gradually enlarging the receptive field.

C. Effect of λ in Point-to-Point Loss Lp

The analysis studies how the point-to-point loss weight λ affects USIP. Repeatability is stable across a wide λ range, but excessive weighting prevents convergence and omitting the loss reduces localization assurance.

  • USIP’s keypoint repeatability remains almost unchanged across a wide range of point-to-point loss weights λ.
  • The network fails to converge when λ is too large because the point-to-point loss dominates training.
  • Training without point-to-point loss does not ensure keypoints remain close to the input point cloud.With λ = 6, the detected keypoints are close to the input point cloud.

D. Our Descriptor a.k.a “Our Desc.”

“Our Desc.” extracts descriptors around USIP proposals and trains them with weak or strong supervision, weighting descriptors by USIP saliency uncertainty. Its weakly supervised formulation uses triplet learning over anchor, overlapping positive, and non-overlapping negative point clouds.

  • Descriptor extraction: Given FPN outputs (Q, Σ), “Our Desc.” builds a radius-r neighborhood around each proposal and extracts a descriptor for every keypoint.The FPN provides proposal locations Q and saliency uncertainties Σ used during descriptor processing.
  • Weak supervision: Weak supervision uses anchor point clouds, overlapping pairs as positives, and non-overlapping pairs as negatives in a triplet-loss framework.For each anchor descriptor, training minimizes distance to the nearest positive descriptor and maximizes distance to the nearest negative descriptor.
  • Uncertainty weighting: The triplet loss weights each anchor descriptor using normalized weight w_m derived from its USIP saliency uncertainty σ_m.σ_m indicates the reliability of the corresponding keypoint and descriptor.
  • Strong supervision: Strong supervision trains on overlapping point-cloud pairs with ground-truth poses, matching descriptors whose transformed keypoint locations lie within distance ρ.Hard negatives are sampled partly at random and partly from the closest keypoints beyond the distance threshold.

E. More Point Cloud Registration Results

The registration evaluation compares USIP and “Our Desc.” with existing methods on Oxford RobotCar and KITTI. USIP achieves strong registration performance and maintains performance better as RANSAC iterations decrease, while uncertainty-weighted descriptor training improves over random-keypoint training.

  • Experimental setup: The evaluation uses 256 extracted keypoints per cloud, omits NMS, and filters keypoints with large saliency uncertainty σ.Experiments use Oxford RobotCar and KITTI point clouds, with random up-axis rotations applied to evaluation clouds in Oxford RobotCar.
  • Registration performance: USIP plus “Our Desc.” achieves the lowest registration failure rate, RTE, and RRE, while attaining the highest inlier ratio among compared methods.Table 5 reports around 2× the inlier ratio of the comparison methods.
  • RANSAC robustness: USIP’s registration failure rate drops little when maximum RANSAC iterations decrease, whereas other methods and random sampling degrade rapidly.The comparison is shown in Figure 12, whose x-axis uses a logarithmic scale.
  • Uncertainty ablation: Descriptors trained with USIP keypoints and saliency uncertainty outperform descriptors trained with randomly sampled keypoints in registration failure rate.Table 6 compares “Desc. w. USIP” against “Desc. w. RS”.
  • Parameter effects: Registration performance remains strong across broad M, K, and λ ranges, but worsens when M is too small or K is too large.A large K increases the receptive field; the point-to-point loss remains important for keeping keypoints close to the input cloud.

F. Qualitative Visualization of USIP Keypoints

Visualizations show USIP keypoints on ModelNet40, KITTI, Oxford RobotCar, and Redwood, alongside analyses of receptive-field-related FPN degeneracy. Highly symmetrical or smooth objects can yield few or no retained keypoints.

  • Limitation: Highly symmetrical or smooth objects may produce no or very few retained keypoints because their saliency uncertainties are large.These keypoints are discarded by saliency-uncertainty thresholding.
  • FPN degeneracy: FPN degeneracy visualizations vary K from 9 to 64 with M fixed at 64, increasing the receptive field from left to right.The figure examines how changing K affects the receptive field.
  • FPN degeneracy: A second degeneracy visualization varies M from 64 to 9 with K fixed at 9, again increasing the receptive field from left to right.This figure examines the effect of changing M while holding K constant.
Loading 1904.00229v1…