Source-linked AI summary
Learning Compact Geometric Features
Marc Khoury, Qian-Yi Zhou, Vladlen Koltun
TL;DR
Existing local descriptors for unstructured point clouds do not jointly achieve precision, compactness, and robustness, despite their importance for geometric registration. The paper learns compact Euclidean features by embedding high-dimensional spherical histograms with deep networks. The resulting CGF family is more precise and compact than hand-crafted descriptors and can replace them in registration pipelines.
Problem
Existing local geometric descriptors do not jointly provide high discriminative ability, compactness, and robustness for geometric registration.
Method
The approach represents local neighborhoods with spherical histograms and trains a deep network to embed them into a low-dimensional Euclidean space.
Results
CGF descriptors are more precise and compact than hand-crafted features and yield higher registration accuracy when replacing FPFH in existing pipelines.
Takeaways & Limitations
CGF provides a drop-in learned feature for robotics, 3D vision, and computer graphics applications using Euclidean nearest-neighbor search.
Takeaways & Limitations
Training assumes overlapping point-cloud fragments and known rigid transformations aligning them to a common coordinate frame.
Abstract
from arXiv · showhide
We present an approach to learning features that represent the local geometry around a point in an unstructured point cloud. Such features play a central role in geometric registration, which supports diverse applications in robotics and 3D vision. Current state-of-the-art local features for unstructured point clouds have been manually crafted and none combines the desirable properties of precision, compactness, and robustness. We show that features with these properties can be learned from data, by optimizing deep networks that map high-dimensional histograms into low-dimensional Euclidean spaces. The presented approach yields a family of features, parameterized by dimension, that are both more compact and more accurate than existing descriptors.
1. Introduction
Local geometric descriptors are central to registration but existing hand-crafted features do not jointly provide discriminability, compactness, and robustness. The paper learns Euclidean features directly from unstructured point clouds and reports improved precision and compactness over prior descriptors.
- Motivation: Local geometric descriptors encode neighborhood geometry for registration in robotics and 3D vision.Euclidean representations support efficient search, while discriminability and compactness are also required.
- Problem: No existing descriptor jointly satisfies high discriminative ability, compactness, and robustness.The difficulty includes manually optimizing parameters of high-dimensional representations.
- Results: Figure 1 presents CGF as a dimension-parameterized family evaluated on the SceneNN test set.The dimensionality axis is logarithmic, and the caption reports greater compactness and precision than baselines.
- Approach: The learned descriptor operates directly on unstructured point clouds without requiring surface parameterization, volumetric representations, or auxiliary depth images.Its Euclidean feature space supports dense correspondences through nearest-neighbor queries.
- Results: CGF yields more discriminative and compact descriptors than state-of-the-art hand-crafted features.Experiments report more accurate matches at lower query times and highest reported Redwood recall without dataset-specific training or fine-tuning.
2. Background
Prior work includes hand-crafted descriptors for rigid registration, learned methods using specialized surface or volumetric inputs, and image-descriptor techniques adapted conceptually to point clouds. Local geometric feature learning remains distinct from global shape classification and retrieval.
- Hand-crafted descriptors: Classic rigid-registration descriptors include Spin Images, 3D Shape Context, PFH, FPFH, SHOT, and USC.A comprehensive evaluation of existing local geometric descriptors is also cited.
- Registration setting: Descriptors for nonrigid registration commonly assume clean meshed surfaces and invariance to isometric deformations.This paper instead targets rigid registration of noisy, unstructured point sets, where sensitivity to isometric deformations is required.
- Learned descriptors: Learned matching methods have used ensembles of depth images, Riemannian manifolds, or volumetric signed distance fields as inputs.These approaches differ from the paper’s direct treatment of unstructured point clouds.
- Connection to image descriptors: Deep networks for image patches motivate this work’s investigation of output dimensionality for point-cloud descriptors.The paper reports that extremely low-dimensional descriptors can represent local geometry while accelerating correspondence search.
- Task distinction: Shape classification and retrieval use volumetric or multi-view representations but do not address accurate, compact local geometric features.The distinction parallels local image features versus image classification and retrieval.
3. Overview
The approach converts local point-cloud neighborhoods into spherical histograms, embeds them with a deep network into a low-dimensional Euclidean space, and uses nearest-neighbor search for correspondence. CGF can replace FPFH in existing registration pipelines without other modifications.
- Parameterization: Spherical histograms encode local geometry around each point using radial, elevation, and azimuth subdivisions aligned with local reference frames.The parameterization incorporates rotational invariance through normal and tangent spaces.
- Feature embedding: A deep network maps high-dimensional spherical histograms into a very low-dimensional Euclidean feature space.Triplet embedding loss trains similar geometric neighborhoods to map nearby in feature space.
- Correspondences: Correspondences reduce to nearest-neighbor queries between learned features using a k-d tree.The paper reports more accurate correspondences than those computed using its feature-space alternatives.
- Applications: Replacing FPFH with CGF in existing geometric registration pipelines yields higher registration accuracy without other modifications.This demonstrates CGF’s use as a drop-in descriptor replacement.
4. Input Parameterization
The input is a normalized spherical histogram of points within a local neighborhood, using a normal-aligned local frame and logarithmic radial bins. This representation captures multi-scale context while avoiding wasteful uniform voxelization.
- Histogram construction: A sphere centered at each point is subdivided into radial, elevation, and azimuth bins, with neighboring points accumulated into the resulting histogram.The two-dimensional illustration uses 24 bins, while the real three-dimensional histogram has approximately two thousand bins.
- Motivation: Uniform voxel grids waste cells because smooth surfaces intersect only O(C^2) of C^3 three-dimensional cells.A clean underlying surface parameterization is also unavailable in general.
- Reference frame: The local reference frame uses the estimated normal and an orientation sign check to provide rotational invariance.The representation is a histogram of local point distributions within a sphere.
- Bin structure: The azimuth, elevation, and logarithmic radial directions use A, E, and R bins, respectively.The total histogram dimensionality is N = R × E × A.
- Radial discretization: Exponential radial thresholds produce multi-scale context, while rmin prevents excessive binning near the center.The first threshold evaluates to rmin.
- Normalization: Neighboring points are found with a k-d tree, binned in constant time, and normalized by the neighborhood size before nonlinear embedding.The normalized histogram becomes the input to a lower-dimensional Euclidean representation.
5. Feature Embedding
The feature embedding maps high-dimensional spherical-histogram inputs into a low-dimensional Euclidean space where similar geometric neighborhoods are nearby. Triplet-loss training supports this embedding, while the network architecture enables efficient descriptor computation.
- Feature embedding: The network f maps input histograms from R^N into a lower-dimensional Euclidean feature space R^n.This mapping replaces arbitrary distances in the histogram space with learned geometric similarity and accelerates nearest-neighbor search.
- Feature embedding: The learned mapping improves search efficiency because nearest-neighbor search dominates geometric-registration runtime.
- Feature embedding: Triplet embedding loss trains the mapping using anchors, similar positive examples, and dissimilar negative examples.The objective encourages each anchor to be closer to its positive than its negative by a margin.
- Feature embedding: A fully connected network with 5 hidden layers and 512 nodes per layer computes descriptors through matrix multiplications and elementwise operations.Each hidden layer is followed by max(·, 0) truncation.
6. Training
Training triplets are constructed from aligned, overlapping point-cloud pairs by pairing nearby points as positives with harder, more distant neighbors as negatives. The procedure defines overlap geometrically and trains on many randomly partitioned triplets.
- 6. Training: Negative examples are sampled from the annulus N_p,2τ \ N_p,τ, producing difficult negatives that are more similar than randomly selected points.The innermost sphere supplies positive correspondences, while the outer region supplies negatives.
- 6. Training: Training assumes point clouds and rigid transformations that align overlapping fragments to a common coordinate frame.Nearby points representing the same latent scene region are close after transformation.
- 6. Training: Point-cloud pairs are declared overlapping when min(α_i,j, α_j,i) ≥ 0.3, corresponding to at least 30% surface overlap.
- 6. Training: For an anchor point, neighbors within distance τ are sampled as good correspondences from the overlapping cloud.
- 6. Training: The resulting triplets are generated in large numbers, randomly permuted, and partitioned into minibatches.
- 6. Training: Training uses minibatches of size 512, Adam, learning rate 10^-4, and three epochs.The first- and second-moment decay parameters are β_1 = 0.9 and β_2 = 0.999.
7. Experiments
Experiments evaluate CGF across laser scans, SceneNN fragments, qualitative visualizations, and registration pipelines. Across these settings, learned CGF descriptors provide accurate correspondences, compact representations, fast queries, and improved registration results.
- Laser scan data: CGF-32 yields 41.4% precision within 1% of model diameter, versus 32.2% for SI, while being 4.7 times more compact.
- Laser scan data: CGF-32 averages 0.42 ms per query on laser scans, 3.9 times faster than SI at 1.62 ms; CGF-12 reaches 0.05 ms and 33.2% precision.
- SceneNN data: On SceneNN, CGF-32 achieves 50.6% precision within 10 cm, compared with 29.8% for USC and lower precision for the other listed descriptors.
- Geometric registration: Replacing FPFH with CGF-32 improves FGR alignment from 82.96% to 92.27% on laser scans and from 88.54% to 91.19% on SceneNN.
- Geometric registration: With CGF-32, CZK achieves 72% recall on Redwood, while robust optimization produces 71.1% recall and 95.1% precision for pairwise alignments.
8. Conclusion
The paper presents learned discriminative features for local geometry in unstructured point clouds. These features achieve state-of-the-art accuracy in a low-dimensional Euclidean space and can replace hand-crafted descriptors in several applications.
- The learned descriptor represents local geometry in unstructured point clouds with discriminative, low-dimensional features.
- The descriptor is more precise and compact than hand-crafted features while achieving state-of-the-art accuracy.
- Its Euclidean structure enables drop-in replacement of existing features in robotics, 3D vision, and computer graphics.