Source-linked AI summary
PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors
Haowen Deng, Tolga Birdal, Slobodan Ilic
TL;DR
3D local descriptors have lacked discriminative power and have often required supervision, rotation-sensitive processing, or handcrafted preparation. PPF-FoldNet addresses these gaps by auto-encoding rotation-invariant 4D point pair features with a folding-based network. It reports state-of-the-art benchmark performance, including stronger results under rotations and reduced point density, while offering interpretable feature visualization.
Problem
3D local descriptors have lacked discriminative power and repeatability, while existing deep methods often required labels, were sensitive to 6DoF rotations, used handcrafted inputs, or performed unsatisfactorily.
Method
PPF-FoldNet uses a PointNet-like encoder and FoldingNet-like decoder to auto-encode 4D point pair features from local patches into rotation-invariant descriptors without supervision.
Results
PPF-FoldNet outperforms state-of-the-art descriptors across standard benchmarks and shows larger advantages under arbitrary rotations and sparse inputs.
Takeaways & Limitations
The approach provides an unsupervised, rotation-invariant, sparse, fast, and interpretable route to 3D local feature learning.
Takeaways & Limitations
The paper identifies adaptation to classification and object pose estimation as future work rather than an evaluated application.
Abstract
from arXiv · showhide
We present PPF-FoldNet for unsupervised learning of 3D local descriptors on pure point cloud geometry. Based on the folding-based auto-encoding of well known point pair features, PPF-FoldNet offers many desirable properties: it necessitates neither supervision, nor a sensitive local reference frame, benefits from point-set sparsity, is end-to-end, fast, and can extract powerful rotation invariant descriptors. Thanks to a novel feature visualization, its evolution can be monitored to provide interpretable insights. Our extensive experiments demonstrate that despite having six degree-of-freedom invariance and lack of training labels, our network achieves state of the art results in standard benchmark datasets and outperforms its competitors when rotations and varying point densities are present. PPF-FoldNet achieves $9\%$ higher recall on standard benchmarks, $23\%$ higher recall when rotations are introduced into the same datasets and finally, a margin of $>35\%$ is attained when point density is significantly decreased.
1 Introduction
PPF-FoldNet addresses limitations of prior 3D local descriptors with an unsupervised, rotation-invariant architecture that operates directly on sparse point sets. It auto-encodes 4D point pair features using a PointNet-like encoder and FoldingNet-like decoder, while supporting interpretable feature visualization.
- Prior 3D deep-learning descriptors commonly required supervision, were sensitive to 6DoF rotations, relied on handcrafted input preparation, or achieved unsatisfactory performance.
- The architecture operates directly on point sets, accounts for sparsity and permutation invariance, handles density variations, and is designed to be fast and end-to-end.
- PPF-FoldNet converts local patches into 4D point pair features, encodes them into compressed codewords, and reconstructs the full PPFs through folding.The decoder’s reconstruction objective encourages codewords to retain critical and discriminative information.
- Extensive evaluations report superiority over state-of-the-art descriptors on standard benchmarks, with larger margins under arbitrary rotations and sparse inputs.The paper also reports qualitative analyses that visualize PPF reconstruction and monitor network progress.
- The network learns rotation-invariant 3D local features without supervision by combining established 4D PPFs with a modified auto-encoder.
2 Prior Art
Prior 3D descriptor research spans voxel, projection, graph, point-set, and folding-based approaches, but generic local descriptors remain relatively scarce. Existing methods commonly retain supervision, rotation sensitivity, handcrafted inputs, or limited invariance.
- Early 3D deep-learning methods used dense voxel grids, later replaced or complemented by sparse octree and kd-tree representations for efficiency and robustness.
- Projection-based methods reduce 3D surfaces to two-dimensional representations so they can exploit developments in 2D convolutional networks.
- Graph-based approaches model point clouds through neighbor edges, but the cited efforts remained supervised and did not outperform unstructured point-processing methods.
- Generic 3D local descriptors are uncommon; 3DMatch, PointNet-based methods, CGF, and PPFNet each retain limitations involving supervision, rotation invariance, input preparation, or descriptor scope.
- FoldingNet introduced folding as an unsupervised decoder alternative that deforms a low-dimensional grid into a point cloud instead of using costly voxelization or point-set interpolation.
- PPFNet improved descriptor performance with global context but still relied on supervision and achieved only slight resilience to Euclidean isometries.
3 PPF-FoldNet
PPF-FoldNet encodes local patches as rotation-invariant point pair features and learns compact descriptors through unsupervised folding-based auto-encoding. Its permutation-invariant encoder and grid-based decoder reconstruct PPF sets while supporting interpretability through feature visualization.
- PPF Auto-Encoder and Folding: The network auto-encodes each PPF patch with a PointNet-like encoder and FoldingNet-like decoder, using one low-dimensional codeword to reconstruct the full PPF set.The decoder deforms a low-dimensional grid under codeword guidance rather than directly upsampling or interpolating point sets.
- Local Patch Representation: PPF-FoldNet represents each local patch as a collection of 4D point pair features computed between a reference point and neighboring points.The representation excludes orientation-dependent points and normals, using three angles and pair distance instead.
- Local Patch Representation: PPF representations explain oriented point pairs up to rotation and reflection about the reference normal, avoiding canonical reference-frame computation.Ignoring reflections leaves one rotational degree of freedom around the normal.
- PPF Auto-Encoder and Folding: The encoder is permutation invariant because point-wise MLPs, nonlinearities, max-pooling, and their composition preserve invariance to input ordering.Reordering the PPF set therefore does not affect the learned representation.
- PPF Auto-Encoder and Folding: Folding makes the decoder interpretable by allowing the evolving grid and reconstructed PPF sets to be monitored during training.PPF sets can be visualized through geometrically meaningful signatures and latent-space embeddings.
- PPF Auto-Encoder and Folding: The Chamfer metric compares reconstructed and input PPF sets when their cardinalities differ and point correspondences are unavailable.The grid size M need not equal the input size N.
4 Experimental Evaluation
PPF-FoldNet is evaluated on 3DMatch using fragment-pair recall, with tests covering standard accuracy, rotations, sparsity, alternative PPF constructions, runtime, qualitative matching, and interpretability.
- Datasets and preprocessing: The evaluation uses 3DMatch fragments, with 54 scenes reserved for training and validation and 8 scenes for benchmarking.Fragments are downsampled, and local patches are formed from points within a 30 cm vicinity.
- Accuracy assessment: Feature quality is measured by recall of overlapping fragment pairs, using τ1 = 10 cm and τ2 = 5% in the primary benchmark.Fragment pairs with more than 30% overlap are considered matching; τ1 controls point distance error and τ2 the required inlier ratio.
- Standard benchmark results: 68.04% recall is achieved by PPF-FoldNet with 2K sample points, versus 62.32% for PPFNet; using 5K points adds 3% improvement.Across the standard benchmark, PPF-FoldNet exceeds the state of the art by more than 9% on average, except in Kitchen and Home.
- Rotation robustness: 18.78% and 23.24% are PPF-FoldNet’s margins over CGF on the rotated benchmark with 2K and 5K points, respectively.PPF-FoldNet achieves the best results in all scenes, while 3DMatch and PPFNet fail under arbitrary rotations.
- Sparsity robustness: At 6.25% of the original points, PPF-FoldNet retains recall above 50%, while PPFNet remains around 12% and other methods almost fail.The evaluation decreases fragment point counts from 100% to 6.25%, testing robustness to sparsity, density changes, and noise.
- Runtime and qualitative analysis: PPF-FoldNet computes fragment features in 3.969 seconds, a 10× speed-up over FPFH at 31.678 seconds on the reported hardware.The reported PPF extraction and inference times are 2.616 and 1.353 seconds, respectively.
5 Concluding Remarks
PPF-FoldNet is presented as an unsupervised, rotation-invariant network for learning 3D local features from point geometry, with interpretable visualizations and strong performance under challenging conditions. Its broader application beyond local feature extraction remains prospective.
- PPF-FoldNet learns 3D local features solely from point geometry without supervision and is described as rotation invariant, low complexity, intuitive, and interpretable.
- Feature visualizations expose latent codewords, associated PPFs, clustered patches, and color-coded latent spaces on fused fragments.
- The network outperforms state-of-the-art descriptors, including supervised ones, under standard benchmarks with varying point density.
- The architecture is proposed as a promising approach to unsupervised 3D local feature extraction and a step toward broader unsupervised 3D vision.
- Extending the features to classification and object pose estimation is identified as a promising direction, with transfer to related problems remaining a hypothesis.
A.1 Evaluations on Generalizability
The generalizability evaluation addresses whether unsupervised learning can operate across datasets without annotated matches. PPF-FoldNet is trained on a small scene subset and evaluated on that scene and six unseen scenes.
- Annotated data can prevent supervised methods from immediately extending to different datasets without fine-tuning and may be prohibitive to obtain.
- PPF-FoldNet learns an unsupervised representation that can operate on any available dataset without auxiliary label information.
- The experiment trains PPF-FoldNet from scratch on part of the Chess scene and measures test loss on Chess plus six scenes excluded from training.
- The generalizability test is presented as an evaluation of the network's behavior across scenes using loss curves.
A.2 Additional Visualizations of Matching
Additional visual analysis examines qualitative matching across fragments and under rotations. The reported comparison highlights PPF-FoldNet's larger number of satisfactory matches relative to related methods in rotated settings.
- Figure 9 presents further qualitative analysis of matching rotated fragments across all evaluated methods.
- Qualitative results compare matching across different fragments and methods when transformations include rotations.
- Under rotations, only hand-crafted algorithms, CGF, and PPF-FoldNet achieve satisfactory matches, while PPF-FoldNet produces significantly more matches.