Source-linked AI summary
Local Spectral Graph Convolution for Point Set Feature Learning
Chu Wang, Babak Samari, Kaleem Siddiqi
TL;DR
Pointnet++ learns neighboring point features independently and uses max pooling that can discard information from disjoint subsets. This paper introduces dynamically constructed local spectral graph convolution with recursive spectral clustering and pooling, and reports state-of-the-art recognition and segmentation results across diverse datasets. The method is designed for end-to-end point-set feature learning without offline graph precomputation.
Problem
Pointnet++ independently learns features for each neighbor and uses greedy max pooling, limiting joint modeling of neighborhood structure and preservation of disjoint point-set information.
Method
The method dynamically builds local k-NN graphs, applies spectral filtering with learnable parameters, and aggregates representations through recursive clustering and pooling.
Results
The proposed architecture achieves new state-of-the-art object recognition and segmentation results on diverse datasets.
Takeaways & Limitations
Local spectral feature learning with recursive clustering and pooling captures local structural and geometric information for feature abstraction from unorganized 3D point sets.
Takeaways & Limitations
The spectral filtering implementation uses eigenvalue decomposition, although its local k-NN computation is reported not to significantly affect runtime.
Abstract
from arXiv · showhide
Feature learning on point clouds has shown great promise, with the introduction of effective and generalizable deep learning frameworks such as pointnet++. Thus far, however, point features have been abstracted in an independent and isolated manner, ignoring the relative layout of neighboring points as well as their features. In the present article, we propose to overcome this limitation by using spectral graph convolution on a local graph, combined with a novel graph pooling strategy. In our approach, graph convolution is carried out on a nearest neighbor graph constructed from a point's neighborhood, such that features are jointly learned. We replace the standard max pooling step with a recursive clustering and pooling strategy, devised to aggregate information from within clusters of nodes that are close to one another in their spectral coordinates, leading to richer overall feature descriptors. Through extensive experiments on diverse datasets, we show a consistent demonstrable advantage for the tasks of both point set classification and segmentation.
1 Introduction
The paper extends pointnet++ with local spectral graph convolution and recursive cluster pooling to learn structural information from unorganized point clouds. Extensive experiments report state-of-the-art recognition and segmentation results across diverse datasets.
- Motivation: Point-cloud processing remains challenging because sensed depth points vary in spatial density and may be incomplete from occlusion or perspective.
- Prior approach: Pointnet++ hierarchically samples centroids, forms neighborhoods, applies multilayer perceptrons, and pools features for classification and segmentation.
- Contribution: The proposed architecture combines local spectral graph convolution with a different pooling strategy to address isolated feature learning and greedy max pooling.
- Contribution: Local spectral graph convolution incorporates structural information from each point’s neighborhood, while the layer is dynamically constructed and trained end-to-end without offline computation.
- Contribution: Recursive clustering and pooling aggregates features using spectral coordinates, replacing winner-take-all aggregation with a richer neighborhood abstraction.
- Results: The proposed architecture achieves new state-of-the-art object recognition and segmentation results on diverse datasets.
2 Challenges in point set feature learning
The paper identifies two pointnet++ limitations: independently learned neighbor features and max pooling that can discard information from disjoint point subsets. It addresses them with joint local spectral convolution and recursive spectral clustering and pooling.
- Feature-learning limitations: Pointnet++ applies a hidden function independently to each k-nearest neighbor, so it does not model their joint relationship.
- Feature-learning limitations: Joint convolution over a neighborhood can capture topological information from geometric layout alongside features such as color and texture.
- Pooling limitation: Max pooling across neighborhood activations cannot preserve information from disjoint point sets, such as an ant’s legs.
- Proposed solution: The proposed recursive spectral clustering and pooling module provides an improved set activation function for k-nearest-neighbor neighborhoods.
- Spectral convolution: Spectral graph convolution jointly processes all points in a local neighborhood, incorporating neighboring features and graph-topology structure rather than using point-wise MLPs.
- Combined operation: The combined abstraction uses convolution outputs for each point as inputs to the proposed set activation function.
3 Graph Convolution
The method builds local k-NN graphs, applies spectral graph convolution to jointly process neighborhood features, and transforms the filtered signals back to the vertex domain. Graphs, Laplacians, and pooling hierarchies are computed during runtime, with shared spectral and feature-filter parameters across neighborhoods.
- Spectral filtering: Spectral convolution transforms point features into the graph Fourier basis, applies learnable spectral modulation and feature filtering, then uses an inverse transform.The Fourier transform is ˜X = U^T X, and the inverse is X = U ˜X.
- Local graph construction: Local k-NN graphs represent neighboring points with distance-based weighted edges before graph-spectrum computation and Fourier-domain filtering.The graph may use spatial xyz distances or distances in a learned feature space.
- Graph Fourier transform: The normalized Laplacian is symmetric positive semidefinite, and its orthonormal eigenvectors form the graph Fourier basis.Its eigenvalues provide the graph frequencies used to construct the diagonal spectrum matrix.
- Spectral filtering: Feature filtering expands the input dimension from m to m′ through Q = PW, where W contains learnable filter parameters.The output feature qk,i is formed by combining the spectrally modulated input features with the filter coefficients.
- Runtime implementation: The method dynamically builds each local graph and computes its Laplacian and pooling hierarchy on the fly, avoiding offline precomputation.Spectral and feature-filter weights are shared across local neighborhoods within a graph-convolution layer.
- Runtime implementation: Local eigendecompositions keep the overhead manageable because k is small and the computations can be parallelized on GPUs.The authors report that this overhead does not significantly affect runtime in their implementation.
4 Pooling on Local k-NN Graph
The pooling module hierarchically clusters local graph nodes using spectral coordinates and pools features within clusters rather than relying only on greedy max pooling. Recursive coarsening and alternating pooling aggregate structurally distinct but locally related information.
- Pooling strategy: The proposed pooling abstracts a k-vertex local graph to one vertex through hierarchical clustering and within-cluster feature pooling.It replaces the greedy max-pooling strategy with pooling performed inside clusters of similar abstract point features.
- Motivation: Clustering can preserve information from disjoint feature groups, such as multiple shape parts, that greedy max pooling may discard.The stated motivation is to capture several distinct feature types that jointly contribute to a salient point-set property.
- Spectral coordinates: Spectral coordinates derived from low-frequency Laplacian eigenvectors encode coarse local shape properties and organize geometrically related features.These coordinates provide a discriminative embedding of local object geometry.
- Recursive clustering: Recursive clustering sorts Fiedler-vector entries and evenly partitions them into k1 sections, grouping points whose entries fall in the same section.The first iteration produces k1 clusters with cluster size c = k/k1.
- Recursive clustering: Each recurrence pools the clustered features to form a coarsened graph, then repeats the process to obtain progressively fewer clusters.The algorithm terminates when the remaining vertex count is no larger than the prescribed cluster size, followed by regular full-stride pooling.
- Pooling configuration: Alternating max and average pooling across recurrences is used to increase the discriminative power of the graph-pooling algorithm.The reported experiments use max pooling first, two clustering recurrences, and a final pooling size of 2.
5 Experiments
Experiments across classification and segmentation datasets compare the proposed spectral graph convolution and recursive cluster pooling against pointnet++ and other methods. The results show consistent performance gains, state-of-the-art-level results, and comparable training time.
- Datasets: The evaluation covers MNIST, ModelNet40, McGill Shape Benchmark, ShapeNet part segmentation, and ScanNet indoor scene segmentation.Experiments use varying point counts, surface normals, and task-specific architectures.
- Ablation Study: The ablation study shows spectral graph convolution and recursive cluster pooling improve pointnet++ on both classification and segmentation.The study separately evaluates local spectral filtering, recursive cluster pooling, scalability, additional features, and training time.
- Ablation Study: Spectral graph convolution on local k-NN graphs outperforms point-wise MLP, while recursive cluster pooling further increases learned-feature discriminative power.The authors attribute the first improvement to encoded topological information and the second to aggregation organized by spectral coordinates.
- Classification Experiments: 0.7% additional overall instance-level accuracy is obtained on the McGill Shape Benchmark when recursive cluster pooling replaces max pooling.Spectral graph convolution also consistently improves average instance-level and category-level accuracy.
- Classification Experiments: 20% error rate reduction over pointnet++ is reported on MNIST, with performance also surpassing the Network in Network model.The comparison uses 784 input points and the 1k experimental network.
- Classification Experiments: The method provides consistent improvement over pointnet++ and state-of-the-art-level classification performance on ModelNet40.Results are reported for 1024 xyz points and for 2048 xyz points with surface normals.
- Segmentation Experiments: State-of-the-art-level performance is achieved on both ShapeNet part segmentation and ScanNet indoor-scene semantic segmentation.ScanNet experiments remove RGB information and evaluate semantic voxel label prediction accuracy.
6 Conclusion
The method combines spectral graph convolution on local point neighborhoods with recursive cluster pooling to improve feature learning from unorganized 3D point sets.
- Spectral graph convolution on local point neighborhoods captures local structural information and geometric cues.
- Recursive cluster pooling operates on resultant representations as an alternative strategy for feature learning from unorganized 3D point sets.
- The approach demonstrates strong experimental performance, acceptable runtime, and versatility.