Source-linked AI summary
Octree guided CNN with Spherical Kernels for 3D Point Clouds
Huan Lei, Naveed Akhtar, Ajmal Mian
TL;DR
Irregular point clouds challenge conventional convolutional networks because they lack regular grids and can make neighborhood construction, memory use, and dynamic filtering expensive. The paper introduces an octree-guided network with spherical, translation-invariant asymmetric kernels that quantize local neighborhoods and avoid dynamic kernel generation. Experiments cover classification and segmentation, with new state-of-the-art reported on ShapeNet and RueMonge2014.
Problem
Irregular point clouds do not fit standard CNN grids, while existing approaches can incur large memory footprints, costly searches, or dynamic-filter overhead.
Method
Ψ-CNN uses octree partitioning for hierarchical coarsening and neighborhood construction, while spherical kernels assign learnable weights to volumetric bins around each point.
Results
The method achieves new state-of-the-art results on ShapeNet part segmentation and RueMonge2014 semantic segmentation, with effectiveness also demonstrated on 3D object classification.
Takeaways & Limitations
Octree-guided spherical convolution provides an efficient way to process large, high-resolution 3D point clouds for classification and segmentation.
Takeaways & Limitations
Part-segmentation performance is affected by inconsistent ground-truth labels and small parts without clear boundaries.
Abstract
from arXiv · showhide
We propose an octree guided neural network architecture and spherical convolutional kernel for machine learning from arbitrary 3D point clouds. The network architecture capitalizes on the sparse nature of irregular point clouds, and hierarchically coarsens the data representation with space partitioning. At the same time, the proposed spherical kernels systematically quantize point neighborhoods to identify local geometric structures in the data, while maintaining the properties of translation-invariance and asymmetry. We specify spherical kernels with the help of network neurons that in turn are associated with spatial locations. We exploit this association to avert dynamic kernel generation during network training that enables efficient learning with high resolution point clouds. The effectiveness of the proposed technique is established on the benchmark tasks of 3D object classification and segmentation, achieving new state-of-the-art on ShapeNet and RueMonge2014 datasets.
1. Introduction
The paper addresses the difficulty of applying convolutional networks to irregular point clouds by combining translation-invariant, asymmetric spherical kernels with octree-guided processing. The resulting approach hierarchically coarsens point-cloud data, avoids costly neighborhood searches and dynamic kernel generation, and achieves new state-of-the-art results on ShapeNet and RueMonge2014.
- The architecture addresses irregular point-cloud processing, where standard CNNs require regular grids and adapted methods can impose excessively large memory footprints.
- Associating kernels with network neurons avoids dynamic kernel generation during training and supports efficient processing of high-resolution point clouds.
- Spherical kernels partition local 3D regions into volumetric bins with learnable weights, providing translation-invariant and asymmetric point-wise feature learning.
- Ψ-CNN uses octree partitioning to hierarchically coarsen point clouds and construct neighborhoods without time-consuming K-NN or range searches.
- New state-of-the-art results are reported on ShapeNet part segmentation and RueMonge2014 semantic segmentation, alongside experiments on ModelNet classification.
2. Related Work
Prior point-cloud methods include point-wise networks, tree-based architectures, graph convolutions, and volumetric CNNs, but face limitations involving geometric context, tree randomness, graph construction, dynamic filters, or resolution.
- PointNet omits geometric context, while PointNet++ adds local regions through K-NN and range searches.
- Kd-network performance depends heavily on randomized tree construction, unlike approaches using deterministic geometric relationships between points.
- Spectral graph networks require fixed graph structures, making them challenging to apply when point-cloud graphs vary.
- ECC dynamically generates filters and relies on range searches for neighborhood construction and graph coarsening, creating computational overhead.
- Volumetric 3D CNNs historically processed only low-resolution inputs because memory and computation grow cubically with volumetric resolution.
3. Spherical Convolutional Kernel
The spherical convolutional kernel quantizes each point’s local 3D neighborhood into learnable volumetric bins, while preserving translation invariance and asymmetric weight assignment. Its multi-scale spherical partition supports detailed local geometry and avoids the uniform-cell constraints of conventional 3D kernels.
- Each target point uses a radius-ρ spherical neighborhood partitioned along azimuth, elevation, and radial dimensions, plus a self-convolution bin, with one learnable weight matrix per bin.Neighboring points are transformed into spherical coordinates relative to the target point and assigned to the corresponding weight matrix.
- The spherical partition allows non-uniform radial splitting, placing finer-grained bins near the origin to encode detailed local geometry without requiring uniformly smaller 3D cells.This multi-scale granularity is contrasted with 3D kernels that rely on increased input resolution, which can create memory issues.
- The kernel is translation-invariant because neighborhoods are represented relative to each target point, while its binning prevents distinct point pairs from sharing weights symmetrically.The asymmetry property is intended to support finer geometric feature learning.
- The asymmetry lemma rules out applying one weight matrix symmetrically to distinct point pairs under the stated angular bin conditions, preserving directional distinctions in the neighborhood.The argument uses opposite displacement vectors for reversed point pairs and concludes symmetric assignment is possible only for identical points.
- For a comparable region, the described spherical construction yields 4 × 4 × 3 + 1 = 49 bins, identified as the coarsest multi-scale quantization allowed by the asymmetry lemma.The additional one accounts for the origin or self-convolution bin.
4. Neural Network
The network maps an octree hierarchy onto hidden layers and uses spatially associated neurons to apply spherical convolutions while coarsening point-cloud features. Classification aggregates multilevel features globally, whereas segmentation retains ancestor-path features for each point.
- Core Architecture: Octree-guided layers hierarchically coarsen point clouds, reducing neighborhood construction costs compared with K-NN or range searches.The octree partitions 3D space, assigns non-empty nodes to network neurons, and reduces node counts across layers.
- Core Architecture: The architecture replicates shallow leaf nodes to maximum depth and ignores empty nodes, preserving tree-to-network alignment with computational and memory benefits.The tree depth determines the hidden-layer count, while empty nodes are omitted during implementation.
- Core Architecture: Each neuron is associated with a spatial location that selects a spherical kernel; the network has as many hidden layers as the octree depth.Leaf-node locations use the mean of points assigned to maximum-depth leaves, while parent locations are derived from child locations.
- Core Architecture: Inter-layer convolutions replace intra-layer operations, substantially reducing the operations required for large point clouds relative to graph-based networks.Nodes with a single child require only self-convolutions.
- Classification and Segmentation: Classification concatenates root and pooled intermediate features, while segmentation concatenates each point’s raw, MLP, and ancestor-layer features before fully connected prediction.The shared core architecture is preceded by an MLP and uses features from all octree levels.
5. Experiments
Experiments evaluate Ψ-CNN on classification, part segmentation, semantic segmentation, and computational efficiency using octree-guided processing of point clouds. The method achieves strong benchmark performance while supporting large inputs and avoiding normal computation.
- 5.1. Classification: Ψ-CNN achieves the best ModelNet classification performance while using half as many parametric layers as Kd-Net and OctNet.The reported classification results use xyz coordinates as raw features.
- 5.2. Part Segmentation: 86.8% mIoU makes Ψ-CNN the top ShapeNet part-segmentation method, outperforming alternatives on 11 of 16 categories and exceeding SSCN and SGPN by about 1%.The comparison includes methods that sometimes use normals in addition to xyz coordinates.
- 5.4. Discussion: Low-quality segmentation cases mainly involve inconsistent ground-truth labels or small parts without clear boundaries, such as skateboard axles and bag handles.Figure 4 reports computed mIoU for representative high- and low-quality cases.
- 5.3. Semantic Segmentation: Ψ-CNN achieves better RueMonge2014 semantic-segmentation performance than OctNet with 7 parametric layers versus OctNet’s 20.The real-world data uses xyz+rgb features and is evaluated under the protocol of.
- 5.4. Discussion: For large inputs, octree neighborhood computation is more efficient than K-NN and range search, and more efficient than Kd-tree at large sizes.The comparison measures neighborhood-computation timings against octree construction.
6. Conclusion
The proposed network combines spherical convolutional kernels with octree-guided structure to process large 3D point clouds efficiently, achieving strong performance on classification and segmentation tasks.
- The network efficiently processes large 3D point clouds while achieving excellent performance on 3D classification and segmentation tasks.
- Spherical kernels divide occupied neighborhoods into bins, each associated with a learnable weight matrix.
- Octree nodes govern neuron locations and neighborhood convolutions throughout the network.