Source-linked AI summary
ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics
Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung
TL;DR
Existing point-cloud methods can be accurate but often involve complex architectures or slow training, while local feature learning must also handle point-order ambiguity. This paper introduces ShellConv, which summarizes concentric spherical shells to enable permutation-invariant convolution, and builds ShellNet for direct point-cloud learning. ShellNet reports state-of-the-art results across classification and segmentation tasks, with 93.1% classification accuracy at shell size 32.
Problem
Point-cloud learning needs local features while handling point-order ambiguity, and existing methods trade accuracy against network complexity or training speed.
Method
ShellConv partitions local neighborhoods into concentric spherical shells and uses order-invariant shell statistics for convolution, forming the ShellNet architecture.
Results
ShellNet achieves state-of-the-art results on object classification, object part segmentation, and semantic scene segmentation, including 93.1% classification accuracy with shell size 32.
Takeaways & Limitations
ShellConv supports efficient point-cloud convolution with larger receptive fields without increasing the number of network layers.
Takeaways & Limitations
Robustness to sparse and partial data requires further investigation; partial-data segmentation is less accurate at boundary points.
Abstract
from arXiv · showhide
Deep learning with 3D data has progressed significantly since the introduction of convolutional neural networks that can handle point order ambiguity in point cloud data. While being able to achieve good accuracies in various scene understanding tasks, previous methods often have low training speed and complex network architecture. In this paper, we address these problems by proposing an efficient end-to-end permutation invariant convolution for point cloud deep learning. Our simple yet effective convolution operator named ShellConv uses statistics from concentric spherical shells to define representative features and resolve the point order ambiguity, allowing traditional convolution to perform on such features. Based on ShellConv we further build an efficient neural network named ShellNet to directly consume the point clouds with larger receptive fields while maintaining less layers. We demonstrate the efficacy of ShellNet by producing state-of-the-art results on object classification, object part segmentation, and semantic scene segmentation while keeping the network very fast to train.
1. Introduction
Point-cloud methods must balance order-invariant local feature learning with accuracy, network complexity, and training speed. ShellConv and ShellNet address this trade-off using concentric-shell statistics and report strong results across three scene-understanding tasks.
- 1. Introduction: Point-cloud networks face a trade-off among network complexity, training speed, and accuracy when capturing local features.PointNet++ improves results with hierarchical local processing but is more complicated and slower; other methods achieve high accuracy with slow training or convergence.
- 1. Introduction: ShellConv partitions local neighborhoods into concentric spherical shells and extracts representative shell statistics to resolve point-order ambiguity.The shell structure supports efficient neighbor querying and defines convolution order from inner to outer shells.
- 1. Introduction: ShellNet is an efficient network built on ShellConv for direct point-cloud learning without point-order ambiguity.The architecture targets object classification, object part segmentation, and semantic scene segmentation.
- 1. Introduction: ShellNet applies to object classification, object part segmentation, and semantic scene segmentation with state-of-the-art accuracy.The paper presents these applications as core contributions of the proposed network.
2. Related Works
3D learning has shifted toward compact point clouds, but their set structure makes convolution difficult because point order is ambiguous. ShellConv organizes local neighborhoods into concentric shells and summarizes each shell for conventional convolution.
- 2. Related Works: Volume representations face large memory footprints, while multi-view images store depth implicitly and complicate view-independent feature learning.These limitations motivate alternatives for learning from 3D data.
- 2. Related Works: Point clouds are compact and intuitive, but their set structure requires fundamental changes to convolution.Efficient point-cloud convolution is described as both challenging and important.
- 2. Related Works: PointNet achieves point-order invariance with a symmetric function but learns features globally without local-region features.Later methods therefore focus on convolution operators that capture local point-cloud features.
- 2. Related Works: Subsequent methods capture local structure through hierarchical PointNet, learned transformations, parameterized kernels, or neighbor-weighted convolution.These approaches represent different strategies for adapting neural operators to unstructured point clouds.
- 2. Related Works: ShellConv instead partitions neighborhoods into concentric spherical shells and aggregates each shell with a permutation-invariant statistic.The supplied Figure 2 passage illustrates representative-point sampling, shell construction, maxpooling, feature lifting, and inner-to-outer 1D convolution.
3. The ShellConv Operator
ShellConv defines an efficient point-cloud convolution by partitioning each neighborhood into ordered concentric shells and aggregating shell features in an order-invariant way. The operator also incorporates sampling to support larger receptive fields without requiring many layers.
- The convolution problem is to efficiently map a point set to a feature vector while handling point-order ambiguity and point-cloud density.
- ShellConv combines point sampling with convolution, producing sparser point sets at deeper layers and larger receptive fields without increasing the number of layers.
- The resulting operator treats point clouds as mathematical sets, making the convolution independent of the ordering of points.
- Neighbor points around representative points are partitioned into concentric spherical shells using distance from the center.Shells are constructed by sorting distances and assigning a fixed number of points to each shell.
- Because shells are ordered from inner to outer, each shell receives a fixed convolution weight and the shell sequence is unambiguous.
- Within each shell, maximum pooling across feature channels produces an order-invariant representative feature.The maximum is a crude distribution approximation, but many feature channels can preserve information from multiple points.
4. ShellNet
ShellNet uses ShellConv in a shared encoder for point-cloud classification and segmentation. Classification applies three ShellConv layers, while segmentation adds a U-net-style decoder with skip connections to recover the original resolution.
- ShellNet replaces traditional 2D convolution with ShellConv and can consume point sets directly because the operator is permutation invariant.
- Classification: The classification network applies three ShellConv layers, progressively reducing representative points from N0 to N2 while increasing feature channels from C0 to C2.
- Classification: After encoding, classification produces an N2 × C2 feature matrix that feeds an MLP and a 128×kcls class probability matrix.
- Semantic segmentation: The segmentation network follows a U-net encoder-decoder, expanding the encoded N2-point representation back to the original resolution.
- Semantic segmentation: Skip connections concatenate earlier-layer features with decoder outputs while decoder layers increase point count and reduce feature channels.
5. Experimental Results
ShellNet achieves strong accuracy across classification and segmentation tasks while using a lightweight, fast-to-train architecture. Experiments also examine receptive-field size, neighbor sampling, and performance on sparse or partial point clouds.
- ShellNet achieves state-of-the-art performance for accuracy and speed across object classification, part segmentation, and semantic segmentation.
- Semantic Scene Segmentation: ShellNet ranks first on ScanNet and S3DIS, and second on Semantic3D for semantic scene segmentation.On Semantic3D, ShellNet uses only 3D coordinates, whereas some prior methods use color or CRF postprocessing.
- Object Part Segmentation: ShellNet trains for 20 hours while producing robust object part segmentation across chair, lamp, skateboard, airplane, and car examples.
- Network Efficiency: ShellNet is more efficient than existing methods in trainable parameters, FLOPs, and running time while converging to state-of-the-art accuracy.Its speed and memory improvements use an MLP within shells and a 1D convolution across shells.
- Receptive Field and Sampling: Shell size 16 performs best in equal-time comparisons, whereas shell size 8 is weaker in equal-epoch tests because of smaller receptive fields.Reducing receptive-field size can make neighbor queries cheaper and computation faster.
- Robustness: For sparse and partial point clouds, mpIoU decreases from 82.4% on original data to 80.2% and 72.6%, with partial-data boundary points less accurate.The authors state that further investigation of robustness is required.
6. Conclusion
The paper introduces ShellConv and ShellNet for efficient deep learning directly on 3D point clouds, using concentric spherical shells to support permutation-invariant convolution. ShellNet achieves state-of-the-art results on object classification and segmentation tasks with pure point cloud inputs, while several other applications remain future work.
- ShellConv efficiently convolves local point sets using concentric spherical shells and their statistics, resolving convolution order ambiguity.The shell structure also supports larger and more overlapped receptive fields without increasing network depth.
- ShellNet is a simple neural network architecture built on ShellConv for learning directly from 3D point clouds without point order ambiguity.
- ShellNet achieves state-of-the-art results on object classification and segmentation tasks using pure point cloud inputs.
- Future work includes applying the approach to object detection, semantic instance segmentation, meshes, and point cloud generation.