Source-linked AI summary
Rotation Invariant Convolutions for 3D Point Clouds Deep Learning
Zhiyuan Zhang, Binh-Son Hua, David W. Rosen, Sai-Kit Yeung
TL;DR
Point-cloud convolutions often lack rotation invariance, and rotation augmentation may not generalize consistently to unseen orientations. The paper introduces RIConv, which uses invariant geometric features and binning to build networks for classification and segmentation. Experiments report good performance and the best consistency in arbitrary-rotation tests, while the method can be less accurate than state of the art without rotation variation.
Problem
Most existing point-cloud convolutions do not guarantee rotation invariance, while rotation augmentation can fail to generalize well to unseen rotations.
Method
RIConv uses translation- and rotation-invariant distances and angles, then combines them with binning to handle point ordering in a single convolution.
Results
The method achieves good classification and segmentation performance with the best consistency under arbitrary rotation test cases.
Takeaways & Limitations
A point-cloud network can be built around Euclidean geometric features to handle arbitrary rotations and point-order variation without requiring a spherical domain.
Takeaways & Limitations
The method can be less accurate than state-of-the-art classification when inputs have no rotation or only simple z-axis rotations because original coordinates are discarded.
Abstract
from arXiv · showhide
Recent progresses in 3D deep learning has shown that it is possible to design special convolution operators to consume point cloud data. However, a typical drawback is that rotation invariance is often not guaranteed, resulting in networks being trained with data augmented with rotations. In this paper, we introduce a novel convolution operator for point clouds that achieves rotation invariance. Our core idea is to use low-level rotation invariant geometric features such as distances and angles to design a convolution operator for point cloud learning. The well-known point ordering problem is also addressed by a binning approach seamlessly built into the convolution. This convolution operator then serves as the basic building block of a neural network that is robust to point clouds under 6DoF transformations such as translation and rotation. Our experiment shows that our method performs with high accuracy in common scene understanding tasks such as object classification and segmentation. Compared to previous works, most importantly, our method is able to generalize and achieve consistent results across different scenarios in which training and testing can contain arbitrary rotations.
1. Introduction
Point clouds support accurate scene understanding but require specialized convolutions because their points lack regular structure. Existing operators often fail to remain rotation invariant, motivating a geometric, order-agnostic convolution.
- 3D point clouds are useful for scene understanding because they are available, compact, and robust compared with volumetric or image representations.
- Point clouds lack regular image- or volume-like structure, so convolution operators require designs that handle this irregularity.
- Most previous point-cloud convolutions do not guarantee rotation invariance, and rotation augmentation may fail to generalize to unseen rotations.
- The proposed convolution uses low-level geometric features that are translation and rotation invariant, together with binning to address point ordering.
- The resulting network targets object classification and part segmentation while remaining agnostic to point-cloud rotations and point orders.
2. Related Works
3D deep learning has evolved from volumetric and multi-view representations toward point-cloud methods, whose irregularity motivates specialized operators. Rotation invariance remains especially important because 3D shapes lack a standard alignment convention.
- Volumetric and multi-view representations are straightforward for neural networks but scale poorly because of memory demands and limited geometric resolution.
- PointNet demonstrated that point-set features can be learned while remaining robust to input point orders, enabling point-cloud classification and semantic segmentation.
- Later point-cloud methods explore geometric kernels, point edges, polynomial convolutions, shape context, lightweight designs, recurrent models, sequence models, and hierarchical structures.
- Rotation invariance is more critical in 3D than in images because there is no convention for aligning 3D shapes.
- Unlike spherical or manifold-based approaches, this method defines rotation-invariant convolution directly on point sets in Euclidean space without requiring a spherical domain or shape surfaces.
3. Rotation Invariant Convolution
RIConv constructs local point features from distances and angles relative to a reference orientation, then uses binning and pooling to obtain convolution inputs invariant to transformations and point order.
- The method seeks local features invariant to translation, rotation, and point order while enabling convolution directly in Euclidean space.
- For each reference point, nearby points and their centroid define a reference vector used to extract translation- and rotation-invariant features.
- Each local point is represented using distances to the reference point and centroid plus angles toward them; these cues remain invariant under rigid transformations.
- The reference vector can degenerate when the centroid coincides with the reference point, so the farthest point is selected as the centroid surrogate.
- RIConv samples representative points, gathers neighborhoods, lifts invariant features with a shared MLP, bins points along the reference vector, and maxpools each bin.
- The reference-point set is also used to implement downsampling and upsampling within the convolution.
4. Neural Networks
The proposed networks use RIConv as their core for classification and part segmentation. Classification stacks three RIConv operators, while segmentation uses an encoder–decoder with skip connections.
- RIConv serves as the core operator for object classification and object part segmentation, two common point-cloud benchmarks.
- Object classification: The classification network applies three rotation-invariant convolution operators followed by a classifier, with batch normalization and ReLU after each convolution.
- Object part segmentation: The part-segmentation network uses an encoder–decoder architecture with skip connections, treating the classification network as its encoder.
- Object part segmentation: During decoding, skip-connected features pass through an MLP before deconvolution, which progressively produces denser points with fewer feature channels.
- Network configuration: The implementation uses task-specific input sizes, progressive downsampling, neighborhood sizes, and bin counts across three convolution layers.
5. Experimental Results
The evaluation tests the method on object classification and part segmentation under multiple rotation settings, along with network-design ablations and limitations. Results show strong accuracy, low rotation sensitivity, and a trade-off between invariant geometric features and discriminative coordinates.
- Evaluation setup: The experiments evaluate object classification on ModelNet40 and object part segmentation on ShapeNet under z/z, SO3/SO3, and z/SO3 rotation settings.The z/SO3 setting trains with gravity-axis rotations and tests with arbitrary SO3 rotations to assess rotation invariance.
- Object classification: Our method has the lowest classification accuracy deviation across the three rotation cases and almost no performance difference between them.Previous methods show especially large deviations in the difficult z/SO3 case, whereas the proposed method remains consistent.
- Object part segmentation: In part segmentation, the method significantly outperforms previous methods in z/SO3 and achieves similar performance in SO3/SO3.It performs best in 15 of 16 categories in z/SO3 and 3 of 16 categories in SO3/SO3.
- Network design: 46.8% accuracy with one convolution layer shows that a single convolution cannot extract effective features; three layers provide the chosen speed–accuracy balance.Additional layers improve accuracy but increase training time.
- Limitations: The method is less accurate than state-of-the-art classification methods for no or simple rotations because invariant feature extraction discards original coordinates.Replacing geometric features with original coordinates yields 91.8% overall accuracy but removes robustness to SO3 rotations.
6. Conclusion
The method handles arbitrarily rotated point clouds through rotation-invariant geometric features and binning, while future work targets learned features and broader transformation invariance.
- The convolution constructs rotation-invariant features from distances and angles using a reference orientation based on a reference point and centroid.
- A binning strategy combines with geometric feature extraction to address rotation invariance and point ordering within one convolution.
- The proposed network supports point-cloud classification and segmentation, with the best consistency under arbitrary rotation test cases.
- Future work includes replacing hand-crafted invariant features with unsupervised learning and extending the method to detection, retrieval, and rigid or non-rigid transformations.
Supplementary Material
The supplementary document identifies the paper’s authors.
- The listed authors are Zhiyuan Zhang, Binh-Son Hua, David W. Rosen, and Sai-Kit Yeung.
- The author list includes four contributors.
- The names are presented with superscript-style affiliation markers for Zhang, Hua, Rosen, and Yeung.
Abstract
The supplementary document expands the paper’s quantitative and qualitative evaluations, emphasizing convergence in the challenging z/SO3 setting and adding class-level and visualization results.
- The supplement provides additional quantitative and qualitative evaluation details from the main paper.
- It reports testing accuracy and mIoU convergence over training epochs for classification and part segmentation.
- The evaluations use the z/SO3 scenario, identified as the most challenging case for rotation-invariant convolution.
- Additional materials include per-class classification accuracy and more object-part-segmentation visualizations.
1. Object Classification
In the z/SO3 setting, the proposed method substantially outperforms prior approaches, while PointCNN and PointNet-based methods remain weak in several categories or fail to adapt to unseen rotations.
- The proposed method significantly outperforms previous methods in the z/SO3 classification scenario.
- PointCNN is the closest competitor, yet its accuracy remains below 50% for categories including desk, laptop, and mantel.
- PointNet-based techniques perform poorly despite transformation networks intended to rotate inputs into a standard pose.
- The z/SO3 scenario indicates that learned transformation networks do not generally adapt well to unseen rotated data.
2. Object Part Segmentation
For object part segmentation, the proposed method generalizes to unseen rotations and significantly outperforms previous methods, including at early training epochs.
- The method significantly outperforms previous methods on object part segmentation, even at early training epochs.The evaluation tracks testing mIoU across training epochs.
- Figure 2 plots testing mIoU against training epochs for the z/SO3 scenario.
- The method's performance demonstrates generalization to unseen rotations in the object part segmentation task.
- Additional visualizations of object part segmentation are provided in Figure 3.