PaperlayerView exact revision on arXiv

Source-linked AI summary

A-CNN: Annularly Convolutional Neural Networks on Point Clouds

Artem Komarichev, Zichun Zhong, Jing Hua

arXiv:1904.08017v1cs.CV

TL;DR

Deep learning on 3D point clouds remains challenging because their geometric structures are irregular, sparse, and represented in different formats. A-CNN introduces annular convolutions with regular and dilated rings, applies them in hierarchical networks, and reports state-of-the-art performance across classification, part segmentation, and large-scale semantic segmentation.

  • Problem

    Deep learning methods for 3D data face challenges from irregular, sparse geometric structures and varied representations of 3D objects.

  • Method

    A-CNN defines convolutions directly on point clouds using regular and dilated ring-shaped neighborhoods, ordering local points and applying hierarchical annular convolutions and pooling.

  • Results

    The proposed networks achieve state-of-the-art performance on object classification, part segmentation, and large-scale semantic segmentation across benchmark datasets.

  • Takeaways & Limitations

    Annular convolutions provide a framework for capturing local geometric information in 3D shapes across multiple point-cloud analysis tasks.

Abstract

from arXiv · show

Analyzing the geometric and semantic properties of 3D point clouds through the deep networks is still challenging due to the irregularity and sparsity of samplings of their geometric structures. This paper presents a new method to define and compute convolution directly on 3D point clouds by the proposed annular convolution. This new convolution operator can better capture the local neighborhood geometry of each point by specifying the (regular and dilated) ring-shaped structures and directions in the computation. It can adapt to the geometric variability and scalability at the signal processing level. We apply it to the developed hierarchical neural networks for object classification, part segmentation, and semantic segmentation in large-scale scenes. The extensive experiments and comparisons demonstrate that our approach outperforms the state-of-the-art methods on a variety of standard benchmark datasets (e.g., ModelNet10, ModelNet40, ShapeNet-part, S3DIS, and ScanNet).

1. Introduction

3D deep learning is challenged by irregular, sparse point-cloud sampling and the computational cost of regular representations. A-CNN addresses this with annular convolutions and hierarchical networks evaluated across three segmentation and classification tasks.

  • Motivation: Irregular and unstructured 3D representations such as point clouds make direct deep-learning computation challenging.Regular alternatives can be computationally heavy and memory-intensive.
  • Approach: Annular convolutions define efficient convolutions directly on point clouds using regular and dilated ring-shaped structures and directions.The operator is designed to capture local neighborhood geometry while adapting to geometric variability and scalability.
  • Applications: A-CNN targets object classification, part segmentation, and semantic segmentation in large-scale scenes.The framework is presented as a model for all three task types.
  • Contributions: The method permits arbitrary kernel sizes on each local ring-shaped region to improve geometric representations of 3D shapes.
  • Contributions: A multi-level hierarchical strategy based on dilated rings captures and abstracts shape geometric details across scales.The dilated strategy supports the proposed closed-loop convolutions and pooling operations.
  • Results: The proposed network models report state-of-the-art performance across these tasks and varied standard benchmark datasets.

2. Related Work

Related work spans volumetric, point-cloud, and mesh-based methods, each offering different ways to process 3D geometry. A-CNN focuses on learning relationships among neighboring points through a point-cloud convolution operator.

  • Volumetric Methods: Volumetric methods use regular occupancy grids or adaptive structures, but computation, memory, and resolution limit their treatment of geometric detail.
  • Point Cloud based Methods: PointNet processes points independently, while PointNet++ learns multiscale local structures but still treats points within each region independently.
  • Point Cloud based Methods: A-CNN defines convolutions that learn relationships between neighboring points in local regions to capture local geometric properties.
  • Point Cloud based Methods: Other point-cloud methods use kd-trees, learnable point-set kernels, extension and restriction operators, self-organizing maps, spectral convolutions, attention, or canonical ordering.
  • Large-scale Point Clouds: Large-scale point-cloud methods aggregate spatial context but generally do not define convolutions for learning local geometric features.
  • Mesh based Methods: Mesh-based approaches generalize convolution through geodesic coordinates, anisotropic diffusion kernels, parametric kernels, or mesh-specific operations.

3. Method

A-CNN defines point-cloud neighborhoods with non-overlapping regular or dilated rings, orders projected neighbors directionally, and applies annular convolution to capture local geometry.

  • Annular Convolution: Annular convolution processes the ordered neighbors on each ring, with regular and dilated rings supporting larger spatial context.The framework applies convolution and pooling on ring-based neighborhoods to capture geometric features at multiple spatial extents.
  • Regular and Dilated Rings: Regular and dilated rings partition each query point’s neighborhood into unique neighboring points without overlap.Dilated rings leave empty space between rings and cover larger areas than regular rings with the same number of neighbors and operation parameters.
  • Neighborhood Search: Constraint-based k-NN searches for the closest points within each ring defined by inner and outer radii.The search constrains neighborhoods to ring-shaped regions while using Euclidean distance to select closest neighbors.
  • Neighbor Ordering: Neighbors are projected onto a tangent plane and sorted by angular direction using dot and cross products.The normal supports projection; the resulting angles determine clockwise or counterclockwise ordering around the query point.
  • Empirical Evidence: Ordering local points improves classification accuracy compared with using unordered points in the authors’ experiments.The result supports ordering as an important component of the proposed framework.

3.4. Annular Convolution on Rings

Annular convolution converts each ring’s ordered neighbors into a circular sequence, applies standard convolution, and aggregates features separately across non-overlapping rings.

  • Circular Ordering: The ordered neighbor array is looped by copying initial elements to the end, enabling convolution across the ring boundary.For a 1 × 3 kernel, the sequence is extended with its first two neighbors before standard convolution.
  • Orientation Invariance: Closed-loop ordering makes annular convolution invariant to the orientation and random starting position of a local patch.The beginning of the neighbor sequence is concatenated with its end, so any starting position can be used.
  • Dilated Context: Applying the same kernel size on dilated rings covers larger areas while preserving the number of convolution parameters.Dilated rings therefore provide larger spatial contextual information than regular rings.
  • Ring Pooling: After sequential annular convolutions, features are aggregated separately on each ring using max pooling.Separate aggregation preserves information about neighboring proximity and spatial remoteness while reducing duplicate information across regions.

4. A-CNN Architecture

A-CNN uses a shared hierarchical encoder for classification and segmentation, sampling centroids, extracting ring neighborhoods, and propagating learned features to task-specific outputs.

  • Shared Encoder: Each abstract layer uses Farthest Point Sampling, constrained k-NN, annular convolutions, and pooling to produce fewer points with learned features.The architecture builds hierarchical representations from local ring neighborhoods.
  • Classification Network: The classification encoder processes regular and dilated rings independently, concatenates ring features, and feeds them to fully connected layers for class probabilities.The architecture uses two rings per layer and applies sequential 1 × 3 annular convolutions in the first layers.
  • Segmentation Network: The segmentation network shares the encoder and directly propagates features from multiple encoder levels toward the original point-cloud resolution.This multi-level propagation lets the network combine features from different hierarchical levels.
  • Feature Propagation: Segmentation restores each level’s features by inverse squared Euclidean distance interpolation from three nearest neighbors.The upsampled features are concatenated and passed through a 1 × 1 convolution before per-point class prediction.

5. Experiments

The experiments evaluate A-CNN on classification, part segmentation, and large-scale semantic segmentation using standard benchmark datasets. Results report strong performance across tasks, while ablations examine the roles of rings, ordering, and annular convolutions.

  • 5.1. Point Cloud Classification: A-CNN achieves better accuracy than point-cloud methods using 1024 points on ModelNet10 and ModelNet40.It is slightly better than Point2Sequence on ModelNet10 and comparable on ModelNet40.
  • 5.2. Point Cloud Segmentation: A-CNN with point locations and normals achieves the best reported ShapeNet-part performance among the compared methods.The point-locations-only model outperforms several point-based methods but is slightly worse than PointGrid and PointCNN.
  • 5.3. Semantic Segmentation in Scenes: A-CNN outperforms the listed competing methods on S3DIS except PointCNN and achieves state-of-the-art performance on ScanNet.On S3DIS, the authors attribute the slight disadvantage to PointCNN’s non-overlapping padded block sampling strategy.
  • 5.4. Ablation Study: The full combination of non-overlapping rings, ordering, and annular convolutions achieves the best ModelNet40 ablation result.The ring-shaped scheme contributes the most, and reducing overlap or redundancy also improves existing methods.

6. Conclusion

The conclusion presents A-CNN as a framework for capturing local geometric information in point clouds and reports state-of-the-art performance across several tasks. Future work targets improved large-scale scene architectures and outdoor datasets.

  • 6. Conclusion: A-CNN captures local geometric information in 3D shapes and achieves state-of-the-art performance on classification and segmentation benchmarks.The reported tasks include point-cloud classification, part segmentation, and large-scale semantic segmentation.
  • 6. Conclusion: Future work will explore architectures for large-scale scenes and applications to large-scale outdoor datasets.

A. Ball Query vs Ring-based Scheme

The paper compares conventional multi-scale neighborhoods with regular and dilated ring-based schemes. Ring structures avoid duplicated neighbors, and experiments associate reduced redundancy with improved classification performance.

  • A. Ball Query vs Ring-based Scheme: Ring-based neighborhoods assign unique neighboring points to each ring, unlike overlapping multi-scale regions.The comparison keeps the number of neighboring points per region the same across methods.
  • A. Ball Query vs Ring-based Scheme: PointNet++ with proposed regular rings without redundant points outperforms its ball-query multi-scale version with redundant points.

B. Training Details

The experiments use different A-CNN configurations for classification versus segmentation and train with Adam under task-specific schedules. Training is conducted on a single Titan Xp GPU.

  • B. Training Details: A-CNN-3L is used for classification, while A-CNN-4L is used for part and semantic segmentation.Regular rings are used in L1, with dilated rings added in later layers according to the architecture.
  • B. Training Details: Adam training uses learning rate 0.001, with decay rates of 0.7 for classification and 0.5 for segmentation.Training lasts 250 epochs for classification, 200 for part segmentation, and task-specific schedules for scene segmentation.

C. Feature Visualization

Feature visualizations examine which points contribute to classification and how learned global shape features organize by semantic category. The A-CNN configurations also specify ring and feature-map structure across layers.

  • Local Feature Visualization: Gradient magnitude visualizations treat high-gradient points as more influential during training and therefore as object-saliency indicators.Blue denotes low gradient magnitude, while red denotes high magnitude.
  • Global Feature Visualization: t-SNE reduces 1024-dimensional global shape features to two dimensions for ModelNet10 and ModelNet40 test splits.The visualizations show semantic clustering of the learned representations.
  • Global Feature Visualization: Similar shapes cluster together by semantic category, while visually similar dresser, night stand, and table classes can overlap.The ModelNet10 and ModelNet40 perplexities are 15 and 50, respectively.
  • Network Configuration: A-CNN-3L and A-CNN-4L denote encoder models with three and four layers, respectively.Each layer specifies centroids, ring radii, neighbors, and feature-map sizes.
  • Network Configuration: The A-CNN-4L ScanNet configuration doubles the number of centroids per layer because each block contains twice as many points as an S3DIS block.Its first layer uses two rings with radii [0.0, 0.1] and [0.1, 0.2], containing 16 and 48 neighbors.

D. Data Preparation Details

The preparation details define block sampling, point representations, normals, augmentation, and benchmark configurations for S3DIS and ScanNet experiments. The supplied materials also include segmentation-table and visualization references.

  • S3DIS Data Preparation: S3DIS rooms are divided into 1 m × 1 m × 2 m blocks with 0.5 m stride, sampling 4096 points per block.The prepared dataset contains 23,585 blocks across six areas.
  • S3DIS Data Preparation: S3DIS points use six-dimensional vectors containing normalized global coordinates and RGB colors.Normals are calculated for each room and used only to order points in local regions.
  • Evaluation References: The supplied references identify ShapeNet-part tables for XYZ-only and XYZ-plus-normals inputs, reporting per-category and mean IoUs.The S3DIS table reports overall accuracy and mean IoU over 13 classes.
  • ScanNet Data Preparation: ScanNet uses 1,201 scenes for training and 312 for testing, sampling 8192 points from each 1.5 m × 1.5 m block.Each point contains block-centered coordinates and normals, without RGB information.

E. More Experimental Results

Additional experiments compare A-CNN with established point-cloud baselines on part and semantic segmentation. The reported visual and quantitative results favor A-CNN overall, with category-specific and sampling-strategy qualifications on S3DIS.

  • ShapeNet-part: The ShapeNet-part evaluation reports results under both XYZ-only and XYZ-plus-normal inputs.The two settings are presented in Tables 6 and 7.
  • ShapeNet-part: A-CNN produces better ShapeNet-part segmentation results than PointNet++ in additional visual comparisons.Figure 12 marks mislabeled points when comparing predictions with ground truth.
  • S3DIS Semantic Segmentation: On S3DIS, A-CNN achieves state-of-the-art performance for segmenting walls and chairs.The visualizations compare rooms from all six areas with PointNet results and ground truth.
  • S3DIS Semantic Segmentation: On other S3DIS categories, A-CNN performs slightly worse than PointCNN because PointCNN uses non-overlapping padded block sampling that A-CNN does not use.Table 8 reports overall accuracy and mean IoU over 13 classes.
  • ScanNet Semantic Segmentation: On ScanNet, A-CNN outperforms PointNet++, PointNet, TangentConv, and PointCNN according to the main-paper quantitative results.Figure 14 visualizes six challenging scenes alongside PointNet++ predictions and ground truth.
Loading 1904.08017v1…