PaperlayerView exact revision on arXiv

Source-linked AI summary

Spherical CNNs on Unstructured Grids

Chiyu "Max" Jiang, Jingwei Huang, Karthik Kashinath, Prabhat, Philip Marcus, Matthias Niessner

arXiv:1901.02039v1cs.CVcs.AIcs.LG

TL;DR

Spherical signals are difficult to process with planar CNNs because projection introduces distortions, motivating efficient convolutions on unstructured grids. The paper replaces conventional kernels with learnable linear combinations of differential operators, which can be estimated locally on meshes. Across spherical vision and climate tasks, the model matches or outperforms state-of-the-art architectures while using significantly fewer parameters.

  • Problem

    Planar CNNs face distortion and boundary challenges on spherical signals, while spherical applications require processing panoramic, planetary, and climate data.

  • Method

    The method reparameterizes convolution kernels as learnable linear combinations of differential operators estimated from one-ring mesh neighborhoods.

  • Results

    Across spherical classification and segmentation tasks, the model matches or outperforms state-of-the-art architectures with significantly fewer parameters.

  • Takeaways & Limitations

    Parameterized differential operators provide an efficient convolution approach for spherical signals on unstructured grids.

Abstract

from arXiv · show

We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals. To this end, we replace conventional convolution kernels with linear combinations of differential operators that are weighted by learnable parameters. Differential operators can be efficiently estimated on unstructured grids using one-ring neighbors, and learnable parameters can be optimized through standard back-propagation. As a result, we obtain extremely efficient neural networks that match or outperform state-of-the-art network architectures in terms of performance but with a significantly lower number of network parameters. We evaluate our algorithm in an extensive series of experiments on a variety of computer vision and climate science tasks, including shape classification, climate pattern segmentation, and omnidirectional image semantic segmentation. Overall, we present (1) a novel CNN approach on unstructured grids using parameterized differential operators for spherical signals, and (2) we show that our unique kernel parameterization allows our model to achieve the same or higher accuracy with significantly fewer network parameters.

1 INTRODUCTION

The paper targets spherical signals that are difficult to process with planar CNNs and introduces parameterized differential operators for efficient CNNs on unstructured grids. The resulting approach uses four parameters per kernel and reports higher performance with fewer parameters than competing methods.

  • Spherical signals arise in panoramic vision, self-driving sensors, and climate science, but planar projections introduce polar distortions and boundary challenges.
  • The paper introduces PDOs for CNNs on arbitrary manifolds and topologies discretized by unstructured grids, focusing on icosahedral spherical meshes.
  • 4 parameters per kernel enable the proposed parameterization to achieve better performance than competing methods with much fewer parameters.
  • The approach provides orientable CNNs on unstructured grids using parameterized differential operators.
  • The spherical model achieves significantly higher parameter efficiency than state-of-the-art architectures for 3D classification and spherical image semantic segmentation.

2 BACKGROUND

Prior work addresses spherical and non-Euclidean convolution through projection adjustments, tangent-plane sampling, rotational equivariance, diffusion kernels, spectral methods, and manifold-based operators. The paper positions its method as an efficient convolutional operator for spherical image segmentation and related unstructured-grid settings.

  • Spherical CNN research seeks to reduce distortions from mapping curved surfaces onto planar images, using enlarged polar kernels or tangent-plane sampling.
  • Other approaches use diffusion kernels for efficient learning on graphs, non-Euclidean domains, and regular-grid images.
  • Geometric deep learning includes mesh convolutions, while graph methods parameterize kernels spectrally or use geodesic cross-correlation on manifolds.
  • Semantic segmentation research spans planar images, equirectangular omnidirectional images, 360 panoramas, point clouds, and voxels.
  • The paper targets spherical image segmentation, including panoramic images, with a more efficient convolutional operator.

3 METHOD

The method replaces conventional mesh convolutions with parameterized combinations of differential operators, enabling efficient CNNs on spherical unstructured grids. Icosahedral meshes provide the discretization and hierarchical structure for classification and segmentation architectures.

  • Parameterized Differential Operators: Parameterized Differential Operators reparameterize learnable convolution kernels as linear combinations of differential operators.This reduces parameters per convolution kernel while preserving local feature extraction.
  • Parameterized Differential Operators: Differential operators on meshes can be estimated efficiently from one-ring neighborhoods, avoiding extensive geodesic computations and interpolations.The implementation uses mesh-based estimation methods such as finite elements or discrete exterior calculus.
  • Parameterized Differential Operators: The PDO basis uses the identity, two orthogonal first derivatives, and the second-order Laplacian, with parameters optimized by standard backpropagation.For spherical signals, the derivative directions are chosen as east-west and north-south components.
  • Icosahedral Spherical Mesh: Icosahedral spherical meshes are generated by recursively subdividing and reprojecting icosahedron faces, providing uniform discretization and natural coarsening and refinement.These properties support pooling and unpooling operations in CNN architectures.
  • Evaluation: The Spherical MNIST experiment validates PDOs, with the model reported to achieve state-of-the-art performance using a comparable number of training parameters.The supplied table caption reports the qualitative comparison but no metric values.
  • Network Architecture: The architecture uses a shared encoder for classification and regression, while segmentation adds a U-Net-inspired decoder with transpose convolutions and skip layers.Figure 2 describes MeshConv, MeshConvT, DownSamp, residual blocks, and concatenation within these networks.

4 EXPERIMENTS

Across spherical MNIST, 3D object classification, and semantic segmentation tasks, the model performs competitively or better while emphasizing parameter efficiency. Experiments cover spherical, omnidirectional, and climate-related signals.

  • Spherical MNIST: The model achieves the best performance among comparable spherical MNIST algorithms with a comparable number of parameters.The reported advantage is attributed to orientation information, which helps distinguish digits such as “6” and “9”.
  • Omnidirectional Image Segmentation: The spherical segmentation model outperforms planar and point-based counterparts across all parameter regimes on 2D3DS.The comparison includes U-Net-style planar baselines and PointNet++-based point processing.
  • 3D Object Classification: Approximately 60% accuracy is achieved on the 40-class ModelNet40 task with 2000+ parameters.The model achieves the best performance across all parameter ranges and is on-par with comparable state-of-the-art models at peak performance.
  • Omnidirectional Image Segmentation: The omnidirectional segmentation experiment uses 1,413 equirectangular RGB-depth images spanning 13 semantic classes and six areas.Evaluation follows the dataset’s official 3-fold cross-validation split, with spherical signals sampled at level-5 mesh vertices.
  • Climate Pattern Segmentation: The climate experiment segments Atmospheric Rivers and Tropical Cyclones in global climate simulations, with the model achieving better accuracy than baseline models.Unlike the baseline’s random crops, the model processes the entire global field and predicts at the input resolution.

5 ABLATION STUDY

The ablation study evaluates differential-operator combinations in a 250k-parameter ModelNet40 model. The full operator combination performs best, and adding operators improves performance.

  • Kernel Components: The kernel containing all differential-operator components achieves the best accuracy in the ablation study.The study uses a 250k-parameter ModelNet40 classification model to compare operator combinations.
  • Kernel Components: Network performance improves as more differential operators are included, providing more degrees of freedom for the kernel.The result supports the operator combination selected in Eqn. 4.

6 CONCLUSION

The paper presents convolution on unstructured grids using parameterized differential operators as convolution kernels. Experiments demonstrate applicability to spherical-signal machine learning and improvements in performance and parameter efficiency.

  • Conclusion: Parameterized differential operators provide a convolution method for unstructured grids and spherical signals.The conclusion highlights applications to machine learning problems involving omnidirectional signals from 3D or LIDAR panorama sensors.

A ADDITIONAL DETAILS FOR IMPLEMENTING MESHCONV OPERATOR

The MeshConv implementation constructs differential operators on a triangular mesh, representing scalar signals with piecewise-linear basis functions and aggregating local gradients to vertices.

  • Identity acts as the unchanged input signal and requires no additional computation.
  • Gradient Operator: Piecewise-linear hat functions represent scalar values at mesh vertices, taking value 1 at their associated vertex and 0 elsewhere.
  • Gradient Operator: A single linear operator G computes constant per-face gradients from the per-vertex scalar function.
  • Gradient Operator: Per-vertex gradients are formed by averaging per-face three-dimensional gradients with face-area weighting.
  • Directional Components: Spherical gradient components use east-west and north-south unit direction fields obtained from normalized gradients of longitudinal and latitudinal functions.
  • Laplacian Operator: The mesh Laplacian is computed with the standard cotangent formula.

B NETWORK ARCHITECTURE AND TRAINING DETAILS

The paper provides detailed network architectures and training parameters for reproducing the experimental results, using Figure 2 as the architectural reference.

  • Architectures and training settings are specified to support reproduction of the results in Section 4.

B.1 SPHERICAL MNIST

The appendix specifies MeshConv network configurations, parameter counts, resolutions, and optimization settings across spherical classification and segmentation experiments.

  • Spherical MNIST: The spherical MNIST input uses a level-4 mesh and a MeshConv, downsampling, residual-block, pooling, and MLP architecture with 61,658 parameters.
  • Classification Architecture: The level-5 classification setup includes full and lean models with 3,737,160 and 70,192 parameters, respectively.
  • Omnidirectional Segmentation: The omnidirectional segmentation network uses a level-5 encoder-decoder with MeshConvT upsampling and 5,180,239 parameters.
  • Segmentation Training: Segmentation training uses weighted cross-entropy with zero weight for the invalid and unknown classes excluded from performance evaluation.
  • Climate Pattern Segmentation: The climate-pattern network reuses the omnidirectional segmentation architecture with feature layers reduced by one quarter and 328,339 parameters.

C DETAILED STATISTICS FOR 2D3DS SEGMENTATION

The 2D3DS analysis reports per-class accuracy and mean IoU against baselines, alongside runtime comparisons with PointNet++ for models of comparable accuracy.

  • Segmentation Statistics: 2D3DS segmentation statistics report per-class performance against benchmark models, averaged over 3-fold cross-validation.
  • Segmentation Statistics: Mean IoU is the reported aggregate comparison metric for segmentation performance against baseline models.
  • Runtime Analysis: 5x speedup over PointNet++ is achieved near the high-accuracy regime, with runtime exceeding 339 fps for real-time applications.
  • Runtime Analysis: The runtime analysis compares the proposed classification model across sizes with PointNet++ on a single NVIDIA GTX 1080 Ti using averaged inference batches.

E IMPLEMENTION DETAILS FOR SEGMENTATION BASELINES

The segmentation baselines use minimally modified planar models, PointNet++, and a bandwidth-constrained S2CNN variant. Their implementations specify input channels, point sampling and augmentation, pooling, and a representative S2CNN architecture.

  • Planar baselines: FCN8S and U-Net change their first convolution to accept 4-channel RGBD input and are trained from scratch without further modifications.The models use available open-source implementations.
  • Point-cloud baseline: PointNet++ uses its official implementation with 8192 points, z-axis rotation augmentation, and sub-region sampling during training.The same code is used for the ScanNet task.
  • Spherical baseline: The modified S2CNN produces scalar fields by average pooling only along the gamma dimension and maintains bandwidth 64 throughout the network.No transpose-convolution operator is defined for S2CNN, and its current implementations are described as memory inefficient.
  • Spherical baseline: The listed S2CNN configuration is [S2Conv(4, 64) +BN+ReLU]b64 followed by [SO3Conv(64, 15)]b64 and AvgPoolGamma.This specifies the compared model's principal convolution, normalization, activation, and pooling stages.
Loading 1901.02039v1…