Source-linked AI summary
Gauge Equivariant Convolutional Networks and the Icosahedral CNN
Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling
TL;DR
The paper addresses how to construct convolutional networks on manifolds when global symmetries are unavailable. It introduces gauge-equivariant convolutions and implements them as icosahedral CNNs, which perform well across tasks and are highly scalable, with computational cost substantially below Spherical CNNs at higher resolutions.
Problem
Manifold CNNs need a principled way to handle local frame choices because general manifolds lack global symmetries.
Method
The paper defines gauge-equivariant convolutions for general manifold feature fields and implements them on the icosahedron using regular-representation features and R2R convolutions.
Results
Icosahedral CNNs achieve up to 99.43% test accuracy on IcoMNIST and outperform competing methods on climate-pattern segmentation.
Takeaways & Limitations
The resulting method performs well across different problems and provides a highly scalable approach to learning with spherical signals.
Takeaways & Limitations
The demonstrated implementation is specialized to the icosahedron, while implementation of gauge CNNs on general manifolds remains future work.
Abstract
from arXiv · showhide
The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be extended beyond global symmetries to local gauge transformations. This enables the development of a very general class of convolutional neural networks on manifolds that depend only on the intrinsic geometry, and which includes many popular methods from equivariant and geometric deep learning. We implement gauge equivariant CNNs for signals defined on the surface of the icosahedron, which provides a reasonable approximation of the sphere. By choosing to work with this very regular manifold, we are able to implement the gauge equivariant convolution using a single conv2d call, making it a highly scalable and practical alternative to Spherical CNNs. Using this method, we demonstrate substantial improvements over previous methods on the task of segmenting omnidirectional images and global climate patterns.
1. Introduction
The paper extends equivariance from global symmetries to local gauge transformations, enabling intrinsic-geometry-dependent CNNs on manifolds. It develops a general gauge-equivariant framework and instantiates it efficiently on the icosahedron.
- Motivation: Intuition-guided architecture design has produced successes but leaves limited understanding of why architectures work, making application-specific search costly.The paper frames equivariance as a rational design principle for reducing this dependence on experimentation.
- Motivation: Manifolds generally lack global symmetries, so transforming filters by those symmetries does not generally provide weight sharing across all points.Parallel transport introduces path-dependent filter-orientation ambiguity, while rotation-invariant filters limit expressivity.
- Gauge-equivariant framework: Local gauge symmetries provide the basis for manifold convolutions that depend only on intrinsic geometry, despite the arbitrary choice of tangent frame.Gauge changes alter coefficients predictably while preserving the represented geometrical quantities.
- Icosahedral CNN: The framework is applied to the icosahedron, whose regularity and local flatness enable an efficient conv2d implementation for spherical-signal learning.The icosahedron also has discrete rotational symmetries, exposing the interplay between local and global symmetries.
- Gauge-equivariant framework: Gauge-equivariant convolution uses arbitrary smooth local gauges, positions filters relative to them, and constrains kernels so outputs transform equivariantly.Multiple overlapping charts make the operation globally well-defined because chart switches correspond to gauge transformations.
3. Related work
Prior equivariant and geometric deep-learning methods extend neural networks to structured domains, but manifold convolutions remain limited by symmetry, feature-type, and kernel-parameterization constraints. Gauge equivariant convolution provides a general framework intended to address these limitations while connecting to existing approaches.
- Equivariant networks have been developed for sets, graphs, planes, 3D space, and homogeneous spaces using the global symmetries of each domain.
- Manifold convolution methods generalize convolutional networks beyond regular domains, but intrinsic methods are commonly limited to particular feature types or non-maximally-flexible kernels.
- Isotropic spectral filters and orientation pooling discard or reduce orientation information, whereas orientation-preserving convolutions can be more effective experimentally.
- Multi-directional convolution handles directional functions but uses a scalar tangent-space kernel, which is less general than a matrix-valued kernel mapping one representation field to another.
- The paper presents the Icosahedral CNN as a fast, simple alternative to Spherical CNNs within a broader gauge-equivariant framework.
- Gauge-equivariant networks can be understood geometrically through principal and associated fiber bundles, with the manifold frame bundle replacing the homogeneous-space bundle used in G-CNNs.
4. Icosahedral CNNs
The Icosahedral CNN exploits the icosahedron’s regular, symmetric grid and overlapping chart atlas to implement gauge-equivariant convolution efficiently. Its pipeline combines gauge-aware padding, kernel expansion, and a single conv2d operation while preserving global and local equivariance.
- 4. Icosahedral CNNs: The icosahedron enables gauge-equivariant convolution without interpolation, with computation concentrated in a single conv2d call.
- 4.1. The Icosahedron: The icosahedron has 20 faces, 30 edges, 12 vertices, and 60 rotational symmetries.
- 4.1. The Icosahedron: Grid refinement subdivides each triangular face into four triangles repeatedly, producing Hr with N = 5 × 2^(2r+1) + 2 points.
- 4.1. The Icosahedron: Non-corner grid points have six neighbours, while the icosahedron’s corners have five, yielding hexagonal and pentagonal pixel types.
- 4.1. The Icosahedron: The refined grid is exactly equivariant under icosahedral rotations, and gauge equivariance therefore also gives equivariance to these global transformations.
- 4.3. The Atlas of Charts: Five overlapping charts cover the icosahedron, each mapping selected triangular-face regions invertibly to planar integer grids.
- 4.3. The Atlas of Charts: Chart interiors are arranged so every non-corner pixel belongs to exactly one interior, while exterior strips provide the redundancy needed for valid local convolutions.
- 4.4. Gauges: Each chart uses a constant axis-aligned planar gauge, while overlaps may require C6 gauge transformations rotating frames by integer multiples of 2π/6.
5. Experiments
Experiments evaluate icosahedral CNNs on transformed MNIST, climate-pattern segmentation, and omnidirectional RGB-D segmentation. The models achieve strong accuracy, outperform competing methods, and show benefits from gauge-equivariant components.
- IcoMNIST: The full R2R model performs best across IcoMNIST conditions, indicating that gauge padding and kernel expansion contribute to performance.
- IcoMNIST: Exactly equivariant S2S, S2R, and R2R models generalize perfectly to icosahedral rotations absent rotated training examples, while continuous rotations require augmentation.
- Climate Pattern Segmentation: R2R and S2R models outperform both competing methods on climate-pattern segmentation across per-class and mean accuracy.
- Climate Pattern Segmentation: Mean average precision shows a clear advantage for R2R over S2R despite a small accuracy difference on climate-pattern segmentation.
- Stanford 2D-3D-S: The R2R residual U-Net outperforms Jiang et al. (2018) on omnidirectional RGB-D segmentation, while Jiang et al. outperforms standard planar U-Net.
6. Conclusion
The paper presents a general theory of gauge-equivariant convolutional networks on manifolds and demonstrates it with icosahedral CNNs for spherical signals. The method performs well across multiple problems and is highly scalable, while broader manifold implementations remain future work.
- Gauge-equivariant convolutional networks provide a general manifold theory, demonstrated concretely through icosahedral CNNs for spherical signals.
- The icosahedral method performs well across a range of problems and is highly scalable.
- Implementations on general manifolds and further scaling of spherical CNNs are identified as future work.
7. Recommended reading
The paper recommends introductory and advanced references for manifolds, fiber bundles, and related geometric concepts. These materials are intended for further study of the mathematical foundations.
- Schuller’s lectures and Nakahara’s book are recommended for accessible explanations of manifolds, fiber bundles, connections, parallel transport, and the exponential map.
- Additional references are recommended for further study of the mathematical theory.
8. Mathematical Theory & Physics Analogy
The mathematical framework uses principal and associated bundles to represent manifolds, frames, and fields, with gauge transformations acting on fields. Replacing the homogeneous-space bundle with the frame bundle yields a unified geometric perspective on manifold and homogeneous-space CNNs.
- Principal and Associated Bundles: A principal bundle consists of a base manifold, fibers, a projection map, and a structure group acting freely and transitively on each fiber.
- Principal and Associated Bundles: The fiber at each point can be viewed as a space of frames without a natural distinguished frame.
- Gauge Transformations and Fields: Gauge transformations are principal-bundle automorphisms that map fibers in a G-equivariant manner, sometimes fixing the base space.
- Gauge Transformations and Fields: An associated bundle is constructed from a principal bundle and a representation space, with fibers isomorphic to the representation space.
- Gauge Transformations and Fields: Matter fields are sections of the associated bundle, although they can be represented globally as functions into the representation space only when the bundle is trivial.
- Gauge Transformations and Fields: The gauge-transformation group acts on fields, and gauge-equivariant CNNs are designed to be equivariant to this action.
- Connection to Geometric Deep Learning: Using the frame bundle of a manifold instead of the homogeneous-space bundle unifies geometrically natural methods and G-CNNs on homogeneous spaces.
- Connection to Geometric Deep Learning: On homogeneous spaces, gauge equivariance can under mild conditions imply equivariance to the space’s global symmetry group.
9. Deriving the kernel constraint
The gauge-equivariant convolution is derived by tracking gauge transformations through the input field, sampling map, and kernel, yielding a kernel constraint that ensures equivariant outputs.
- The derivation starts from the gauge-equivariant convolution definition and examines how it transforms under a gauge transformation.
- The sampled manifold point qv remains unchanged under the simultaneous transformation of the exponential-map coordinates and gauge.
- Imposing the kernel constraint makes the convolution output transform as a ρout-field under gauge transformations.
10. Additional information on experiments
The experiments use gauge-equivariant architectures for classification, climate prediction, and omnidirectional semantic segmentation, with regular-to-regular layers and equivariant processing throughout.
- 10.1. MNIST experiments: The main MNIST model uses one scalar-to-regular convolution, six regular-to-regular layers, and three fully connected layers.
- 10.1. MNIST experiments: Equivariant batch normalization averages over groups of six feature maps, while ReLU remains equivariant for the scalar and regular representations used.
- 10.1. MNIST experiments: Global pooling over spatial and orientation channels produces an invariant representation before three fully connected layers and softmax.
- 10.1. MNIST experiments: The Spherical CNN baseline replaces the corresponding layers with spherical and SO(3) convolutions while matching channels and strides.
- 10. Additional information on experiments: The climate model is a U-Net with a scalar-to-regular first layer and five stride-2 regular-to-regular downsampling layers.
- 10. Additional information on experiments: The 2D-3D-S model is a residual U-Net using residual blocks, bilinear upsampling, and hexagonal max pooling.
11. Computational complexity analysis of Spherical and Icosahedral CNNs
The Icosahedral CNN is designed to improve scalability over Spherical CNNs through a homogeneous grid, fixed six orientations, and efficient spatial convolution, with lower observed cost and memory at high resolution.
- Spherical CNNs use SOFT grids with inhomogeneous sampling and couple spatial and angular resolution through the bandwidth B.
- The Icosahedral CNN uses a spatially homogeneous grid and applies filters in six orientations regardless of spatial resolution.
- With fixed small filters, the Icosahedral CNN has complexity comparable to O(B^2), because it is linear in the number of grid pixels.
- Figure 7 compares wallclock time across increasing grid resolutions for S2CNN and IcoNet, showing the comparison most directly at higher resolutions.
- The Spherical CNN was not evaluated beyond resolution r = 6 because it did not fit in GPU memory even with batch size 1.
- At resolution r = 6, the Spherical CNN uses about 10GB of memory, whereas the Icosahedral CNN uses about 1GB.
- The Icosahedral CNN benefits operationally from implementing its main computation with a single conv2d call and associated hardware optimization.
- For very high-resolution climate or weather data, feature maps may need to be split across local charts and compute nodes for scalability.
12. Details on G-Padding
G-padding transfers values across chart boundaries while applying the gauge transformations needed to preserve the manifold convolution represented by planar conv2d operations.
- Chart boundaries require padding even though the icosahedron itself has no boundary.
- Padding values are copied from another chart rather than filled with zeros, and a gauge transformation may be required during copying.
- Planar conv2d assumes a fixed global x-y gauge, whereas corresponding frames on different icosahedral charts can differ.
- A vector at a shared chart pixel can therefore have different planar directions when the charts are laid out together.
- Feature transformations act through ρ(gij(p)); scalar features use the trivial representation, while regular features use a six-element cyclic permutation.
- G-padding precomputes chart-transition elements and indexing operations for the top, bottom, left, and right chart sides.