Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs

Pim de Haan; Maurice Weiler; Taco Cohen; Max Welling

Source-linked AI summary

Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs

Pim de Haan, Maurice Weiler, Taco Cohen, Max Welling

arXiv:2003.05425v3Mar 11, 2020cs.LGcs.CVstat.ML

TL;DR

Mesh graph convolutions lose local orientation because isotropic kernels treat neighbours identically. GEM-CNN restores geometric sensitivity with anisotropic gauge-equivariant kernels and parallel-transport message passing, achieving stronger results than conventional GCNs and other methods while retaining intrinsic and deformation-related properties.

Problem
Graph convolutions on meshes discard neighbour orientations and use isotropic kernels, limiting the expressiveness of mesh features.
Method
GEM-CNN minimally modifies GCNs with anisotropic gauge-equivariant kernels and parallel transport over mesh edges.
Results
GEM-CNN outperforms conventional GCNs and other prior work on shape correspondence, with state-of-the-art performance on FAUST when gauge equivariance is combined with final-layer symmetry breaking.
Takeaways & Limitations
The model provides orientation-sensitive mesh features while producing equivalent outputs under arbitrary kernel orientations and generalizing across non-isometric deformations.
Takeaways & Limitations
Although radial-distance dependence can be added to the kernel, the authors omit it because it did not improve empirical performance.

Abstract

from arXiv · show

A common approach to define convolutions on meshes is to interpret them as a graph and apply graph convolutional networks (GCNs). Such GCNs utilize isotropic kernels and are therefore insensitive to the relative orientation of vertices and thus to the geometry of the mesh as a whole. We propose Gauge Equivariant Mesh CNNs which generalize GCNs to apply anisotropic gauge equivariant kernels. Since the resulting features carry orientation information, we introduce a geometric message passing scheme defined by parallel transporting features over mesh edges. Our experiments validate the significantly improved expressivity of the proposed model over conventional GCNs and other methods.

1 INTRODUCTION

Meshes can be processed as graphs, but graph convolutions discard local orientation and therefore use isotropic kernels. GEM-CNN restores geometric sensitivity with anisotropic, gauge-equivariant kernels and parallel-transport message passing, outperforming conventional GCNs and prior methods on shape correspondence.

Motivation: Graph representations discard incident-edge angles and ordering, so conventional GCN outputs are permutation-invariant and use isotropic kernels.This makes their features less expressive than orientation-aware anisotropic filters.
Method: GEM-CNN minimally modifies GCNs to use anisotropic filters while sharing weights across positions and respecting local mesh geometry.Gauge equivariance handles the arbitrary choice of reference neighbour and relates responses under different kernel orientations.
Method: The forward pass combines edge-based parallel transport with equivariant kernels evaluated at neighbour orientations, while conventional GCNs appear as the isotropic identity-transporter case.This preserves the GCN framework while adding orientation-aware geometric messaging.
Results: GEM-CNNs outperform conventional GCNs and other prior work on a shape correspondence task.The introduction attributes this result to the enhanced expressiveness of the gauge-equivariant model.

2 CONVOLUTIONS ON GRAPHS WITH GEOMETRY

Graph convolutions treat all neighbours identically, whereas mesh geometry supports orientation-dependent kernels. GEM-CNN adds anisotropic gauge-equivariant filtering and parallel transport while retaining linear layer complexity and intrinsic geometric properties.

Graph convolutions: Conventional graph convolution applies the same neighbour map before summation, so it cannot distinguish geometrically different neighbourhoods.Its isotropic kernel ignores the relative orientation of incident edges.
Gauge-equivariant convolution: GEM-CNN introduces anisotropic kernels K(θ) whose values depend on neighbour orientation, a minimal modification that adds direction sensitivity to graph convolution.Neighbour angles are defined in tangent planes using an arbitrary reference neighbour.
Gauge-equivariant convolution: Features at different vertices are parallel transported to a common vertex before contraction with orientation-dependent equivariant kernels.The resulting message-passing operation differs from conventional graph convolution only through anisotropic kernels and parallel transport.
Gauge equivariance: Gauge equivariance constrains kernels so outputs remain equivalent under changes of reference orientation, with complete basis-kernel sets linearly combined by learned parameters.The paper states that this parameterization spans the full equivariant kernel space.
Implementation: The layer has linear time and space complexity in the number of vertices and supports automatic-differentiation gradients.The algorithm computes neighbour angles, parallel transporters, and the forward pass within this complexity.
Geometric properties: GEM-CNN depends only on intrinsic mesh shape, and its convolution is equivariant to distance- and orientation-preserving self-maps.The paper states that these properties are proven analytically and shown empirically.

3 GAUGE EQUIVARIANCE & GEOMETRIC FEATURES

Gauge equivariance makes anisotropic mesh convolutions independent of arbitrary local reference orientations. The construction represents geometric features in gauge-dependent coefficients, constrains kernels accordingly, and parallel-transports features before combining them.

Changing the arbitrary reference neighbour rotates local coordinates, so gauge-equivariant outputs transform by a known rule rather than changing arbitrarily.
Anisotropic convolution evaluates neighbour-dependent kernels on tangent-plane orientations, while gauge transformations require compatible output transformations.
Geometric feature coefficients transform under representations of SO(2), with scalars and tangent vectors corresponding to distinct feature types.
3.1 KERNEL CONSTRAINT: Equivariant kernels are constructed from basis kernels satisfying angular constraints, and their complete linear parameterization determines the allowed self- and neighbour interactions.Table 1 summarizes solutions to the angular kernel constraint for mappings between irreducible representations.
3.2 GEOMETRY AND PARALLEL TRANSPORT: Mesh tangent spaces come from normals, neighbour directions are represented in local frames, and edge transporters move feature coefficients between independently gauged vertices before summation.

4 NON-LINEARITY

The proposed RegularNonlinearity addresses the incompatibility between point-wise nonlinearities and non-scalar geometric feature representations. It uses Fourier-domain sampling to obtain a practical approximation to gauge-equivariant nonlinear processing.

RegularNonlinearity uses Fourier transformations, point-wise nonlinearities, and inverse transformations to process features with multiple geometric representation components.
For arbitrary gauge angles, the sampled discrete implementation is only approximately equivariant, while transformations by multiples of 2π/N remain exactly equivariant.
The per-vertex computational complexity is O(bN) for the implemented transforms and nonlinearity, with an FFT alternative achieving O(N log N).

5 RELATED WORK

Prior mesh convolutions use graph, intrinsic-geometric, spectral, spiral, or embedding-space constructions with different limitations. GEM-CNN instead combines intrinsic mesh geometry with gauge-equivariant steerable features and nontrivial parallel transport.

Graph-based methods often use isotropic convolutions, while related anisotropic approaches include dynamically computed attention kernels.
Intrinsic geodesic methods support larger neighbourhoods but retain ambiguity in kernel orientation, whereas spiral methods work well mainly when graph structures are similar.
Spectral convolutions can be unstable under mesh deformations and correspond to isotropic kernels, motivating alternatives that retain local geometric orientation.
Voxel-based embedding-space approaches can require high memory and computation, while GEM-CNN operates on the mesh’s intrinsic geometry.
On flat Euclidean spaces, GEM-CNN corresponds to steerable CNNs, with parallel transport becoming trivial because the geometry is flat.

6 EXPERIMENTS

Experiments test GEM-CNN across changing mesh geometries and on FAUST shape correspondence. The model combines strong performance on flat meshes with geometry-dependent generalization and outperforms the compared correspondence baselines without their additional preprocessing pipelines.

6.1 EMBEDDED MNIST: The embedded-MNIST study trains on flat, isometric, and intrinsically curved meshes across controlled roughness levels, then evaluates every model across multiple test geometries.
6.1 EMBEDDED MNIST: At training roughness 2.5, performance on smooth meshes degrades to random guessing, showing that geometry-dependent generalization has a practical boundary.
6.1 EMBEDDED MNIST: 0.60 ± 0.05% test error on flat MNIST meshes contrasts with 19.80 ± 3.43% for an equally sized isotropic graph CNN.
6.2 SHAPE CORRESPONDENCE: On FAUST, GEM-CNN outperforms prior works and a non-gauge-equivariant CNN, while isotropic graph CNNs are unable to solve the correspondence task.The comparison uses three-run means and standard errors in Table 2; the model also benefits from breaking gauge symmetry in final layers for this registered-topology dataset.
6.2 SHAPE CORRESPONDENCE: GEM-CNN avoids geodesic computation, pooling, SHOT precomputation, and prediction refinement required by some baselines, though SpiralNet++ requires similarly meshed inputs.

7 CONCLUSIONS

GEM-CNNs extend mesh graph convolutions with anisotropic, gauge-equivariant kernels and parallel transport, preserving geometric consistency across local frames. The model is invariant to isometric deformations and achieves state-of-the-art FAUST correspondence performance.

7 CONCLUSIONS: GEM-CNNs add anisotropic kernels and parallel transport to graph convolutions, making mesh features sensitive to geometry while remaining equivalent under arbitrary kernel orientations.The method uses gauge-equivariant kernels and transports neighboring features before combining them.
7 CONCLUSIONS: Gauge equivariance resolves the arbitrary choice of reference neighbor, allowing anisotropic kernels to share weights across mesh positions.Neighbor orientations and edge transporters are computed relative to arbitrary local gauges.
7 CONCLUSIONS: Parallel transport first aligns tangent spaces on curved meshes and then corrects feature coefficients for differing local gauges before message passing.On general meshes, alignment uses the rotation between source and target normals; the resulting connection can be precomputed.
7 CONCLUSIONS: The convolution combines transported neighbor features with kernels evaluated at their gauge-dependent orientations under explicit representation constraints.Feature coefficients transform through representations, while transported features are expressed at the target vertex before contraction.
7 CONCLUSIONS: GEM-CNNs depend on intrinsic mesh geometry rather than ambient embedding, and are equivariant to orientation-preserving mesh isometries.This includes unaffected outputs under simultaneous rigid motions and equivariant signal movement under orientation-preserving mesh isometries.

D.1 PROOF OF MESH ISOMETRY EQUIVARIANCE

The proof establishes equivariance of GEM-CNN convolutions under orientation-preserving mesh isometries. It shows that neighborhood structure, local frames, and parallel transport transform compatibly with the feature push-forward.

D.1 PROOF OF MESH ISOMETRY EQUIVARIANCE: An orientation-preserving isometry maps faces, edges, and neighborhoods bijectively while preserving orientation and distances through an orthogonal differential.These properties define the geometric transformation used in the proof.
D.1 PROOF OF MESH ISOMETRY EQUIVARIANCE: Under the isometry, neighbor angles and parallel transporters transform with gauge offsets, preserving the convolution’s geometric relations.The proof uses the transformed transporter relation and the corresponding angular shift.
D.1 PROOF OF MESH ISOMETRY EQUIVARIANCE: Features are pushed forward by applying the representation of the negative local gauge angle at the mapped vertex.This defines the action of the isometry on features used in the equivariance statement.
D.1 PROOF OF MESH ISOMETRY EQUIVARIANCE: For an orientation-preserving mesh isometry φ, GEM-CNN convolution commutes with feature push-forward: K⋆φ∗(f)=φ∗(K⋆f).The theorem follows by combining neighborhood bijectivity, representation functoriality, and the kernel constraints.

E ADDITIONAL DETAILS ON THE EXPERIMENTS

The additional experiment details describe how curved mesh geometries and a shared GEM-CNN architecture are constructed for cross-geometry evaluation. The model’s receptive field and gauge-invariant output are fixed across these geometries.

E ADDITIONAL DETAILS ON THE EXPERIMENTS: Curved grids are generated by independently displacing flat-grid vertices along Z and smoothing those displacements with a Gaussian kernel.The evaluated roughness values correspond to smoothing standard deviations from 2.5 to 0.5 pixels.
E ADDITIONAL DETAILS ON THE EXPERIMENTS: The same seven-block GEM-CNN architecture is used across geometries, with convolution, RegularNonlinearity, batch normalization, and dropout in each block.Seven layers yield a 15-pixel field of view because information propagates only between direct neighbors.
E ADDITIONAL DETAILS ON THE EXPERIMENTS: Scalar inputs and outputs use the trivial representation, ensuring gauge-invariant predictions for the geometry experiments.The network uses multiplicities 1 and 64 for input and output scalar fields.
E ADDITIONAL DETAILS ON THE EXPERIMENTS: The experiments were run on a single TitanX GPU.

E.2 SHAPE CORRESPONDENCE EXPERIMENT

The shape-correspondence experiment evaluates a non-gauge-equivariant baseline alongside computational costs for the FAUST model. The reported cost study finds runtime largely unchanged by RegularNonlinearity sampling, while memory usage varies.

E.2 SHAPE CORRESPONDENCE EXPERIMENT: The experiments were run on single RTX 2080TI GPUs, requiring 3 seconds per epoch.
E.2 SHAPE CORRESPONDENCE EXPERIMENT: The non-gauge-equivariant baseline uses SHOT local reference frames and an unconstrained band-limited spherical convolution.With B = 2 and seven layers, hyperparameter search selected 128 channels as the best-performing configuration.
E.2 SHAPE CORRESPONDENCE EXPERIMENT: The FAUST model’s computational comparison varies the number of samples used by RegularNonlinearity while using batch size 1.The table measures one epoch of training and validation and maximum memory usage.
E.2 SHAPE CORRESPONDENCE EXPERIMENT: Runtime is not significantly affected by the RegularNonlinearity, whereas memory usage changes with the number of samples.Table 3 reports one-epoch training and validation runtime together with maximum memory usage for varying sample counts.

F.2 EQUIVARIANCE ERRORS

The experiments test gauge, ambient-coordinate, and isometry equivariance on regular and deformed icosahedra. GEM-CNN generally approaches the expected equivariance as Regular NonLinearity sampling increases, while heuristic SHOT frames fail under local symmetries.

Experimental setup: The experiments compare equivariance errors across gauge changes, ambient transformations, and orientation-preserving mesh isometries.Gauge tests use sampled gauge transformations, while ambient tests use rotations and translations in R3.
Gauge equivariance: GEM-CNN approaches gauge equivariance as Regular NonLinearity samples increase, while equivariance error accumulates with more layers on both icosahedra.The non-equivariant model is not gauge-equivariant, and the regular and deformed icosahedra behave similarly.
Ambient equivariance: GEM-CNN is invariant to joint rotations and translations of vertex coordinates, whereas SHOT-based frames fail on the symmetric icosahedron.SHOT cannot disambiguate axes when local symmetry makes its first two singular vectors ambiguous; the issue disappears on the deformed icosahedron.
Isometry equivariance: GEM-CNN becomes equivariant to icosahedral isometries when the Regular NonLinearity sample count is a multiple of 5, but not on the deformed icosahedron.The multiple-of-5 behavior reflects the order-5 stabilizer subgroup at icosahedron vertices.

G EQUIVARIANCE ERROR BOUNDS ON REGULAR NON-LINEARITY

The regular non-linearity approximates gauge-equivariant pointwise nonlinear processing by sampling Fourier modes in space, applying a pointwise non-linearity, and transforming back. Its equivariance error decreases with sampling and vanishes asymptotically under the stated band-limit conditions.

Construction: The regular non-linearity maps Fourier modes to N spatial samples, applies a pointwise non-linearity, and maps the samples back to Fourier modes.The representation is treated as discrete Fourier modes with band limit M, using inverse and forward DFT matrices.
Equivariance behavior: The procedure is exactly equivariant for gauge rotations that are multiples of 2π/N and approximately equivariant for intermediate rotations.Nyquist sampling requires N ≥ 2B + 1 to avoid information loss.
Bound: Theorem G.1 bounds regular-nonlinearity gauge-equivariance error for band-limited input and output, sample count N, and Lipschitz constant Lf.The bound is derived by controlling Fourier-mode differences using the Lipschitz behavior of sine and cosine.
Bound: The resulting error is O((BB′ + B′^2)/N ||x||1), so it vanishes as N approaches infinity.This follows from ||∂x||1 = O(B||x||1) under the stated band-limit assumptions.