PaperlayerView exact revision on arXiv

Source-linked AI summary

General $E(2)$-Equivariant Steerable CNNs

Maurice Weiler, Gabriele Cesa

arXiv:1911.08251v2cs.CVcs.LGeess.IV

TL;DR

The paper tackles the proliferation of rotation- and reflection-equivariant CNN designs by providing a unified E(2)-steerable framework. It reduces arbitrary representation constraints to irreducible ones, solves them for E(2) and its subgroups, and finds significant improvements on CIFAR-10, CIFAR-100, and STL-10 without additional hyperparameter tuning. The study also introduces group restriction for depth-varying equivariance.

  • Problem

    Many rotation- and reflection-equivariant CNN architectures exist, but systematic empirical comparison and a general framework for their kernel constraints are still needed.

  • Method

    The paper reduces kernel constraints for arbitrary representations to irreducible representations, solves them for E(2) and its subgroups, and uses the resulting framework to implement diverse architectures and group restrictions.

  • Results

    Equivariant convolutions used as drop-in replacements produced significant improvements over conventional convolutions on CIFAR-10, CIFAR-100, and STL-10.

  • Takeaways & Limitations

    The unified framework supports systematic comparison and architectures whose equivariance decreases with depth to match symmetries at different feature scales.

  • Takeaways & Limitations

    The authors note that further research is needed to determine whether the findings generalize to different settings.

Abstract

from arXiv · show

The big empirical success of group equivariant networks has led in recent years to the sprouting of a great variety of equivariant network architectures. A particular focus has thereby been on rotation and reflection equivariant CNNs for planar images. Here we give a general description of $E(2)$-equivariant convolutions in the framework of Steerable CNNs. The theory of Steerable CNNs thereby yields constraints on the convolution kernels which depend on group representations describing the transformation laws of feature spaces. We show that these constraints for arbitrary group representations can be reduced to constraints under irreducible representations. A general solution of the kernel space constraint is given for arbitrary representations of the Euclidean group $E(2)$ and its subgroups. We implement a wide range of previously proposed and entirely new equivariant network architectures and extensively compare their performances. $E(2)$-steerable convolutions are further shown to yield remarkable gains on CIFAR-10, CIFAR-100 and STL-10 when used as a drop-in replacement for non-equivariant convolutions.

1 Introduction

The paper develops a unified E(2)-steerable CNN framework to describe and compare rotation- and reflection-equivariant models for planar images. It solves the general kernel constraint, supports adaptive equivariance, and reports gains from equivariant convolutions on several image benchmarks.

  • The paper addresses the missing empirical survey comparing the many proposed rotation- and reflection-equivariant models for planar images.
  • Steerable CNNs represent feature spaces through group representations and impose representation-dependent linear constraints on convolution kernels.
  • The authors solve the kernel constraint for E(2) and its subgroups, unifying existing models and enabling hybrid architectures that mix field types.
  • Group restriction enables equivariance to decrease with depth, adapting feature transformations to symmetries that vary across feature-field length scales.
  • The benchmark compares groups, representations, and nonlinearities across MNIST variants and reports significant gains over non-equivariant baselines on CIFAR-10, CIFAR-100, and STL-10.

2 General E(2) - Equivariant Steerable CNNs

The section develops E(2)-equivariant steerable CNNs by associating feature fields with group representations and constraining layers to preserve their transformation laws. It provides general kernel-space solutions, supports subgroup-based symmetry choices, and reports no test-time computational overhead relative to conventional CNNs.

  • Group actions: E(2) models planar translations, rotations, and reflections, making it relevant to image features appearing at arbitrary positions and orientations.The framework also considers subgroups such as continuous rotations, discrete rotations, and reflection-containing groups.
  • Feature fields and representations: Steerable CNN feature fields carry representation types that specify how their channels mix under transformations, giving features an explicit notion of orientation.Scalar fields use the trivial representation, whereas vector fields use the standard representation and transform their orientations.
  • Feature fields and representations: Network layers must be equivariant so that the transformation law of steerable feature spaces is preserved across the network.Feature spaces can stack multiple fields with distinct representations, whose direct-sum structure lets the individual fields transform independently.
  • Kernel constraints: The kernel-space constraint can be reduced to irreducible representations, and this strategy applies beyond subgroups of O(2) while improving computational efficiency for large representations.The resulting angular kernel bases are harmonic and each is associated with a unique angular frequency; explicit solutions are provided for several groups.
  • Group restriction: Symmetry can be reduced at a chosen network depth by restricting equivariance from a group G to a subgroup H and reinterpreting subsequent feature fields accordingly.The paper gives the example of restricting rotations-and-reflections equivariance to reflection equivariance.
  • Implementation: E(2)-steerable convolutions have no computational overhead at test time compared with conventional CNNs because the expanded kernel can be reused in evaluation mode.For reasonably sized images, actual convolution time outweighs the cost of kernel expansion.

3 Experiments

Experiments compare steerable CNN groups, representations, nonlinearities, and restriction strategies across transformed MNIST and CIFAR datasets. Performance generally improves when equivariance matches local or global symmetries without discarding task-relevant orientation information.

  • MNIST benchmarking: Regular steerable CNNs outperform the non-equivariant baseline on MNIST O(2), with gains saturating at approximately 8 to 12 orientations.DN models perform better than same-order CN models because the dataset contains reflected digits.
  • MNIST benchmarking: On MNIST rot, restricted dihedral models DN|5CN outperform pure CN models by exploiting local reflections without imposing global reflection invariance.Pure DN models cannot distinguish reflected digits and perform worse than the baseline for N = 1.
  • Group restriction: On MNIST 12k, restricted models DN|5{e} and CN|5{e} outperform globally invariant CN and DN models by preserving global orientation information.Restricted models exploit local symmetries while avoiding confusion between approximately symmetry-related digits.
  • Architectural comparisons: Gated nonlinearities produce the best results among the evaluated models, while isotropic kernels can underperform conventional CNNs and regular steerable CNNs remain stronger than several compressed alternatives.Induced O(2) representations improve over O(2) irreducible-representation models, but the best induced model remains below large-N DN models.
  • Group restriction: Restricting equivariance later in the network improves accuracy, and all restricted models outperform corresponding globally invariant models on transformed MNIST.The restriction operation adapts equivariance to the symmetry level present at the dataset’s global scale.
  • Training dynamics: Higher-equivariance steerable CNNs converge significantly faster than conventional CNNs, with convergence-rate gains saturating at approximately N = 8.The models have approximately the same number of parameters; equivariance reduces the transformed-image variability that must be learned.
  • Competitive MNIST experiments: A D16-equivariant model restricted to C16 achieves a new state of the art on MNIST rot, while quotient representations provide a smaller improvement over the replicated regular C16 model.The reimplementation matches the previous state-of-the-art accuracy before these modifications.
  • CIFAR experiments: On CIFAR, combining global and local rotational or reflectional equivariance improves results, with the best non-AutoAugment performance obtained by the D8 D4 D4 model.A smaller D8 D4 D1 model shows a remarkable gain over the non-equivariant baseline without increased computational cost.

4 Conclusions

The paper unifies rotation- and reflection-equivariant CNNs through a general E(2)-steerable framework, achieving significant gains across image benchmarks while identifying open design questions.

  • STL-10 data ablations show that equivariant models yield significantly improved results across all dataset sizes.
  • The framework analytically solves kernel constraints for arbitrary O(2) representations and subgroups, enabling systematic comparison of many equivariant architectures.
  • Group restriction adapts equivariance with network depth to match symmetries present at different feature scales.
  • Drop-in G-steerable convolutions significantly improve CIFAR-10, CIFAR-100 and STL-10 results without additional hyperparameter tuning.
  • Future work must investigate steerable CNN design choices more deeply and test whether the findings generalize to different settings.

A Local gauge equivariance of E(2)-steerable CNNs

E(2)-steerable CNNs use representation-dependent feature transformations and kernel constraints that extend from global Euclidean symmetries to local gauge transformations and two-dimensional manifolds.

  • Figure 5 distinguishes active signal rotations, equivalent passive global gauge transformations, and independent local gauge transformations.
  • Classical E(2)-steerable CNNs guarantee equivariance under active translations, rotations and reflections, but not independently rotated or reflected image patches.
  • Gauge equivariant CNNs model local transformations by changing position-dependent reference frames, with each feature vector expressed relative to a local gauge.
  • Local gauge equivariance yields the same kernel constraint as Eq. (2) and therefore implies equivariance under active Euclidean isometries.
  • The kernel constraint applies to arbitrary two-dimensional Riemannian manifolds with structure groups G ≤ O(2), covering spherical CNNs and networks on triangulated meshes.
  • Feature-field transformation laws are encoded by linear group representations, which specify how feature channels mix under group actions.
  • Representations can be decomposed into irreducible components, reducing kernel-constraint analysis to invariant subspaces and enabling efficient general solutions.
  • Regular, quotient, restricted and induced representations provide alternative feature transformation laws based on group elements, cosets, subgroups and induced subgroup actions.

C An intuition for quotient representation fields

Quotient representations remove redundant responses when feature patterns have known symmetries, reducing computational, memory, and parameter costs while retaining selected field expressiveness. Their use is beneficial only when the assumed symmetries are present and relevant.

  • Quotient fields: Representations such as ρCN/C2 and ρCN/C4 encode features invariant under rotations by π and π/2, respectively.The corresponding examples include lines and crosses.
  • Quotient fields: Quotient representations store N/M non-redundant orientation responses instead of N regular-field channels when CM symmetry is assumed.For symmetric patterns, this removes repeated responses across orientations.
  • Hybrid representations: Mixing regular, quotient, and trivial representations preserves expressiveness while incorporating known symmetries and compressing the model.The described C16 model combines 5 regular, 2 quotient, and 4 trivial components.
  • Limitations: Quotient representations can severely harm performance when their assumed symmetry is absent or unimportant for inference.Because many quotient combinations are possible, architecture search may be needed; regular representations are recommended by default.
  • Steerable operations: A G-steerable kernel constraint guarantees equivariance of convolutional mappings, and its analytical solutions are given for irreducible representations across E(2) and its subgroups.The framework also supports equivariant pointwise nonlinearities and concatenated feature fields.

F.4 Derivations of the kernel constraints

The derivation solves kernel constraints for irreducible representations of subgroups G ≤ O(2), separating cases by the dimensions of input and output irreps.

  • Case structure: The derivation distinguishes mappings between two-dimensional irreps, mixed-dimensional irreps, and one-dimensional irreps.Positive radial components are handled first, while the origin requires additional considerations deferred to another section.

F.4.1 Derivation for SO(2)

For SO(2), the kernel constraints restrict angular Fourier components according to input and output representation frequencies while leaving radial parts unrestricted. The resulting solution spaces have dimensions depending on the irrep pair.

  • General strategy: SO(2)-equivariant kernels can be analyzed using angular bases at fixed radius because the constraint leaves radial parts unrestricted.The derivation projects kernel constraints onto angular basis elements in L2(S1).
  • Two-dimensional irreps: For two-dimensional irreps with m,n > 0, frequency matching imposes µ′ = m − sn′, yielding a four-dimensional angular kernel basis.Both phase γ and parity s remain unrestricted.
  • One-dimensional irreps: For trivial input and output representations, SO(2) equivariance requires rotation-invariant kernels, producing a one-dimensional isotropic basis.The constraint reduces to κ(φ + θ) = κ(φ) for all rotations.
  • Mixed-dimensional irreps: For a one-dimensional input and two-dimensional output with m > 0, the frequency condition µ′ = m yields a two-dimensional equivariant-kernel basis.The transposed case with two-dimensional input and one-dimensional output has the analogous basis.

F.4.2 Derivation for the reflection group

Reflection-equivariant kernel bases are derived in angular Fourier form and then combined with rotation constraints to obtain O(2)-equivariant kernels. Reflection symmetry preserves only part of the unrestricted basis.

  • Reflection action: The reflection group acts through a selectable reflection axis β, and its kernel constraint can be reduced to the β = 0 case by rotating coordinates.Kernels for a general axis are recovered by transforming the kernel relative to β.
  • Irreducible representations: The reflection-group irreps are one-dimensional, so only scalar 1×1 kernels occur in this derivation.The two irreps are distinguished by the sign carried by the reflection action.
  • Reflection constraints: Only half of the β = 0 Fourier basis is preserved by reflection equivariance.For positive frequencies, the surviving phase is γ′ = (i + j mod 2)π/2.
  • Combining constraints: O(2)-equivariant kernels are obtained by intersecting the SO(2)-equivariant and reflection-equivariant kernel spaces.This follows from O(2) decomposing into rotations and reflections.
  • Mixed-dimensional irreps: For mixed one- and two-dimensional O(2) irreps, imposing both constraints fixes the phase and frequency and yields a one-dimensional basis.For n = 0 and m > 0, the combined conditions are γ = jπ/2 and µ = m.

F.4.4 Derivation for CN

The CN derivation solves discrete-angle kernel constraints by projecting onto Fourier bases and reducing validity to frequency congruences modulo N.

  • Discrete-angle constraints: CN constraints must hold for discrete angles θ, while the projection on the left-hand side matches the SO(2) result.The derivation reuses SO(2) projection operators and handles discrete rotations separately.
  • Representation cases: When N is even, CN has an additional one-dimensional irrep of frequency N/2, requiring separate one-dimensional cases.The one-dimensional representations take values in {±1} at the discrete rotation angles.
  • Basis qualification: For µ = 0, some candidate basis elements vanish and are omitted from the basis.This is stated as a basis-construction qualification for the zero-frequency case.
  • Frequency solutions: For discrete rotations, valid output frequencies satisfy µ′ = ±m ± n + tN, with t ∈ Z.Considering p = 1 is sufficient because it implies the congruence constraints for the remaining discrete angles.
  • Frequency solutions: The resulting basis includes aliased frequencies obtained by adding integer multiples of N to the frequency combinations.These solutions are summarized in the explicit CN basis and Table 11.
  • Representation cases: The 1-and-2-dimensional and 2-and-1-dimensional cases produce corresponding bases, with the latter obtained from the former up to transposition.The derivation projects the kernel constraint onto the relevant Fourier bases.

F.4.5 Derivation for DN

The DN derivation adapts the O(2) procedure to discrete rotations and reflections, while accounting for reflection-axis equivalence classes and the resulting kernel bases.

  • Derivation strategy: DN solutions are derived by repeating the O(2) reflection analysis using the CN kernel bases as the starting point.This transfers the discrete-rotation solutions into the dihedral setting.
  • Reflection axes: Unlike O(2), DN does not act transitively on reflection axes, so the reflection-axis choice β matters up to discrete equivalence classes.The derivation can nevertheless use β = 0 and recover a general axis by angular shifting.
  • Reflection axes: For general reflection axis β, the equivariant kernel is obtained from the β = 0 kernel as κ′(r, φ) = κ(r, φ − β).This shift provides the stated reduction without loss of generality.
  • Representation cases: The DN basis is organized by representation pair, with explicit solutions listed for 1-and-2-dimensional, 2-and-1-dimensional, and 2-and-2-dimensional cases.The corresponding cases are located in the indicated cells of Table 12.
  • Radial dependence: The angular kernel constraints can be solved independently at each radius, but r = 0 requires special treatment because polar angles are undefined there.The derivation preserves consistency with the r > 0 solutions while addressing the origin separately.
  • Complex representations: The real-valued implementation of complex Harmonic Networks uses only a 2-dimensional kernel space, whereas the real SO(2)-steerable basis has 4 dimensions.The missing solutions correspond to frequency m + n.
  • Complex representations: Implementing the complex solution on real feature fields implicitly adds negative frequencies, enabling the larger real kernel basis without overhead.The paper states that the complex-field solution itself is complete, while the real implementation accesses the larger basis.

G Alternative approaches to compute kernel bases and their complexities

The paper contrasts numerical null-space computation for arbitrary representations with an irreducible-decomposition strategy that uses analytical E(2)-subgroup kernel solutions and scales better for large representations.

  • General constraint solution: The kernel constraint can be vectorized into Sylvester equations and decomposed through Clebsch–Gordan representations into independent constraints.Solving each independent constraint yields basis elements that are transformed back into kernel space.
  • Numerical approach: A numerical approach solves the stacked generator equations by a null-space computation, whose solutions correspond to Clebsch–Gordan coefficients.Generators suffice because solutions for them automatically satisfy the constraints for all group elements.
  • Numerical approach: The numerical method supports arbitrary representations without analytical Clebsch–Gordan coefficients but can be extremely expensive for large representations.Its cost is driven by the dimensions of the stacked Sylvester system and the change-of-basis matrices.
  • Irreducible decomposition: The paper’s method decomposes input and output representations independently into irreps and applies analytical kernel solutions for groups G ≤ O(2).This reduces arbitrary-representation constraints to irreducible kernel problems.
  • Complexity and scale: The improved implementation makes large representations such as dim(ρD20 reg) = 40 computationally feasible.The paper attributes this feasibility to the improved complexity of its implementation.

H Additional information on the training setup

The MNIST benchmark models derive from a shared base architecture, with widths expressed relative to a parameter-matched regular C16 model.

  • Base architecture: Each MNIST benchmark model is derived from a basic architecture whose convolution blocks contain convolution, batch normalization, and a nonlinearity.The first fully connected layer is followed by batch normalization and ELU.
  • Width convention: Layer widths are expressed as numbers of regular C16 feature fields with approximately the same number of parameters.This convention is used for both the basic MNIST models and the final MNISTrot models.
  • MNISTrot architecture: The final MNISTrot architecture is replicated from prior work and places dropout with p = 0.3 before each fully connected layer.The first two fully connected layers are followed by batch normalization and ELU.

H.1 Benchmarking on transformed MNIST datasets

The experiments use parameter-matched equivariant models and specify training adaptations for transformed MNIST and CIFAR benchmarks. CIFAR models replace conventional convolutions with G-steerable convolutions while preserving total parameters.

  • Experimental setup: Models are derived from a common architecture with layer widths scaled to match total parameter counts.The relative layer widths follow the architecture reported in Table 13.
  • Transformed MNIST: The transformed-MNIST comparison replicates a prior C16-equivariant architecture while changing its frequency bandlimit and Gaussian radial-profile width.The replicated architecture is summarized in Table 14.
  • Training: MNIST training uses 40 epochs, Adam with batch size 64, learning-rate decay after 15 epochs, L1/L2 regularization, and dropout probability p = 0.3.The reported setup uses an initial learning rate of 0.015 and regularization weight 10^-7.
  • CIFAR experiments: CIFAR-10 and CIFAR-100 models replace WideResNet convolutions with G-steerable convolutions while preserving total parameters.D8- and C8-equivariant blocks use 5 × 5 kernels instead of 3 × 3 kernels.
  • Training: CIFAR models are trained for 200 epochs with SGD, momentum 0.9, initial learning rate 0.1, and weight decay 5 · 10^-4.The learning rate is reduced by a factor of 0.2 every 60 epochs, with standard crop, flip, and normalization augmentation.

H.4 STL-10 experiments

The STL-10 experiments adapt an existing model and training setup, including corrected implementation details, while evaluating data efficiency across balanced reduced datasets. The ablation averages results over four runs at five training-set sizes.

  • Model adaptation: STL-10 models are adapted from a prior architecture using corrected settings reported in the authors’ GitHub repository.Changes include 60 × 60 cutout patches, stride 2 in the first block’s first convolution, and CIFAR-10 input statistics.
  • Model design: The small model preserves channel counts rather than parameter counts and expands the first convolution from 16 to 64 output channels.This yields four regular fields for the initially D8-equivariant layer.
  • Kernel basis: The kernel basis uses ring width σ = 0.6, outermost-ring width σ = 0.4, and frequency cutoffs 0, 3, 6, … .These settings are used for the STL-10 model’s kernel basis.
  • Training: STL-10 training uses only labeled data for 1000 epochs with SGD, Nesterov momentum 0.9, weight decay 5 · 10^-4, and scheduled learning-rate reductions.The learning rate starts at 0.1 and is reduced at epochs 300, 400, 600, and 800.
  • Data ablation: The data ablation averages four runs for balanced training sets of 250, 500, 1000, 2000, and 4000 images.The validation and test sets contain 1000 and 8000 images, respectively.

I Additional information on the irrep models

The irrep-model analysis compares nonlinearities, invariant maps, induced representations, and O(2)-specific operations. It reports that gated nonlinearities and induced representations improve performance within the tested models.

  • SO(2) models: Gated nonlinearities perform significantly better than other tested nonlinearities on SO(2) irrep models.Sharing gates across fields does not significantly affect performance, whereas norm-based invariant mapping consistently outperforms conv2triv.
  • O(2) models: O(2)-conv2triv produces invariant outputs from scalar and sign-flip fields by taking absolute values of the reflection-sensitive fields.Direct convolution to O(2) trivial representations is avoided because the corresponding steerable kernel space does not allow certain mappings, causing dead neurons.
  • O(2) models: O(2)-specific experiments include models operating on IndO(2) fields induced from SO(2) irreducible representations.These models are reported among the O(2)-equivariant variants in rows 50–53, 56, and 57.
  • Induced representations: Induced representations are constructed from SO(2) irreducible representations and act on functions over the quotient O(2)/SO(2).Their subfields transform through reflection-dependent permutations and rotations.
  • Invariant maps: Ind-conv2triv maps rotation-invariant but reflection-transforming scalar subfields to scalar fields using G-pooling over the two subfields.Ind-norm instead computes subfield norms and applies G-pooling to the result.
  • Induced nonlinearities: For induced fields, separate norm-ReLUs or gates are applied to subfields with shared bias or equivariantly permuting gates.Gates are computed from IndO(2)_0-fields containing two permuting scalar fields.
  • Induced representations: IndO(2)-SO(2) models perform much better than pure irrep models despite being equivalent up to a change of basis.The induced representation separates reflection actions into permutations of subfields.
Loading 1911.08251v2…