PaperlayerView exact revision on arXiv

Source-linked AI summary

On Making $\mathrm{SE}(2)$-Invariant Networks Optimal

Tomas Karella, Emily Shinkle, Alice Elisabeth Anastasia Allen, Pieter J. Swart, Nicholas Lubbers, Roxana Bujack

arXiv:2610.17424v1cs.LGcs.AI

TL;DR

Direct moment-invariant constructions can become numerically unstable and inefficient, motivating practical invariant maps for SE(2)-symmetric data. The paper derives an optimal moment-based map and implements it as a plug-and-play CNN module; across rotation-symmetric benchmarks, the models achieve competitive or improved accuracy with fewer parameters and lower runtime than steerable equivariant baselines.

  • Problem

    Direct moment-invariant constructions involve high-order combinations that can cause rapidly growing magnitudes and numerical instability, as well as inefficient complex-valued operations.

    Source: §2.2 · p. 5

  • Method

    The method combines rotation-invariant moment magnitudes with reference-normalized relative phases in a minimal, information-preserving map implemented as a plug-and-play CNN module.

    Source: §2.2 · p. 5, §4 · p. 8

  • Results

    Across rotation-symmetric benchmarks, the models achieve competitive or improved accuracy with substantially fewer parameters and lower runtime than steerable equivariant baselines.

    Source: §4 · p. 8

  • Takeaways & Limitations

    Invariant maps provide an effective, computationally efficient alternative to fully equivariant features, while relative orientation can still be inferred from invariant features over the continuous spatial domain.

    Source: §4 · pp. 8–9

    The approach is most beneficial when rotational symmetry is strong and orientation is a nuisance; invariance is only approximate in discretized implementations.

    Source: §4 · p. 9

Abstract

from arXiv · show

Respecting a-priori known symmetries of underlying data is a key principle in designing efficient neural architectures. Convolutional neural networks are a canonical example, encoding translation equivariance. Extending this inductive bias to richer symmetry groups such as $\mathrm{SE}(2)$ in the most effective way possible is an ongoing architectural and computational challenge. Building on classical invariant theory, we derive invariant feature maps that are provably optimal: minimal and information-preserving with respect to the symmetry group. Our construction is designed to integrate seamlessly into standard architectures, requiring no modification to nonlinearities or normalization layers. As a concrete implementation, we introduce a plug-and-play module for convolutional neural networks that enforces $\mathrm{SE}(2)$ invariance, capturing both translations and rotations within standard architectures. The resulting models are both efficient and expressive. Experiments show competitive or improved accuracy compared to state-of-the-art equivariant methods at significantly lower computational cost. Finally, we provide a theoretical and empirical analysis of the relationship between equivariance and invariance, offering insight into their roles in controlling robustness and flexibility.

1 Background: Discriminative and Minimal Invariant Maps

The section develops invariant feature maps from moments, aiming to retain all information up to the group action without redundant features. It introduces the Flusser basis and formalizes discriminability and minimality.

1.1 Complex Moments and Irreducible Representations of SO(2)

  • Moments project patterns onto bases with defined transformation properties; rotational bases use radial and angular components, with angular frequency determining each moment’s type.Examples include complex and Zernike polynomials and steerable bases [15] [17]; irreducible representations and the Peter-Weyl Theorem [28] provide the framework.Source: §1.1 · pp. 2–3
  • Moments transform with type-dependent rotation phases, and combining them so those phases cancel produces rotation-invariant features.Equivariance means transforming consistently with the group action, whereas an invariant feature remains unchanged under that action.Source: §1.1 · pp. 2–3

1.2 Flusser Basis as a Minimal and Discriminative Invariant Map

  • The Flusser basis normalizes moments of arbitrary type by a nonzero type-1 reference moment, yielding non-redundant features that discriminate patterns up to rotation.Its dimension equals the moment manifold’s dimension minus the group dimension, capturing the signal’s degrees of freedom up to the group action.Source: §1.2 · p. 3
  • A minimal invariant map is discriminative, and no proper subset of its features remains discriminative; the Flusser construction’s stated result assumes a nonzero reference moment.When m1 = 0, the flexible invariant basis can be used instead.Source: §1.2 · p. 3

2 Building SE(2)-Invariant Networks with Moment Invariants

The method builds SE(2)-invariant CNNs by projecting local signals onto moment bases and combining the responses into invariant features. Its optimal basis and moment–invariant blocks are designed to preserve information while integrating into standard CNN architectures.

2 Building SE(2)-Invariant Networks with Moment Invariants

  • Each layer preserves spatial SE(2) equivariance while producing rotation-invariant values; pointwise nonlinearities and normalization already fit, but ordinary convolutions must be redesigned.The network constraint is stated up to discretization errors for standard operations.Source: §2 · p. 4

2.1 Moment Layer: Projection to a basis

  • Moment projections form steerable feature fields, but the proposed block projects them directly to invariant type-0 features, avoiding explicit type-to-type interactions and scaling linearly with the number of moments.Steerable convolutions incur O(NinNout) complexity from interactions among input and output types.Source: §2.1 · p. 4

2.2 Optimal Bases: Preserving All Information from Moments

  • The optimal basis combines invariant moment magnitudes with reference-normalized relative phases, avoiding polynomial magnitude growth and suppressing unstable phase contributions near zero magnitude.The phase-aligned interaction is represented by a real-valued invariant per pair of moments.Source: §2.2 · p. 5

2.3 Plug-and-Play Invariant Moment Layer

  • The Mom/Inv block combines moment and invariant layers and stacks into CNN architectures; its basis is more numerically stable and efficient than alternatives, while extra nonlinearities bring no consistent gains.The implementation follows the illustrated block design with a 1 × 1 convolution, normalization, and nonlinearity after invariant features.Source: §2.3 · pp. 5–6, Figure 1 · p. 5

3 Experiments

The experiments evaluate SE(2)-invariant models across rotation-relevant benchmarks and examine their accuracy, rotation consistency, sample efficiency, runtime, and design choices. Results show competitive or better accuracy with fewer parameters, alongside lower rotation-consistency error and faster layer execution than equivariant baselines.

3 Experiments

  • Evaluation spans rotated digits, remote sensing, textures, scientific imaging, and a curated NFFA microscopy benchmark designed for scarce, structurally constrained data.The reported baselines include continuous-SO(2)-equivariant E(2)-CNNs; additional CNN and transformer baselines appear in Appendix G.Source: §3 · pp. 6–7
  • Rotation consistency is measured by the L2 distance between logits for upright and rotated inputs, with lower values indicating greater consistency.The paper reports that this metric was particularly informative when comparing invariant and equivariant models.Source: §3 · p. 6

3.1 Quantitative results

  • Across benchmarks, the method achieves competitive or better accuracy than E(2)-CNNs while using substantially fewer parameters.Table 1 reports mean ± standard deviation over multiple runs; DTD uses top-five accuracy.Source: §3.1 · p. 7, Table 1 · p. 7
  • The method yields lower rotation consistency index values across datasets, with the largest differences on explicitly rotated benchmarks and an order-of-magnitude reduction.The reported advantage persists on Galaxy Zoo, where accuracy is comparable or slightly lower.Source: §3.1 · p. 7

3.2 Ablation Studies

  • The ablations also examine degenerate reference moments, radial-function parameterizations, and adding an equivariant type-1 branch.These are listed as design choices studied; this passage does not report their outcomes.Source: §3.2 · p. 7
  • In NFFA's low-data regime, the model consistently outperforms an augmented ResNet18, and does so with approximately 300k parameters versus 11M.The accuracy gap is most pronounced at 6.2% and 12.5% of the data, then narrows as data increases.Source: §3.2 · p. 8
  • Layer forward-pass runtime is lower than for steerable CNNs, with speedups reaching 14× at higher moment orders.For NFFA at maximum type k = 3, the full model takes 98 seconds per epoch versus 227 seconds for e2cnn on one A100 GPU.Source: §3.2 · p. 8, Figure 4 · p. 8
  • Moment magnitudes combined with relative phase interactions outperform representations using the real and imaginary parts of the Flusser basis.The Flusser-based representation requires additional normalization to prevent instability from large magnitudes.Source: §3.2 · p. 8, Table 2 · p. 8

4 Conclusion and Limitations

The proposed invariant CNNs achieve competitive or improved accuracy with fewer parameters and lower runtime than steerable equivariant baselines. Their benefits are strongest when rotational symmetry is strong and orientation is a nuisance, while discretization and small moment magnitudes constrain the implementation.

4 Conclusion and Limitations

  • The architecture uses a minimal, discriminative invariant map and a plug-and-play CNN module that projects moments into invariant type-0 features for standard layers.The module integrates with pointwise nonlinearities, normalization layers, and 1 × 1 convolutions.Source: §4 · p. 8
  • Across rotation-symmetric benchmarks, the models achieve competitive or improved accuracy with substantially fewer parameters and lower runtime than steerable equivariant baselines.The authors also report strong performance despite local invariance, with relative orientation inferable from invariant features over the continuous spatial domain.Source: §4 · pp. 8–9
  • The approach is most beneficial for tasks with strong rotational symmetry where orientation is a nuisance variable; in high-data regimes, augmentation and increased capacity can often achieve similar robustness.Continuous-transformation theory yields only approximate invariance in discretized implementations, and very small moment magnitudes can make features unreliable.Source: §4 · p. 9
  • Future work may explore adaptive moment selection, broader transformation groups, combinations of invariant and equivariant representations, and extensions to three dimensions.The paper identifies projecting 3D features through 2D as one possible direction.Source: §4 · p. 9

5 Related Work

Prior work handles rotations through equivariant feature spaces or by constructing invariant representations, each with distinct architectural or representational trade-offs. Flexible invariant bases address degeneracies from fixed reference moments, but require prior knowledge or multiple frames.

5 Related Work

  • Equivariant CNNs extend convolutional symmetry handling to groups including planar rotations, but steerable feature spaces add computational and architectural complexity.Examples include harmonic networks and steerable CNN frameworks built on irreducible representations [11] [15] [17] [49].Source: §5 · p. 9
  • Deep-learning invariant methods include pooling, canonicalization, orientation-aware features, Bessel kernels, and invariant coordinates, but do not explicitly construct classically complete, non-redundant invariant spaces.The approaches described rely on pooling, weight sharing, learned canonicalization, or specific filter parameterizations [58] [59] [60] [61].Source: §5 · p. 9

A Flexible Bases

  • The Flusser basis can degenerate when its fixed type-1 reference moment vanishes; flexible bases instead select a nonzero reference moment, but may require prior pattern knowledge or multiple frames.Figure 5 depicts rotated three-fold and four-fold structures labeled by minimal distance and their projection into Flusser complex moments.Source: A Flexible Bases · p. 15, Figure 5 · p. 15

A.1 Synthetic flexibility benchmark

The appendix tests when flexible references matter and develops invariant descriptors that remain discriminative and minimal. Flexibility is valuable in sparse or highly symmetric settings, while degeneracies are rare in the dense feature representations tested in CNNs.

A.1 Synthetic flexibility benchmark

  • With valid convolution, non-flexible Flusser and optimal bases achieve approximately 50% accuracy, while the flexible optimal basis reaches approximately 99%.Adding padding lets the non-flexible bases recover comparable performance by introducing boundary context, but does not resolve the underlying reference-moment limitation.Source: §A.1 · p. 15

A.2 Implications for Dense Invariant CNNs

  • Candidate reference moments are rarely near zero in dense CNN feature representations, unlike sparse or highly symmetric inputs where flexibility can prevent degeneracy.The appendix reports this pattern across datasets and moment orders, with the dense-feature observation tested before and after training; it contrasts with sparse geometric graph settings [66].Source: §A.2 · p. 16, Table 3 · p. 16

B Optimal Invariant Descriptors

  • An optimal descriptor is discriminative up to rotation and minimal, meaning removing any feature breaks that property; the paper proves invariance and minimality for O(2) and SO(2).The minimal discriminative map is derived without a-priori knowledge from Bézout maps.Source: B Optimal Invariant Descriptors · p. 16

B.1 From Complex to Fourier Moments

  • The appendix separates angular and radial parts of complex moments to make notation concise and accommodate other radial choices, including Zernike moments.It then reduces the construction to Fourier moments.Source: §B.1 · pp. 16–17

B.2 Bézout Map

  • The Bézout map uses descriptors selected for a known input pattern to distinguish it from other moment sets.The paper contrasts this with a flexible map that can discriminate patterns without prior knowledge.Source: B Optimal Invariant Descriptors · p. 16, §B.2 · p. 17

B.2.1 Invariance

The section moves from known-pattern Bézout descriptors to a minimal discriminative map for arbitrary unknown inputs. It establishes rotation invariance, discriminability, and minimality while noting limits involving reflections and descriptor dependence.

B.2.1 Invariance; B.2.2 Discriminability; B.2.3 Minimality

B.2.1 Invariance

  • Replacing complex descriptors with magnitudes and real parts adapts the Bézout map to reflections and retains magnitudes when a support change zeros its normalizer.Source: §B.2.1 · p. 18

B.2.4 Relation to Flusser’s Basis

  • For Flusser’s basis, the flexible variant is optimal among one-phase-per-order descriptors, preserving all magnitude information at fixed cardinality but remaining not fully discriminative.Source: §B.2.4 · p. 19

B.3 The Minimal Discriminative Map; B.3.4 Dependence

  • For arbitrary unknown inputs, Φ is the smallest map that discriminates any two inputs, although its descriptors are algebraically dependent.Source: §B.3 · p. 20, §B.3.4 · p. 22

B.3 The Minimal Discriminative Map

  • The minimal flexible set Φ consists of moment magnitudes and all pairwise combinations of moments.Source: §B.3 · p. 20

B.3.1 Invariance

  • The minimal discriminative map Φ is invariant under planar rotations.Source: §B.3.1 · p. 20

B.3.2 Discriminability

  • Φ is discriminative because every Bézout invariant can be constructed from it.Source: §B.3.2 · pp. 20–21

B.3.3 Minimality

C Equivariant Structure

The paper tests whether explicitly retaining equivariant information improves invariant architectures. Across benchmarks and on RotMNIST, the reported results show no consistent benefit from adding equivariant components.

C Equivariant Structure

D Radial Basis Design

The radial-basis study finds that performance is broadly stable across tested discretizations and parameterizations, with a slight optimum around four rings.

D Radial Basis Design

  • The study varies radial discretizations and parameterizations because the moment layer depends on the choice of radial basis functions.Source: D Radial Basis Design · p. 23
  • Performance remains stable across radial-ring counts, with a slight optimum around four rings.Source: D Radial Basis Design · p. 23
  • B-splines, monomials, and orthogonal Legendre polynomials yield consistent performance across radial-basis parameterizations.Source: D Radial Basis Design · p. 23

E Filter Size Trade-off

The additional benchmark study reports strong performance with substantially lower parameter counts, alongside a filter-size trade-off between upright precision and rotational robustness. It also evaluates the architecture on an imbalanced SEM dataset suited to low-data symmetry studies.

E Filter Size Trade-off

F NFFA-Europe

  • NFFA-Europe provides an SEM benchmark with arbitrary, semantically irrelevant object orientation, limited samples, intra-class variability, and class imbalance.Source: F NFFA-Europe · p. 25, Figure 8 · p. 25

G Additional Quantitive Study

  • In compact, low-parameter comparisons, the invariant moment architecture achieves strong performance with substantially fewer parameters than equivariant and standard alternatives.Larger-benchmark experiments were omitted because runtimes often exceeded those of the method by a factor of 5–20.Source: G Additional Quantitive Study · p. 25

H Minimal Implementation

The appendix gives a minimal moment-based invariant-feature implementation and relates its representation dimension to the group action on moment space.

H Minimal Implementation

  • The provided implementation illustrates computing invariant features from moments and reuses existing complex-moment or steerable-convolution implementations for moment projection.It is presented as framework-agnostic code for reproducibility.Source: H Minimal Implementation · p. 25
  • Table 7 reports accuracy ± standard deviation and parameter count for a quantitative comparison of the CNN with other models.Source: Table 7 · p. 26
  • A type-1 reference moment supplies the angular offset used to combine magnitudes with cosine-adjusted relative angles before concatenation.Source: H Minimal Implementation · p. 26

I Dimension of Quotient

  • When m1 ≠ 0, the action is free on the corresponding moment-space subset, and the invariant representation parametrizes a quotient of dimension dim(M) − dim(G).The cited quotient-manifold theorem assumes a smooth, free, and proper Lie-group action.Source: I Dimension of Quotient · p. 26

J Experimental Settings

The experimental settings specify shared architecture hyperparameters, dataset-specific training configurations, and the computing resources used.

J Experimental Settings

  • Shared architectural hyperparameters for Mom/Inv models and ESCNN baselines are listed separately from dataset-specific resolutions, batch sizes, learning rates, and preprocessing.The shared settings are in Table 8; dataset-specific settings are in Table 9.Source: J Experimental Settings · p. 26, Table 8 · p. 27, Table 9 · p. 27

K Computational Resources

  • Experiments used a single GPU, with training on NVIDIA A100 GPUs and local experimentation and development on NVIDIA RTX 5090 GPUs.All models were implemented in PyTorch.Source: K Computational Resources · p. 27

Related papers

Loading 2610.17424v1…