PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Steerable Filters for Rotation Equivariant CNNs

Maurice Weiler, Fred A. Hamprecht, Martin Storath

arXiv:1711.07289v3cs.LGcs.CV

TL;DR

CNNs provide translational equivariance but must learn responses to other transformations, motivating models that encode rotation behavior directly. SFCNNs use steerable filters, group convolutions, and adapted initialization to achieve joint translation-rotation equivariance, with state-of-the-art benchmark results.

  • Problem

    CNNs implement translational equivariance by construction but must learn the proper mapping for other input transformations.

  • Method

    SFCNNs combine steerable filters, group convolutions, and generalized He initialization to encode joint translation and rotation equivariance.

  • Results

    The approach achieves state-of-the-art results on rotated MNIST and the ISBI 2012 2D EM segmentation challenge.

  • Takeaways & Limitations

    SFCNNs generalize learned patterns over orientations and achieve lower sampling complexity than CNNs in rotation-equivariant recognition tasks.

  • Takeaways & Limitations

    Strided pooling can reduce equivariance to a subgroup determined by the stride, and initialization assumptions include i.i.d. activations and gradients.

Abstract

from arXiv · show

In many machine learning tasks it is desirable that a model's prediction transforms in an equivariant way under transformations of its input. Convolutional neural networks (CNNs) implement translational equivariance by construction; for other transformations, however, they are compelled to learn the proper mapping. In this work, we develop Steerable Filter CNNs (SFCNNs) which achieve joint equivariance under translations and rotations by design. The proposed architecture employs steerable filters to efficiently compute orientation dependent responses for many orientations without suffering interpolation artifacts from filter rotation. We utilize group convolutions which guarantee an equivariant mapping. In addition, we generalize He's weight initialization scheme to filters which are defined as a linear combination of a system of atomic filters. Numerical experiments show a substantial enhancement of the sample complexity with a growing number of sampled filter orientations and confirm that the network generalizes learned patterns over orientations. The proposed approach achieves state-of-the-art on the rotated MNIST benchmark and on the ISBI 2012 2D EM segmentation challenge.

1. Introduction

The paper introduces SFCNNs to share weights across filter orientations, using steerable filters and group convolutions to achieve translation and rotation equivariance. The approach targets improved generalization and sample efficiency, with strong results on two benchmarks.

  • SFCNNs share weights over filter orientations to improve generalization and reduce sample complexity.
  • Steerable filters represent learned filters as linear combinations of fixed atomic filters, enabling exact rotations without interpolation artifacts.
  • Group convolutions ensure equivariant feature-map mappings, while the architecture supports arbitrary angular resolution beyond four sampled orientations.
  • The paper generalizes He’s weight initialization to filters represented by atomic-filter coefficients rather than direct pixel weights.
  • The network achieves state-of-the-art results on rotated MNIST and ranks among the top three entries in the ISBI 2012 EM segmentation challenge.

2. Equivariance properties of CNNs

Equivariance means that a function commutes with transformations acting on its input and output. Incorporating known transformation behavior into CNNs reduces the hypothesis space and can improve generalization over transformation-related inputs.

  • A function is equivariant when applying a group transformation before or after the function produces corresponding transformed outputs.
  • Invariance is a special case of equivariance in which the output remains unchanged under the transformation.
  • For segmentation, translation equivariance maps image orbits to corresponding label-image orbits.
  • Encoding transformation behavior by construction restricts models to mappings between quotient spaces, reducing the hypothesis space.
  • Extending equivariance beyond translations to rotations, mirroring, or dilations can improve generalization over larger transformation orbits.

3. Steerable Filter CNNs

SFCNNs combine steerable filters, rotational weight sharing, and group convolutions to achieve translation- and rotation-equivariant processing while avoiding interpolation artifacts. Their parameterization also supports arbitrary sampled orientation resolution and adapted weight initialization.

  • SFCNNs extend translation-equivariant CNNs by sharing weights over filter orientations, improving sample complexity and generalization across translated and discretely rotated images.
  • Steerable filters represent rotations as linear combinations of fixed atomic filters, enabling exact orientation responses without interpolation artifacts.Each rotated response can be synthesized from the atomic convolution responses.
  • Circular harmonics ψjk use radial parts τj(r) and angular frequencies k; the learned filters combine these atoms with complex weights whose phases enable steering.The experiments use Gaussian radial parts and limit angular frequencies to avoid aliasing.
  • Group convolutions preserve equivariance through intermediate layers, while orientation pooling produces rotation-invariant outputs for classification or segmentation.The network samples Λ equidistant filter orientations, with higher orientation resolution obtained by expanding phase-factor tensors.
  • Compared with independently learning filters in Λ orientations, an SFCNN uses Λ times fewer parameters for the same representation; its computational cost remains equivalent when ICNN = ΛISFCNN.The overhead for constructing kernels from the circular-harmonics basis is reported as negligible.
  • The paper generalizes He-style initialization to atomic-filter expansions, accounting for differing basis-filter energies and orientation channels.The proposed scheme normalizes filters and adjusts weight variance for the additional orientation dimension.

4. Prior and related work

Prior approaches incorporate transformation handling through augmentation, pooling, resampling, transformed feature dimensions, group convolutions, or steerable representations. Their reported limitations include restricted orientation sets and equivariance only to global transformations.

  • Data augmentation enriches training sets with transformed samples along group orbits without modifying network architectures.
  • Other approaches obtain invariance through transformed inputs and pooling, or achieve equivariance by resampling inputs according to a symmetry group.
  • Earlier rotation-equivariant CNNs use transformed batch or feature dimensions, group-convolutional feature maps, or steerable representations, sometimes including reflections.
  • Some group-convolutional approaches restrict rotations to four orientations, while transformed-image pipelines provide equivariance only to global transformations.

5. Experimental results

Experiments on rotated MNIST and ISBI 2012 evaluate sampled-orientation resolution, rotational generalization, initialization, and segmentation performance. SFCNNs improve with more orientations, generalize rotations better than conventional CNNs, and achieve strong benchmark results.

  • Rotated MNIST: Accuracy improves significantly as sampled orientations increase, then saturates around 12 to 16 orientations.Increasing angular resolution from 2 to 4 orientations often yields more gain than doubling the training set while retaining 2 orientations.
  • Rotated MNIST: SFCNNs generalize learned digit patterns over rotations substantially better than conventional CNNs across augmentation settings.The comparison trains on unrotated digits and tests on held-out digits rotated by different angles.
  • Rotated MNIST: For Λ = 16, augmentation with π/2 rotations does not change accuracy because those samples lie on the network’s invariant group orbit.Unlike conventional CNNs, SFCNNs also avoid increased error near zero angles when augmentation is used.
  • Rotated MNIST: The proposed coefficient initialization reduces rotated-MNIST test error from 0.957% with He initialization to 0.880%, reaching 0.714% with continuous-rotation augmentation.The paper reports a 29-fold reduction relative to the best previously published error.
  • ISBI 2012 2D EM segmentation challenge: On the ISBI 2012 EM segmentation challenge, the SFCNN pipeline achieves top-tier leaderboard results and improves upon the established comparison method.Predictions are ranked using the foreground-restricted Rand score V Rand and information score V Info.

6. Conclusion

SFCNNs achieve rotation equivariance by learning steerable filters and using group convolutions, while generalized initialization supports filters formed from atomic bases. Experiments show improved performance with more sampled orientations and state-of-the-art results on two benchmarks.

  • SFCNNs learn steerable filters and use group convolutions to obtain layerwise rotation equivariance.
  • Generalized He initialization applies to filters represented as linear combinations of atomic filters and empirically increases accuracy.
  • Performance improves as the number of sampled filter orientations grows, before reaching saturation.
  • Experiments confirm that SFCNNs generalize learned patterns across orientations, reducing sampling complexity relative to CNNs.
  • SFCNNs achieve state-of-the-art results on rotated MNIST and the ISBI 2012 2D EM segmentation challenge.

A.2. Group-convolutional layers

Group-convolutional layers map transformed feature maps equivariantly under the group action. Orientation pooling preserves equivariance and can produce rotation-invariant classification outputs.

  • Group-convolutional layers: Group convolutions map Rαζ(l)(x, θ) to Rαζ(l+1)(x, θ) under the group action.
  • Group-convolutional layers: Bias addition and activation remain equivariant because they do not depend on spatial position or orientation channel.
  • Group-convolutional layers: Max-pooling over orientations is used after the final group-convolutional layer for rotation-invariant segmentation or classification.
  • Group-convolutional layers: For classification, pooling out the remaining spatial structure makes the output invariant to input transformations.
  • Group-convolutional layers: Average pooling could replace maximum pooling while retaining equivariance by an analogous derivation.

B. Derivation of the generalized He weight initialization scheme

The generalized He initialization derives weight-variance conditions for filters composed from atomic filters, aiming to keep gradient and activation variances stable across layers. The forward-pass approximation uses independence assumptions and the central limit theorem.

  • B. Derivation of the generalized He weight initialization scheme: The derivation considers ReLU activations and pre-nonlinearity responses formed by convolving and summing filters across input channels.
  • B. Derivation of the generalized He weight initialization scheme: Filters are represented as combinations of Q real-valued atomic filters, with independent zero-mean weights that need not be identically distributed.
  • B. Derivation of the generalized He weight initialization scheme: The initialization requires gradient variance to remain constant across layers, preventing vanishing or exploding gradients from inappropriate initialization.
  • B. Derivation of the generalized He weight initialization scheme: Symmetric zero-mean weights and ReLU nonlinearities yield the stated relationship between activation and pre-activation variances under the derivation’s assumptions.
  • B. Derivation of the generalized He weight initialization scheme: The resulting initialization condition assigns contributions across the Q summands and is obtained from the variance-preservation requirement.
  • B. Derivation of the generalized He weight initialization scheme: The forward-pass calculation targets unamplified activation variance and approximates it using the central limit theorem for many independent finite-variance terms.

B.3. Normalization of complex atomic filters

The scheme initializes atomic filters after normalization, while complex filters require normalization that preserves the relative amplitude of their real and imaginary parts. Table 2 describes the baseline architecture used in initial experiments.

  • B.3. Normalization of complex atomic filters: The derived scheme initializes weights uniformly after normalizing each atomic filter to ∥ψq∥2 = 1.
  • B.3. Normalization of complex atomic filters: Complex-filter normalization must preserve the relative amplitude of real and imaginary parts to retain steerability.
  • B.3. Normalization of complex atomic filters: For sampled circular harmonics, real and imaginary parts may have unequal norms, preventing their independent normalization.
  • B.3. Normalization of complex atomic filters: Table 2 presents the SFCNN architecture used in initial experiments on sampled-orientation resolution and rotational generalization.

C. Details on the experimental setup

This section provides further details on the network architectures and training setup used in the experiments.

  • The section introduces additional details about the network architectures used in the experiments.
  • It also covers the training setup for the experiments.
  • These details supplement the preceding experimental analyses.

C.1. Rotated MNIST

The rotated MNIST experiments use a baseline architecture to study orientation dependence and rotational generalization, then select Λ = 16 for final benchmarking and further tuning.

  • The experiments first use the Table 2 architecture as a baseline for studying sampled orientations and rotational generalization.
  • Λ = 16 sampled orientations are fixed before further tuning with the larger Table 3 architecture.Table 3 specifies the architecture used in the final rotated MNIST benchmarking experiments.
  • 24 learned filters with Λ = 16 orientations produce 384 effective responses in the first layer of the smaller network.Rotational weight sharing makes extracting this comparatively large response set possible without overfitting.
  • All networks are trained for 40 epochs with Adam, exponential learning-rate decay from epoch 15, elastic-net regularization, and fully connected-layer dropout.The initial learning rate is 0.015, and dropout uses p = 0.3 in fully connected layers.

C.2. ISBI 2012 EM segmentation challenge

The ISBI 2012 EM segmentation network is a steerable-filter encoder–decoder that combines multiscale context with spatial localization and produces a pixel-wise membrane probability map.

  • The architecture is a symmetric encoder–decoder with skip-connections between stages of the same resolution.This design extracts semantic information from a large field of view while preserving precise spatial localization.
  • The network uses steerable group-convolution layers with Λ = 17 orientations and reflect-padded inputs to mitigate boundary artifacts.The topology is inspired by U-Net and FusionNet.
  • After decoding, max-pooling over orientations produces locally invariant features before central cropping and two 1 × 1 convolutions generate the probability map.
  • Training minimizes spatially averaged binary cross-entropy with Adam, elastic-net regularization, and dropout probability p = 0.4 in steerable and 1 × 1 convolution layers.
Loading 1711.07289v3…