B-Spline CNNs on Lie Groups

Erik J Bekkers

Source-linked AI summary

B-Spline CNNs on Lie Groups

Erik J Bekkers

arXiv:1909.12057v4Sep 26, 2019cs.LGstat.ML

TL;DR

G-CNN implementations were previously limited in the groups they could handle, motivating a more general construction. This paper defines B-spline kernel expansions on Lie groups, enabling localized, atrous, and deformable convolutions, and reports stronger performance than classical 2D CNNs on rotation- and scale-sensitive tasks.

Problem
Practical G-CNN implementations were limited to discrete groups or continuous compact groups, restricting frameworks for arbitrary Lie groups.
Method
The paper defines B-splines on Lie groups through the Lie algebra to expand and sample group-convolution kernels.
Results
G-CNN architectures outperform classical 2D counterparts on cancer detection and facial landmark localization, while localized and atrous group convolutions add evaluated capabilities.
Takeaways & Limitations
The framework supports localized, atrous, and deformable G-convolutions for arbitrary Lie groups, with localization important in scale-translation experiments.
Takeaways & Limitations
Uniform B-splines covering a Lie group generally cannot be constructed, and the experiments focus on affine groups acting on Euclidean input data.

Abstract

from arXiv · show

Group convolutional neural networks (G-CNNs) can be used to improve classical CNNs by equipping them with the geometric structure of groups. Central in the success of G-CNNs is the lifting of feature maps to higher dimensional disentangled representations, in which data characteristics are effectively learned, geometric data-augmentations are made obsolete, and predictable behavior under geometric transformations (equivariance) is guaranteed via group theory. Currently, however, the practical implementations of G-CNNs are limited to either discrete groups (that leave the grid intact) or continuous compact groups such as rotations (that enable the use of Fourier theory). In this paper we lift these limitations and propose a modular framework for the design and implementation of G-CNNs for arbitrary Lie groups. In our approach the differential structure of Lie groups is used to expand convolution kernels in a generic basis of B-splines that is defined on the Lie algebra. This leads to a flexible framework that enables localized, atrous, and deformable convolutions in G-CNNs by means of respectively localized, sparse and non-uniform B-spline expansions. The impact and potential of our approach is studied on two benchmark datasets: cancer detection in histopathology slides in which rotation equivariance plays a key role and facial landmark localization in which scale equivariance is important. In both cases, G-CNN architectures outperform their classical 2D counterparts and the added value of atrous and localized group convolutions is studied in detail.

1 INTRODUCTION

G-CNNs use group geometry to disentangle feature representations and guarantee equivariance, but prior practical implementations were limited in group type. This paper introduces Lie-group B-spline convolutions and evaluates them on rotation- and scale-sensitive tasks.

1 INTRODUCTION: G-CNNs lift feature maps to group domains, disentangling pose parameters and learning high-level patterns through relative transformations.This lifting supports representations organized by the group structure rather than only spatial coordinates.
1 INTRODUCTION: Prior G-CNN implementations were limited to discrete groups or continuous compact unimodular groups that support Fourier methods.The paper identifies this restriction as the central implementation limitation it addresses.
1 INTRODUCTION: The proposed framework defines B-splines on Lie groups through the Lie algebra and uses them to expand and sample group-convolution kernels.The Log-map provides the connection from the group to the flat Lie-algebra vector space.
1 INTRODUCTION: Localized, atrous, and deformable G-convolutions arise from localized, sparse, and non-uniform B-spline expansions, respectively.The framework is intended for arbitrary Lie groups, while the paper focuses experimentally on affine groups acting on Euclidean data.
1 INTRODUCTION: On histopathology cancer detection and facial landmark localization, G-CNNs outperform classical 2D counterparts while localized and atrous convolutions are studied.These tasks emphasize rotation and scale equivariance, respectively.

2 RELATED WORK

Related work spans discrete, regular continuous, and steerable continuous G-CNNs, while this paper extends B-spline constructions to Lie-group settings. Its scale-equivariant formulation connects to scale-space, semigroup, and dilation-based approaches.

2 RELATED WORK: Existing G-CNN research includes discrete, regular continuous, and steerable continuous approaches, with steerable methods restricted to unimodular groups such as roto-translations.The literature is presented as a progression from grid-preserving discrete groups to continuous and steerable constructions.
2 RELATED WORK: The paper presents scale-translation G-CNNs as a direct G-CNN application achieving equivariance beyond roto-translations.Related approaches address scale equivariance through scale-space, semigroup, weight-sharing, or dilation mechanisms.
2 RELATED WORK: The proposed Lie-group B-spline construction generalizes earlier B-spline SE(2) CNNs and differs from graph and manifold methods in how relative points are represented.Here, group points relate through the logarithmic map, whereas related methods use Euclidean embeddings or gauge-frame exponential maps.

3 LIE GROUP CNNS

This section develops the group-theoretic foundations of G-CNNs, defining equivariant feature-map transformations and group-correlation layers. It then introduces B-spline kernel expansions that make these constructions applicable to affine Lie groups.

Affine Lie groups: For affine groups G = R^d ⋊ H, the group product combines translation addition after transforming the second translation by h1 with multiplication in H.The special Euclidean group SE(2) results when H = SO(2), whose elements act on R^2 by matrix multiplication.
Preliminaries: G-CNNs represent feature maps as functions on spaces carrying group actions, with transformations composed according to the group product.Lie groups additionally provide differential structure and infinitesimal generators through their Lie algebras.
Equivariance: An operator is equivariant when transforming the input and then applying the operator agrees with applying the operator and then transforming the output.For equivariant neural networks, information is shifted under transformations rather than lost.
Group-convolutional layers: Equivariant layers include lifting, group correlation, and projection layers, which respectively create group-indexed features, correlate over the group, and integrate over H.The lifting layer matches kernels with the input under transformations in G, while the group-correlation layer transforms kernels and performs spatial cross-correlation.
B-spline kernels: Kernel operators for G-CNNs are expanded using shifted cardinal B-splines, localized polynomial functions that can be sampled under arbitrary group transformations.Cardinal B-splines are piece-wise polynomials with finite support, and multivariate versions are formed by tensor products.
Modular design: The resulting modular construction requires group-specific definitions for H’s product, inverse, action on R^d, and logarithmic map.This framework is designed for equivariant G-CNNs on Lie groups of the form G = R^d ⋊ H.

4 EXPERIMENTS

Experiments evaluate roto-translation G-CNNs for rotation-sensitive tumor classification and scale-translation G-CNNs for facial landmark localization, comparing kernel specifications with 2D baselines. G-CNNs show benefits from equivariant multi-scale or localized designs, while excessive rotation sampling can degrade performance.

4.1 Roto-translation CNNs: 327,680 RGB PCam patches are binary-labeled for metastasis, making the classification problem rotation invariant across image features.The dataset consists of histopathologic lymph-node patches, with arbitrary rotations occurring from edges through tissue-level structures.
4.1 Roto-translation CNNs: Roto-translation G-CNNs are compared with a 2D baseline while varying local, global, and atrous kernels on the SO(2) component.The networks share the same architecture, differing in how kernels are defined over rotations.
4.1 Roto-translation CNNs: Finer SO(2) sampling improves results up to Nh = 12, after which performance slightly degrades, potentially because spline detail exceeds the 5x5 kernel signal resolution.Atrous kernels generally outperform dense kernels, and localized kernels perform better in the low Nk regime.
4.1 Roto-translation CNNs: 90° augmentation is crucial for the 2D model but has little effect on the SE(2) model; the SE(2) model without augmentation outperforms the augmented 2D model.The result is consistent with G-CNN equivariance being guaranteed by construction rather than supplied through augmentation.
4.2 Scale-translation CNNs: CelebA scale-translation G-CNNs use either all-scale, neighboring-scale, or no-scale interactions, while 2D baselines use fixed-scale kernels tested separately.Localized kernels use Nk = 3 and no-scale interaction uses Nk = 1; each G-CNN remains multi-scale because kernels operate across a range of scales.
4.2 Scale-translation CNNs: For large scale ranges, G-CNNs outperform 2D CNNs by considering small and large features simultaneously, and localized interactions outperform dense and no-scale interactions.Scale augmentation moderately benefits 2D CNNs but does not reach G-CNN performance; the authors state that a multi-scale approach seems essential.

5 CONCLUSION

The paper presents B-spline basis functions as a flexible foundation for G-CNNs on arbitrary Lie groups. Their local support enables localized, atrous, and deformable kernels, with experiments demonstrating benefits for localized and atrous group convolutions.

5 Conclusion: The framework builds G-CNNs for arbitrary Lie groups using B-spline basis functions to represent convolution kernels.The basis functions have properties that classical Fourier-based basis functions cannot achieve.
5 Conclusion: Localized, atrous, and deformable convolution kernels arise from localized, sparse, and non-uniform B-spline constructions.The paper describes B-spline basis functions as smooth pixels on Lie groups, supporting familiar CNN design choices.
5 Conclusion: Experiments on two applications and two groups demonstrate the added value of localized and atrous group convolutions.In scale-translation G-CNNs specifically, kernel localization was important.

A PROOF OF THEOREM 1

The proof derives the structure of equivariant linear operators by representing them as integral operators and imposing kernel constraints. It then relates homogeneous spaces to quotient groups and explains why lifting functions to the group removes a kernel symmetry constraint.

A Proof of Theorem 1: A linear bounded operator K is an integral operator by the Dunford-Pettis theorem.This establishes the integral-operator form used in the proof.
A Proof of Theorem 1: Left-equivariance imposes bi-left-invariance on the kernel ˜k.The constraint is stated for every g ∈ G and every f ∈ L2(X).
A Proof of Theorem 1: Transitivity of the G-action on Y allows every y to be written as y = gy y0 for a fixed origin y0.This representation supports reducing the operator description to a chosen origin.
A Proof of Theorem 1: Every homogeneous space Y is identified with a quotient G/H, where H is the stabilizer of a selected origin y0.The stabilizer satisfies H = StabG y0.
A Proof of Theorem 1: When Y = G, the symmetry constraint vanishes, so lifting functions to G permits equivariant maps without kernel constraints.The group is identified with G/{e}, where e is the identity element.
A Proof of Theorem 1: For groups G = R^d ⋊ H, the action of H on R^d supplies the group structure and the representations needed to construct B-spline G-CNNs.The subsequent examples derive the required tools from these core definitions.

B.2 THE 2D ROTATION GROUP H = SO(2)

The rotation group SO(2) is integrated with translations through the semidirect product SE(2), while its logarithmic map and representations provide the ingredients for rotation-aware B-spline kernels. Figure 6 illustrates how oriented feature collections and kernels transform under the group action.

B.2 The 2D Rotation Group H = SO(2): SO(2) consists of 2×2 orthogonal matrices with determinant 1, representing rotations.Its elements are parameterized by an angle θ.
B.2 The 2D Rotation Group H = SO(2): Combining translations with SO(2) yields the roto-translation group SE(2) = R^2⋊SO(2).The group structure follows from the semidirect-product construction.
B.2 The 2D Rotation Group H = SO(2): The SO(2) representations implement periodic shifts θ′ − θ mod 2π along the rotation axis.These representations describe how SE(2) convolution kernels transform.
B.2 The 2D Rotation Group H = SO(2): The logarithmic map on SO(2) is given by the matrix logarithm and supports the B-spline construction in the Lie algebra.The basis is centered around rotation elements and computed using the Lie-algebra representation.
B.2 The 2D Rotation Group H = SO(2): Figure 6 contrasts a transformed collection of oriented features with convolution kernels on SE(2) whose weights encode expected local orientations.The feature collection transforms by left multiplication, while kernels transform through the group representation.

B.3 SCALING GROUP H = (R+, ×)

The section develops scaling and related transformation groups, then uses logarithmic maps and Lie-algebra coordinates to construct B-spline kernels on these spaces and their quotients.

The scaling group R+ uses multiplication, with identity element e = (1), and acts on Rd by scalar multiplication.
Combining translations with scaling yields the scale-translation group R2 ⋊R+, whose action transforms spatial locations and scales of features.
Under this group action, kernel scaling combines planar scaling with a logarithmic shift along the scale axis, with Jacobian determinant | det h| = sd.
For the scaling group, the logarithmic map is the natural logarithm, Log h = ln s, providing coordinates for B-spline construction.
The framework also treats rotations, roto-translations, and spherical quotients using Haar measures, matrix logarithms, exponential coordinates, and tangent-space B-splines.
The sphere is represented as the quotient SO(3)/SO(2), with B-splines defined through a logarithmic map into a two-dimensional tangent space.
For spherical kernels, setting αi = 0 neglects an approximately isotropic rotation effect, while shifted spline superpositions remain non-isotropic by construction.
The related scale-space construction samples kernels and images on a discrete grid and interprets scale-slice correlations as discrete atrous correlations.

C.1.2 RELATION TO LIFTING CORRELATIONS (EQ. (6)) WITH B-SPLINES

This section relates B-spline lifting correlations to scale-space lifting by expressing kernels in a B-spline basis and approximating Gaussian kernels with scaled B-splines.

The lifting kernel is represented in a B-spline basis whose centers receive weights through a coefficient map c.
Approximating the Gaussian kernel Gs(x) with a scaled B-spline makes the proposed lifting correlation coincide with the corresponding scale-space lifting followed by non-scale-interacting correlation.
The comparison is framed through group kernels K and their Lie-algebra representations ˜K, with integration governed by the Haar measure on G.
Gauge-equivariant correlation instead integrates over the Lie algebra and maps tangent-space vectors to the group through the exponential map.

C.2.2 RELATION TO G-CORRELATIONS (EQ. (7)) WITH B-SPLINES

The section establishes an identity between localized B-spline group correlations and gauge-equivariant correlations by relating group integration through Log to tangent-space integration through Exp.

For Lie groups, localized B-spline group correlation and gauge-equivariant correlation are identical under the stated mappings between the group and Lie algebra.
The derivation uses a substitution by left multiplication and the left-invariance of the Haar measure.
The final change switches integration from a region in the Lie group to the corresponding region in the Lie algebra.

D G-CNN ARCHITECTURES

The experiments use sequential G-CNNs with lifting, group-correlation, pooling, and projection layers, adapting details for metastasis classification and landmark detection.

G-CNN architectures: Both task-specific architectures begin with a lifting layer, continue through group-correlation layers with optional spatial max-pooling, and end with projection over H and a 1 × 1 output stage.
PCam metastasis classification: For PCam, 64 × 64 inputs are cropped to 88 × 88, unpadded layers reduce the spatial size to 1 × 1, and the final layer produces two softmax classes.
PCam metastasis classification: The first five PCam layers, including rotation pooling, encode images into a 64-dimensional rotation-invariant feature vector before classical classifier layers.
Kernel support: To reduce orientation bias, kernels approximate disk-shaped support by retaining only spline centers within a radius and discarding corner basis functions.
Group-domain sampling: The group-domain grid is uniformly sampled, with dense and atrous kernels using different scale parameters determined by the sampling resolutions Nk and Nh.
Data augmentation: Training follows geometric augmentations such as 90° rotations and horizontal flips, alongside brightness, saturation, hue, and contrast variations.
PCam configuration: Table 1 reports PCam SE(2) G-CNN settings and free-parameter counts, with Nk specifying the number of basis functions on the SO(2) kernel component.

D.2 CELEBA

The CelebA landmark detector uses scale-aware feature processing followed by heatmap prediction, with landmarks selected from the resulting heatmaps. Performance is measured by the fraction of detections within 10 pixels of the true landmark.

Architecture: The ten-layer architecture uses zero padding to preserve 128 × 128 spatial dimensions, with batch normalization and ReLU after each layer except the final one.Layer 10 instead adds a bias vector and applies a logistic sigmoid activation.
Feature processing: The first six layers, including average pooling over scale, assign locally scale-invariant feature vectors to each pixel.The final layers convert these feature maps into heatmaps using regular 2D convolutions.
Landmark localization: Landmarks are localized by applying argmax independently to each heatmap.Each heatmap corresponds to a landmark prediction.
Evaluation: A detection counts as successful when its distance from the actual landmark is less than 10 pixels.The reported success rate averages successful detections across all five landmarks and all images.