PaperlayerView exact revision on arXiv

Source-linked AI summary

E2PN: Efficient SE(3)-Equivariant Point Network

Minghan Zhu, Maani Ghaffari, William A. Clark, Huei Peng

arXiv:2206.05398v3cs.CVcs.AIcs.RO

TL;DR

Learning from 3D point clouds is challenging because rotations add variation that conventional convolutions do not handle equivariantly. E2PN combines discretized group convolutions with quotient feature maps and a permutation layer. It reports strong performance across multiple tasks with substantially lower resource use than comparable group-convolution networks.

  • Problem

    3D point-cloud learning faces greater transformation variation, while discretized equivariant methods can incur high memory and computational costs.

    Source: §1 · p. 1

  • Method

    E2PN uses quotient feature maps on S2 × R3 and a permutation layer to recover rotation information in an SE(3)-equivariant point-cloud convolution structure.

    Source: §1 · p. 2

  • Results

    E2PN delivers state-of-the-art performance across multiple tasks while using only a fraction of the memory and computation of a group-convolution network with similar performance.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The quotient-space design is presented as an efficient, practical convolution approach for point-cloud learning.

    Source: §1 · p. 2

    E2PN does not outperform EPN on keypoint matching, and its permutation layer needs improvement for inputs without a clear pose definition.

    Source: §5 · p. 8

Abstract

from arXiv · show

This paper proposes a convolution structure for learning SE(3)-equivariant features from 3D point clouds. It can be viewed as an equivariant version of kernel point convolutions (KPConv), a widely used convolution form to process point cloud data. Compared with existing equivariant networks, our design is simple, lightweight, fast, and easy to be integrated with existing task-specific point cloud learning pipelines. We achieve these desirable properties by combining group convolutions and quotient representations. Specifically, we discretize SO(3) to finite groups for their simplicity while using SO(2) as the stabilizer subgroup to form spherical quotient feature fields to save computations. We also propose a permutation layer to recover SO(3) features from spherical features to preserve the capacity to distinguish rotations. Experiments show that our method achieves comparable or superior performance in various tasks, including object classification, pose estimation, and keypoint-matching, while consuming much less memory and running faster than existing work. The proposed method can foster the development of equivariant models for real-world applications based on point clouds.

1. Introduction

Learning from 3D point clouds is challenging in part because rotations add variation that conventional convolutions do not handle equivariantly. E2PN uses quotient feature maps and a permutation layer to make SE(3)-equivariant point-cloud convolutions more efficient.

1. Introduction

  • Rotations create more variation in 3D than in 2D, while conventional convolutions are not rotation-equivariant.The paper notes that 2D rigid transformations have 3 degrees of freedom, compared with 6 in 3D.Source: §1 · p. 1
  • Equivariant learning has not become the default for 3D tasks partly because continuous-group operations can be hard to integrate and discretized-group feature maps can be costly.Source: §1 · p. 2
  • The method enables SE(3)-equivariance up to SO(3) discretization by replacing KPConv and using quotient feature maps on S2 × R3.A final permutation layer recovers full 6 DoF information.Source: §1 · p. 2
  • A permutation layer recovers full SE(3) information from the quotient space.Source: §1 · p. 2
  • E2PN achieves comparable or better performance with significantly reduced computational cost than existing equivariant models.Source: §1 · p. 2

2. Related work

Related work spans group convolutions, steerable CNNs, theoretical formulations of equivariant networks, and applications in perception. The paper positions E2PN among practical equivariant methods for 3D point-cloud learning.

Group convolutions

  • Discretized group convolutions offer a relatively simple structure, but enlarged feature and kernel domains increase memory and computation costs, especially for large groups.Continuous-group Monte Carlo approaches can also have a large memory burden as network depth grows.Source: §2 · p. 2

Steerable CNNs

  • Steerable CNNs generalize group convolutions through feature representations, but continuous-group Fourier analysis can make them complex to apply.Some empirical work found irreducible-representation steerable CNNs underperform regular representations on certain tasks.Source: §2 · p. 2

Theoretical progress

  • Theoretical work characterizes equivariant maps as convolutions and develops equivariant-kernel formulations for groups including E(2) and compact groups.Other work addresses Monte Carlo group convolutions and equivariant nonlinear layers.Source: §2 · pp. 2–3

Applications of equivariant learning in perception tasks

  • Equivariant networks have been applied to point-cloud learning, registration, manipulation, image processing, and 3D detection.EPN is an SE(3)-equivariant group-convolution network based on KPConv and serves as a major baseline for this work.Source: §2 · p. 3

3. Methodology

E2PN extends KPConv with discretized group convolutions, quotient features that reduce computation, and a permutation layer that recovers distinguishable rotation features. The method is framed as an equivariant CNN on homogeneous spaces, balancing simplicity, efficiency, and expressiveness. [p43, p50, p54, p35]

3.1.1 Improved efficiency with quotient features

  • Feature fields move from SO(3)′×R3 to S2′×R3 by quotienting with SO(2)′; |S2′|=|SO(3)′|/|SO(2)′| reduces feature-map and kernel sizes and convolution costs.The quotient representation groups rotations by the destination of a sphere point.Source: §3.1.1 · p. 3

3.1.2 Information recovery with permutations

  • Because quotient features merge rotations sharing a sphere destination, an injective rotation-to-permutation map lets the permutation layer recover distinguishable SO(3)′ features.Each rotation corresponds to a unique permutation of the S2′ elements.Source: §3.1.2 · pp. 3–4

3.2.1 Recap of KPConv

  • KPConv gathers neighboring input features at kernel-point coordinates to align them before convolution; this paper does not use deformable KPConv.The conventional convolution is discretized over kernel points, with gathered features denoted ˆf.Source: §3.2.1 · p. 4

3.2.2 KPConv on group and on quotient space

  • Group KPConv defines features on SO(3)′×R3 and kernels on SE(3)′, while quotient convolution uses a section map to act on S2′×R3.The full group domain scales as |SO(3)′||R3′|, making feature storage and convolution expensive.Source: §3.2.2 · pp. 4–5

3.2.3 Symmetric kernels

  • The kernel is closed under SO(3)′ and satisfies the SO(2)′ steerability constraint, reducing feature-gathering positions to |R3′|.The kernel points are Platonic-solid vertices at radius r plus the origin; rotations permute them without changing their positions as a set.Source: §3.2.3 · pp. 5–6

3.2.4 Choices of the discretization of SO(3) and S2

  • Experiments use the icosahedron for the finest SO(3)′ discretization, a smaller S2′ than the dodecahedron, and direct comparison with existing methods.For small rotation groups, additional edge or face centers or multiple polyhedra can provide more kernel points.Source: §3.2.4 · p. 6

3.2.5 Other aspects of the network

  • The network uses standard nonlinearities, point subsampling with neighbor aggregation, batch normalization, and attentive pooling over S2′ to produce invariant features.Task-specific heads use cross-entropy for classification, L2 loss for residual pose regression, and batch-hard triplet loss for keypoint matching.Source: §3.2.5 · p. 6

3.3. Relation to existing work

  • E2PN adapts KPConv within equivariant CNNs on homogeneous spaces, balancing simplicity, efficiency, and expressiveness while using permutation to distinguish rotations.The paper also frames the method as a steerable CNN on S2×R3 with SO(2) stabilizer, or on R3 with SO(3) stabilizer.Source: §3.3 · p. 6

4. Experiments

Across classification, pose estimation, and keypoint matching, E2PN aims to retain competitive performance while reducing memory and computation relative to equivariant baselines. The experiments also examine its classification behavior, pose-estimation reliability, and limits on local patches without a canonical pose.

4. Experiments

  • The study evaluates ModelNet40 classification and pose estimation alongside 3DMatch keypoint matching, comparing E2PN chiefly with EPN, a KPConv-style SE(3)-equivariant network.EPN uses features on I × R3, whereas E2PN uses S2′ × R3.Source: §4 · p. 6
  • E2PN's quotient-space convolutions use much less memory and run much faster than EPN across all three tasks under matched input sizes, channel counts, and layer counts.The comparison uses one NVIDIA A40 GPU; training and inference numbers are not directly comparable because batch sizes may differ.Source: §4 · pp. 6–7
  • Symmetric kernels improve feature-gathering efficiency by gathering at |R3′| locations and permuting features instead of gathering at |S2′||R3′| locations.The reported further efficiency gain is especially in computational speed.Source: §4 · p. 7

4.1. Object classification on ModelNet40

  • On ModelNet40 classification, E2PN is the best listed equivariant model and is on par with PCT among all models.Without rotational-augmentation interpolation, its performance falls between non-equivariant and continuously SO(3)-equivariant models; the paper also reports that the permutation layer outperforms GA pooling with negligible overhead.Source: §4.1 · pp. 7–8

4.2. Pose Estimation on ModelNet40

  • For airplane pose estimation, E2PN has similar overall rotation accuracy to EPN, with a slightly larger mean error but lower median error, maximum error, and standard deviation.The authors suggest the results could indicate greater registration robustness when rotations are represented as feature permutations rather than a single feature-map element.Source: Table 3 · p. 7, §4.2 · p. 8

4.3. Keypoint matching on 3DMatch

  • On 3DMatch keypoint matching, E2PN is comparable to top methods while using a fraction of EPN's computational resources, but it does not outperform EPN on this task.GA pooling is used because local patches lack a canonical pose for permutation-layer pose supervision; global-scan or matching-scan context is left for future work.Source: §4.3 · p. 8

5. Conclusion

E2PN combines quotient-space feature maps, symmetric kernels, and a permutation layer in an efficient SE(3)-equivariant point-cloud convolution design. Its experiments report strong multi-task performance with reduced resources, while keypoint matching without a clear pose remains a limitation.

5. Conclusion

  • E2PN uses feature maps on S2 × R3 with stabilizer SO(2), symmetric kernels, and a permutation layer to recover SO(3)′ information from S2′ features.The paper reports strong performance across multiple tasks while using only a fraction of the memory and computation of a group-convolution network with similar performance.Source: §5 · p. 8
  • The paper presents the design as a way to bring SE(3)-equivariance to mainstream point-cloud networks for various tasks.This is framed as a prospective opportunity rather than a demonstrated deployment result.Source: §5 · p. 8
  • The authors identify keypoint matching without a clear pose definition as a limitation, noting that the network, especially the permutation layer, needs improvement in this setting.They also leave alternative feature designs, reduced discretization effects, and extensions to other groups as future directions.Source: §5 · p. 8

E2PN: Efficient SE(3)-Equivariant Point Network (Appendix)

The supplied appendix material identifies the paper's authors and institutional affiliations and gives its arXiv version date.

E2PN: Efficient SE(3)-Equivariant Point Network (Appendix)

1. Preliminaries and notation

The preliminaries introduce group actions and representations, define equivariance, and relate quotient spaces to subgroups and homogeneous spaces.

1. Preliminaries and notation

  • A group is a set with an associative operation, an identity, and an inverse for every element.The text gives integers under addition as an example.Source: §1 · p. 11
  • A representation describes a group action on a linear space using linear transformations; in finite dimensions these transformations can be written as invertible matrices.The text uses two-dimensional rotations acting on R2 as an example.Source: §1 · p. 11
  • A map is G-equivariant when applying a group action before the map agrees with applying the corresponding action after it.Source: §1 · p. 11
  • For a subgroup H of G, the quotient space G/H consists of cosets; the stabilizer of the quotient's identity coset is H.Source: §1 · p. 11
  • A homogeneous space is one on which the group acts transitively, and a quotient space G/H is homogeneous.Source: §1 · p. 11

2. Definition of the section functions

A section function selects a group element representing each point in the quotient space, making the convolution well-defined. For the continuous SO(3)/SO(2) case, the paper uses a specific Euler-angle choice and fixes discrete choices for consistency.

2. Definition of the section functions

  • A section s maps each quotient-space point x to a group element in its coset, satisfying s(x)H = x.This choice is needed to define the convolution in a homogeneous space.Source: §2 · p. 11
  • For the continuous case, H = SO(2), G = SO(3), and X = S2; the section maps each sphere point x to a rotation taking the north pole to x.The chosen Euler-angle form s(Rn) = Rz(α)Ry(β) removes the final z-axis rotation.Source: §2 · pp. 11–12
  • In the discretized setup, the continuous section may not map into the discretized rotation group, so an arbitrary coset representative is selected and then fixed.Fixing the selection makes behavior deterministic and consistent across inputs.Source: §2 · p. 12

3. The derivation of our proposed convolution

The derivation applies existing equivariance results to construct the convolution, then specializes its kernel domain to SE(3) with SO(2) as stabilizer. The resulting valid kernels occupy a constrained domain and obey an invariance condition.

3.1. The equivariance of our convolution

  • The derivation builds on existing results and applies them to a convolution specification not previously discussed, rather than establishing new theorems.The paper starts from the induced-representation conclusions in.Source: §3.1 · p. 12
  • The function h relates s(gx) and gs(x), which lie in the same coset but may differ by an SO(2) element.It captures the representative-group-element twist beyond moving to another coset.Source: §3.1 · p. 12
  • Equivariant linear maps are expressed as twisted cross-correlations over G/H, and scalar-type features make the subgroup representations identity maps.With identity subgroup representations, the convolution simplifies to the form used in the main paper.Source: §3.1 · p. 12

3.2. The specific form of our kernel

  • For G = SE(3) and H = SO(2), the double-coset domain is isomorphic to S2 × R+ × R, with sphere coordinates and radial-height coordinates.Each (rg, zg) pair corresponds geometrically to a circle around the z-axis at height zg.Source: §3.2 · pp. 12–13
  • The valid kernel domain is a subset of S2 × R3, with the scalar-feature constraint reducing to κ(hx) = κ(x).The paper states that this invariance condition is equivalent to its main-text Eq. (6).Source: §3.2 · p. 13

4. Equivariance of element-wise non-linear layers and normalization layers

The paper argues that element-wise nonlinear and normalization layers preserve G-equivariance because the induced representation acts by changing coordinates without changing feature values.

4. Equivariance of element-wise non-linear layers and normalization layers

  • BatchNorm and InstanceNorm apply a constant scale and shift across spatial points and spherical anchors, with their statistics taken over B × N × A or N × A.The feature map is treated as a function from N × A to RB×C.Source: §4 · p. 13
  • The induced representation acts like a regular representation by changing coordinates without modifying function values.Under this representation, element-wise operations commute with the group action.Source: §4 · p. 13
  • Element-wise nonlinearities such as ReLU, as well as BatchNorm and InstanceNorm, are G-equivariant.The paper derives this from the commutation of these operations with the induced representation.Source: §4 · p. 13

5. Prediction heads and loss functions

The prediction heads use anchor permutations and matching to handle rotation in pose estimation and classification, while keypoint matching omits the permutation layer. Each task uses a loss suited to its prediction targets.

5.1. Pose estimation task

  • Pose estimation tests 60 anchor permutations, matches anchor pairs, selects the highest-scoring rotation, then regresses the residual rotation with quaternions.Anchor-pair matching is binary rather than rotation-class classification, since some anchors can align under an incorrect rotation; the losses are binary cross entropy and L2.Source: §5.1 · pp. 13–14, Figure 1 · p. 14

5.2. Object classification task

  • Object classification compares permuted input features with learned class reference features, chooses the highest-scoring class, then uses that class's best rotation for invariant features.This assumes only the correct class yields high-quality matching under the actual rotation; training uses cross entropy and binary cross entropy.Source: §5.2 · pp. 14–15, Figure 2 · p. 15

5.3. Keypoint matching task

  • Keypoint matching omits the permutation layer because local patches have no canonical pose or relative-pose pairing, and instead uses GA pooling with attention over S2′.The task uses batch-hard triplet loss, consistent with.Source: §5.3 · p. 15

6. More specifications in the experiments

The experiments largely follow EPN settings, with a reduced-width classification backbone that does not harm accuracy. Additional details clarify rotational augmentation, efficiency-comparison inputs, and the steerable CNN baseline.

6. More specifications in the experiments

  • For keypoint matching, efficiency-comparison input size is defined by the number of global scans and local patches per scan; Table 5 reports the comparison batch sizes.The notation is ng × nl, where ng is the number of global scans and nl is the number of local patches extracted from each scan.Source: §6 · p. 15, Table 5 · p. 16
  • The experiments generally follow EPN's optimizer, learning-rate schedule, network depth, and width, except ModelNet40 classification uses a backbone half as wide without harming accuracy.The first-layer width changes from 64 to 32, with later backbone layers adjusted accordingly; widths are aligned across KPConv, EPN, and E2PN for valid comparison.Source: §6 · pp. 15–16
  • Training uses random rotations to support interpolation among discretized rotations and residual-rotation regression; testing uses fixed rotations for repeatable, comparable results.Rotational augmentation is applied during training, while each test input receives fixed rotations.Source: §6 · p. 16
  • The steerable CNN baseline was reimplemented because its task-specific implementation was unavailable, with hand-tuned voxel-generation and learning-rate settings.The implementation uses σ = 0.03 and an initial learning rate of 10^-3; it uses norm pooling as the invariant layer.Source: §6 · p. 16
  • The steerable CNN baseline underperforms both EPN and E2PN, but efficiency is not compared because their inputs and network structures differ.The passage suggests possible explanations, including auxiliary rotation-estimation training and voxel inputs losing detail, but does not establish either cause.Source: §6 · p. 16
Loading 2206.05398v3…