PaperlayerView exact revision on arXiv

Source-linked AI summary

Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties

Benjamin Kurt Miller, Mario Geiger, Tess E. Smidt, Frank Noé

arXiv:2008.08461v4cs.LGphysics.chem-phphysics.comp-phstat.ML

TL;DR

For scalar molecular-property prediction, the paper examines what rotationally equivariant layers add beyond invariant layers. It compares distance-only with distance-and-angle convolutions in an e3nn ablation on QM9, finding a larger average error reduction from adding rotating features than from adding depth.

  • Problem

    The paper asks what invariant-only layers leave out when neural networks regress scalar molecular properties.

    Source: §1 · p. 1

  • Method

    The authors ablate rotating features in L1Net against L0Net on QM9, comparing networks with the same architecture except for the internal feature equivariance.

    Source: §2.1 · p. 3, §3 · p. 5

  • Results

    Adding rotating features improved performance by 23% on the mean absolute error, while an added invariant block reduced error by an average of 4%.

    Source: §2.1 · pp. 3–4

  • Takeaways & Limitations

    For the tested QM9 models, rotating features improved parameter efficiency by reducing error without changing the number of parameters.

    Source: §2.1 · p. 3

    It remains unclear whether any class of targets is fundamentally better suited to rotating-feature architectures; SchNet had the lowest error on the targets most impacted by the ablation.

    Source: §2.1 · p. 5

Abstract

from arXiv · show

Equivariant neural networks (ENNs) are graph neural networks embedded in $\mathbb{R}^3$ and are well suited for predicting molecular properties. The ENN library e3nn has customizable convolutions, which can be designed to depend only on distances between points, or also on angular features, making them rotationally invariant, or equivariant, respectively. This paper studies the practical value of including angular dependencies for molecular property prediction directly via an ablation study with \texttt{e3nn} and the QM9 data set. We find that, for fixed network depth and parameter count, adding angular features decreased test error by an average of 23%. Meanwhile, increasing network depth decreased test error by only 4% on average, implying that rotationally equivariant layers are comparatively parameter efficient. We present an explanation of the accuracy improvement on the dipole moment, the target which benefited most from the introduction of angular features.

1 Introduction

The introduction asks what rotationally equivariant internal layers add when predicting scalar molecular properties, and frames a QM9 comparison of distance-only and distance-plus-angle convolutions. It situates that comparison among invariant and equivariant molecular networks and identifies architectures and property types outside the study’s scope.

1 Introduction

  • The paper asks what invariant-only networks miss for scalar molecular-property regression compared with networks that include equivariant layers.The study uses QM9 properties, which are invariant to molecular rotation and translation.Source: §1 · p. 1
  • The comparison holds the ENN task to scalar, rototranslationally invariant outputs while varying whether convolutions use distances alone or distances and angles.The authors also examine whether adding depth to rotationally invariant networks reduces test error.Source: §1 · pp. 1–2
  • L0Nets use non-rotating features and invariant layers, whereas L1Nets use rotating features and equivariant layers, with non-rotating output features for invariant targets.The introduction relates this framing to spherical harmonics, whose feature degrees are indexed by ℓ.Source: §1 · p. 2

1.1 Related Work

  • The study builds on SchNet’s continuous convolutions, Tensor Field Networks’ and Cormorant’s angular equivariance, and se3cnn’s gated nonlinearity within e3nn.e3nn is described as a superset of Tensor Field Networks, SchNet, and se3cnn; its support for Cormorant’s two-body interaction is not tested here.Source: §1.1 · p. 2
  • DimeNet, despite leading QM9 regression, is excluded because its directional message passing is not trivially compatible with the frameworks studied.The authors leave investigation of incorporating its spherical Fourier-Bessel edge features for future work.Source: §1.1 · p. 2
  • QM7-X offers tensor properties for prediction, while SchNet and DimeNet in their current forms cannot predict tensor quantities.Source: §1.1 · p. 2

2 Methods

The section specifies a molecular network with shared atom-wise weights and invariant aggregation, then tests angular features against distance-only and deeper variants.

Atom-wise

  • Each atom receives a dense layer with shared parameters, which also serves as a learned embedding of its element.Source: Atom-wise · p. 2

Radial Basis Function (rbf)

Convolution

  • L0Net restricts input and output degrees to 0, while L1Net allows degrees 0 and 1 for scalar and rotating features.The convolution output is normalized to have component-wise unity second moments under the stated input assumptions.Source: Convolution · p. 3

Gated Block

  • Scalar features use Softplus, while degree-1 vector features are multiplied by an activated scalar to introduce nonlinearity while preserving equivariance.The preceding layer supplies extra learned scalar features for the gating.Source: Gated Block · p. 3

Final Atom-wise and Shift, Scale, Aggregate

2.1 Experiment

  • On unseen QM9 test data, adding rotating features (L0Net to L1Net) reduced mean absolute error by 23% relative to L0Net, with architecture and parameter count unchanged.Source: §2.1 · p. 3, Table 1 · p. 4
  • 4% average error reduction came from adding an invariant convolutional block; L0Net Deep also worsened predictions on Cv and μ.By contrast, L1Net improved on every target relative to L0Net.Source: §2.1 · p. 4
  • For a two-dipole example, predicting magnitude squared requires angular dependence for unbiased estimates under the stated assumptions; QM9 distance-only sufficiency remains open.The experiments used random QM9 splits of 109,000 training, 1,000 validation, and 23,885 test molecules.Source: §2.1 · pp. 3–5

3 Conclusion

The ablation study finds that rotationally equivariant internal layers improve molecular-property prediction more than adding depth. The conclusion recommends these layers when angular contributions matter in geometric-tensor magnitudes, while noting that the most suitable targets remain difficult to identify generally.

3 Conclusion

  • Rotating internal features improved performance by 23% on average, while adding layers helped by a mean of 4%.The study used QM9 molecular-property regression and compared rotationally equivariant internal layers with rotationally invariant ones.Source: §3 · p. 5
  • A two-constituent dipole example provides physical intuition for the gains; dipole moment benefited most from rotating features relative to increased depth.Source: §3 · p. 5
  • The authors recommend equivariant internal layers for geometric-tensor magnitudes when angular contributions matter, but say suitable targets remain hard to identify generally.They suggest angular contributions also matter for other properties, but to a lesser degree.Source: §3 · p. 5

4 Supplementary Material

The supplementary material describes the normalization, target shift-and-scale operation, and multi-target hyperparameter search used to configure the networks. It also reports the search winner and compares validation learning curves across network variants.

4.1 Normalization Coefficient

  • The normalization coefficient is chosen so unit-second-moment inputs and normally distributed radial components yield unit-second-moment output features.Source: §4.1 · p. 8
  • The convolution is described as a kernel followed by a kernel-feature convolution, producing intermediate features before the gated block.Source: §4.1 · p. 8

4.2 Shift & Scale Function

  • Target prediction combines a reference bias, an atom-wise sum, and a scaled contribution from each atom, with reference values keyed by element and target.Training-set statistics of deviations from the reference values are used to normalize the regression output.Source: §4.2 · pp. 9–10

4.3 Hyperparameter Search Technique

  • Multi-target training cut hyperparameter-search time by a factor of 12 versus 12 separate searches, trading away single-target accuracy; it was not used for reported performance results.The authors note that simultaneously achieving good performance across targets is often more difficult.Source: §4.3 · pp. 10–11
  • Figure 3 depicts possible architectures, output blocks, and the multi-target normalized loss; output blocks receive the same learned featurization but do not share gradients.Each block transforms the shared input features in parallel to predict one target.Source: Figure 3 · p. 10, §4.3 · pp. 10–11
  • The shared featurization combines an embedding and convolution–gated layers; output blocks process order-zero and order-one features before scalar prediction and shift-and-scale.Scalar features then pass through atom-wise layers while preserving rotational invariance of the network output.Source: §4.3 · p. 11
  • The loss normalizes target variances to one using training statistics and assumes each output block predicts mean zero at initialization.Source: §4.3 · p. 11
  • L1Net had the minimum test loss averaged over normalized losses across all targets among the 40 hyperparameter configurations.Each configuration was trained for ten epochs in the multi-target search.Source: §4.3 · pp. 11–12
Loading 2008.08461v4…