PaperlayerView exact revision on arXiv

Source-linked AI summary

A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems

Alexandre Duval, Simon V. Mathis, Chaitanya K. Joshi, Victor Schmidt, Santiago Miret, Fragkiskos D. Malliaros, Taco Cohen, Pietro Liò, Yoshua Bengio, Michael Bronstein

arXiv:2312.07511v2cs.LGcs.AIq-bio.QMstat.ML

TL;DR

Geometric graphs encode atomic systems with attributes that transform under physical symmetries, so treating them as ordinary graph features does not preserve their physical meaning. This survey provides background and a pedagogical taxonomy of Geometric GNNs, alongside applications and research directions. It concludes that the field faces open questions about symmetry choices and their trade-offs.

  • Problem

    Generic treatment of geometric attributes as ordinary node features does not retain their physical meaning and transformation semantics [Bronstein et al., 2021, Bogatskiy et al., 2022].

    Source: §1 · p. 6

  • Method

    The survey synthesizes Geometric GNN foundations, modeling-pipeline components, architecture families, datasets, applications, and research directions.

    Source: §1 · p. 7, §8 · p. 48

  • Results

    The paper presents a four-family taxonomy: invariant, Cartesian-basis equivariant, spherical-basis equivariant, and unconstrained Geometric GNNs.

    Source: §8 · p. 48

  • Takeaways & Limitations

    Choosing how strictly to enforce symmetry remains an open question involving trade-offs between unconstrained local messaging and strictly equivariant methods.

    Source: §8.1 · p. 48

    Unconstrained or approximately symmetry-preserving models may sacrifice precision and physical validity; ForceNet-based simulators were reported unstable over long timesteps and large systems [Fu et al., 2023].

    Source: §6 · p. 41

Abstract

from arXiv · show

Recent advances in computational modelling of atomic systems, spanning molecules, proteins, and materials, represent them as geometric graphs with atoms embedded as nodes in 3D Euclidean space. In these graphs, the geometric attributes transform according to the inherent physical symmetries of 3D atomic systems, including rotations and translations in Euclidean space, as well as node permutations. In recent years, Geometric Graph Neural Networks have emerged as the preferred machine learning architecture powering applications ranging from protein structure prediction to molecular simulations and material generation. Their specificity lies in the inductive biases they leverage - such as physical symmetries and chemical properties - to learn informative representations of these geometric graphs. In this opinionated paper, we provide a comprehensive and self-contained overview of the field of Geometric GNNs for 3D atomic systems. We cover fundamental background material and introduce a pedagogical taxonomy of Geometric GNN architectures: (1) invariant networks, (2) equivariant networks in Cartesian basis, (3) equivariant networks in spherical basis, and (4) unconstrained networks. Additionally, we outline key datasets and application areas and suggest future research directions. The objective of this work is to present a structured perspective on the field, making it accessible to newcomers and aiding practitioners in gaining an intuition for its mathematical abstractions.

Notation

The notation distinguishes scalar, geometric, and tensor quantities, while visual conventions mark collections, learnable quantities, and geometric meaning. Special symbols and indices specify graph structure, transformations, and message-passing quantities.

Notation

  • Bold characters denote collections, underlining marks learnable quantities, and arrows identify geometric meaning.Geometric quantities may also carry component, node, channel, and representation indices.Source: Notation · p. 4
  • Scalars use lowercase Latin letters, geometric vectors carry arrows and lie in 3D, and higher-order geometric tensors use uppercase letters with arrows.Spherical tensor type and Cartesian tensor type use different superscript styles.Source: Notation · p. 4

Special symbols

  • The symbol list assigns meanings to G, σ, f, W, ψ, ⊙, ⊗, and ||, including weights, basis functions, channel-wise products, tensor products, and concatenation.The notation also defines rotations, permutations, translations, angles, graph quantities, and message-passing variables.Source: Special symbols · pp. 4–5
  • The paper uses “tensor” for arrays that follow specified transformation rules, rather than simply for multidimensional arrays of numbers.Lists mean ordered collections, not computer-science data structures.Source: Special symbols · p. 5
  • Figure 1 presents a partial selection of representative architectures, organized by the intermediate representations used within their layers.An exhaustive list is available on GitHub.Source: Figure 1 · p. 6

1 Introduction

Geometric graphs carry spatial attributes whose behavior under physical transformations makes generic treatment as ordinary node features unsuitable. The paper guides readers through the field with background, a four-family architecture taxonomy, applications, and future directions.

1 Introduction

  • Molecular properties such as potential energy remain invariant under rotations and translations, while atomic forces transform equivariantly with the molecule.Geometric graphs represent atoms as nodes with scalar and geometric attributes.Source: §1 · p. 6
  • Treating geometric attributes like ordinary node features fails to retain their physical meaning and transformation semantics [Bronstein et al., 2021, Bogatskiy et al., 2022].The survey frames this as a modeling challenge specific to geometric graphs.Source: §1 · p. 6
  • The survey aims to help newcomers and experienced researchers navigate Geometric GNNs through a comprehensive, pedagogical overview of their modeling pipeline and future directions.Its roadmap includes background, pipeline components, datasets and applications, and future research.Source: §1 · p. 7
  • The taxonomy groups architectures into invariant, Cartesian-tensor equivariant, spherical-tensor equivariant, and unconstrained families.The paper presents the taxonomy as linking existing architectures.Source: §1 · p. 7

2 Preliminaries

The preliminaries establish how ordinary graphs and message passing represent relational structure, then extend that framework to geometric attributes and their transformation behavior. They define invariant, equivariant, and unconstrained functions and preview the common Geometric GNN pipeline.

2.1 Graphs and Graph Neural Networks

  • Graphs represent entities as nodes and relationships as edges, with node features stored alongside adjacency structure.The ordering of nodes is arbitrary, so graph models must account for permutation symmetry.Source: §2.1 · p. 8
  • GNNs update node features by computing neighbor messages, aggregating them with a permutation-invariant operator, and updating the node representation.Stacked message-passing layers propagate information beyond local neighborhoods; readouts produce node-, edge-, or graph-level predictions.Source: §2.1 · pp. 8–9, Figure 3 · p. 8

2.2 Geometric Graphs

  • Geometric graphs augment relational structure with coordinates and optionally vector features, representing systems such as atoms in 3D space.Unlike 2D molecular graphs, geometric graphs include spatial locations and other geometric attributes.Source: §2.2 · p. 9
  • A radial cutoff can connect atoms whose pairwise distance is at most c, producing a local graph from the point cloud.This construction is distinguished from the 2D molecular graph representation, which has no spatial information.Source: §2.2 · p. 9
  • Permutation shuffles node attributes, rotations or reflections transform coordinates and vector features, and translations shift coordinates while leaving vector features unchanged.Scalar features remain invariant under rotations; whether reflections should be respected depends on the application.Source: §2.2 · pp. 9–10
  • Invariant functions preserve outputs under input transformations, equivariant functions transform outputs correspondingly, and unconstrained functions have no predictable output transformation.The common inference pipeline creates a geometric graph, embeds nodes and edges, updates them in interaction blocks, and predicts at node, edge, or graph level.Source: §2.2 · p. 10, Figure 6 · p. 11

3 From GNNs to Geometric GNNs

Geometric GNNs build physical symmetries into models of atomic systems, whose geometric attributes must transform consistently under Euclidean transformations. The section traces the pipeline from graph construction through atom representations to task-specific predictions.

3 From GNNs to Geometric GNNs

  • Geometric GNNs encode physical symmetries as an inductive bias because standard GNNs can treat geometric attributes in ways that violate their transformation semantics.The paper presents symmetry-aware models as aligning predictions with physical principles and enhancing generalization and data efficiency.Source: §3 · p. 11, §3.2.2 · p. 14

3.1 Input preparation

  • Cutoff graphs focus on local interactions and reduce computational overhead, while complete graphs capture all pairwise interactions at greater computational cost.Smooth cutoffs are used in molecular dynamics to avoid abrupt energy changes when atoms cross the cutoff; long-range connections offer another way to represent nonlocal interactions.Source: §3.1 · p. 12
  • Periodic boundary conditions represent crystal repetition through neighboring-cell offsets, while preprocessing may normalize features, center coordinates, rescale targets, or augment samples.The paper notes that whether periodic boundary conditions suffice for crystals remains an active research question.Source: §3.1 · p. 13

3.2 Learning representations of atoms

  • Embedding and Interaction blocks initialize learnable atom representations, then iteratively update them with geometric GNN layers to capture neighborhood information.The resulting representations are built from the geometric graph defined during input preparation.Source: §3.2 · p. 13, §3.2.2 · pp. 14–15

3.2.1 Embedding block: initialising latent representations

  • The embedding block concatenates learnable atom embeddings into initial scalar features; embeddings may include chemical element, group or period, and physical properties.For biomolecules, practitioners may represent residues with alpha Carbon atoms and initialize node representations using geometric quantities such as displacements and torsion angles.Source: §3.2.1 · p. 14

3.2.2 Interaction blocks: learning geometric and relational features

  • Interaction blocks use message passing to update atom features, and the survey groups architectures into invariant, Cartesian-equivariant, spherical-equivariant, and unconstrained families.The taxonomy compares expressive power in terms of body order, tensor order, and depth; enforcing symmetries can constrain expressivity, while unconstrained models do not strictly enforce intermediate-feature equivariance.Source: §3.2.2 · pp. 14–15, Figure 9 · p. 14

3.3 Output block: making predictions

  • Output heads map learned representations to task-specific node, edge, or graph predictions, with invariant or equivariant forms chosen to match the prediction type.Graph-level predictions commonly use permutation-invariant sum or average readouts.Source: §3.3 · p. 15
  • Energy-gradient forces conserve energy and support stable simulations, but increase memory usage by 2-4× and reduce modeling performance on several datasets.The paper leaves whether to enforce energy conservation as an open question that likely depends on the task.Source: §3.3 · p. 15

4 Invariant Geometric GNNs

Invariant GNNs achieve transformation-invariant predictions by processing scalarized geometric features rather than frame-dependent coordinates. Their expressivity can be increased with higher-body scalars or canonical frames, but higher-order methods face computational and completeness limits.

Overview

  • Invariant GNNs use scalar geometric quantities unchanged by Euclidean transformations, yielding invariant intermediate representations and predictions.Translation invariance can also be enforced by centering coordinates or using relative displacements.Source: §4 · p. 16

Distance-based invariant GNNs

  • SchNet encodes pairwise distances with learnable radial basis functions and combines them with neighboring atom representations through continuous convolutions.This provides an efficient way to integrate atom-identity and local geometric information.Source: §4 · p. 16, Figure 10 · p. 17

Many-body scalars

  • DimeNet adds triplet bond angles to pairwise distances, while GemNet extracts four-atom torsion angles using local reference frames.Both approaches increase scalar body order beyond distance-only message passing.Source: §4 · p. 17, Figure 10 · p. 17
  • Higher-body descriptors increase expressivity at computational cost; localized four-body methods can miss non-local distinctions, and the body order needed for unique identification remains open.GemNet considers direct, two-hop, and three-hop neighbors; practical GemNet-OC variants are often restricted to three-body scalars, while SphereNet and ComENet avoid looping over all three-hop neighbors.Source: §4 · pp. 17–18

Universality

  • GemNet’s universality statement applies to a theoretical model under conditions including infinite cutoffs and suitable discretization, not necessarily to its practical final architectures.The paper identifies universality for sparse graphs with finite tensor rank as an open question.Source: §4 · p. 18

Canonical frame-based invariant GNNs

  • Canonical-frame methods scalarize geometry using reference frames; AlphaFold2’s IPA defines local frames at protein residues and aggregates neighbor features through global-frame attention.Aggregated features are rotated back to local frames before invariant features are updated.Source: §4 · p. 18

5 Equivariant Geometric GNNs

This section presents equivariant geometric GNNs for 3D systems, covering Cartesian and spherical-tensor features and operations for maintaining symmetry-aware representations. It explains how these features support equivariant message passing and outputs, and discusses higher tensor orders and approaches to expressivity and scalability.

5 Equivariant Geometric GNNs

  • Equivariant layers build task-relevant invariants during message passing, with additional layers enabling more complex information from potentially non-local neighborhoods.Unlike invariant GNNs, their hidden features transform with the input, allowing them to predict equivariant quantities as well.Source: §5 · p. 19
  • Equivariance requires tracking feature types: add only matching types, use tensor products to preserve transformation behavior, and generally apply component-wise nonlinearities only to scalars.Naive component-wise multiplication of equivariant features generally does not transform like its factors.Source: §5 · p. 20

5.1 Equivariant GNNs with Cartesian tensors

  • Cartesian GNNs restrict operations on geometric features to preserve equivariance, often updating scalar and vector messages in parallel; scalar-vector models are relatively fast but cover only part of the design space.Their familiar operations include scalar products, scalar-scaled vectors, and vector dot products yielding scalars.Source: §5.1 · pp. 20–22
  • Cartesian tensor types extend scalars and vectors to higher ranks, with tensor products constructing higher-rank features and contractions producing lower-rank features while preserving predictable transformations.Cartesian tensors of a given rank transform consistently under rotations and reflections, and linear combinations of the same type retain that type.Source: §5.1 · pp. 23–25
  • Higher-order Cartesian GNNs are uncommon because tensor storage costs grow exponentially, and generality requires asymmetric contractions; TensorNet is a recent exception using tensors up to rank 2.The paper presents Cartesian tensor construction mainly for didactic purposes.Source: §5.1 · p. 26

5.2 From Cartesian tensors to spherical tensors – reducible to irreducible representations

  • Any finite-dimensional SO(3) representation decomposes into irreps of dimension 2l + 1; a rank-2 Cartesian tensor splits into l = 0, 1, 2 components of dimensions 1, 3, and 5.These components correspond respectively to the trace (scalar), the antisymmetric part (vector/cross-product space), and the symmetric traceless part.Source: §5.2 · pp. 27–28, Figure 15 · p. 28

5.3 Equivariant GNNs with spherical tensors – Irreducible representations

  • Spherical EGNNs track tensor types using spherical harmonics to project directions, Clebsch-Gordan coefficients to decompose products, and Wigner D-matrices to rotate features.Examples include TFN, NequIP, SEGNN, MACE, and Equiformer; many use e3nn for these operations.Source: §5.3 · p. 32

5.3.1 An example spherical EGNN

  • In the example spherical EGNN, each node’s features form a list of spherical tensors across types l, organized along channel, tensor-type, and 2l + 1-component axes.The discussion focuses on node features, while noting that the same concepts extend to edge feature tensors.Source: §5.3.1 · pp. 32–33
  • Spherical EGNNs decompose displacement vectors into radial distances and directions, map directions to spherical harmonics, and use Clebsch-Gordan tensor products to construct equivariant filters and messages.Radial functions can map distances to bounded ranges; learnable weights are assigned to valid tensor-product paths, and the resulting filter supports permutation-invariant aggregation.Source: §5.3.1 · pp. 33–35
  • A spherical EGNN layer combines equivariant message passing, channel mixing, and gated nonlinearities, then supports invariant scalar or equivariant dipole and quadrupole readouts.The readout can select scalar channels for invariant predictions or higher-degree channels for equivariant quantities.Source: §5.3.1 · p. 36

5.3.2 Optimisations and improvements.

  • Later approaches add equivariant attention or steerable updates, factor many-body terms for efficiency, or reduce SO(3) convolutions to SO(2) for faster higher-order computation.SE(3)-Transformers use equivariant self-attention, SEGNN uses steerable MLPs, and Equiformer interleaves attention with equivariant updates while remaining local. MACE factorizes higher-order terms, while Allegro uses the same idea without message passing for node-local, parallel computation; eSCN aligns embeddings with edge vectors, and EquiformerV2 uses this trick to scale to hundreds of millions of parameters and improve performance on large-scale catalysis tasks.Source: §5.3.2 · pp. 36–38
  • Higher-order tensors improve data efficiency and resolve orientations that lower-order features cannot distinguish when structure symmetry exceeds the tensor order.NequIP experiments found steeper learning curves and improved data efficiency when evaluated on the same systems used for training; the spherical-harmonic explanation is that higher orders capture finer angular detail. [2022]Source: §5.3.2 · p. 37

6 Unconstrained Geometric GNNs

Unconstrained Geometric GNNs relax architectural symmetry constraints to allow more flexible optimization and expressive models, while raising questions about accuracy, physical validity, and inference cost. The section reviews data augmentation, soft constraints, and frame-based canonization as ways to handle symmetries.

Overview

  • Relaxing equivariance constraints may allow more diverse optimization paths, but whether this benefit outweighs reduced sample complexity or generalization from symmetry inductive bias depends on the application and data scale.Source: §6 · pp. 39, 41

Data augmentation and soft constraints

  • For invariant prediction on 3D images, data augmentation matched invariant-network accuracy at significantly lower inference cost, while results were less successful for equivariant tasks [Gerken et al., 2022].Source: §6 · p. 39
  • SCN reached state-of-the-art performance on OC20 after relaxing strict equivariance for more expressive nonlinear transformations of spherical channels.It projects spherical channels onto a grid and applies pointwise convolutions and a nonlinearity [Zitnick et al., 2022].Source: §6 · pp. 39–40

Globally and locally canonized GNNs

  • FAENet uses PCA-based canonization to map data into a canonical space, letting an otherwise unconstrained model process relative atom positions and preserve symmetries rigorously or empirically.The pipeline maps the input to a global reference frame, applies FAENet, then aggregates predictions.Source: §6 · p. 40, Figure 22 · p. 40
  • Stochastic Frame Averaging samples one frame at training and inference instead of averaging all frame members, reducing inference cost but not guaranteeing invariance or equivariance.Exhaustive averaging requires 8 inferences for E(3) or 4 for SE(3).Source: §6 · pp. 40–41
  • Frame-based unconstrained models can use nonlinear functions of relative positions, distinguish known counterexamples, and iteratively incorporate information from higher-order neighborhoods.The section also describes learned canonization and local coordinate systems as alternatives to hand-designed global frames [Kaba et al., 2023; Mondal et al., 2024; Du et al., 2022; Pozdnyakov and Ceriotti, 2023].Source: §6 · p. 41

Summary

  • Approximate symmetry enforcement may reduce precision or cause physical violations; ForceNet-based simulators were unstable over long timesteps and for large systems, leaving when to impose strict equivariance open.The section also identifies implicitly learning exact symmetries from data as a future direction.Source: §6 · p. 41

7 Applications

Geometric GNNs appear in property prediction, molecular dynamics, generative modeling, and structure prediction, with task-specific data and graph choices. The section also surveys datasets and argues for organized, maintained benchmarks to support evaluation and progress.

7 Applications

  • Geometric GNN applications span property prediction, molecular dynamics simulation, generative modeling, and structure prediction, with learned representations used within task-specific pipelines.Figure 23 summarizes these application categories.Source: §7 · p. 42, Figure 23 · p. 42

7.1.1 Property Prediction

  • Property prediction uses regression or classification for graph-level properties such as energy and band gaps or node-level properties such as atomic charges, learning from simulation- or experiment-labeled datasets.Predictions can accelerate screening of molecule and material libraries; applications include drug and material discovery, environmental assessment, and process optimization.Source: §7.1.1 · pp. 42–43

7.1.2 Interatomic Potentials for Molecular Dynamics Simulation

  • For molecular dynamics, GNN-based potentials predict energy whose gradients yield forces, or a GNN head predicts forces directly; MAE-based training does not guarantee simulation stability, likely due to distribution shift.Training on trajectory dynamics is noted as a promising alternative, while further work is needed to understand capabilities and limitations.Source: §7.1.2 · p. 43

7.1.3 Generative Modeling and Design

  • Generative methods use Geometric GNNs in sequential construction or as full-graph encoders, but MatterGen does not adequately account for periodic crystal symmetries and most models are evaluated computationally.Sequential approaches build graphs stepwise; full-graph approaches can use VAEs, GANs, flows, or diffusion models. The section calls for more meaningful in-silico evaluation metrics.Source: §7.1.3 · p. 44

7.1.4 Structure Prediction

  • Structure prediction refines residue coordinates from sequence representations using a structure module with invariant or equivariant Geometric GNN message passing; related tasks include conformer and crystal structure prediction.Structure prediction estimates positions for a given sequence, whereas generative modeling learns a dataset distribution and samples new systems. Property prediction and dynamics tend toward local graphs, while generation and structure prediction generally need fully connected graphs.Source: §7.1.4 · p. 45

7.2 Datasets

  • The paper provides a selected dataset list and advocates benchmark repositories with transparent splits, metrics, leaderboards, domain expertise, and continual updates to support fair comparisons and assess generalization.The dataset table summarizes application task and properties including sample counts, prediction targets, input domain, split method, metric, release date, and source.Source: §7.2 · pp. 46–47, Table 1 · p. 47

8 Conclusion and Future Research Directions

The survey’s conclusion frames symmetry choices, graph construction, and scaling as open design questions for Geometric GNNs, while its appendix supplies geometric vocabulary for readers. It presents these directions as opportunities for research rather than settled prescriptions.

8 Conclusion and Future Research Directions

  • The survey organizes Geometric GNNs into invariant, Cartesian-equivariant, spherical-equivariant, and unconstrained families, and highlights applications including medicine, energy-efficient materials, and green chemistry.It aims to help readers understand the methods and inspire further modeling work.Source: §8 · p. 48

8.1 To what extent should physics and symmetry be ‘baked in’ to Geometric GNNs?

  • For molecular dynamics, energy-gradient forces improve simulation stability, whereas direct force prediction offers memory and runtime gains and sometimes performance gains.The authors call for better metrics of energy conservation and work on both stability and compute efficiency at scale.Source: §8.1 · p. 48
  • Strict equivariance may suit precise geometric prediction, while approximately symmetric or unconstrained models may suit property prediction or data with canonical order and orientation.The authors identify data efficiency and generalization as possible benefits of strict symmetry enforcement, and expressivity and efficiency as possible benefits of relaxing constraints when data are sufficient.Source: §8.1 · p. 48
  • Open theoretical questions include characterizing expressive power, optimization, and generalization, while higher-order tensor models’ practical utility is constrained by memory use and slow tensor products.The paper calls for universal equivariant GNNs on sparse graphs with finite tensor and body order, alongside fair compute-budgeted benchmarks and domain-specific evaluation.Source: §8.1 · p. 49

8.2 How to construct geometric graphs?

  • Graph construction remains domain-specific, with open questions about neighborhood design, long-range interactions, implicit solvent entities, and whether coordinate noise is physically justified.The authors also suggest investigating information flow beyond physics-based radial cutoffs and interaction thresholds.Source: §8.2 · p. 49
  • Representations of molecules and materials should account for temporal dynamics as well as static geometry, including conformational ensembles across small molecules, crystals, proteins, and RNA.The paper identifies dynamics and conformational flexibility as relevant to protein functionality and material behavior.Source: §8.2 · pp. 49–50

8.3 How to scale up Geometric GNNs?

  • General-purpose atomic-system models raise open questions about cross-domain generalization, architecture scalability, and whether interactions across molecules, proteins, and materials are semantically similar.The paper points to large predicted-protein-structure collections and calls for public, use-case-specific datasets and benchmarks; it also notes promising early work on full biological assemblies.Source: §8.3 · p. 50

A.1 Geometric vocabulary

  • The appendix distinguishes geometric vectors, which transform by standard rotation matrices, from tensors, which follow consistent group-action transformations and may use Cartesian or spherical bases.It also describes tensor order and body order as ways to characterize feature transformation structure and the number of atoms involved in computations.Source: §A.1 · pp. 64–65

A.2 Group theory

The section introduces groups as a formal language for symmetries and identifies the transformation groups relevant to 3D atomic systems. It relates these choices to the transformations Geometric GNNs may need to respect.

A.2 Group theory

  • Group theory provides a formal way to describe transformations and symmetries in atomic systems, including rotations, reflections, and translations.The section defines a group through associativity, an identity element, and inverses.Source: §A.2 · p. 65
  • E(d) includes translations, rotations, and reflections; O(d) includes rotations and reflections, while SO(d) includes rotations without reflections.The section also places O(d) within E(d) and SO(d) within O(d).Source: §A.2 · p. 65
  • SE(d) combines translations and rotations, and T(d) represents translations alone, with T(d) a subgroup of SE(d).Both are presented as subgroups of E(d).Source: §A.2 · p. 65
  • S_n is the group of all permutations of the set of n atoms.Source: §A.2 · p. 66
  • The section’s symmetry discussion links these transformation groups to desired Geometric GNN equivariance: reflections matter differently for isolated molecules and molecules interacting with their surroundings.E(3)-equivariance is described as desirable for isolated molecules, while SE(3)-equivariance is desirable when mirror-image properties may differ through intermolecular interactions.Source: §A.2 · pp. 65–66, Figure 24 · p. 66

A.3 Data structures

The appendix groups molecules, proteins, and materials under the term “atomic systems,” also referring to them as “molecules and materials.”

A.3 Data structures

  • “Atomic systems” is the appendix’s umbrella term for molecules, proteins, and materials.The text also uses “molecules and materials” as an alternative.Source: §A.3 · p. 66

A.3.1 (Small) Molecules

The appendix describes small molecules, proteins, and solid-state materials as distinct data settings for geometric modeling. It also outlines periodic boundary conditions and quantum-chemistry concepts relevant to these systems.

A.3.1 (Small) Molecules

  • Molecules can be represented through 1D descriptions, 2D topology graphs, or 3D conformations, whose consideration is increasingly recognized as important for property prediction.The appendix’s figure contrasts molecular representations including Kekulé structures, molecular graphs, SMILES, 3D graphs, and molecular surfaces.Source: §A.3.1 · p. 67, Figure 26 · p. 67

A.3.2 Proteins

  • Proteins are amino-acid chains whose structures include primary sequence, local secondary folds, overall tertiary arrangement, and sometimes multi-chain quaternary complexes.The text notes that protein 3D structure is essential for function and can be analyzed with machine learning to study interactions and tasks.Source: §A.3.2 · pp. 67–68

A.3.3 Solid-State Materials

  • Crystals have periodic atomic arrangements described using a unit cell and translational axes, and machine-learning methods seek to harness that periodicity for accurate predictions.The appendix contrasts crystalline solids with amorphous materials lacking large-scale positional order.Source: §A.3.3 · p. 68

A.4 Periodic boundary conditions

  • Periodic boundary conditions represent an infinite crystal through a finite simulation cell, with edge cell offsets used to calculate interatomic distances across neighboring cells.The distance expression accounts for the offset vector and cell geometry: d_ij = ||(x_i − x_j) + o_ij · c||.Source: §A.4 · pp. 68–69

A.5 Quantum chemistry background

  • Density Functional Theory approximates electronic structure using electron density and functionals, but remains computationally expensive for intricate systems, motivating machine-learning approximations.The potential energy surface relates molecular potential energy to nuclear coordinates and provides information about stability, reactions, and molecular geometry.Source: §A.5 · p. 69

A.6 Energy conservation

This section reviews core Geometric GNN building blocks, from energy-conserving force prediction to message passing, activation functions, and geometric basis functions.

A.6 Energy conservation

  • Energy-conserving models derive forces from predicted energy and can include force errors in training, but that constraint may hamper performance.On Open Catalyst, non-energy-conserving models outperformed energy-conserving ones; energy conservation alone does not guarantee accurate force approximation.Source: §A.6 · p. 70

A.7.1 Message Passing

  • Message passing computes each node’s message from its and its neighbor’s representations, then updates the node using aggregated messages and its existing representation.The aggregation operator is not learnable and often uses sum or mean.Source: §A.7.1 · p. 70

A.7.2 Activation function

  • Swish is commonly used because its smooth output and gradient suit atomic systems, though the authors encourage searching for better alternatives.The passage associates smoothness with gradual output changes under small input perturbations.Source: §A.7.2 · p. 71

A.7.3 Basis functions

  • Identity, radial basis, and sine functions encode geometric inputs as scalarized coordinates, distance-based Gaussian features, or Fourier-inspired features of unit directions.The radial basis uses evenly spaced Gaussian means on [0, 1], while the sine basis is applied to each dimension of the unit vector.Source: §A.7.3 · pp. 71–72
  • Spherical harmonics encode angular information equivariantly under SO(3), whereas an MLP basis is flexible but does not preserve equivariance.Spherical harmonics are often used as convolutional filters or input features.Source: §A.7.3 · p. 72

A.7.4 Examples of architecture

The examples contrast SchNet’s continuous-filter convolution with GemNet’s more complex architecture, illustrating different ways to process atomic interactions.

GemNet

  • GemNet illustrates that high-performing invariant methods can have complex architectures, with two-hop and one-hop message passing blocks shown in its full architecture.The figure also marks shared-weight layers and embedding sizes.Source: §A.7.4 · p. 72, Figure 27 · p. 73

SchNet

  • SchNet maps atom positions to continuous convolution filters, allowing convolution over atoms at arbitrary positions.Its filter-generating function is learnable and maps positions to discrete filter values.Source: §A.7.4 · p. 72
  • SchNet applies feature-wise convolutions for efficiency and handles interactions between feature maps with separate atom-wise layers.The layer uses element-wise multiplication, a two-layer MLP with SoftPlus, and a radial basis function.Source: §A.7.4 · p. 72

A.8 Expressive power of Geometric GNN

The section describes geometric expressivity through symmetry-aware graph comparison and discusses inductive biases as a way to shape learning.

A.8 Expressive power of Geometric GNN

  • Geometric graph isomorphism requires both topological isomorphism and geometric attributes equivalent up to global symmetries such as rotations and reflections.The Geometric Weisfeiler-Leman framework characterizes Geometric GNN expressivity by whether distinct geometric graphs receive distinct representations up to group actions.Source: §A.8 · p. 74
  • Counterexamples show that distinct configurations can share the same k-body scalar features, so models using only 2-, 3-, or 4-body features may fail to distinguish them.The figure progresses from pairwise distances through neighbor distances, angles, and tetrahedra.Source: §A.8 · p. 74, Figure 29 · p. 74
  • Under GWL, maximal expressivity requires injective aggregation, update, and readout, while universal equivariant GNNs on sparse graphs with finite tensor and body order remain an open question.Some architectures claim universality or completeness only under specific conditions.Source: §A.8 · p. 74

A.9 On inductive biases

  • Inductive biases restrict or favor regions of the accessible function space and are often designed to improve data efficiency or generalization.Examples include parameter regularization, transformation invariance or equivariance, and choices built into network architecture.Source: §A.9 · p. 75

B An opinionated history of methods

The history traces a shift from handcrafted molecular representations to geometric GNNs, followed by invariant, equivariant, and unconstrained architectural approaches.

B An opinionated history of methods

  • Early molecular energy and force models used handcrafted physical representations before research moved toward end-to-end GNN models.The paper places this shift alongside expansion into solid-state and molecular materials tasks.Source: B An opinionated history of methods · p. 75
  • Invariant models progressed from pairwise distances in SchNet to bond angles in DimeNet and torsion angles in SphereNet and GemNet.Adding geometric scalars helps distinguish more atomic systems, but GemNet’s torsion information requires 3-hop neighborhoods at training and inference.Source: B An opinionated history of methods · p. 75
  • Spherical equivariant GNNs use spherical tensor embeddings and constrained operations to ensure SO(3) equivariance, with Clebsch-Gordan products contributing to high computational cost.The paper also characterizes these methods as expressive and generalizing well, but difficult to implement and restrictive of the functional space.Source: B An opinionated history of methods · pp. 75–76
  • Cartesian equivariant GNNs update scalar and vector representations, avoiding costly spherical operations but lacking compact hierarchical representations and cross-direction information.The paper links those missing properties to efficiency, memory use, and sensitivity to angular variations and dependencies.Source: B An opinionated history of methods · p. 76
  • Unconstrained GNNs address architectural restrictions by learning symmetries through augmentation, relaxing them, or mapping inputs to a canonical Euclidean space.The paper presents these approaches in response to concerns that restricting model operations may hamper learning capacity in practice.Source: B An opinionated history of methods · p. 76

C.1 Data splits

The appendix describes dataset-splitting choices and their trade-offs, gives examples of predicted molecular and material properties, and explains atom-type rescaling as a preprocessing approach.

C.1 Data splits

  • Random splits provide a same-distribution baseline, while stratified splits preserve class or property representation but require labels and are not optimal for measuring generalization.Random splits are suggested for well-balanced, representative datasets; small or imbalanced datasets can remain challenging for stratification.Source: §C.1 · pp. 76–77
  • Extrapolation splits test generalization to unseen chemical environments, but curated out-of-distribution examples may introduce biases or artifacts.They are suggested when out-of-distribution generalization matters, including materials or drug discovery applications.Source: §C.1 · p. 77
  • Time splits train on earlier data and test on later data, reflecting temporal variation but assuming both periods represent the same underlying distribution.The appendix recommends them when meaningful temporal data are present.Source: §C.1 · p. 77
  • Group splits evaluate specified subsets, such as targets, protein families, or chemical series, but require group information and may introduce representativeness biases.They may also fail to preserve the underlying data distribution or capture split-specific challenges.Source: §C.1 · pp. 76–77

C.2 Examples of predicted properties

  • Predicted-property examples span material characteristics such as band gap, dielectric constant, and exfoliation energy, as well as molecular quantities including dipole moment and HOMO.The examples also include forces and relaxed energy, alongside properties such as toxicity, solubility, and heat capacity.Source: §C.2 · pp. 77–78

C.3 Atom-type rescaling

  • Rescaling can factor out atomic contributions from targets or adjust input features with differing scales, and the preprocessing and shift values should be reported.The appendix describes target shifting as a way to focus on interaction-energy variation and notes its use on QM7-X [Hoja et al., 2021].Source: §C.3 · pp. 78–79
  • Atom-type rescaling estimates per-element shifts from training-set atom counts and target values, then subtracts their summed contribution from each ground-truth energy.The procedure constructs a count matrix A and solves Ax = q for shifts x before calculating shifted energies y′.Source: §C.3 · pp. 78–79

Related papers

Loading 2312.07511v2…