PaperlayerView exact revision on arXiv

Source-linked AI summary

Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems

Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, Tianfan Fu, Yucheng Wang, Alex Strasser, Haiyang Yu, YuQing Xie, Xiang Fu, Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hannah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards, Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang, Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang, Tuong Phung, Minkai Xu, Chaitanya K. Joshi, Simon V. Mathis, Kamyar Azizzadenesheli, Ada Fang, Alán Aspuru-Guzik, Erik Bekkers, Michael Bronstein, Marinka Zitnik, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovec, Heng Ji, Jimeng Sun, Regina Barzilay, Tommi Jaakkola, Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess Smidt, Shuiwang Ji

arXiv:2307.08423v6cs.LGphysics.comp-ph

TL;DR

AI for science is an enormous, interdisciplinary field for which a unified technical treatment is needed but challenging. This paper focuses on quantum, atomistic, and continuum systems and frames AI for science as a new paradigm for scientific discovery.

  • Problem

    AI for science is an enormous, emerging interdisciplinary field, making a unified technical account challenging.

    Source: §1.5 · p. 11

  • Method

    The paper provides a technically thorough account focused on AI for quantum, atomistic, and continuum systems.

    Source: §1.5 · p. 11

  • Results

    The authors frame AI for science as opening a new paradigm of scientific discovery and interdisciplinary innovation.

    Source: §12 · p. 229

  • Takeaways & Limitations

    AI's mission in science is to solve complex scientific problems accurately and efficiently.

    Source: §12 · p. 229

    The work is not comprehensive or conclusive, covers selected areas, and may omit relevant literature as the field evolves.

    Source: §1.5 · p. 11

Abstract

from arXiv · show

Advances in artificial intelligence (AI) are fueling a new paradigm of discoveries in natural sciences. Today, AI has started to advance natural sciences by improving, accelerating, and enabling our understanding of natural phenomena at a wide range of spatial and temporal scales, giving rise to a new area of research known as AI for science (AI4Science). Being an emerging research paradigm, AI4Science is unique in that it is an enormous and highly interdisciplinary area. Thus, a unified and technical treatment of this field is needed yet challenging. This work aims to provide a technically thorough account of a subarea of AI4Science; namely, AI for quantum, atomistic, and continuum systems. These areas aim at understanding the physical world from the subatomic (wavefunctions and electron density), atomic (molecules, proteins, materials, and interactions), to macro (fluids, climate, and subsurface) scales and form an important subarea of AI4Science. A unique advantage of focusing on these areas is that they largely share a common set of challenges, thereby allowing a unified and foundational treatment. A key common challenge is how to capture physics first principles, especially symmetries, in natural systems by deep learning methods. We provide an in-depth yet intuitive account of techniques to achieve equivariance to symmetry transformations. We also discuss other common technical challenges, including explainability, out-of-distribution generalization, knowledge transfer with foundation and large language models, and uncertainty quantification. To facilitate learning and education, we provide categorized lists of resources that we found to be useful. We strive to be thorough and unified and hope this initial effort may trigger more community interests and efforts to further advance AI4Science.

1 INTRODUCTION

The paper surveys AI for quantum, atomistic, and continuum science, organizing selected research areas by physical scale and examining shared technical challenges. It also describes integrative, educational, and community resources while acknowledging that its coverage is selective and evolving.

1 INTRODUCTION

  • Deep learning can accelerate solutions to known scientific equations using simulator-generated training data, while experimentally generated data enables modeling in areas such as biology.Once trained, models can predict faster than simulators; biological examples include protein structure prediction from experimentally acquired structures.Source: §1 · p. 5

1.1 Scientific Areas

  • The survey reviews neural wavefunctions for quantum mechanics and deep-learning predictions of quantum tensors and density-functional learning for molecules and materials.It frames quantum mechanics as solving the Schrödinger equation and DFT as a practical but computationally expensive first-principles approach.Source: §1.1 · p. 7
  • For small molecules and proteins, the survey covers molecular representation, generation, simulation, and dynamics, alongside learning from protein structures and designing novel proteins.Small molecules typically have tens to hundreds of atoms; protein structure prediction has recently advanced through machine learning.Source: §1.1 · p. 7
  • The survey covers crystal-material property prediction and structure generation, molecular interactions, and deep-learning surrogate models for continuum PDEs.Across its seven domains, it sets up problems, surveys approaches and benchmarks, and identifies challenges and future directions.Source: §1.1 · pp. 7–8

1.2 Technical Areas of AI

  • Shared technical challenges include encoding physical symmetries, interpreting models, generalizing across distribution shifts, and using foundation models when labeled data are scarce.The paper treats symmetry, expressed through invariance or equivariance, as a common thread across the surveyed areas.Source: §1.2 · p. 8
  • The paper reviews uncertainty quantification for robust decisions under data and model uncertainty and compiles categorized resources to support AI-for-science learning and education.It also offers perspectives on how the community can better integrate AI with science and education.Source: §1.2 · pp. 8, 10

1.3 Integrative Multi-Scale Analysis

  • The survey connects quantum, DFT, molecular-dynamics, and continuum analyses across spatial and temporal scales, noting that theoretical level depends on the phenomenon and computational complexity.It presents different analyses as potentially complementary and integrative.Source: §1.3 · pp. 10–11

1.4 Online Resources; 1.5 Scope and Feedback

  • Online resources include a regularly updated portal and mindmap, plus the AIRS software library, benchmarks, and curated literature; the survey remains selective and welcomes community feedback.The authors expect to add methods and benchmarks as the field develops and acknowledge that relevant work may be missing.Source: §1.4 · p. 11, §1.5 · p. 11

2 SYMMETRIES, EQUIVARIANCE, AND THEORY … 2.4 Equivariant Data Interactions

The section explains why scientific learning must respect symmetries and reviews equivariant methods spanning discrete rotations, continuous 3D geometry, and feature interactions. It progresses from motivating examples through mathematical foundations to formal network theory and open directions.

2 SYMMETRIES, EQUIVARIANCE, AND THEORY

  • The section builds from discrete and continuous symmetry examples to intuitive foundations, formal theory, a general equivariant-network formulation, and open research directions.Some formal-theory subsections can be skipped depending on reader background and interests.Source: §2 · p. 14, Figure 4 · p. 15

2.1 Overview

  • Arbitrary reference frames can confound physical representations, making symmetry-aware learning essential; data augmentation has capacity, coverage, depth, and robustness drawbacks, motivating symmetry-adapted models.The reviewed symmetry-adapted models require no data augmentation and have been applied to molecules, proteins, and crystalline materials.Source: §2.1 · pp. 14–15

2.2 Equivariance to Discrete Symmetry Transformations

  • For 90°, 180°, and 270° rotations, G-CNNs use lifting and group convolutions to maintain rotation-indexed features, then pool over rotations to produce equivariant outputs.The lifting convolution creates feature maps for rotated kernels; intermediate group convolutions preserve their rotation structure, and final pooling handles feature-map permutations.Source: Figure 5 · p. 17, §2.2 · pp. 17–18

2.3 Equivariant Featurization of 3D Geometries

  • For continuous SE(3) molecular geometry, targets transform by order-ℓ Wigner-D matrices; spherical harmonics encode relative coordinates as order-ℓ equivariant features.Order ℓ=0 corresponds to invariant properties such as energy, while ℓ=1 includes quantities such as force fields that rotate with the structure.Source: §2.3 · pp. 18–19

2.4 Equivariant Data Interactions

  • Spherical-harmonic-based operations encode 3D geometry so it can interact with higher-dimensional features while preserving equivariance.The section reviews equivariant data interactions and operations used in machine-learning models on 3D structures.Source: §2.4 · p. 19

2.4.1 Equivariant Data Interactions via Tensor Product.

  • Tensor-product message passing combines spherical-harmonic encodings of relative positions with node features; Clebsch-Gordan coupling ensures equivariance, and such operations are necessary for SE(3)-equivariance.Equivariance is preserved when node features are updated with linear or tensor-product operations, enabling stacked message-passing layers.Source: §2.4.1 · pp. 19–20

2.4.2 Approximately Equivariant Data Interactions via Spherical Channel Networks.

  • SCNs update spherical-function features through sampled surface values and pointwise convolution, while eSCN reduces tensor-product cost by edge alignment and sparse Clebsch-Gordan operations; both are only approximately equivariant.SCN and eSCN are strictly equivariant only when input features rotate by angles exactly sampled on the spherical grid, which may not be feasible for continuous rotations.Source: §2.4.2 · p. 21
  • Scalarization reduces interaction costs by computing high-order feature messages as scalars before returning to high-order representations through attention or gating; work also studies tensor-product expressiveness and explainability.The passage describes scalarization methods as promising a balance between efficiency and expressiveness in tensor-product networks.Source: §2.4.2 · p. 21

2.5 Intuitive Physics and Mathematical Foundations

This section builds intuition for symmetry-equivariant learning through irreducible representations, tensor-product decompositions, and spherical harmonics, connecting the mathematical framework to geometric encoding and angular momentum.

2.5.1 Overview.

  • Changing basis decomposes the quadratic-polynomial space into invariant L0 and L2 subspaces, showing that equivariantly learned spaces are composed of irreps and tensor products produce and decompose them.L0 is the invariant scalar x^2 + y^2 + z^2, while L2 is spanned by the remaining five basis elements; L2 corresponds to the ℓ = 2 real spherical harmonics.Source: §2.5.1 · p. 22

2.5.2 Illustration of Irreducible Representations via A Discrete Example.

  • A quadratic vector-valued outcome on a four-node square has only two independent learnable weights, illustrating how equivariance constrains mappings and can reduce parameters.The passage contrasts this with the 10 independent weights in a general quadratic polynomial; the allowed vector-valued form comes from tensor products of linear irreps.Source: §2.5.2 · p. 23

2.5.3 Tensor Products and Clebsh-Gordan Coefficients.

  • For 3D rotations, the 9-dimensional coefficient space for bilinear maps decomposes into stable 1D, 3D, and 5D subspaces, with Clebsch-Gordan coefficients providing the change of basis.The subspaces correspond to orders ℓ = 0, 1, and 2; their decomposition depends on the transformation group.Source: §2.5.3 · pp. 25–26

2.5.4 Spherical Harmonics Projections and Equivariant Networks. 𝑚:

  • Spherical harmonics provide a complete orthogonal basis for functions on the sphere; projecting a direction’s Dirac delta uses radial and directional components, with finite ℓmax yielding an approximation.As ℓmax increases, the reconstructed 3D blob becomes thinner and more closely approximates the Dirac delta.Source: §2.5.4 · pp. 26–27, Figure 7 · p. 27

2.5.5 Spherical Harmonics Functions and Angular Momentum.

  • In quantum mechanics, spherical harmonics describe hydrogenic electron wavefunctions, while tensor products couple the angular momenta of separate electrons under Coulomb forces.The real spherical harmonics are indexed by orbital angular momentum ℓ and magnetic quantum number m and form a complete orthogonal basis for spherical functions.Source: §2.5.5 · pp. 27–28

2.6 Group and Representation Theory

The section introduces group and representation theory as a foundation for describing how symmetries act on data and features in equivariant neural networks. It connects these actions to invariant and equivariant functions and to representations of feature spaces.

2.6.1 Symmetry Groups.

  • Groups formalize transformations under composition via inverse, identity, and associativity; planar rotations are abelian, while SO(n) rotations exclude the reflections included in O(n).Source: §2.6.1 · pp. 28–29

2.6.2 Group Actions and Equivariant Maps.

2.6.3 Group Representations.

  • Group representations describe actions on vector spaces as matrices, with a homomorphism ensuring group composition matches matrix multiplication.Source: §2.6.3 · p. 31

2.6.4 Irreducible Representations.

  • Reducible representations decompose into independent irreps; these serve as natural data types in many equivariant networks, while Schur’s lemma simplifies equivariance constraints.Source: §2.6.4 · pp. 31–32
  • Equivalent representations need basis-independent labels: finite groups use characters, while semisimple Lie groups use highest weights, including angular momentum ℓ for SO(3).Source: §2.6.4 · p. 32

2.7 𝑆𝑂(3) Group and Spherical Harmonics · 2.8 A General Formulation of Equivariant Networks via Steerable Kernels

The section presents steerable CNNs as a representation-theoretic framework for equivariant networks: feature fields carry group representations, and equivariant linear maps are convolutions with symmetry-constrained kernels. This framework connects earlier equivariant operations, supports mixed field types, and extends to other spaces and local gauge transformations.

2.8 A General Formulation of Equivariant Networks via Steerable Kernels

  • Steerable CNNs unify equivariant convolutions as maps between feature fields with representation-defined transformation laws, using symmetry-constrained steerable kernels.The framework applies across choices of spaces, symmetry groups, and representations; Fig. 8 illustrates scalar and vector fields and equivariant maps between them.Source: §2.8 · pp. 34–35, Figure 8 · p. 35

2.8.1 Feature Vector Fields.

  • Feature vector fields assign vectors to points in Euclidean space and can be discretized on pixel grids or point clouds.Affine groups combine translations with matrix-group transformations, while a representation specifies how transformations act on each feature vector; the trivial and defining representations recover scalar and tangent vector fields.Source: §2.8.1 · pp. 35–36
  • Regular representations explain group convolutions, while irreducible SO(3) representations describe the 2ℓ+1-dimensional features used in tensor field networks.The four-channel regular representation of C4 shifts channels cyclically; group convolutions generally correspond to regular representations.Source: §2.8.1 · p. 36

2.8.2 Steerable Convolutions.

  • The most general linear equivariant maps between specified input and output field types are convolutions whose kernels satisfy the corresponding G-steerability constraint.Convolution provides translational equivariance, while steerability enforces equivariance under G and kernels are matrix-valued when mapping between vector-valued features.Source: §2.8.2 · pp. 36–37
  • Steerable kernels lie in a linear subspace, so they can be parameterized by a basis with learnable coefficients; bases and implementations cover broad group classes.Bases were derived for SO(3) irreps, representations of groups G≤O(2), and representations of arbitrary compact groups G≤O(d); escnn is available for PyTorch and JAX.Source: §2.8.2 · p. 37
  • The formulation derives equivariant maps from fixed group actions, establishes completeness of admissible kernel solutions, and enables hybrid models combining different field types.Tensor field network tensor-product operations correspond one-to-one with steerable-kernel solutions; other approaches may use only a subset of admissible kernels [Weiler et al. 2021].Source: §2.8.2 · p. 38
  • Steerable CNNs extend to homogeneous spaces and Riemannian manifolds, where coordinate-independent convolutions use G-steerable kernels and are equivariant to local gauge transformations.The account also connects steerable kernels to quantum-mechanical representation operators via the Wigner–Eckart theorem, and covers steerable partial differential operators [Jenner and Weiler 2022].Source: §2.8.2 · p. 38

2.9 Open Research Directions

The section surveys open directions in symmetry-aware learning, from symmetry breaking and equivariant-model costs and expressivity to frame averaging and approximate equivariance. It highlights limitations and trade-offs that shape the choice of method.

2.9.1 Symmetry Breaking.

  • Equivariant models cannot select a single lower-symmetry output from a symmetric input; loss gradients can reveal missing input, while representing all valid outcomes remains open.For a particular desired outcome, gradients of the loss can indicate what additional input is needed to break symmetry.Source: §2.9.1 · p. 39

2.9.2 Empirical Benefits and Expense of Equivariance versus Invariance.

  • Increasing spherical-harmonic order has improved reported SE(3)- and E(3)-equivariant model performance, but tensor-product contractions dominate equivariance’s computational expense.For voxel models, precomputing the contraction can reduce cost; other potential workarounds include eSCN-like operations, optimized kernels, domain-specific compilers, and tailored hardware [Passaro and Zitnick 2023].Source: §2.9.2 · p. 39

2.9.3 Universality of Equivariant Neural Architectures.

  • Many equivariant architectures are universal, but graph neural networks are generally not, and universality alone does not predict relative architecture performance.Universality results include polynomial invariant/equivariant networks, tensor-product architectures [Bogatskiy et al. 2022], and SE(3)-transformers and tensor field networks [Dym and Maron 2021]; graph-network expressivity analysis remains active [Xu et al. 2018; Joshi et al. 2023].Source: §2.9.3 · pp. 39–40

2.9.4 Frame Averaging as an Alternative for Equivariance.

  • Frame averaging can wrap generic architectures in equivariant pipelines, offering a potentially efficient alternative to tensor-based architectures, but frame selection and its trade-offs remain open.Moving-frame constructions require backpropagation through a smooth frame; proposed approaches include smooth weighted averaging over neighbor-defined frames [Pozdnyakov and Ceriotti 2023], random frame sampling for efficiency [Duval et al. 2023], learned lightweight frames [Kaba et al. 2022], and minimal frame averaging [Lin et al. 2024]. Frame methods can also break permutation symmetry or be discontinuous [Cen et al. 2024; Dym et al. 2024].Source: §2.9.4 · pp. 40–41

2.9.5 Approximate Equivariance.

  • Relaxed equivariance can outperform exact and unconstrained models on approximate-symmetry tasks, but its long-term advantage on genuinely symmetric tasks remains open.Residual Pathway Priors favor equivariance while allowing unconstrained weights [Finzi et al. 2021]; relaxed group convolution also showed superior performance on synthetic smoke-plume and experimental jet-flow datasets. Spherical channel networks on Open Catalyst were later surpassed by fully equivariant networks [Passaro and Zitnick 2023].Source: §2.9.5 · p. 42

3 AI FOR QUANTUM MECHANICS

This section surveys neural-wavefunction methods for quantum modeling, including variational optimization and Monte Carlo approaches to quantum spin and many-electron ground states, as well as molecular potential energy surfaces across geometries. It covers modeling, optimization, generalization, and evaluation, with attention to symmetry, fermion antisymmetry, size consistency, sign structure, lattice geometry, and computational efficiency.

3.1 Overview

  • Variational optimization makes ground-state approximation feasible when direct eigendecomposition is impractical, while variational Monte Carlo estimates energy and gradients from wavefunction-distributed samples.The methods also extend to excited states and time-dependent quantum dynamics; the latter updates neural-wavefunction parameters using the time-dependent variational principle.Source: §3.1 · pp. 43, 45, 46, Figure 10 · p. 45

3.2.2 Technical Challenges.

  • Spin-system wavefunctions span 2^N configurations, and neural networks use variational Monte Carlo to learn them despite challenges from symmetries, difficult sign structures, and diverse lattice geometries.These challenges matter because lattice symmetries constrain ground states, frustrated regimes complicate sign learning, and geometry affects magnetic properties.Source: §3.2.1 · p. 46, §3.2.2 · p. 47

3.2.3 Existing Methods.

  • Neural quantum states use restricted Boltzmann machines, feed-forward networks, CNNs, graph networks, and autoregressive or recurrent networks to represent spin-system states.Approaches preserve symmetries through averaging or group-equivariant convolutions, model sign through complex-valued or separate amplitude-and-phase networks, and address varied geometries with graph networks or lattice convolution.Source: §3.2.3 · pp. 47–49, Table 2 · p. 48

3.2.4 Optimization Methods.

  • Ground-state optimization uses direct energy-gradient descent or stochastic reconfiguration, whose cost can be reduced with iterative solvers or smaller sampled-configuration matrices.Imaginary-time supervised wavefunction optimization is another approach, maximizing overlap with an imaginary-time-evolved target state through repeated gradient updates.Source: §3.2.4 · pp. 49–50

3.2.5 Datasets and Benchmarks.

  • Spin-system models are trained concurrently for a specified lattice and Hamiltonian using dynamically sampled configurations, with system energy serving as the ground-state accuracy measure.Common test systems include square, honeycomb, triangular, and kagome lattices, and the J1-J2 quantum Heisenberg model is frequently used.Source: §3.2.5 · p. 50

3.2.6 Open Research Directions.

  • Open directions include expressive wavefunctions for frustrated, strongly correlated systems, comprehensive benchmarks, and more efficient sampling; autoregressive models can bypass MCMC and support exact sampling [Sharir et al. 2020].MCMC samples can remain correlated and make energy estimates inaccurate, while adding annealing steps to reduce autocorrelation increases computational cost for larger systems.Source: §3.2.6 · p. 50

3.3 Learning Ground States for Many-Electron Systems

  • Many-electron wavefunctions operate on continuous electron coordinates and must satisfy fermionic antisymmetry, while their search space still grows exponentially with electron count.Molecular methods use either discrete antisymmetric basis functions (second quantization) or continuous-space wavefunctions (first quantization); this section focuses on the latter.Source: §3.3 · p. 51

3.3.1 Problem Setup.

  • Ground-state VMC minimizes Monte Carlo-estimated energy over wavefunctions antisymmetric to same-spin electron exchanges; the variational principle bounds the energy below by the ground-state value.For multiple geometries, a generalized wavefunction conditions on molecular structure to model the PES, avoiding a separate optimization for each nuclear configuration.Source: §3.3.1 · pp. 52–53

3.3.2 Technical Challenges.

  • Key challenges are enforcing fermion antisymmetry, modeling interacting-electron orbitals, reaching chemical accuracy, and generalizing across molecular geometries while preserving energy symmetries and size-consistency.Chemical accuracy is 1 kcal/mol (1.594 mEh or 0.043 eV); estimating energies from wavefunctions also requires costly numerical integration.Source: §3.3.2 · pp. 53–54

3.3 Learning Ground States for Many-Electron Systems

  • Deep neural VMC can outperform DFT or CCSD(T) in strongly correlated settings, but DFT is much faster and CCSD(T) often has lower relative errors.CCSD(T) scales less favorably in theory, and can be unreliable for challenging multireference systems where deep VMC often succeeds.Source: §3.3.3 · p. 54

3.3.3 Existing Methods.

  • Slater determinants enforce same-spin antisymmetry, while neural orbital models such as FermiNet and PauliNet encode electron context and can incorporate physics-informed features.Designs include pairwise interactions, distance-based convolutions, attention, decaying envelopes, cusp treatments, and classical-orbital information.Source: §3.3.3 · pp. 55, 57, 58
  • VMC optimization uses gradients that account for the changing sampling distribution, Metropolis-Hastings sampling, and natural-gradient updates, with KFAC or conjugate gradients to make updates tractable.The proposal scale is adjusted for an acceptance ratio around 0.5.Source: §3.3.3 · pp. 58–59
  • DMC refines VMC wavefunctions through imaginary-time-weighted Langevin sampling, while DiffVMC directly learns the quantum score using a score-matching loss [Ren et al. 2023; Wilson et al. 2021; Zhang et al. 2023c].DMC uses the quantum drift ∇_r log ψ; DiffVMC combines VMC and DMC by parameterizing and updating the score instead of sample weights.Source: §3.3.3 · p. 59
  • For multiple geometries, DeepErwin shares most parameters across structures, whereas PESNet adapts orbitals with a nuclei-conditioned network optimized once for a molecule’s PES [Scherbela et al. 2022; Gao and Günnemann 2021].The weight-sharing approach can transfer across different atom sets, but PESNet cannot.Source: §3.3.3 · pp. 59–60
  • For different compounds, TAO uses Hartree–Fock-derived coefficients to combine atomic orbitals, while Globe learns localized orbitals by spatial message passing [Scherbela et al. 2023; Gao and Günnemann 2023a].TAO performs better at transfer to new molecules, while Globe shows strong ability to learn ground states of various compounds simultaneously.Source: §3.3.3 · p. 60
  • Generalized wavefunctions need equivariance for invariant energies; methods use equivariant coordinate frames or rotation augmentation, and nuclear permutation symmetry is commonly built in through orbital summations [Gao and Günnemann 2021; Gao et al. 2022; Gao and Günnemann 2023a; Scherbela et al. 2023].Restricting wavefunctions to be invariant can limit the function class and typically yield higher energies. Size consistency improves generalization by requiring orbitals with decaying receptive fields, as implemented by Moon [Gao and Günnemann 2023a].Source: §3.3.3 · pp. 60–61
  • PlaNet models a PES by fitting functions to observed energies at each time step and temporally averaging them through a first-order Taylor approximation [Gao et al. 2022].This adapts the running-average energy estimate used in single-structure calculations to geometries sampled from a continuous distribution.Source: §3.3.3 · p. 61

3.3.4 Datasets and Benchmarks.

  • Evaluations estimate accuracy from sampled mean energy and standard deviation, with lower energy indicating greater accuracy; benchmarks include atoms, molecules, compounds, and transition energies.Examples include N and Fe, N2 and CCl4, H10, the benzene dimer, cyclobutadiene automerization, and Fe ionization [Ren et al. 2023; Spencer et al. 2020].Source: §3.3.4 · p. 61

3.3.5 Open Research Directions.

  • Open challenges include optimizing fermion antisymmetry without Slater determinants, implicit wavefunction modeling, and scaling beyond the current limit of 80 electrons as complexity grows as O(N^4).Potential routes to improved scale include more efficient sampling and optimization, greater weight sharing, and faster energy evaluation implementations such as Jax.Source: §3.3.5 · p. 61

4 AI FOR DENSITY FUNCTIONAL THEORY

This section examines how machine learning can support density functional theory, including equivariant quantum-tensor prediction and learning exchange-correlation functionals. It surveys physical constraints, benchmark datasets, and directions for improving accuracy, generalization, optimization, and applications beyond atomistic systems.

4.1.1 Density Functional Theory.

  • Exact many-body wavefunctions become impractical as electron counts grow, while DFT uses ground-state density and a Kohn-Sham system of noninteracting electrons to obtain energies and properties.The Hohenberg-Kohn results make ground-state density determine the external potential and Hamiltonian, and make the ground-state energy a functional minimized by the ground-state density.Source: Recommended Prerequisites: Section 3.1 · p. 62, §4.1.1 · p. 62, Figure 13 · p. 64

4.1.2 The Kohn-Sham Equation.

  • Approximate exchange-correlation functionals cause delocalization, self-interaction, and static and dynamical correlation errors, motivating more accurate machine-learned functionals.Hybrid functionals partly reduce self-interaction error by including exact Hartree-Fock exchange, but cost more than typical GGA and LDA functionals.Source: §4.1.2 · pp. 65–66

4.1.3 Density Functional Theory In Practice.

  • Localized atomic-orbital bases approximate electronic wavefunctions, and self-consistent-field iterations solve for the Hamiltonian; quantum tensor networks can predict it directly to accelerate calculations.The direct-prediction pipeline avoids iterative optimization, while other approaches use optimization strategies such as stochastic gradient descent to replace the SCF loop.Source: §4.1.3 · pp. 66–67, Figure 14 · p. 67
  • LDA, GGA, hybrid, and meta-GGA functionals trade increasing accuracy for higher cost, but the exact exchange-correlation functional remains unknown and is a target for machine learning.Approaches include neural networks, MLPs, DNNs, symbolic regression, kernel ridge regression, voxel DNNs, and Δ-Learning, with or without physical constraints.Source: Figure 12 · p. 63, §4.1.3 · pp. 67–68

4.2.1 Problem Setup.

  • Quantum tensor learning predicts Hamiltonian matrices and other tensors from molecular atom types and coordinates, enabling physical properties such as total energy, charge density, and polarization to be derived.The Hamiltonian matrix is a key target for accelerating DFT calculations.Source: §4.2 · p. 68, §4.2.1 · p. 68

4.2.2 Technical Challenges.

  • Quantum tensor models must preserve permutation, translation, and rotation equivariance, handle system-dependent tensor sizes, and manage the computational cost of equivariant operations.Upon rotation, each Hamiltonian block transforms as Dℓ1(R) B Dℓ2(RT).Source: §4.2.2 · p. 68, Figure 15 · p. 69

4.2.3 Existing Methods.

  • Existing quantum tensor networks combine node-wise interaction, pairwise feature construction, and matrix assembly; SchNorb and DeepH use invariant features, while PhiSNet and QHNet construct equivariant features.PhiSNet and QHNet use spherical harmonics and tensor products; QHNet reduces tensor-product operations and is more efficient than PhiSNet.Source: §4.2.3 · pp. 69–70, Figure 16 · p. 70
  • Pairwise-feature designs differ: DeepH uses rotation-invariant local-coordinate features, SchNorb scales edge directions to form equivariant features, and PhiSNet and QHNet use tensor products for equivariant features.QHNet uses tensor products for diagonal and non-diagonal pairs, whereas PhiSNet uses them only for non-diagonal pairs.Source: §4.2.3 · p. 71
  • Tensor expansion and related basis transformations enable equivariant quantum-matrix prediction, while SchNorb instead uses rotation augmentation without guaranteeing equivariance.PhiSNet assigns irrep channels to atom-orbital quadruples, and spin-orbital or magnetic time-reversal equivariance can be handled by transforming back to an integral basis.Source: §4.2.3 · pp. 71–72

4.2.4 Datasets and Benchmarks.

  • 83.12 × 10−6 Eh MAE was achieved by QHNet trained on QH9 when predicting Hamiltonian matrices on mixed MD17.QH9 provides precise Hamiltonians and includes stable geometries split for IID or atom-count-based OOD evaluation, plus dynamic trajectories split by molecule or geometry.Source: §4.2.4 · p. 72

4.2.5 Open Research Directions.

  • Accurate quantum tensors can supply physics-based features for molecular-property prediction and may accelerate materials discovery, especially in low-energy regimes.OrbNet uses approximate tensors from GNF-xTB, while quantum-tensor networks could predict accurate tensors for downstream models such as force fields.Source: §4.2.5 · p. 73

4.3 Density Functional Learning.

  • Machine learning targets exchange-correlation, kinetic-energy, universal, and correction functionals through numerical or symbolic methods, from scratch or with physical constraints.Approaches also include starting from established functionals or incorporating exact physical constraints into the functional form.Source: §4.3 · p. 74

4.3.1 Machine Learning Exchange-Correlation Energy Functionals.

  • Machine-learned exchange-correlation functionals address a major source of KS-DFT error, with exact constraints incorporated analytically or through training data.DM21 imposes fractional-charge and spin constraints and performed strongly on bond-breaking, QM9, and GMTKN55, though its generalizability has been challenged and transition-metal calculations can face SCF convergence issues. Other constrained approaches report improved generalizability or convergence, while iteration-level training information has also improved dissociation-curve accuracy and equilibrium convergence.Source: §4.3.1 · pp. 74–76

4.3.2 Machine Learning Kinetic Energy Functionals.

  • Chemical accuracy has been achieved by learned kinetic-energy functionals, including a voxel neural network for graphene and a 1D model trained on 20 examples.Orbital-free DFT avoids KS orbitals and scales quasi-linearly, but noisy kinetic-energy functional derivatives complicate optimization; imposing exact constraints improved a 1D Hooke’s-atom case but not 1D H2 bond stretching.Source: §4.3.2 · pp. 76–77

4.3.3 Datasets and Benchmarks.

  • 10,049 structures and 8,656 unique reference data points comprise ACCDB, a high-quality resource for training and benchmarking learned density functionals.The datasets evaluate functionals across applications such as atomization energies, barrier heights, bond energies, and noncovalent interactions; the review notes improving accuracy, sometimes approaching chemical accuracy, at lower computational cost than higher-level calculations.Source: §4.3.3 · p. 78

4.3.4 Open Research Directions.

  • Symbolic regression produced GAS22 with improved error on MGCDB84, while automated feature engineering with a deep Q-network improved scores and reduced computation time on three materials databases.These methods offer routes toward symbolic exchange-correlation functionals through evolutionary search or equation-like feature generation and selection.Source: §4.3.4 · pp. 78–79
  • Training on constraint-satisfying data improved DM21 and DM21mu over DM21m, while Bayesian optimization can predict Hubbard U more accurately than linear response [Kirkpatrick et al. 2021; Yu et al. 2020].DM21 used fractional-charge and fractional-spin systems; DM21mu used data satisfying the uniform electron gas constraint. Bayesian optimization has been applied to transition-metal oxides, europium chalcogenides, and narrow-gap semiconductors.Source: §4.3.4 · p. 79
  • Further KEDF work should establish exact physical constraints and assess their effect on accuracy; gradient-free optimization and richer training data also remain open directions [Wrighton et al. 2023; Ryczko et al. 2022b; Li et al. 2021a; Kalita et al. 2022; Dick and Fernandez-Serra 2021; Brockherde et al. 2017].Monte Carlo optimization avoids evaluating the kinetic-energy functional derivative but still needs extension to three dimensions; proposed training-data approaches include KS regularization and automatic differentiation, while non-uniform density sampling remains an issue.Source: §4.3.4 · p. 79
  • ML density functionals need further testing on solids and broader molecular groups, with LC20 and SOL62 proposed as useful datasets [Sun et al. 2011; Trepte and Voss 2022; Zhang et al. 2018c; Bystrom and Kozinsky 2023].Nagai et al. 2020 trained on three small molecules and showed promising results across first- and second-row molecules, but has not been tested on transition-metal or extended systems.Source: §4.3.4 · p. 80
  • Stochastic gradient descent on energetic quantities with wave-function orthonormality in the objective can accelerate KS-DFT ground-state energy and magnetic-state prediction [Li et al. 2023d].The approach targets improved convergence and/or reduced computational complexity.Source: §4.3.4 · p. 80
  • Quantum tensor learning and accurate ML density functionals may support applications across chemistry, physics, engineering, biology, and pharmaceutical research.AI/ML advances in quantum mechanics may enable more accurate dataset generation, furthering model development for molecules, proteins, materials, and molecular docking.Source: §4.3.4 · p. 80

5 AI FOR SMALL MOLECULES

This section surveys AI methods for small molecules, spanning symmetry-aware 3D representation learning, conformer reconstruction and generation, generation of 3D molecules from scratch, and machine-learning approaches to molecular dynamics. It reviews methods, datasets, and benchmarks while highlighting challenges involving stereochemistry, conformational flexibility, physical symmetry, chemical validity, sampling, scale, data, and efficiency.

5.1 Overview

  • The section covers 3D molecular representation learning, conformer generation, molecule generation from scratch, molecular dynamics simulation, and learning stereochemistry and conformational flexibility.Representation learning targets downstream molecule- and atom-level predictions; the generative tasks include sampling low-energy conformers from 2D graphs and generating 3D molecules from scratch.Source: §5.1 · p. 81, Figure 17 · p. 82

5.2.1 Problem Setup.

  • 3D molecular representation learning uses atom types and coordinates to learn features for molecule-level or atom-level predictions, whose scalar or tensor targets must respect reference-frame symmetry.A 3D molecule is represented as a point cloud of atom types and coordinates; conformer geometry matters for properties such as energy and electric dipole moment.Source: §5.2.1 · p. 83, §5.2 · p. 83

5.2.2 Technical Challenges.

  • Key challenges are enforcing task-appropriate invariance or equivariance, distinguishing enantiomers and conformers, and maintaining computational efficiency.Energy prediction calls for SE(3)-invariant representations, whereas force predictions should be SO(3)-equivariant; expressive models must distinguish geometries with different interactions, energies, or forces.Source: §5.2.2 · p. 84

5.2.3 Overview of Existing Methods.

  • Existing 3D GNNs are categorized by tensor order—scalar invariant, vector equivariant, or higher-order equivariant features—and by the body order of their interactions.Body-order expansions represent local many-body interactions, while standard message passing uses two-body interactions between a central atom and one neighbor.Source: §5.2.3 · pp. 84–85, Figure 18 · p. 85, Figure 19 · p. 86

5.2.4 Invariant Methods (ℓ= 0 Scalar Features).

  • Invariant methods trade off geometric discriminability against efficiency, ranging from distance-only SchNet to richer many-body features and ComENet’s complete local geometric features.ComENet is described as distinguishing all different 3D molecules while retaining O(nk) complexity through reference-node selection within the 1-hop neighborhood.Source: §5.2.4 · pp. 86–87

5.2.5 Equivariant Methods (ℓ= 1 Vector Features).

  • Equivariant methods with ℓ=1 vector features preserve symmetry through operations such as vector scaling, summation, linear maps, and products.Examples include EGNN’s vector scaling and summation, PaiNN’s linear transformations and scalar products, and EQGAT’s cross products and attention.Source: §5.2.5 · pp. 87–88

5.2.6 Equivariant Methods (ℓ≥1 Tensor Features).

  • Higher-order equivariant models use tensor products to build features across rotation orders, with constrained operations and model-specific attention or nonlinearities.Their linear weights are constant across the representation index to maintain equivariance, and tensor products combine feature orders; examples include TFN, NequIP, SE(3)-Transformer, Equiformer, Cormorant, and SEGNN.Source: §5.2.6 · pp. 88–90, Figure 20 · p. 89

5.2.6 Equivariant Methods (ℓ≥1 Tensor Features)

  • Higher-order equivariant models sum neighbor messages; SE(3)-Transformer and Equiformer weight them with attention, while residual connections help retain chemical information such as atom type.SE(3)-Transformer uses dot-product attention; Equiformer uses MLP attention computed from ℓ=0 features to preserve equivariance. TFN and SE(3)-Transformer lack residual connections.Source: §5.2.6 · p. 90

5.2.7 Higher Body Order Methods

  • Atomic cluster expansion (ACE) reduces many-body interaction cost to linear growth in neighbor count by expanding interactions in products of atomic basis functions.ACE removes neighbor-order constraints, then uses the density trick to simplify many-body terms; its basis uses Bessel functions and spherical harmonics.Source: §5.2.7 · pp. 91–93
  • Linear ACE, MACE, PACE, and Allegro implement higher-body interactions differently, using layered message passing, edge operations, or boosted tensor products.Linear ACE is single-layer; MACE adds multiple ACE layers, Allegro stacks edge-centered operations, and PACE uses edge-boosting to approximate higher-degree equivariant polynomials. ACE-family methods aggregate symmetrized basis features, while Allegro sums neighboring-edge messages.Source: §5.2.7 · pp. 93–94

5.2.8 Model Outputs

  • Invariant models can derive energy-conserving SO(3)-equivariant forces from energy gradients, while equivariant models can also predict forces directly or more readily handle targets such as Hamiltonians.Invariant approaches need additional operations to ensure equivariance for targets such as Hamiltonian matrices.Source: §5.2.8 · p. 95

5.2.9 Datasets and Benchmarks

  • Small-molecule benchmarks cover property prediction and energy-and-force prediction, with dataset setups ranging from individual molecules to shared models across isomers.QM9 contains over 130k molecules with up to nine heavy atoms and quantum-chemistry properties; MD17 and rMD17 provide molecular conformations with energies and forces, while ISO17 covers 129 isomers and Molecule3D about 4 million molecules. MAE and MSE are common metrics.Source: §5.2.9 · pp. 96–97

5.2.10 Open Research Directions

  • Open directions include joint 2D–3D learning and more expressive, efficient geometric GNNs; higher tensor and body orders improve expressivity but raise costs, while depth risks oversquashing.Invariant GNNs can be more scalable and may match equivariant GNNs on fully connected graphs or with precomputed non-local features, motivating study of their trade-offs.Source: §5.2.10 · pp. 97–98

5.3.1 Problem Setup

  • Because accurate 3D geometries are costly to obtain, conformer reconstruction seeks either low-energy geometry distributions or the equilibrium geometry from a 2D molecular graph.3D structure matters for molecular properties and applications; generation models learn p(C|G), whereas prediction models estimate the equilibrium ground-state geometry.Source: Recommended Prerequisites: Section 5.2 · p. 98, §5.3.1 · pp. 98–99

5.3.2 Technical Challenges

  • Conformer reconstruction must ensure geometrically and chemically valid structures while modeling position distributions invariant to rotation and translation.Symmetric atoms can collapse to identical embeddings, and distance-derived coordinates may not form a valid Euclidean distance matrix; chemical constraints include planar aromatic rings and non-planar small rings or macrocycles.Source: §5.3.2 · p. 99

5.3.3 Existing Methods.

  • Conformer generation spans equivariant diffusion and torsion-based generative methods as well as predictors, with chemical validity and dependence on RDKit conformers as limitations.ConfGF and GeoDiff generate coordinates but may violate constraints such as ring planarity; torsional diffusion preserves RDKit-generated local bond lengths and angles but cannot refine local ring structures. GeoMol predicts local structures with geometric constraints and symmetric-atom matching, while other predictors output distances or coordinates.Source: §5.3.3 · pp. 99–100, Table 19 · p. 100

5.3.4 Datasets and Benchmarks.

  • Benchmarks span small molecules, flexible drug-like molecules, biomolecules, and materials; force and energy errors alone can misrepresent MD simulation performance.GEOM-QM9 covers small molecules with up to 9 heavy atoms, whereas GEOM-Drugs includes larger, flexible molecules. QM9 and GEOM-Drugs also benchmark from-scratch generation; MD studies assess observables including free-energy surfaces, radial distribution functions, and diffusivity.Source: §5.3.4 · p. 101, §5.4.4 · pp. 103–104, §5.5.4 · p. 107

5.3.5 Open Research Directions.

  • Future conformer-generation work should model solvent effects, learn from limited data, and generate high-energy transition states relevant to chemical reactions.Existing benchmarks are simulated in vacuum, and current methods focus mainly on low-energy conformers; transfer and few-shot learning are proposed for underrepresented compounds.Source: §5.3.5 · p. 101

5.4.2 Technical Challenges.

  • From-scratch generation targets a distribution over 3D molecules without molecular-graph inputs, requiring symmetry-aware modeling and chemically valid structures that distinguish chirality.Coordinate outputs require explicit SE(3)-invariant probability modeling; invariant distances or angles avoid that requirement but must be valid and sufficient to reconstruct coordinates. Many methods use E(3) symmetry, which also identifies reflections, and satisfying local chemical constraints remains challenging.Source: §5.4.1 · p. 102, §5.4.2 · p. 102, §5.4.5 · p. 104

5.4.3 Existing Methods.

  • From-scratch 3D molecule generators either produce coordinates with E(3)-invariant models or build structures from invariant geometric features; only EDM supports property-conditioned generation.E-NFs and EDM zero-center coordinates and use a center-of-mass-free Gaussian prior; GeoLDM adds latent diffusion. EDMNet generates distance matrices, while G-SchNet and G-SphereNet place atoms autoregressively using geometric features.Source: §5.4.3 · pp. 102–103, Table 20 · p. 103

5.5.1 Problem Setup.

  • MD simulations yield trajectories and macroscopic observables but are costly because accurate forces may require quantum chemistry and long processes require many femtosecond-scale steps.ML force fields predict energy and forces from coordinates to replace expensive quantum calculations; simulations can then produce observables such as radial distribution functions, stress, mean-squared displacement, and free-energy surfaces. Sampling and coarse-graining methods target observables rather than exact trajectory recovery.Source: Recommended Prerequisites: Section 5.2 · p. 104, §5.5.1 · p. 105

5.5.3 Existing Methods.

  • ML force fields replace costly quantum calculations in MD, while learned reaction coordinates and coarse-grained models support enhanced sampling and more efficient simulation.Force fields predict energies and forces from atomic coordinates and are designed to respect E(3) energy invariance and force equivariance. Coarse-graining reduces complexity but trades off accuracy and may require mapping back to fine-grained states.Source: §5.5.1 · p. 105, §5.5.3 · pp. 106–107

5.5.2 Technical Challenges.

  • MD learning must address complex energy surfaces and costly long simulations, while improving force-field efficiency, stability, long-range interactions, coarse-graining, and rare-event sampling.Lower energy or force prediction error does not necessarily mean more stable simulations or accurate observables. Proposed directions include active learning from underperforming states and coarse-graining that preserves useful information within a computational budget.Source: §5.5.2 · p. 106, §5.5.5 · pp. 107–108

Recommended Prerequisites: Section 5.2

  • Stereoisomers can share a 2D graph yet differ chemically, while observable properties may depend on thermally accessible conformers or particular active geometries.Stereochemical forms include tetrahedral and axial chirality and E/Z isomerism; conformers interconvert at rates that depend on barriers, environment, and temperature.Source: Recommended Prerequisites: Section 5.2 · pp. 108–110, Figure 22 · p. 109

5.6.1 Problem Setup.

  • Conformer-ensemble learning formalizes conformers as geometries grouped by practical interconversion timescale, with tasks including stereoisomer classification, property prediction, Boltzmann averaging, and active-conformer identification.The ensemble is represented discretely using an RMSD threshold, and conformers may be assigned statistical weights.Source: §5.6.1 · p. 110

5.6.2 Technical Challenges.

  • 2D GNNs struggle to distinguish stereoisomers, while some E(3)-invariant 3D GNNs cannot distinguish enantiomers; ensemble inputs also raise compute, optimization, inference-cost, and data-quality challenges.Using inexpensive or sampled conformers can reduce accuracy or miss property-active geometries, whereas high-quality ensembles may require costly simulations at both training and inference.Source: §5.6.2 · pp. 110–111

5.6.3 Existing Methods.

  • Existing chirality methods range from SMILES or graph tags to permutation-based and torsion-aware models; ChIRo is conformer-invariant, while 3D GNNs can require many conformers and underperform tagged 2D GNNs.Global CIP tags may change after small graph edits, and atom-ordering-based features can break permutation invariance; Pattanaik et al. [2020] and Adams et al. [2021] report improved chiral-property prediction over simple tags at higher computational cost.Source: §5.6.3 · pp. 111–112
  • Conformer ensembles are commonly encoded by multiple-instance learning with pooling or attention, while single effective structures and conformer augmentation offer alternatives that may miss key conformers.Attention can assign relative importance to conformers, and implicit approaches avoid encoding multiple conformers at inference time.Source: §5.6.3 · pp. 112–113

5.6.4 Datasets and Benchmarks.

  • Existing benchmarks are limited: experimental datasets often lack multiple stereoisomers per molecule, and ensemble MIL models did not outperform single-conformation baselines on GEOM-DRUGS or random-forest fingerprints on the synthetic ligand task.The GEOM-DRUGS task had only 426 bioactive hits, and MIL models required orders of magnitude more training resources; CREMP and MISATO are newer ensemble datasets.Source: §5.6.4 · pp. 113–114

5.6.5 Open Research Directions.

  • Future work should develop stereochemistry-sensitive benchmarks and models that need not see multiple stereoisomers in training, extend representation beyond tetrahedral chirality, and make conformer-ensemble learning more effective and efficient.Proposed directions include testing alternative ensemble architectures, reducing the cost of generating and encoding conformers, and evaluating transfer from inexpensive or generated ensembles.Source: §5.6.5 · pp. 114–115

6 AI FOR PROTEIN SCIENCE

This section surveys protein science, covering protein folding, representation learning, and backbone generation alongside protein datasets and open problems. It focuses on structure-based methods, geometric and physical constraints, and diffusion-based backbone generation, including its representations, methods, and evaluation benchmarks.

6.1 Overview

  • The section covers folding, representation learning, and backbone generation; it focuses on individual proteins and structure-based methods, with diffusion models prominent in generation.Backbone generation aims to model real backbone distributions while meeting geometric constraints; the survey excludes protein–small-molecule interactions and inverse folding as dedicated topics.Source: §6 · p. 116, §6.1 · pp. 116, 118

6.2.1 Problem Setup.

  • Protein folding predicts a protein’s 3D atomic structure from its amino acid sequence, while methods must address equivariance, physical constraints, and computationally demanding structural search.Relevant physical constraints include characteristic bond lengths and avoiding clashes between arbitrarily paired atoms.Source: §6.2.1 · p. 119, §6.2.2 · p. 119

6.2.3 Existing Methods.

  • Folding methods shifted from two-stage distance-and-orientation prediction followed by energy minimization to end-to-end geometric learning, exemplified by AlphaFold2’s equivariant structure module.AlphaFold2 generates backbone coordinates sequentially as relative positions parameterized by rotations and transition vectors; RoseTTAFold jointly models sequence, distance-map, and coordinate information.Source: §6.2.3 · p. 120, Table 21 · p. 121

6.2.4 Datasets and Benchmarks.

  • At CASP14, AlphaFold2 achieved nearly 90 GDT scores; ESMFold reached up to 60× speedup while maintaining accuracy, and AlphaFold3 extended prediction to interactions among biomolecules.AlphaFold2 was described as the first computational method to predict protein structures with near-experimental accuracy; AlphaFold3 showed improvement over traditional and state-of-the-art approaches.Source: §6.2.4 · p. 121

6.2.5 Open Research Directions.

  • Folding generalization remains difficult for multi-chain proteins and proteins unlike training data; predicting structures in biological contexts is a key direction, with PDB serving as a major experimental structure resource.The PDB houses over 180,000 structures; AlphaFold2’s training set contains around 475K structures from self-distillation and PDB sources.Source: §6.2.4 · pp. 120–121, §6.2.5 · p. 122

6.3.2 Technical Challenges.

  • Protein representations support protein-level property prediction and node-level amino-acid prediction, but models must balance scale, structural levels, SE(3)-invariant outputs, and expressiveness.Protein-level examples include enzyme reaction classification; a node-level example is inverse folding.Source: §6.3.1 · p. 123, §6.3.2 · p. 123

6.3.3 Existing Methods.

  • Representation methods reduce protein graph size by using amino acids rather than atoms as nodes, capture structures at different levels, and incorporate symmetry-aware features; higher-order methods remain absent.IEConv uses atom nodes and hierarchical pooling, while amino-acid-node approaches are more efficient; ProNet captures backbone and side-chain detail, and MaSIF and dMaSIF emphasize surfaces.Source: §6.3.3 · pp. 123–125

6.3.4 Datasets and Benchmarks.

  • Protein representation methods are evaluated on tasks including amino-acid-type and protein-function prediction, using datasets such as CATH 4.2, Fold, Enzyme Reaction, EC, and Gene Ontology.The surveyed dataset statistics include sample counts and maximum and average amino-acid counts per protein.Source: §6.3.4 · p. 125, Table 23 · p. 125
  • Protein benchmarks cover inverse folding, fold classification, function prediction, and representation-learning tasks, including CATH, Gene Ontology, and Atom3D datasets.Inverse folding evaluates perplexity and recovery; fold classification uses accuracy, and the Gene Ontology dataset uses the Enzyme Commission metrics.Source: §6.3.4 · pp. 125–126

6.3.5 Open Research Directions.

  • Protein representation learning still needs to capture local secondary structures and their tertiary arrangement, incorporate surfaces and domains, and explore higher-order equivariant methods.MaSIF models solvent-excluded surfaces as meshes, while dMaSIF uses oriented point clouds.Source: §6.3.5 · p. 126
  • Protein dynamics, circuit design, and mutation-effect prediction are open directions; ProteinGym provides a benchmark for protein fitness prediction and design.ProteinGym includes deep mutational scanning assays, a curated clinical dataset with expert-annotated mutation effects, and an evaluation framework.Source: §6.3.5 · pp. 126–127

6.4 Protein Backbone Structure Generation

  • The section focuses on diffusion models that noise protein backbones and denoise them with a learned network, while noting flow-matching alternatives.Figure 27 illustrates the noise-to-denoising pipeline and examples using coordinate, frame, and internal-angle representations.Source: §6.4 · p. 127, Figure 27 · p. 128

6.4.1 Problem Setup.

  • Backbone generation learns the distribution of real protein backbones so it can sample novel structures from that distribution.Source: §6.4.1 · p. 127

6.4.2 Technical Challenges.

  • Generation must model an unknown, sparse structure distribution efficiently while preserving rotation and translation invariance through equivariant networks.The text notes that reflection invariance can impose an incorrect bias because proteins are chiral; it also emphasizes efficient representations to manage the vast structure space.Source: §6.4.2 · pp. 127–128

6.4.3 Existing Methods.

  • Protein backbone diffusion methods use coordinate, angle, or frame representations and design diffusion processes to support sampling while respecting distribution symmetries.The surveyed approaches include E(3)-invariant methods, which treat chiral structures as equivalent, and SE(3)-invariant methods.Source: §6.4.3 · pp. 128–130, Table 24 · p. 129, Table 25 · p. 130

6.4.4 Datasets and Benchmarks.

  • Backbone-generation evaluation lacks standard datasets and commonly uses scTM score and novelty relative to training structures.Earlier studies evaluated models on selected structures or datasets including PDB, CATH, SCOPe, and curated AlphaFold DB data.Source: §6.4.4 · p. 131

7 AI FOR MATERIALS SCIENCE

This section surveys AI and machine-learning approaches across crystalline, disordered, and amorphous materials, covering property prediction and generation, diffraction-based crystal prediction, experimental characterization, phonon calculations, and structural analysis. It highlights symmetry-aware methods alongside challenges in data availability, generalization, representation, inverse design, accuracy, and computational efficiency.

7.1 Overview

  • Crystals are represented by atom types, coordinates, and lattice vectors; equivalent materials must yield symmetry-invariant property predictions and generation distributions.The relevant transformations include atom permutations, E(3) transformations, and periodic coordinate shifts.Source: §7.1 · pp. 132, 134

7.2 Material Representation Learning

  • Crystal property prediction must handle periodic invariance and repeating-cell patterns while balancing higher-order interactions against unit cells that can contain over 200 atoms.The task predicts real-valued or categorical material properties; crystal structures also have unit-cell E(3) invariance and periodic invariance.Source: §7.2.2 · p. 134, §7.2.1 · p. 134

7.2.3 Existing Methods

  • Multi-edge crystal graphs provide periodic and E(3) invariance, while ALIGNN, M3GNet, and CHGNet add higher-order information and Matformer, PotNet, and Ewald-MP encode periodic patterns.Matformer encodes lattice information with self-connecting edges; PotNet and Ewald-MP use Ewald-summation approaches to capture long-range periodic interactions.Source: §7.2.3 · pp. 135–137, Table 26 · p. 136

7.2.4 Datasets and Benchmarks / 7.2.5 Open Research Directions

  • Materials Project, JARVIS, and MatBench provide crystal-property benchmarks, but current prediction methods leave higher-order rotation properties and geometric completeness open.The benchmark suites include regression and classification tasks; properties are mostly calculated using DFT-based methods.Source: §7.2.4 · p. 137, §7.2.5 · p. 137

7.3 Material Generation

  • Crystal generation must assign equal probability to symmetry-equivalent structures, including periodic transformations that molecule-generation methods and distance-, angle-, or torsion-based targets do not handle.Periodic transformations can change pairwise distances, so incorporating periodic invariance into E(3)-invariant generative models is challenging.Source: Recommended Prerequisites: Section 7.1 · p. 137, §7.3.2 · p. 138

7.3.3 Existing Methods

  • Voxel-grid and fractional-coordinate methods capture only subsets of crystal symmetries, whereas SyMat captures permutation, E(3), periodic, and translation invariance; CDVAE lacks translation invariance.CDVAE applies score matching to coordinates, while SyMat applies it to pairwise distances.Source: §7.3.3 · pp. 138–139, Table 28 · p. 139

7.3.4 Datasets and Benchmarks / 7.3.5 Open Research Directions

  • Perov-5, Carbon-24, and MP-20 provide generation benchmarks, but efficient sampling and standard measures of synthesizability remain unresolved.Recent score-matching methods incur high coordinate-refinement costs, and the literature lacks a standard metric for whether generated crystals can be synthesized.Source: §7.3.4 · pp. 139–140, §7.3.5 · p. 140

7.4 Materials Characterization

  • Machine learning can help accelerate experimental characterization by predicting crystal structures from scattering or spectroscopy data and reconstructing spectra from structures.Characterization addresses local as well as long-range disorder, and experimental data processing can be time-consuming; the review notes that it only scratches the surface of possible applications.Source: Recommended Prerequisites: Section 7.1 · pp. 140–141, Figure 30 · p. 141, §7.4.1 · p. 141

7.4.3 Existing Methods.

  • For diffraction spectra, CNNs and conventional classifiers support symmetry or lattice prediction, but lower-symmetry cases remain difficult; equivariant networks also predict INS spectra from structures [Greasley and Hosein 2023; Cheng et al. 2023b].SVM and Complement Naive Bayes performed as well as a neural network for multi-phase identification; Cheng et al. predict 1D and 2D spectra using a latent-space autoencoder and Euclidean neural network.Source: §7.4.3 · pp. 142–143

7.4.4 Datasets and Benchmarks.

  • Synthetic data are necessary for scattering models, but experimental data differ and no experimental benchmark dataset is identified, motivating augmentation for peak shifts, broadening, texture, and noise [Jain et al. 2013; D et al. 2019; Groom et al. 2016].The section also notes that training on simulated data and testing on experimental data does not tend to produce satisfactory results.Source: §7.4.4 · p. 143

7.4.5 Open Research Directions.

  • Future work includes symmetry- and Fourier-aware scattering models, expressive invariant local-environment descriptors, and machine-learning tools such as computer vision to accelerate characterization; amorphous materials pose distinct challenges because they lack long-range order [Liu et al. 2024].Suggested applications include Bragg-peak detection and artifact identification, while local descriptors should be invariant to translation, rotation, and same-species atom permutation.Source: §7.4.5 · p. 143, §7.5.5 · pp. 145–146, Recommended Prerequisites: Section 7.1 · p. 149

7.5.2 Technical Challenges.

  • Disorder and short-range order complicate structural characterization: large DFT supercells and demanding experiments limit training data, requiring data-efficient models that capture local structure and tensor properties.PDF and NMR measurements can probe local structures, which models may use to predict atomic positions or tensorial chemical-shift properties.Source: §7.5 · pp. 143–144, §7.5.2 · p. 144, §7.5.1 · p. 144

7.5.3 Existing Methods.

  • Equivariant Si chemical-shift tensor models outperformed symmetry-invariant models by 53%, while DeepStruc predicted some simple monometallic nanoparticle structures from experimental PDFs.DeepStruc uses a conditional variational autoencoder to generate structures conditioned on pair distribution functions.Source: §7.5.3 · pp. 144–145

7.5.4 Datasets and Benchmarks.

  • Available local-structure datasets remain limited: the cited PDF set has 3,742 nanoparticle structures and the NMR set has 421 silicate structures, motivating broader datasets [Kjær et al. 2023; Sun et al. 2020; Venetos et al. 2023].The phonon section likewise calls for a standard ML database, noting representative Γ-phonon and PDOS datasets of about 146,000 materials and 1,521 semiconductor compounds, respectively [Okabe et al. 2023].Source: §7.5.4 · p. 145, §7.6.4 · p. 148

7.6.2 Technical Challenges.

  • Phonon calculations are costly for complex or low-symmetry materials, while periodicity, symmetry, real-to-reciprocal-space differences, and variable band counts challenge learning PDOS and dispersion relations.Lower computational settings can reduce DFT accuracy or produce nonphysical frequencies, and experimental PDOS measurements also require scarce high-intensity sources.Source: §7.6.1 · pp. 146–147, §7.6.2 · p. 147

7.6.3 Existing Methods.

  • Active-learning MTPs reduced DFT training needs and reproduced DFT phonon quantities, while VGNNs handle variable-dimensional bands and E(3)NNs reproduce experimental PDOS features [Ladygin et al. 2020; Okabe et al. 2023; Chen et al. 2021a].The E(3)NN workflow may support high-throughput screening for thermal-material candidates [Chen et al. 2021a].Source: §7.6.3 · p. 148, §7.6.5 · p. 149

7.7.1 Technical Approaches and Challenges.

  • Amorphous-material modeling combines melt-quench molecular dynamics and Monte Carlo sampling with experimental constraints and refinement to determine structures.Reverse Monte Carlo uses experimental metrics such as the radial distribution function, while force-enhanced atomic refinement can improve nonunique structures [Madanchi et al. 2024a].Source: §7.7.1 · p. 149
  • Special quasirandom structures approximate random atomic or defect distributions, and cluster expansion fits effective Hamiltonians for subsequent simulations.Tools include ATAT and CASM.Source: §7.7.1 · p. 149

7.7.2 Existing Methods.

  • MLIPs model amorphous carbon and silicon, while GNNs predict glassy dynamics and support inverse design of glass landscapes [Bapst et al. 2020; Wang and Zhang 2021].Bapst et al. [2020] report long-time dynamics and propensity-map prediction and a structural length scale; Wang and Zhang [2021] combine GNNs with swap Monte Carlo to predict plastic resistance and identify a geometrically ultrastable but energetically metastable state.Source: §7.7.2 · p. 150
  • An autoencoder and Gaussian mixture models applied to bond-order descriptors reveal hidden local structural heterogeneities in supercooled liquids without dynamical information [Boattini et al. 2020].The resulting learned structural order parameter detects the largest structural variation in those liquids.Source: §7.7.2 · p. 150
  • Generative models extrapolate structures from a correlation-length context kernel to arbitrary sizes, extending from 2D to 3D disordered systems and complex bulk metallic glasses [Kilgour et al. 2020; Madanchi et al. 2024b; Zhou et al. 2023b].Source: §7.7.2 · p. 150

7.7.3 Open Directions.

  • Open directions include inverse design of amorphous materials, extending diffusion models and LLMs to large supercells, improving GNN accuracy and sampling, and reducing data-generation and model computation and memory costs.Efficient models are needed for large-scale, long-time dynamics across scales and for understanding short- and potentially intermediate-range order; crystalline-material generation methods are promising, but their extension to amorphous materials remains largely unexplored [Xie et al. 2022b; Jiao et al. 2023; Antunes et al. 2024; Yan et al. 2024; Zeni et al. 2025].Source: §7.7.3 · p. 150

8 AI FOR MOLECULAR INTERACTIONS

This section surveys AI for molecular interactions, covering protein–ligand binding, generative drug design, and molecule–material prediction. It outlines task definitions, datasets, methods, evaluation and benchmarks, symmetry-aware modeling, open challenges, and research directions.

8.1 Overview

  • Protein–ligand tasks include predicting binding poses and affinities or generating ligands, while molecule–material work predicts energies, forces, and relaxed structures; material generation remains unexplored.The overview classifies tasks by their nature rather than by method, and notes that material predictions can support properties such as adsorption and transition-state energies.Source: §8.1 · p. 151, Figure 31 · p. 152

8.2.1 Problem Setup.

  • Docking predicts ligand atom positions in known-pocket or blind settings, whereas binding-strength prediction estimates affinity; evaluations should account for whether protein structures are bound, unbound, or computationally generated.Docking commonly assumes a bound protein structure, an unrealistic condition for applications; affinity prediction may use bound complexes or protein structures and ligand graphs.Source: §8.2.1 · p. 152

8.2.2 Technical Challenges.

  • Docking must model a sparse, multimodal Boltzmann distribution with limited quality training data, and requires SE(3)-equivariant pose predictions while affinity predictions should be SE(3)-invariant.The passages estimate roughly 20k accessible quality docking samples and describe similarly limited, noisy affinity data; a single most-likely pose is difficult to find among plausible configurations.Source: §8.2.2 · p. 154

8.2.3 Existing Methods.

  • Blind docking spans search-based scoring, single-pose regression, and generative sampling; generative methods can represent multiple poses and dock to unbound or computationally generated proteins.Regression methods can produce unphysical clashes by forcing one pose, while DiffDock [Corso et al. 2022] samples translations, rotations, and torsions and NeuralPLexer [Qiao et al. 2023] conditions diffusion on predicted contacts.Source: §8.2.3 · pp. 154–155, Table 30 · p. 155

8.2.4 Datasets and Benchmarks.

  • Docking is commonly evaluated by the fraction of generated poses below a specified RMSDT threshold and steric-clash counts; binding-strength tasks use classification accuracy, ranking correlation, or MAE, ideally with unbound or computationally generated proteins.ChEMBL contains 20 million activity measurements without structures.Source: §8.2.4 · p. 156

8.2.5 Open Research Directions.

  • 22% accuracy for DiffDock on ESMFold structures illustrates that docking remains unsolved; proposed directions include protein conformational change modeling and generative statistical-mechanics methods for interaction strengths.The section also identifies joint use of structural data and larger sequence-based affinity datasets, with measurement noise as a challenge.Source: §8.2.5 · p. 156

8.3.1 Problem Setup.

  • Structure-based drug design learns a conditional distribution of ligand molecules given a protein binding site, confronting a chemical space estimated to exceed 10^60 molecules plus conformational possibilities.Generated ligands should transform equivariantly with rotations or translations of the binding site, while only a minuscule fraction of chemical space is drug-like.Source: §8.3.1 · p. 156, §8.3.2 · p. 157

8.3.3 Existing Methods.

  • Structure-based drug design methods either add atoms or fragments autoregressively using relative-position variables, or generate all ligand coordinates in one shot with diffusion models.Autoregressive methods preserve SE(3)-equivariance through relative geometry; diffusion methods use equivariant denoising and center-of-mass shifting, while VD-Gen [Lu et al. 2023b] iteratively refines particles through virtual dynamics.Source: Table 31 · p. 157, §8.3.3 · p. 158

8.3.4 Datasets and Benchmarks.

  • Structure-based drug-design evaluation spans generated-molecule quality, binding-affinity estimates, and drug-likeness or synthesizability metrics.Datasets include CrossDocked2020, PDBBind, DUD-E, and scPDB; CrossDocked2020 contains 22,584,102 docked complexes across 2,922 pockets and 13,839 ligands, and studies commonly filter by binding-pose RMSD.Source: §8.3.4 · p. 159

8.3.5 Open Research Directions.

  • Fragment and scaffold priors could narrow drug-design search, while jointly optimizing multiple drug-like properties without losing binding affinity remains an open challenge.DiffLinker [Igashov et al. 2022] uses an E(3)-equivariant, protein-pocket-conditioned diffusion model to link fragments; Adams and Coley (2022) propose shape-based 3D generation. Existing works do not explicitly optimize properties such as solubility and permeability during generation [Bickerton et al. 2012].Source: §8.3.5 · pp. 159–160

8.4 Molecule and Material Interactions

  • The section covers molecule–material interaction prediction and identifies generative tasks for this area as unexplored.Its tasks include predicting pair energies and per-atom forces, as well as relaxed structures and energies from initial structures.Source: Recommended Prerequisites: Section 5.2 · p. 160

8.4.1 Problem Setup.

  • The molecule–material tasks include predicting pair energy (S2E), per-atom forces (S2F), relaxed energy (IS2RE), and relaxed positions (IS2RS).Adsorption energy subtracts clean-surface and gas-phase or reference-state energies from the adsorbate–surface system energy; depending on the dataset, energy may mean total or adsorption energy, including a local or global minimum.Source: §8.4.1 · pp. 160–161

8.4.2 Technical Challenges.

  • Molecule–material models must respect rotational invariance for energies and equivariance for forces, while accounting for relaxation from rough initial structures.IS2RE predicts relaxed energy and IS2RS predicts relaxed structure, so both require modeling structural relaxation as well as the appropriate symmetry behavior.Source: §8.4.2 · p. 161

8.4.3 Existing Methods.

  • Energy-derived forces conserve energy but add computation, whereas direct-force prediction can be faster and perform better with sufficiently large training datasets.Indirect relaxation iteratively updates positions using predicted forces and can be accurate but slow; direct methods predict relaxed-coordinate differences and are faster but less accurate. EquiformerV2 improves indirect performance, while Equiformer leads direct methods; efficiency, expressivity, and physical constraints remain tradeoffs.Source: §8.4.3 · pp. 161–162

8.4.4 Datasets and Benchmarks.

  • OC20 provides S2EF, IS2RE, and IS2RS tasks, with in-domain and out-of-domain splits for unseen adsorbates, catalysts, or both.IS2RE/IS2RS training has 460,328 structures; S2EF training has 133,934,018. OC20-Dense samples initial configurations to estimate global minima, while OC22 targets total rather than adsorption energy, broadening properties but increasing difficulty.Source: §8.4.4 · pp. 162–163

8.4.5 Open Research Directions.

  • Generating molecule–material pairs conditioned on molecules could enable periodic-material design, but models must capture adsorbate–material interactions and molecule geometry changes.The proposed task generates periodic materials with desired properties, such as adsorption energies; the molecule’s 3D geometry may change in response to the generated material.Source: §8.4.5 · p. 164

9 AI FOR PARTIAL DIFFERENTIAL EQUATIONS

This section surveys AI methods for PDEs across forward modeling, inverse problems, and inverse design. It covers multi-scale dynamics, rollout stability, symmetry-preserving and physics-informed approaches, and their methods, limitations, benchmarks, and open needs.

9.1 Overview

  • Neural solvers can be faster than classical PDE solvers while generalizing across initial conditions, parameters, and geometries, and learning dynamics from observed data when equations are unknown.They use larger time steps, coarser spatial discretizations, direct mappings from past to future states, and parallel GPU operations; generative models can also make uncertainty estimation more efficient.Source: §9.1 · p. 167

9.2.1 Problem Setup.

  • Forward solvers predict future states from initial conditions or historical observations, and symmetry-aware constraints can improve generalization and sample complexity.Training configurations may vary in initial conditions or PDE parameters; equivariant architectures and data augmentation are approaches to enforcing symmetries.Source: §9.2.1 · pp. 168–169

9.2.2 Technical Challenges.

  • Forward PDE solvers must address multi-scale and multi-resolution dynamics, long-term rollout instability, intrinsic symmetries, and scientifically implausible predictions from unconstrained data fitting.The section identifies five challenges: representing interacting spatial scales, allocating resolution to fast-changing regions, controlling accumulated rollout error, preserving symmetries, and incorporating physical principles.Source: §9.2.2 · p. 169

9.2.3 Existing Methods: Multi-Scale Dynamics.

  • Multi-scale architectures aggregate local-to-global information sequentially or in parallel; SineNet reduces misalignment between U-Net features and improves performance over a single U-Net under a fixed parameter budget.Dil-ResNet uses changing dilation rates and U-Nets down- and up-sampling for sequential processing; FNO processes scales in parallel in the frequency domain.Source: §9.2.3 · p. 170, Figure 34 · p. 170
  • Neural operators can generalize across discretization resolutions; FNO uses Fourier-domain convolutions to process scales in parallel and has been applied to weather forecasting and climate mitigation.FNO learns Fourier-transformed kernels, whose lower-frequency modes are theoretically invariant to resolution changes; cutting off higher modes helps manage computational cost.Source: §9.2.3 · p. 171
  • Weather models including FourCastNet, 3DEST, and GraphCast outperform operational numerical weather prediction models on various tasks at a fraction of the cost; climate and physics foundation models benefit from pretraining and fine-tuning.ClimaX fine-tunes across varied weather and climate tasks, while physics foundation models show sample-complexity advantages on downstream dynamics distinct from their pretraining data.Source: §9.2.3 · p. 172
  • Adaptive meshes allocate fine resolution to fast-changing regions and coarse resolution elsewhere; learned refinement can adjust the mesh as those regions shift over time.Irregular meshes motivate GNN-based simulation because CNN architectures cannot directly model them; learned alternatives to costly re-meshing are proposed by Pfaff et al. [2021] and Wu et al. [2022b].Source: Figure 35 · p. 173

9.2.4 Existing Methods: Multi-Resolution Dynamics.

  • MeshGraphNets autoregressively predict PDE dynamics on irregular meshes and use a second GNN to predict node sizing; LAMP jointly optimizes dynamics error and mesh node count with reinforcement learning.The sizing prediction drives adaptive mesh refinement; LAMP replaces costly re-meshing with a learned policy [Pfaff et al. 2021; Wu et al. 2022b].Source: §9.2.4 · p. 174
  • DeepONet maps an input function and query point to a target solution through branch and trunk encoders; extensions remove shared-discretization constraints, enabling mesh-independent neural operators.The branch encodes the input function and the trunk encodes the query point; neural operators aim to generalize beyond training discretizations [Lu et al. 2021b; Kovachki et al. 2021].Source: §9.2.4 · p. 174
  • Transformer neural operators address irregular meshes: OFormer uses cross-attention for differing input and query grids and latent-space evolution, while GNOT handles multiple input grids with location-dependent gating.OFormer uses keys and values from the input discretization and queries from target points; GNOT’s gating MLP varies the point-wise update with location [Li et al. 2022d; Hao et al. 2023].Source: §9.2.4 · pp. 174–175
  • GKN’s locality limits long-range interactions; MGKN’s multiscale V-cycles improve on GKN, while Geo-FNO and GINO enable Fourier-based long-range processing on irregular meshes.GINO combines GKN encoding and decoding with an FNO processor in latent space, supporting generalization beyond training resolution [Li et al. 2020a; Li et al. 2020b; Li et al. 2022b; Li et al. 2023c].Source: §9.2.4 · pp. 175–176

9.2.5 Existing Methods: Long-Term Stability.

  • Autoregressive errors compound because training uses ground-truth states but rollouts feed predictions; MNO enforces dissipativity, while GNS and MP-PDE train with injected prediction noise.MNO combines soft and hard dissipativity constraints [Li et al. 2021c]; MP-PDE sources its training noise from model predictions to reduce the training–test distribution shift [Brandstetter et al. 2022c].Source: §9.2.5 · pp. 176–177
  • PDE-Refiner’s three-step noise refinement improves rollout stability; ACDM preserves flow statistics and produces diverse physically consistent samples, while GMR-Transformer predicts irregular-mesh fluid rollouts for hundreds of steps.PDE-Refiner also improves sample complexity and detects instability; ACDM’s diverse samples support uncertainty quantification [Lippe et al. 2023; Kohl et al. 2023; Han et al. 2021a].Source: §9.2.5 · p. 178

9.2.6 Existing Methods: Preserving Symmetries.

  • Symmetry-aware PDE methods range from relaxed-equivariance CNNs that expose symmetry breaking in learned weights to Clifford-algebra networks whose geometric-product operations yield O(n)-equivariance and gains across tasks.Relaxed kernels extend to three spatial dimensions with rank-1 decomposition; Clifford methods also model transformations such as rotations, reflections, and translations [Wang et al. 2023c; Ruhe et al. 2023a; Ruhe et al. 2023b].Source: §9.2.6 · pp. 179–180
  • SFNOs and SWSCNNs address spherical forecasting, while IsoGCN applies E(n)-equivariance to irregular meshes and simulates heat fields with over 1 million collocation points.Spherical approaches avoid projection distortions; SWSCNNs also equivariantly model vector fields. IsoGCN connects its adjacency representation to differential operators including gradient, divergence, and Jacobian.Source: §9.2.6 · p. 181

9.2.7 Existing Methods: Incorporating Physics.

  • HNNs [Greydanus et al. 2019] learn Hamiltonians through gradient supervision and preserve energy-consistent dynamics, unlike standard fully connected networks trained on the same data.The learned dynamics are numerically integrated with an explicit fourth-order Runge-Kutta method.Source: §9.2.7 · p. 182
  • APHYNITY [Yin et al. 2021] combines a known PDE operator with a learned residual, penalizing its L2 norm so the network models only dynamics the physical model misses.Predictions are generated by numerical integration of the known operator plus the learned component.Source: §9.2.7 · pp. 183–184
  • PINNs approximate analytical PDE solutions with derivative-based constraints; B-PINNs provide uncertainty quantification and more accurate predictions on noisy data.PINN training can be mesh-free, but generally requires retraining for each PDE configuration.Source: §9.2.7 · pp. 184–185

9.2.8 Datasets and Benchmarks.

  • PDE benchmarks span one to three dimensions and include turbulent or fast dynamics, large-step and conditional prediction, irregular geometries, and Lagrangian materials and structures.PDEBench includes eight PDEs; its compressible Navier-Stokes data includes initial Mach numbers up to 1 and low-viscosity turbulent dynamics.Source: §9.2.8 · pp. 185–187, Table 33 · p. 186
  • Current benchmarks may not reflect real-world PDE settings, so future tasks should explicitly test multi-scale processing, multi-resolution modeling, and rollout stability in complex domains.Real applications can combine complex external forces with large, irregularly shaped domains.Source: §9.2.8 · p. 187

9.2.9 Open Research Directions.

  • Learned solvers need costly training data; principled OOD-dynamics methods could improve sample complexity, with meta-learning and physics foundation-model fine-tuning offering adaptation to unseen dynamics.Prior studies examine OOD domain size, forcing, PDE parameters, initial conditions, and rollout length, but principled approaches remain a gap.Source: §9.2.9 · pp. 187–188
  • Three-dimensional PDE modeling poses memory and optimization challenges and can introduce more chaotic dynamics, motivating scalable solvers with suitable inductive biases and sufficient expressiveness.The energy cascade can make three-dimensional turbulent flows chaotic to a degree unseen in two-dimensional flows.Source: §9.2.9 · p. 188

9.3 Inverse Problem and Inverse Design

  • Inverse problems recover unknown PDE configurations from observations, whereas inverse design optimizes configuration parameters to meet a predefined objective.The unknowns or design variables can include initial conditions, boundary conditions, or PDE parameters.Source: §9.3 · pp. 188–189, Figure 36 · p. 189

9.3.1 Problem Setup.

  • A fixed PDE solver or differentiable learned forward model can support configuration optimization; learned models may require physical-consistency constraints.The formulation distinguishes recovering unknowns from observed, physically plausible data from optimizing a design that may not have an exact solution (Lu et al. 2021c).Source: §9.3.1 · pp. 189–190

9.3.2 Technical Challenges.

  • Inverse tasks inherit forward-modeling costs and can yield physically implausible adversarial solutions; inverse problems also face objective mismatch, ill-posedness, indirect observations, and physics-integration challenges.Jointly optimized forward models may trade PDE consistency for the inverse objective, while sparse observations can make solutions non-unique.Source: §9.3.2 · p. 192

Challenges for Inverse Design

  • Inverse design must represent hierarchical, heterogeneous design spaces and balance conflicting objectives whose relative importance can change across scenarios.The section gives rocket design and launch-stage objectives as examples of these challenges.Source: Challenges for Inverse Design · pp. 192–193

9.3.3 Existing Methods.

  • PINNs combine data and PDE-residual losses and can learn unknown parameters; hPINN enforces hard constraints, while CinDM composes diffusion models to design states and boundaries beyond training examples.The section also describes NeRF-based fluid grounding and system identification, and adjoint or backpropagation-based design methods; latent-space BPTT improves runtime and accuracy over input-space design (Wu et al. [2022b]).Source: §9.3.3 · pp. 193–194
  • Existing AI-assisted inverse-design methods remain limited to idealized scenarios, far from real-world systems with multi-scale physics, hierarchical designs, and diverse tasks.The section identifies deep reinforcement learning for fusion-plasma shaping as an example of feasibility on a complex physical system (Degrave et al. [2022]).Source: §9.3.3 · p. 195

9.3.4 Datasets and Benchmarks.

  • Inverse-design studies use domain-specific datasets, but no standard benchmark supports systematic comparison, and current datasets fall short of real-world physics and design complexity.The section also notes a benchmark suite of 12 full waveform inversion datasets (Deng et al. [2022]).Source: §9.3.4 · p. 195

9.3.5 Open Research Directions.

  • Future work should improve inverse-problem uncertainty quantification and training, and develop inverse-design representations, optimization methods, and approaches that generalize across domains.The proposed design methods must address hybrid discrete and continuous variables in complex design spaces.Source: §9.3.5 · p. 195

10 RELATED TECHNICAL AREAS OF AI

This section surveys interpretability, out-of-distribution generalization, self-supervised and foundation models, language-guided discovery, and uncertainty quantification as technical areas relevant to scientific AI. It covers approaches and challenges across scientific systems, including molecular and protein tasks, PDE solvers, multimodal language models, and quantum systems.

10.1 Interpretability

  • Instance-level interpretability seeks to identify which input-graph components drive each prediction, supporting trust, physical-plausibility checks, scientific insight, and compound design.For geometric deep learning models, explainable AI can help domain experts assess whether predictions reflect physical laws and reveal learned mechanisms that may guide future research.Source: §10.1 · p. 196

10.1.1 Existing XAI Methods.

  • Graph XAI includes gradient/feature, perturbation, decomposition, and surrogate methods, but explanations for geometric and equivariant models remain incomplete.Existing GDL work analyzes architectures and representations, while a 3D-point perturbation method uses learned noise for importance scores but does not address invariant or equivariant explanations; more techniques are needed.Source: §10.1.1 · pp. 196–197

10.1.2 Potential Application Scenarios.

  • XAI can verify scientific consistency, expose model errors, support knowledge discovery, and guide drug, material, and protein design.Examples include checking conservation laws or fermion antisymmetry, identifying molecule substructures or protein residues tied to properties, and locating protein–ligand features relevant to binding affinity.Source: §10.1.2 · pp. 197–198

10.2 Out-of-Distribution Generalization

  • OOD shifts between training and test distributions can degrade scientific prediction; adaptation uses target samples, whereas generalization targets unseen domains.Causal and invariant-learning approaches use environment partitions and assumptions about invariant features, while scientific OOD methods remain relatively underexplored. Benchmarks including OGB, Wilds, GOOD, DrugOOD, and CardioTox provide datasets and evaluation schemes.Source: §10.2 · p. 199, §10.2.1 · pp. 199–200, §10.2.9 · p. 203

10.2.2 OOD in AI for Quantum Mechanics

  • Quantum and molecular OOD challenges include shifts in system size, geometry, and chemical space, while DFT models trained on tens of atoms may face systems with hundreds or thousands.Approaches include shared interaction encodings, supplemental QM descriptors, size-invariant graph representations, and training across sizes using size as an environment; quantum tensor size shifts still lack a general solution. Caro et al. [2022] prove unitary-learning OOD generalization across broad training and test distributions, including learning from product states to act on entangled states.Source: §10.2.2 · pp. 200–201, §10.2.3 · p. 201, §10.2.4 · pp. 201–202

10.2.5 OOD in AI for Protein Science

  • Protein, materials, and interaction OOD work targets scarce data and unfamiliar structures, with domain knowledge, transferable representations, and specialized learning methods among proposed responses.Examples include protein sequence generation and ensemble robustness, materials transfer learning and applicability domains, and subpocket-level, domain-invariant, or meta-learning methods for molecular interactions and dark gene families.Source: §10.2.5 · p. 202, §10.2.6 · p. 202, §10.2.7 · pp. 202–203

10.2.8 OOD in AI for Partial Differential Equations

  • NCLaw generalizes from one motion trajectory to new geometries, conditions, temporal ranges, and multiphysics systems, with orders-of-magnitude gains on typical OOD tasks.It embeds rotation equivariance and undeformed-state equilibrium priors in differentiable simulation; future work can exploit causal factors and physical knowledge, and known shifts such as orientation shifts may be addressed with SE(3)-equivariant models.Source: §10.2.8 · p. 203, §10.2.10 · p. 203

10.3 Foundation and Large Language Models

  • Self-supervised learning and foundation models address scarce scientific labels by learning from unlabeled data, while large language models offer flexible knowledge transfer and generative applications.Scientific labels can be expensive or experimentally difficult to obtain; self-supervised models can learn data priors such as physical rules and symmetries, and pretrained foundation models can adapt to tasks with limited labeled data.Source: §10.3 · p. 204

10.3.1 Self-Supervised Learning.

  • Self-supervised learning spans contrastive and predictive tasks, from molecular and protein representation learning to physics-informed PDE solvers that reduce reliance on costly labeled simulations.For 3D molecules, methods align 2D and 3D representations or train models to estimate noise added to atom coordinates. Physics-informed DeepONet can generalize across PDE families and out-of-distribution regimes, and was reported to outperform its supervised counterpart.Source: §10.3.1 · pp. 204–205

10.3.2 Single-Modal Foundation Models.

  • Protein foundation models support structure prediction, conditional or unconditional generation, and interaction prediction, while molecule models span graph and string representations without a dominant non-language-based model.SMILES-based generation can produce chemically invalid strings; DeepSMILES and SELFIES were proposed to address validity, with SELFIES using a formal grammar. Protein language models also support design from sequences.Source: §10.3.2 · pp. 206–207

10.3.3 Natural Language-Guided Scientific Discovery.

  • Language-guided discovery aims to provide high-level control and connect scientific modalities, with modality choice depending on the task and multimodal data availability.Bi-encoders separately encode text and another modality for cross-modal retrieval and can reuse single-modal models; joint-representation models support prediction or, with decoders, cross-modal generation and molecule editing.Source: §10.3.3 · pp. 207–209
  • Scientific LLM adaptation uses domain-specific pretraining, fine-tuning, or few-shot and zero-shot prompting, with task formulation and domain context central to effective use.These approaches trade off training-data requirements, model access, and flexibility; tools can also provide external knowledge, while scientific modalities and limited domain data complicate adaptation.Source: §10.3.3 · pp. 207, 210, 211, 212, Figure 39 · p. 210

10.3.4 Open Research Directions.

  • Open directions include acquiring diverse, realistic scientific data, building robust SSL across modalities and distribution shifts, and extending foundation models and SSL to underexplored quantum systems.Quantum learning often centers on wavefunctions and lattice-specific architectures, which hinder transfer; unlabeled data may encode symmetry and physical rules that SSL could learn as priors. LLMs may flexibly transfer knowledge across systems and modalities.Source: §10.3.4 · pp. 212–213

10.4 Uncertainty Quantification

  • Reliable uncertainty quantification in physics-constrained predictive and generative models remains necessary for robust decisions, but practical integration requires efficient, scalable inference.The passage identifies approximate Bayesian computation and variational inference as methods needing development to meet predictive and computational requirements.Source: §10.4 · p. 214

10.4.1 Uncertainty Quantification: Introduction and Background.

  • Aleatoric uncertainty arises from irreducible process randomness, whereas epistemic uncertainty reflects limited knowledge and can diminish with additional data or knowledge.UQ addresses uncertainties from omitted system factors, model misspecification, stochastic or non-stationary dynamics, noisy or corrupted observations, and finite-precision computation. Quantifying prediction uncertainty can improve reliability and decisions in safety-critical applications; aleatoric UQ also helps prevent overfitting, while online learning can use uncertainty for discovery.Source: §10.4.1 · pp. 214–215

10.4.2 Uncertainty Quantification in Computational Science.

  • Computational UQ distinguishes forward propagation of input randomness to outputs from inverse calibration of uncertain model parameters or physical inputs.Forward methods include Monte Carlo sampling, Taylor approximations, and surrogate models. Bayesian inverse UQ updates parameter or input distributions to form posterior beliefs; alternatives include probabilistic, interval, confidence-score, distance-based, and prior-robust game-theoretic UQ approaches.Source: §10.4.2 · pp. 215–216

10.4.3 Uncertainty Quantification in Deep Learning.

  • Deep-learning UQ includes Bayesian neural networks, scalable approximate inference, and evidential learning, which models lack of evidence and can detect out-of-distribution data.Bayes-by-backprop, MC dropout, deep ensembles, and Bayesian last layers trade inference cost against uncertainty estimation; evidential methods penalize predictions unsupported by evidence. Conformal prediction and quantile regression offer interval-based alternatives.Source: §10.4.3 · pp. 217–218
  • Graph UQ methods model weight, topology, or structural uncertainty, but uncertainty estimation and analysis remain under-explored in graph learning.Examples include GDC, BGCNN, VGCN, GPN, GPGC, and conformalized GNNs; VGCN can improve robustness to graph-structure perturbations, and GPN supports OOD node detection.Source: §10.4.3 · pp. 219–220

10.4.4 Uncertainty Quantification in AI for Science.

  • Scientific AI UQ applications span molecular properties, compound-protein binding, ground-state density, PDE surrogates, and seismic inversion.For PDE problems, a study of PINNs and DeepONets found uncertainty indicative of prediction error and useful for OOD detection; its proposed GAN-plus-GP functional prior uses historical data to reduce computational cost [Psaros et al. 2023].Source: §10.4.4 · p. 221
  • Scientific AI studies show UQ can flag errors, noise, and out-of-distribution inputs, but no method consistently leads across tasks and setups.For molecular properties, MVE and MC Dropout assess data quality [Ryu et al. 2019], SVGD-based BNNs can mitigate dataset bias and support active learning [Zhang et al. 2019a], EDL uncertainty correlates with prediction error, and MVE and deep ensembles identify noise and OOD data. For DFT-related tasks, uncertainty helps identify inaccurate electron-density predictions [Fowler et al. 2019] and problematic test structures [Mahmoud et al. 2020]; a GP appended to a CNN performed best in one adsorption-energy benchmark [Tran et al. 2020], while localized neural kernel fulfilled six proposed force-field UQ desiderata [Wollschläger et al. 2023]. PDE studies found uncertainty indicative of prediction error and useful for OOD detection [Psaros et al. 2023]. Benchmarks favor different methods depending on tasks and setups [Scalia et al. 2020; Hirschfeld et al. 2020; Greenman et al. 2023; Li et al. 2023b]. UQ can also support active learning and Bayesian experimental design for data-efficient training and molecular discovery [Zhang et al. 2019a; Soleimany et al. 2021; Hie et al. 2020].Source: §10.4.4 · pp. 221–222

10.4.5 Open Research Directions.

  • UQ evaluation remains difficult because scientific AI surrogates stand in for costly physics-based models, making comprehensive stochastic testing computationally prohibitive.The section calls for high-fidelity benchmark datasets and standardized evaluation metrics with guaranteed performance.Source: §10.4.5 · p. 222
  • Future scientific UQ needs methods that encode domain knowledge and scale to large models while balancing computational cost against uncertainty quality.Existing methods may perform inconsistently across tasks and metrics; MC dropout can be simplistic, while deep ensembles and BNNs can be expensive. Approximation, stochastic-gradient, variational, and subspace-based approaches are suggested as avenues for scaling.Source: §10.4.5 · p. 222

11 LEARNING, EDUCATION, AND BEYOND

The section surveys foundational AI and science learning resources and argues that AI for Science needs cross-disciplinary collaboration, a broader community, richer education, and coordinated knowledge-building. It identifies fragmentation and the field’s rapid growth as reasons to develop more systematic resources and programs.

11.1 Existing Resources for Fundamental AI and Science

  • Foundational AI and science resources include books, courses, and computational libraries.The section lays out learning resources for individual fields, including recommended books and open-source resources.Source: §11.1 · p. 223, Table 35 · p. 226, Table 37 · p. 228

11.2 Paradigm Shifts in AI for Science

  • AI for Science should transcend disciplinary boundaries, combining scientific and AI perspectives to support principled advances and discovery.Examples include interpreting neural networks with physical laws and designing generative models using dynamical systems, control, and optimal transport.Source: §11.2 · p. 223, Figure 41 · p. 224
  • AI-related articles in ACS Omega rose 133% from 2020 to 2021, as workshops and symposiums expand dialogue across communities.The section calls for an agile community spanning students, researchers, and practitioners, with local and global collaboration.Source: §11.2 · p. 224, Figure 42 · p. 225
  • AI for Science education is expanding through summer schools, blogs, tutorials, and reading groups, but still needs structured programs across fundamentals and applications.Rapid methodological change and the field’s interdisciplinary nature make it difficult to curate a comprehensive curriculum.Source: §11.2 · p. 224

11.3 Prospective and Proposed Actions

  • Because resources often operate independently, the field needs a unified roadmap, collaborative community platforms, and effective knowledge collection methods.The section encourages individuals to contribute expertise to AI for Science subareas and calls for collective effort.Source: §11.3 · p. 225

12 CONCLUSION

The paper presents a technical, unified review of AI for science, organized by the spatial and temporal scales used to model the physical world. It surveys approaches, challenges, datasets, benchmarks, and future directions, while recognizing that the evolving field makes the review noncomprehensive.

12 CONCLUSION

  • The review organizes AI-for-science areas by spatial and temporal scale, specifying problem setups and discussing key challenges for applying AI.It surveys major approaches and evaluation datasets and benchmarks, then summarizes remaining challenges and future directions for each area.Source: §12 · p. 229
  • The paper compiles categorized learning and education resources to support engagement with AI for science.Source: §12 · p. 229
  • The authors describe the review as noncomprehensive and nonconclusive, and expect to add topics as the field develops while welcoming community feedback.Source: §12 · p. 229

A CLASSIFYING AND COMPUTING IRREDUCIBLE REPRESENTATIONS

The section explains how irreducible representations can be computed and classified: commuting operators enable decomposition, while characters and highest weights identify irreducibles in finite-group and semisimple Lie-algebra settings.

Computing irreducible representations

  • A nullspace computation finds operators commuting with the representation; eigenspaces of a random linear combination yield an irrep decomposition with high probability.Schur’s lemma motivates the method: for a unique copy of an irrep, each commuting operator restricts to a scalar multiple of the identity or zero.Source: A CLASSIFYING AND COMPUTING… · p. 232

Classifying irreps for Finite Groups

  • For finite groups, characters identify representations up to isomorphism and are constant on conjugacy classes; decomposing the regular representation yields all irreps.The regular representation can be constructed from the group’s multiplication table.Source: A CLASSIFYING AND COMPUTING… · p. 232

Classifying irreps for the Semisimple Lie Groups

  • Semisimple Lie-algebra representations decompose into irreps, and a Cartan subalgebra’s commuting elements split representation spaces into weight spaces that support classification.The nonzero weights of the adjoint representation are roots.Source: A CLASSIFYING AND COMPUTING… · pp. 234–235
  • Dominant integral weights classify the irreducible representations of finite-dimensional semisimple Lie algebras: each irrep has a unique such highest weight, and each such weight determines an isomorphism class.For so(3), these labels are nonnegative half-integers; only integer labels correspond to representations of SO(3).Source: A CLASSIFYING AND COMPUTING… · pp. 235–236

Related papers

Loading 2307.08423v6…