SchNet - a deep learning architecture for molecules and materials

Motivation

• Accurate quantum-chemical calculations are too costly for exploring the vast chemical space of molecules and materials.Machine-learning approaches reduce the required reference calculations, but earlier methods often rely on handcrafted descriptors or lack atomic positions.Source: §I · p. 1

Contribution

• SchNet models complex atomic interactions using continuous-filter convolutions and learned representations of atom types and positions.The architecture is a variant of DTNNs and uses filter-generating neural networks to incorporate chemical knowledge and constraints.Source: §I · p. 1, §II · p. 2
• Rotational, translational, and atom-indexing invariance are built into SchNet, supporting predictions across compositional and configurational chemical space.These symmetries are captured by design rather than learned only from examples.Source: §I · p. 1
• SchNet incorporates periodic boundary conditions through its filter-generating networks, enabling accurate formation-energy predictions for diverse bulk crystals.The filters can encode periodic structure while preserving the required property invariances.Source: §I · p. 1

Applications

• SchNet enables efficient molecular-dynamics studies by predicting potential-energy surfaces and energy-conserving force fields.The introduction reports nanoseconds of low-temperature PIMD completed within hours instead of years for an exemplary C20-fullerene study.Source: §I · pp. 1–2

Atom-wise layers

• Shared atom-wise layers transform each atom representation independently while preserving scalability with respect to the number of atoms.The layers also process environmental information incorporated through interaction layers.Source: B. Atom-wise layers · p. 2

Interaction blocks

• Interaction blocks add refinements based on pair-wise interactions with surrounding atoms using continuous-filter convolutional layers.These convolutions generalize discrete convolutions to atomistic systems whose positions are not arranged on a regular grid.Source: C. Interaction blocks · p. 2
• A continuous-filter convolution and three atom-wise layers form the residual mapping of an interaction block.The overall architecture and interaction-block organization are illustrated in Fig. 1.Source: C. Interaction blocks · p. 2

Activation functions

• Shifted-softplus activations satisfy ssp(0) = 0 and have infinite-order continuity, supporting smooth potential-energy surfaces, force fields, and second derivatives.The shift is described as improving convergence.Source: C. Interaction blocks · p. 3

Filter generation

• The filter-generating network maps the vector from an atom to a neighbor to filter values that determine how their interaction is modeled.This fully connected network can constrain the model and incorporate chemical knowledge.Source: D. Filter-generating networks · p. 3

Distance representation

• Rotational invariance is included by computing pairwise distances rather than using relative positions.The distances are then expanded in a Gaussian basis.Source: §1 · p. 3
• Gaussian centers are placed uniformly between zero and the distance cutoff, while the number of Gaussians and γ determine filter resolution.The work uses a grid spacing and scaling parameter γ of 0.1 Å for all models.Source: §1 · p. 3

Periodic boundary conditions

• Periodic repetitions receive equivalent atom-wise feature vectors, so periodic boundary conditions are applied directly to the filters.This accurately describes atom interactions while keeping predictions invariant to the choice of unit cell.Source: §2 · p. 3

Periodic filters

• The periodic filter sums contributions from all periodic images within the cutoff and can be normalized by the number of nearby atoms.Generated filters reflect the lattice structures of diamond and graphite.Source: §2 · p. 3

Property prediction

• SchNet predicts molecular or material properties from atom-wise contributions by summing extensive properties or averaging intensive properties.This construction retains invariance to atom indexing.Source: E. Property prediction · p. 3

Force prediction

• Atomic forces are obtained by differentiating the predicted energy with respect to atomic positions.With a rotationally invariant energy model, this produces rotationally equivariant and energy-conserving force predictions.Source: E. Property prediction · pp. 3–4

• SchNet models each property target by minimizing a squared loss.Source: F. Training · p. 4
• Energy and force trajectory models use a combined loss that balances energy and force errors through ρ.ρ controls the trade-off between the two loss components.Source: F. Training · p. 4
• Models use mini-batch stochastic gradient descent with ADAM and batches of 32 examples.The learning rate decays exponentially by a ratio of 0.96 every 100,000 steps.Source: F. Training · p. 4
• Data are split into training, validation, and test sets, with validation used for early stopping.The remaining data are used to compute test errors.Source: F. Training · p. 4
• Precomputing nearby-atom indices makes each training step scale linearly with system size within the cutoff.Source: F. Training · p. 4

• QM9 experiments train SchNet on 131k small organic molecules with up to nine heavy atoms, using T = 6 and 64-dimensional atomic representations.Source: A. Learning molecular properties · p. 4
• SchNet reaches or improves over enn-s2s on 8 out of 12 QM9 properties where atomic decomposition is appropriate.Its predictions of polarizability α and electronic spatial extent ⟨R2⟩ are noticeably less accurate.Source: A. Learning molecular properties · p. 4
• Interpretable property-specific output layers, such as for the dipole moment, remain future work.Source: A. Learning molecular properties · p. 5
• 0.014 eV is the U0 mean absolute error for SchNet with T ∈ {3, 6} interaction blocks and 110k training examples.With T = 2, the error is 0.015 eV; the figure compares learning curves against best-performing DTNN models with T = 3.Source: Figure 3 · p. 4, A. Learning molecular properties · p. 5
• T = 6 produces the lowest U0 errors when training on fewer examples, despite having the most parameters.With 110k examples, it also reduces convergence from 2400 epochs for T = 2 to fewer than 750 epochs.Source: A. Learning molecular properties · p. 5

Formation energies

• SchNet predicts formation energies for 69,640 Materials Project crystal structures spanning 89 atom types up to Z = 94.The model uses six interaction blocks, 64-dimensional atomic representations, and a 5 Å distance cutoff.Source: B. Learning formation energies of… · p. 5

Learned atom embeddings

• Figure 4 visualizes the two leading principal components of sp-atom embeddings, with colors denoting periodic-table groups.The projection explains only about 20% of embedding variance, so overlapping points may still be distant in the full space.Source: Figure 4 · p. 5, B. Learning formation energies of… · p. 5
• The learned atom embeddings group elements according to periodic-table groups and often order them from lighter to heavier elements.This structure was inferred from geometries and formation-energy targets rather than imposed on the model.Source: B. Learning formation energies of… · p. 5, Figure 4 · p. 5

Local chemical potentials

• Local chemical potentials are computed by adding a virtual probe atom that acts as a test charge to the SchNet input.The probe enables visualization of the learned representation in its molecular environment.Source: C. Local chemical potentials · p. 5, Figure 5 · p. 6
• For QM9 molecules, DTNN and SchNet local potentials capture bond saturation and varying aromaticity, while SchNet produces sharper feature separation.The distinguishing molecular features remain remarkably robust across the two neural-network architectures.Source: C. Local chemical potentials · p. 6, Figure 5 · p. 6
• The relationship between inferred local potentials and traditional observables such as electron density, electrostatic potentials, and orbitals remains unresolved.The paper identifies chemical-reactivity trends as a possible future application.Source: C. Local chemical potentials · p. 6
• SchNet local-potential cuts for graphite and diamond resemble the periodic structures of the corresponding solids.In solids, these potentials could help investigate defect formation and distribution, including vacancies and interstitials.Source: C. Local chemical potentials · p. 6, Figure 6 · p. 6

Benchmark setup

• SchNet predicts potential-energy surfaces and force fields on the MD17 benchmark of eight small organic molecules.Experiments use 1,000 or 50,000 reference calculations and compare against GDML and DTNN.Source: D. Combined learning of energies and… · p. 6, Table III · p. 7, Table IV · p. 7

Combined training

• On small training sets, adding force information reduces SchNet energy and force errors by 12 orders of magnitude versus energy-only training.With 50,000 examples, the improvement is less apparent because the potential-energy surface is already well sampled.Source: D. Combined learning of energies and… · p. 7

Comparison with GDML

• With 1,000 reference calculations, SchNet outperforms GDML on malonaldehyde and ethanol, whereas GDML achieves lower force errors on trajectories containing aromatic rings.The authors suggest SchNet’s indexing invariance may benefit more flexible molecules.Source: D. Combined learning of energies and… · p. 7

Accuracy and scalability

• SchNet achieves energy and force MAEs below 0.12 kcal/mol and 0.33 kcal/mol/Å, respectively.These results are reported for the tested molecular-dynamics trajectories and prediction setup.Source: D. Combined learning of energies and… · p. 7

Combined training

• SchNet performs better with combined energy-and-force training on 1,000 calculations than with energy-only training on 50,000 examples.The comparison supports the value of force information when reference calculations are limited.Source: D. Combined learning of energies and… · p. 7

Model and reference data

• SchNet was trained on DFT reference data to predict energies and forces for C20-fullerene molecular dynamics.The reference calculations used PBE with Tkatchenko-Scheffler van der Waals interactions.Source: E. Application to molecular dynamics… · pp. 7–8
• The C20 model used 20,000 training references and 4,500 examples for early stopping, with test errors reported on the remaining data.Model selection varied interaction blocks, feature dimensions, and the energy-force trade-off.Source: E. Application to molecular dynamics… · p. 8, Table V · p. 8

Potential-energy surface

• The selected model reproduced DFT normal-mode frequencies with a largest error of approximately 1%.The authors conclude that the model successfully reconstructs the C20 potential-energy surface and its symmetries.Source: E. Application to molecular dynamics… · p. 8, Figure 7 · p. 8
• The force-only model was selected to drive molecular dynamics because the study focused on the fullerene's mechanical properties.Differentiating the energy model still guarantees an energy-conserving force field.Source: E. Application to molecular dynamics… · p. 8

Nuclear quantum effects

• At 300 K, nuclear quantum effects increased nearest-neighbor distances by less than 0.5%, while bond-length delocalization remained considerable.The peaks of the pair distribution function were reported to change little.Source: E. Application to molecular dynamics… · pp. 8–9
• 1.25 ns of PIMD reduced runtime relative to DFT by 3–4 orders of magnitude, from about 7 years to less than 7 hours.The simulations used SchNet@DFT and included nuclear quantum effects through PIMD.Source: E. Application to molecular dynamics… · pp. 8–9, Figure 8 · p. 8

Conclusions

• SchNet learns task-adapted representations from first principles rather than requiring painstaking design of mechanistic force fields or descriptors.The architecture is applied from property prediction across chemical space to force fields for individual molecules.Source: §IV · p. 9
• SchNet models rotational, translational, and permutational invariances by design and can directly represent periodic boundary conditions.The paper also examines learned representations and atomic embeddings.Source: §IV · p. 9
• The C20 PIMD study at the PBE+vdWTS level would not have been computationally feasible with common DFT approaches.The authors identify larger molecules, periodic systems, and interpretable architectures as directions for future work.Source: §IV · p. 9

C20 simulation setup

• C20 reference data were generated with classical NVT molecular dynamics at 500 K using PBE plus the Tkatchenko-Scheffler method for van der Waals interactions.The reported timestep was 1 fs.Source: Appendix B · p. 9
• Nuclear quantum effects were introduced with path-integral molecular dynamics, where the number of beads controls their treatment.The simulations used a 0.5 fs timestep and an NVT path-integral Langevin thermostat.Source: Appendix B · p. 9

Error distributions

• QM9 error histograms include absolute prediction errors for all predicted properties and use a logarithmic scale to show distribution tails.The histograms aggregate test errors across all three repetitions.Source: Appendix A · p. 9, Figure 9 · p. 10
• The Materials Project error histogram shows absolute errors for formation energies per atom on a logarithmic scale.The logarithmic presentation is used to visualize the tails of the error distribution.Source: Figure 10 · p. 10