PaperlayerView exact revision on arXiv

Source-linked AI summary

Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules

Niklas W. A. Gebauer, Michael Gastegger, Kristof T. Schütt

arXiv:1906.00957v3stat.MLcs.LGphysics.chem-phphysics.comp-ph

TL;DR

Exhaustive chemical-space search is infeasible, while graph-based generative models lack spatial information for representing spatial isomerism and non-bonded interactions. G-SchNet generates rotationally invariant 3D point sets, and fine-tuning produced more than 3.6k novel structures with small HOMO-LUMO gaps.

  • Problem

    Exhaustive chemical-space search is infeasible, and graph-based generative models lack spatial information to represent spatial isomerism and non-bonded interactions.

    Source: §1 · pp. 1–2

  • Method

    G-SchNet autoregressively generates rotationally invariant 3D point sets, placing points according to learned conditional probabilities based on previously placed points.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    More than 3.6k novel structures with small HOMO-LUMO gaps were generated after fine-tuning G-SchNet toward the target property.

    Source: §6 · p. 9

  • Takeaways & Limitations

    G-SchNet demonstrates a promising strategy for targeted molecular discovery, including discovery of molecules with small HOMO-LUMO gaps relevant to organic solar cells.

    Source: §6 · p. 9

    Future work includes scaling G-SchNet to larger systems, conditioning directly on chemical properties, and incorporating periodic boundary conditions.

    Source: §6 · p. 9

Abstract

from arXiv · show

Deep learning has proven to yield fast and accurate predictions of quantum-chemical properties to accelerate the discovery of novel molecules and materials. As an exhaustive exploration of the vast chemical space is still infeasible, we require generative models that guide our search towards systems with desired properties. While graph-based models have previously been proposed, they are restricted by a lack of spatial information such that they are unable to recognize spatial isomerism and non-bonded interactions. Here, we introduce a generative neural network for 3d point sets that respects the rotational invariance of the targeted structures. We apply it to the generation of molecules and demonstrate its ability to approximate the distribution of equilibrium structures using spatial metrics as well as established measures from chemoinformatics. As our model is able to capture the complex relationship between 3d geometry and electronic properties, we bias the distribution of the generator towards molecules with a small HOMO-LUMO gap - an important property for the design of organic solar cells.

1 Introduction

G-SchNet generates molecular structures directly as rotationally invariant 3D point sets, rather than generating molecular graphs. The paper applies it to QM9 molecule generation and property-directed sampling, including biasing toward a small HOMO-LUMO gap.

1 Introduction

  • Unlike graph-based methods, which cannot distinguish spatial isomers with different 3D structures and properties, G-SchNet generates positions as well as atom types.Its point-based approach can capture local structures such as functional groups and non-bonded interactions without using bonds or rings as generation primitives.Source: §1 · pp. 1–2
  • G-SchNet autoregressively generates atom types and positions as 3D point sets, with rotational invariance and local symmetries built into its conditional probabilities.It uses SchNet continuous-filter convolutional layers to model interactions among already placed points.Source: §1 · p. 2
  • On QM9, G-SchNet yields novel, accurate equilibrium molecules while capturing spatial and structural properties of the training distribution.The generated molecules have arbitrary composition.Source: §1 · p. 2
  • G-SchNet can be biased toward complex electronic properties, including the HOMO-LUMO gap, an important property for organic solar-cell design.Source: §1 · p. 2
  • The work introduces datasets of generated molecules absent from QM9, verified and optimized at the same level of theory as QM9.Source: §1 · p. 2

2 Related work

Prior 3D generators either prioritize point-cloud shape density over precise molecular geometry or generate molecular graphs without atom positions. G-SchNet instead generates spatial structures directly, enabling end-to-end biasing toward 3D-dependent electronic properties.

2 Related work

  • Volumetric point-cloud generators emphasize shape density, not precise point placement, and are not designed to capture rotational invariance, limiting their fit to molecular structures.For molecules, accurate relative atom placement is the key requirement.Source: §2 · p. 2
  • SMILES and graph-based generators omit atom positions, while conformer generation from a fixed graph [44] leaves spatially dependent target properties absent from the intermediate representation.A two-step graph-generation and conformer-sampling approach would require both stages to be biased appropriately.Source: §2 · p. 2
  • G-SchNet directly generates 3D structures without graph or bond information, enabling end-to-end biasing toward complex, geometry-dependent electronic properties.Building on a factorization previously proposed for fixed-composition isomers [45], it supports arbitrary atom compositions and uses auxiliary tokens to improve scalability and distribution robustness.Source: §2 · p. 2

3 Symmetry-adapted factorization of point set distributions

The section factorizes point-set distributions into sequential conditional steps, with symmetry-adapted probabilities for types and positions. Auxiliary tokens, especially a local focus point, support generation of variable-sized sets while keeping each step’s spatial search bounded.

3 Symmetry-adapted factorization of point set distributions

  • Point sets contain a variable number of 3D positions paired with types, and auxiliary tokens are additional positions and types that support generation but are not part of the generated set.For molecules, the types correspond to chemical elements.Source: §3 · p. 3
  • The joint distribution over positions and types is factorized into conditional probabilities for sequential point generation, with symmetry adaptation including permutation invariance and rotation and translation equivariance.The conditional distributions can also capture local symmetries, such as rotations of functional groups.Source: §3 · p. 3
  • A focus token is selected from already placed points, and restricting the next point to its neighborhood limits candidate positions to a fixed-size region.The next type is sampled before its position, and the focus token lets type sampling use positional information; a stop probability accommodates differently sized sets.Source: §3 · p. 3
  • Pairwise distances to preceding points and tokens define position probabilities that are translation-independent and yield rotationally equivariant probabilities in 3D.The focus point keeps the spatial region considered at each step small regardless of generated set size.Source: §3 · p. 3

4 G-SchNet

G-SchNet adapts SchNet to autoregressively generate molecular 3D point sets, predicting atom types and distances from rotation- and translation-invariant features. Focus and origin tokens guide placement, while training uses sampled atom-placement trajectories and cross-entropy losses.

4.1 Neural network architecture

  • G-SchNet uses SchNet features to predict the next atom type and distances to existing points, producing rotationally and translationally equivariant placement distributions.The architecture uses nine interaction blocks and 128 atom features; type prediction uses five dense layers and a softmax, while distances are discretized up to 15 Å in bins of approximately 0.05 Å, with a temperature parameter controlling generation randomness.Source: §4.1 · pp. 4–5

4.2 Auxiliary tokens

  • The focus token breaks placement symmetry and prevents phantom positions, while the fixed origin token encodes global geometry and improves generated structures’ validity and training-data faithfulness.The focus token is placed on the currently focused atom; the origin token remains at the training molecule’s center of mass during generation.Source: §4.2 · p. 5

4.3 Training

  • Training samples a random atom-placement trajectory per epoch and uses cross-entropy losses for atom types and average distance predictions.Distance labels are formed by expanding distances to preceding atoms and tokens with Gaussians centered at bins on the one-dimensional distance grid.Source: §4.3 · p. 5

5 Experiments and results

G-SchNet generates 3D molecules close to equilibrium and captures key QM9 structural statistics. Fine-tuning on low-gap molecules shifts generated molecules toward smaller HOMO-LUMO gaps and semiconductor-related structures.

5 Experiments and results

  • About 77% of 20,000 generated molecules were valid; filtering non-unique structures removed approximately 10%.G-SchNet was trained on 50,000 molecules from QM9, with 5,000 used for validation and early stopping; QM9 contains approximately 134,000 molecules with up to nine heavy atoms.Source: §5 · p. 6

5.1 Accuracy of generated molecules

  • Around 0.21 Å was the median RMSD for both training and unseen test molecules against relaxed structures; novel molecules had slightly higher RMSDs.Carbon-carbon and carbon-oxygen radial distributions and bonded carbon-carbon-carbon and carbon-carbon-oxygen angle distributions aligned well with QM9. RMSD was calculated after energy minimization at the QM9 level of theory.Source: §5.1 · pp. 6–7

5.2 Structural properties of generated molecules

  • G-SchNet accurately matched QM9 atom and bond counts, and reproduced average ring counts better than CGVAE, though it generated more three- and four-membered rings.When trained on a QM9 subset without three- and four-membered rings, it again matched atom and bond statistics and almost perfectly reproduced ring counts, with no such small rings in generated molecules.Source: §5.2 · pp. 7–8, Figure 4 · p. 8

5.3 Targeted discovery of molecules with small HOMO-LUMO gaps

  • 43% of valid molecules generated after fine-tuning had a HOMO-LUMO gap below 5.0 eV, compared with 7% before biasing.Fine-tuning used QM9 equilibrium structures with gaps below 4.5 eV; among 20,000 generated molecules, validity was 69% and 74% of valid molecules were unique. Biased molecules showed more double bonds and a tendency toward six-membered cycles.Source: §5.3 · pp. 8–9, Figure 5 · p. 8

6 Conclusions

G-SchNet generates rotationally invariant 3d point sets and accurate equilibrium structures of organic molecules, and can bias generation toward molecules with small HOMO-LUMO gaps. The conclusions also identify scaling, direct property conditioning, and periodic boundary conditions as directions for future work.

6 Conclusions

  • G-SchNet generates rotationally invariant 3d point sets by using distances to previously placed points to determine the next position, producing an equivariant conditional probability distribution.It incorporates Euclidean-space constraints and spatial invariances, while SchNet point-wise features capture local symmetries.Source: §6 · p. 9
  • G-SchNet yields 79% unseen molecules, a dataset of >9k novel organic molecules absent from QM9, and >3.6k novel structures with small HOMO-LUMO gaps.Generated molecules closely resemble the training data’s spatial and structural distributions; supplementary experiments demonstrate biasing toward other electronic properties.Source: §6 · p. 9
  • Future work includes scaling G-SchNet to larger systems, directly conditioning on chemical properties, and including periodic boundary conditions.The authors expect applications in guided chemical-space exploration and crystal structure prediction.Source: §6 · p. 9

Supplementary material: Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules · Architecture · Training details

The supplement specifies a nine-block, 128-feature interaction architecture with separate type and distance heads, and describes the optimizer schedule and validation-based model selection.

Architecture

  • The feature extractor uses nine interaction blocks with 128 atom features; distances are expanded into 25 Gaussians centered evenly from 0 Å to 10 Å.The model reuses a 128-feature embedding layer across the steps shown in Fig. 1.Source: Architecture · p. 14
  • Both prediction heads use five atom-wise dense layers with shifted-softplus activation and a final softmax layer.The type head has 128, 96, 64, 32, and 1 features; the distance head has 128, 171, 214, 257, and 300.Source: Architecture · p. 14

Training details

  • Training uses stochastic gradient descent with the ADAM optimizer, starting at a learning rate of 10^-4 and halving it after 10 epochs without validation-loss improvement.Source: Training details · p. 14
  • Training stops when the learning rate reaches 10^-6 or lower, and the model with the lowest validation error is selected for generation.Source: Training details · p. 14
  • Atom-type labels qtype_i come directly from training data, while distance-distribution labels qdist_ij are obtained by a rule constrained by j < t + i.The supplied passage gives the condition but omits the rule itself.Source: Training details · p. 14
  • The Gaussian width parameter γ is set to 10% of the bin size, producing very peaky, unimodal label vectors.Source: Training details · p. 14

Controlling randomness with the temperature parameter

The temperature parameter controls sampling randomness, and RMSD comparisons led the authors to use T = 0.1 for their other experiments.

Controlling randomness with the temperature parameter

  • Increasing T smooths the grid distribution and increases randomness, whereas smaller T yields a peakier distribution and less randomness.The temperature was fixed at T = 0.1 for all experiments, following the described comparison procedure.Source: Controlling randomness with the… · p. 14
  • For five temperatures, G-SchNet trained on 50k QM9 equilibrium structures generated 20k molecules per setting, with 1k valid unique molecules selected for RMSD assessment.The selected molecules were relaxed at PBE/def2-SVP; 800 resembled test structures, 100 resembled training structures, and 100 were novel structures with more than 9 heavy atoms.Source: Controlling randomness with the… · p. 14
  • RMSD medians and interquartile ranges were smallest at T < 1; lowering T below 0.1 did not improve RMSD, so subsequent experiments used T = 0.1.The authors chose the highest temperature they found still produced structures close to equilibrium, while noting that very small T could reduce the number of unique samples.Source: Controlling randomness with the… · p. 14, Figure 6 · p. 15

Sampling generation traces for training

Training traces build each molecule from its center of mass, then iteratively place bonded neighbors from randomly selected unfinished atoms until all atoms are finished.

Sampling generation traces for training

Generating molecules · Matching molecules

Molecule generation grows 3D structures by sampling atom types and positions from G-SchNet predictions, then stops when all atoms are finished or a 35-atom limit is reached. Matching uses FP2 fingerprint similarity followed by canonical SMILES comparison, a conservative procedure that can filter out spatial isomers.

Generating molecules

  • The first atom is sampled from a 3D grid extending 1.7 Å in each dimension at 0.05 Å intervals, using G-SchNet predictions for atom type and two distance distributions.Grid-cell probabilities are calculated according to Eq. 5 to sample the atom position.Source: Generating molecules · p. 15
  • At subsequent steps, a random unfinished atom becomes the focus, and G-SchNet predicts the next atom’s type and position from a grid centered on that focus.The origin token remains at the original grid origin.Source: Generating molecules · p. 15
  • When G-SchNet predicts a stop token, the focused atom is marked finished without sampling a position, and generation continues from another unfinished atom.Generation ends when no unfinished atoms remain.Source: Generating molecules · p. 15
  • Structures not finished after 35 placed atoms are discarded as invalid; this affected about 1% of generated molecules.The reported experiments used this stopping limit.Source: Generating molecules · p. 15

Matching molecules

  • To detect duplicates and molecules resembling training or test data, the authors calculate pairwise Tanimoto similarity of FP2 path-based fingerprints using Open Babel.A similarity of one triggers comparison of canonical SMILES representations.Source: Matching molecules · p. 15
  • Molecules with matching canonical SMILES are treated as equal, making the matching procedure conservative because it also filters some spatial isomers that SMILES cannot distinguish.The SMILES comparison follows an FP2 similarity of one.Source: Matching molecules · p. 15

Ablation study · Targeted discovery with respect to further electronic properties

Removing G-SchNet’s origin token weakens the generated-molecule distribution’s fidelity to QM9, while property-biased models shift distributions toward larger polarizability, dipole moment, and electronic spatial extent. Fine-tuning uses high-property QM9 subsets, and separate SchNet predictors estimate those properties.

Ablation study

  • Almost 20 percent lower validity without the origin token accompanied only 13.7% of generated molecules having 9 or fewer heavy atoms, versus 60.4% with it.All QM9 structures have at most 9 heavy atoms.Source: Ablation study · p. 16
  • Without the origin token, significantly fewer generated molecules match QM9 training or test structures, and the model generates less faithful molecules despite equal uniqueness and novelty.The study concludes the origin token positively affects the approximated probability distribution and enables more valid molecules that better capture training-data characteristics.Source: Ablation study · p. 16
  • Atom and bond counts diverge from training data without the origin token, while ring counts are not noticeably better or worse and six-membered rings increase as smaller rings decrease.Both models slightly diverge from the QM9 training ring-count distribution.Source: Ablation study · p. 16

Targeted discovery with respect to further electronic properties

  • Biasing G-SchNet toward larger isotropic polarizability, dipole moment, or electronic spatial extent produces clear shifts in the corresponding property distributions.These experiments fine-tune a G-SchNet previously trained on 50k structures, using small QM9 subsets exhibiting each target property.Source: Targeted discovery with respect to… · p. 16
  • Fine-tuning subsets contain 2100/500 polarizability, 3000/500 dipole-moment, and 4400/500 spatial-extent molecules for training/validation, respectively.The thresholds are ≥91 Bohr3, ≥5.75 Debye, and ≥1785 Bohr2, respectively.Source: Targeted discovery with respect to… · p. 16
  • Separate SchNet predictors trained on QM9 yielded mean absolute test errors of 0.070 Bohr3 for polarizability, 0.016 Debye for dipole moment, and 0.126 Bohr2 for spatial extent.Each predictor used 100k molecules for training, 10k for validation, and the remaining structures for testing.Source: Targeted discovery with respect to… · p. 16

Detailed statistics

This section reports detailed statistics on generated molecules, comparing G-SchNet variants and CGVAE with related graph- and SMILES-based models. It also shows property distributions for unbiased and property-biased G-SchNet models.

Detailed statistics

  • Table 1 reports statistics for standard and ring-excluded G-SchNet, a small-gap fine-tuned model, and CGVAE [22].The G-SchNet models were trained on 50k randomly selected QM9 molecules, except the ring-excluded model; the small-gap model was fine-tuned on 3k molecules with gaps ≤4.5 eV.Source: Detailed statistics · pp. 16–17, Table 1 · p. 17
  • Figure 8 shows QM9, unbiased G-SchNet, and property-biased G-SchNet distributions for three quantum-chemical properties.The biased models were directed toward larger values of the respective property.Source: Figure 8 · p. 17
  • 77%, 78%, 69%, and 100% of 20k generated molecules were valid for the Table 1 models from left to right.The models are standard G-SchNet, ring-excluded G-SchNet, a small-gap fine-tuned model, and CGVAE [22].Source: Detailed statistics · pp. 16–17, Table 1 · p. 17
  • Table 2 compares valid, novel, and unique molecule percentages for G-SchNet and six graph- or SMILES-based models.High novelty or uniqueness can be misleading when validity is low, and direct comparison is limited because 3D structures and graph or string generation are different tasks.Source: Detailed statistics · p. 17, Table 2 · p. 18

Related papers

Loading 1906.00957v3…