Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules

Niklas W. A. Gebauer; Michael Gastegger; Kristof T. Schütt

Source-linked AI summary

Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules

Niklas W. A. Gebauer, Michael Gastegger, Kristof T. Schütt

arXiv:1906.00957v3Jun 2, 2019stat.MLcs.LGphysics.chem-phphysics.comp-ph

TL;DR

Molecular generators based on graphs lack the spatial information needed to represent 3d structure-dependent effects, while exhaustive chemical-space search is infeasible. The paper introduces G-SchNet, which autoregressively generates rotationally invariant 3d point sets using symmetry-adapted spatial predictions, and applies it to targeted molecular discovery. It generates accurate and novel equilibrium structures and supports biasing toward molecules with small HOMO-LUMO gaps.

Problem
Graph-based molecular generators lack 3d spatial information for recognizing spatial isomerism and non-bonded interactions, while exhaustive chemical-space exploration remains infeasible.
Method
G-SchNet autoregressively generates atomic types and positions as rotationally invariant 3d point sets using distances to previously placed points and auxiliary tokens.
Results
G-SchNet generates accurate 3d equilibrium structures resembling training distributions, yields 79% unseen molecules, and produces over 3.6k novel structures with small HOMO-LUMO gaps.
Takeaways & Limitations
The model provides a strategy for targeted discovery of novel organic molecules with desired 3d structure-dependent electronic properties.
Takeaways & Limitations
Future work includes scaling to larger systems, directly conditioning on chemical properties, and incorporating periodic boundary conditions.

Abstract

from arXiv · show

Deep learning has proven to yield fast and accurate predictions of quantum-chemical properties to accelerate the discovery of novel molecules and materials. As an exhaustive exploration of the vast chemical space is still infeasible, we require generative models that guide our search towards systems with desired properties. While graph-based models have previously been proposed, they are restricted by a lack of spatial information such that they are unable to recognize spatial isomerism and non-bonded interactions. Here, we introduce a generative neural network for 3d point sets that respects the rotational invariance of the targeted structures. We apply it to the generation of molecules and demonstrate its ability to approximate the distribution of equilibrium structures using spatial metrics as well as established measures from chemoinformatics. As our model is able to capture the complex relationship between 3d geometry and electronic properties, we bias the distribution of the generator towards molecules with a small HOMO-LUMO gap - an important property for the design of organic solar cells.

1 Introduction

The paper introduces G-SchNet, an autoregressive generator of rotationally invariant 3d molecular point sets, to address the spatial limitations of graph-based generation. It generates accurate and novel equilibrium molecules and can target small HOMO-LUMO gaps.

Contribution: G-SchNet generates 3d point sets directly, incorporating Euclidean-space constraints and rotational invariance into the atom distribution.The model samples atomic types and positions rather than molecular graphs.
Motivation: Unlike graph-based models, the approach captures spatial information needed for spatial isomerism, local structures, and non-bonded interactions.It uses continuous-filter convolutions to model interactions among previously placed points.
Molecular generation: Applied to QM9, G-SchNet generates novel and accurate equilibrium molecules while capturing spatial and structural properties of the training distribution.The application uses organic molecules with arbitrary composition.
Targeted generation: The generator can be biased toward molecules with small HOMO-LUMO gaps, a property relevant to organic solar-cell design.This demonstrates targeting of a complex electronic property.
Generated datasets: The work introduces datasets of generated molecules absent from QM9 and verifies and optimizes them at the same theory level used for QM9.These datasets support subsequent analysis of novel generated structures.

2 Related work

Prior molecular generators mainly use SMILES strings or molecular graphs, while existing 3d point-cloud models target volumetric shapes and generally do not enforce rotational invariance. G-SchNet instead generates 3d structures directly and supports arbitrary compositions and property biasing.

3d point-cloud models: Existing 3d generative models typically represent volumetric shapes as point clouds, where exact individual-point placement is less important than density.Such architectures are not designed for the accurate relative placement required by molecular structures.
Graph- and string-based models: Molecular generators based on SMILES or graphs can produce novel molecular graphs but discard atomic 3d positions during generation.This omits spatial information associated with target properties.
G-SchNet: G-SchNet directly generates 3d structures without graph- or bond-based information and can be trained end-to-end for complex 3d-dependent electronic properties.The spatial representation avoids relying on heuristic bonds and rings.
Advances: Compared with earlier fixed-composition 3d-isomer generation, G-SchNet handles arbitrary atom compositions and uses auxiliary tokens to improve scalability and distribution robustness.The code and generated datasets are publicly available.

3 Symmetry-adapted factorization of point set distributions

G-SchNet factorizes point-set distributions into sequential conditional predictions whose coordinates are symmetry-adapted through distances to previously placed points. Auxiliary tokens, especially the focus point, constrain generation while preserving rotational and translational properties.

Point-set representation: The model represents a variable-size point set as positions and types, with auxiliary tokens supporting generation but excluded from the generated structure.For molecules, point types correspond to chemical elements.
Autoregressive factorization: The joint distribution is factorized into conditional probabilities so new points can be sampled step by step in an autoregressive process.The factorization also includes a stopping probability for variable-sized point sets.
Sequential sampling: At each step, the model samples an atom type and then its position, with the type prediction able to use positional information from the focus token.The focus token identifies a local neighborhood for the next point.
Auxiliary-token design: The focus point limits candidate positions to a fixed-size neighborhood, keeping the space considered at each generation step small as the point set grows.Figure 1 illustrates sampling an atom from two placed atoms together with focus and origin tokens.
Symmetry adaptation: New positions are constructed from distances to preceding points and tokens, producing translation-independent and rotation-equivariant positional probabilities.The distance is d(t+i)j = ||r_t+i − r_j||_2.

4 G-SchNet

G-SchNet generates molecules autoregressively by predicting atom types and distances while encoding rotational and translational symmetries through auxiliary tokens and symmetry-adapted coordinates.

Architecture: SchNet extracts atom-wise features invariant to rotation, translation, and atom ordering for molecular generation.Its architecture uses element embeddings and interaction blocks that update features from atomic environments.
Atom-type prediction: The network predicts each next atom type using element and stop-token embeddings followed by dense layers and a softmax.Feature vectors are copied for each possible type and combined with the corresponding embedding before classification.
Distance prediction: Distances from the new atom to preceding atoms are predicted as distributions discretized on a one-dimensional grid extending to 15 Å.The sampled atom-type features are reused to produce distance distributions for every input atom.
Symmetry adaptation: Symmetry-adapted distance coordinates make position probabilities translation- and rotation-independent, while a focus point keeps each prediction local.In Cartesian coordinates, these probabilities become rotationally equivariant; the focus point also limits the spatial region considered at each step.
Auxiliary tokens: Two auxiliary tokens resolve positional ambiguities: the focus token breaks local symmetry around the currently focused atom, and the origin token encodes global geometry.The origin token remains fixed near the molecule’s center of mass, whereas the focus token moves with each placement step.
Training: Training uses randomly sampled single-atom placement trajectories, with cross-entropy losses for atom types and Gaussian-expanded distance labels.Each training epoch samples a placement trajectory, and the distance targets are expanded over bins of the one-dimensional distance grid.

5 Experiments and results

G-SchNet generates valid, unique 3D molecules whose spatial and structural statistics resemble QM9, while distribution biasing shifts generated molecules toward smaller HOMO-LUMO gaps.

Experimental setup: G-SchNet was trained on 50k QM9 molecules and evaluated by generating 20k molecules filtered for validity and uniqueness.QM9 contains approximately 134k organic molecules with up to nine heavy atoms; about 77% of generated molecules were valid.
Accuracy of generated molecules: Generated structures are generally close to equilibrium, with unseen test molecules having a median RMSD of approximately 0.21 Å.The reported 0.39 Å comparison to another model is not directly comparable because it used a different evaluation procedure.
Structural properties: G-SchNet accurately captures QM9 atom and bond counts, while reproducing average ring counts better than CGVAE despite increased three- and four-membered rings.CGVAE has especially elevated five- and six-membered ring counts compared with the training distribution.
Structural properties: When trained without three- and four-membered rings, G-SchNet almost perfectly reproduces the filtered training distribution and generates no such rings.In this experiment, 77% of generated molecules were valid and 89% were unique.
Targeted discovery: Fine-tuning on QM9 molecules with HOMO-LUMO gaps below 4.5 eV shifts generated gaps downward, increasing the fraction below 5.0 eV from 7% to 43%.The biased molecules also show more double bonds and a tendency toward six-membered cycles, indicating conjugated and aromatic motifs.

6 Conclusions

The paper develops G-SchNet for symmetry-adapted generation of rotationally invariant 3D point sets and demonstrates accurate molecular generation and property-directed biasing. It produces novel equilibrium structures and datasets of molecules with targeted small HOMO-LUMO gaps.

Method: G-SchNet generates rotationally invariant 3D point sets while incorporating Euclidean-space constraints and local symmetries.It determines each next position using distances to previously placed points and uses SchNet point-wise features.
Molecular generation: The model generates accurate organic-molecule equilibrium structures that resemble the spatial and structural distributions of QM9.The paper reports 79% unseen molecules and introduces a dataset containing more than 9k novel organic molecules.
Targeted discovery: Fine-tuning on a small subset biases generation toward molecules with small HOMO-LUMO gaps and yields more than 3.6k novel structures with the desired property.The paper identifies this property as important for designing organic solar cells.
Limitations and future work: Future work includes scaling to larger systems, directly conditioning on chemical properties, and incorporating periodic boundary conditions.These directions define the stated scope boundaries for the current approach.

Supplementary material:

The supplementary material specifies the neural-network architecture, optimization procedure, and distance-label construction used in the experiments.

Architecture: The architecture uses 128 atom features, nine interaction blocks, and distances expanded with 25 Gaussian functions spanning 0 Å to 10 Å.These settings are used in the interaction and filter-generating layers.
Optimization: Training uses ADAM with an initial learning rate of 10^-4, halving after 10 unimproved validation epochs and stopping at 10^-6.The model with the lowest validation error is selected for generation.
Label construction: Atom-type labels come directly from training data, whereas distance-distribution labels are obtained through the stated construction for previously placed points.The supplementary passages identify the labels as qtype_i and qdist_ij and state the condition ∀j < t + i.
Label construction: The Gaussian width is set to 10% of the bin size, producing very peaky, unimodal label vectors.The width is controlled by the γ parameter.

Controlling randomness with the temperature parameter

The temperature parameter controls sampling randomness in G-SchNet, affecting how closely generated structures match relaxed geometries. The authors select T = 0.1 as the highest value that still produces structures close to equilibrium.

Temperature T controls sampling randomness: larger values smooth the grid distribution, while smaller values produce peakier, less-random distributions.
The study generated 20k molecules for each T ∈ {2, 1, 0.1, 0.01, 0.001}, then relaxed selected valid, unique molecules for RMSD evaluation.
RMSD medians and interquartile ranges were smallest for T values below 1, whereas T = 1 and T = 2 increased RMSD.
Decreasing T below 0.1 did not further reduce RMSD, so T = 0.1 was chosen for subsequent experiments.

Sampling generation traces for training

Generation traces initialize focus and origin tokens at the training molecule’s center of mass, then grow structures by selecting atoms relative to the origin and focused atoms.

Generation starts with focus and origin tokens at the training molecule’s center of mass.
The first atom is taken nearest the center of mass, while later atoms are selected from neighbors closest to the origin token.
At each subsequent step, one already placed non-token atom is randomly selected as the focus point.

Generating molecules

G-SchNet samples molecular atoms and positions on a three-dimensional grid, while fingerprint-based comparisons identify duplicate or matching structures. Figure 7 evaluates how the origin token affects generated molecular statistics.

The first atom is sampled from a 3d grid extending 1.7 Å in every dimension with 0.05 Å spacing.
At each generation step, G-SchNet samples atom type and position sequentially using the focus and origin tokens.
Molecular uniqueness and similarity to QM9 structures are assessed with FP2 Tanimoto similarity, followed by canonical SMILES comparison when similarity equals one.
Figure 7 compares 20k molecules generated with and without the origin token across property percentages and average atom, bond, and ring counts.

Ablation study

The ablation study shows that removing the origin token substantially changes validity and molecular-size and bond-count distributions, while biased generation shifts several targeted electronic-property distributions.

Origin-token ablation: Removing the origin token reduced generated-molecule validity by almost 20 percent and decreased matches to QM9 training and test structures.
Origin-token ablation: Only 13.7% of molecules generated without the origin token had 9 or fewer heavy atoms, compared with 60.4% when the token was included.
Origin-token ablation: Without the origin token, atom and bond counts diverged from the QM9 distribution, while ring counts were not noticeably better or worse overall.
Electronic-property biasing: Biasing G-SchNet toward isotropic polarizability, dipole moment, or electronic spatial extent produced clear shifts in the targeted-property distributions.

Detailed statistics

The paper reports detailed statistics for generated molecules across multiple G-SchNet variants, a constrained graph model, and related representations. These include validity, novelty, uniqueness, and distributions of quantum-chemical properties.

Table 1: Table 1 compares standard G-SchNet, a ring-excluded variant, a small-HOMO-LUMO-gap variant, and CGVAE using percentages of validly generated structures.The models generate 20k structures, with percentages reported among valid molecules.
Figure 8: Figure 8 compares QM9, unbiased G-SchNet, and property-biased G-SchNet distributions for three quantum-chemical properties.The biased models target larger values of the respective property.
Table 2: Table 2 reports valid, novel, and unique molecule percentages among 20k generated structures for G-SchNet and six graph- or SMILES-based models.Validity is based on atom valency constraints, while novelty and duplication are identified using molecular fingerprints and canonical SMILES strings.
Table 2: High novelty can result from many invalid structures, and high uniqueness is informative only when most generated structures are valid.The paper therefore cautions against interpreting novelty or uniqueness independently of validity.
Comparison scope: Direct comparison across 3d molecular generation, graph generation, and SMILES generation is limited because their valid outputs can correspond to different numbers of 3d conformers.G-SchNet outputs represent one proper 3d structure, whereas valid graphs or SMILES may require expensive quantum-chemical simulations to obtain conformers.