Inverse design of 3d molecular structures with conditional generative neural networks

Niklas W. A. Gebauer; Michael Gastegger; Stefaan S. P. Hessmann; Klaus-Robert Müller; Kristof T. Schütt

Source-linked AI summary

Inverse design of 3d molecular structures with conditional generative neural networks

Niklas W. A. Gebauer, Michael Gastegger, Stefaan S. P. Hessmann, Klaus-Robert Müller, Kristof T. Schütt

arXiv:2109.04824v2Sep 10, 2021cs.LGphysics.chem-phphysics.comp-phstat.ML

TL;DR

Molecular inverse design requires guided exploration because chemical space is too large for exhaustive search and reference calculations are sparse. The paper introduces cG-SchNet, a conditional generative neural network that sequentially generates 3d molecules while targeting multiple properties, and demonstrates novel stable and electronically targeted structures beyond the training regime.

Problem
Inverse molecular design seeks structures with specified properties, but exhaustive exploration is infeasible and promising regions may have sparse reference calculations.
Method
cG-SchNet generates 3d molecules autoregressively by sequentially sampling atom types and positions conditioned on arbitrary combinations of structural and chemical properties.
Results
cG-SchNet generated novel motifs and compositions, recovered 169 of the 200 lowest-relative-energy test isomers, and sampled more than 800 low-energy molecules with HOMO-LUMO gaps below 4.5 eV.
Takeaways & Limitations
The model enables targeted, data-efficient exploration and systematic improvement of chemical databases using conditional sampling rather than target-specific biasing procedures.

Abstract

from arXiv · show

The rational design of molecules with desired properties is a long-standing challenge in chemistry. Generative neural networks have emerged as a powerful approach to sample novel molecules from a learned distribution. Here, we propose a conditional generative neural network for 3d molecular structures with specified chemical and structural properties. This approach is agnostic to chemical bonding and enables targeted sampling of novel molecules from conditional distributions, even in domains where reference calculations are sparse. We demonstrate the utility of our method for inverse design by generating molecules with specified motifs or composition, discovering particularly stable molecules, and jointly targeting multiple electronic properties beyond the training regime.

I. INTRODUCTION

cG-SchNet addresses inverse molecular design by conditionally generating 3d structures from desired chemical or structural properties, including in sparsely sampled regions. Its autoregressive architecture predicts atom types and positions sequentially while supporting multiple condition types and variable molecular sizes.

I. INTRODUCTION: Inverse design constructs molecular structures corresponding to specified properties, but exhaustive chemical-space exploration and connectivity-based representations remain limiting.Spatial conformations can differ in properties, while bonding ambiguity affects transition-metal complexes, conjugated systems, and metals.
I. INTRODUCTION: cG-SchNet learns target-dependent conditional distributions over 3d molecular structures and can generate molecules without requiring a target composition.The model is designed for arbitrary molecular size and learns relationships between composition and physical properties.
I. INTRODUCTION: The model supports jointly targeting multiple structural and chemical properties after training, including electronic properties, compositions, and molecular fingerprints.Explicit conditioning avoids target-specific retraining or indirect sampling constraints.
I. INTRODUCTION: Molecules are generated autoregressively by placing atoms one at a time, first sampling each atom type and then its distances to preceding atoms and auxiliary tokens.A stop type controls termination, allowing variable-size and variable-composition molecules.
I. INTRODUCTION: Distance-based position distributions provide equivariance to translation and rotation, while conditions are embedded and combined into conditional feature vectors.The architecture accepts scalar- and vector-valued conditions through suitable embedding procedures.

Generating molecules with specified motifs

cG-SchNet can target structural motifs using molecular fingerprints and generate molecules matching motifs from unseen test structures. It also supports composition- and property-conditioned exploration beyond the observed HOMO-LUMO gap range.

Generating molecules with specified motifs: The model conditions on 1024-bit path-based fingerprints encoding linear molecular-graph segments of up to seven atoms.Training used a randomly selected subset of 55k molecules from the QM9 dataset.
Generating molecules with specified motifs: For each unseen fingerprint target, cG-SchNet conditionally samples 20k molecules and compares their Tanimoto similarity with training structures and generated molecules.The figure distinguishes generated sets with and without duplicates and displays highly similar unseen examples.
Generating molecules with specified motifs: For three of four fingerprint targets, the model generates molecules with the same fingerprint as the training data, demonstrating generalization to unseen structural targets.The reported result concerns exact fingerprint matching for three of the four targets.

Generalization of condition-structure relationship

cG-SchNet generalizes conditional structure–property relationships to unseen compositions and sparsely sampled property regions, including simultaneous targeting of low energy and small HOMO-LUMO gaps. It generates novel, structurally diverse candidates while avoiding several unstable motifs and reducing the search burden relative to exhaustive enumeration.

Generalization beyond training distributions: Most generated C7N1O1H11 isomers matched target HOMO-LUMO gaps within ±1 eV, including values outside the training range.The model transferred relationships learned from molecules of other compositions to the target composition.
Discovery of low-energy conformations: 169 of the 200 lowest-energy C7O2H10 test isomers were recovered, alongside 416 novel isomers and 243 novel stereoisomers.Among isomers below −0.05 eV relative atomic energy, 67% of the 1k test structures were recovered.
Discovery of low-energy conformations: cG-SchNet generated 32% more unique C7O2H10 isomers below −0.05 eV relative atomic energy than were contained in QM9.Relative atomic energy normalizes energy per atom within composition, with negative values indicating comparatively higher stability.
Structural diversity: Twelve generated novel low-energy C7O2H10 isomers contained carboxylic acid groups, absent from the QM9 isomers of that composition.The model also discovered acyclic isomers with significantly lower relative atomic energy than those in QM9.
Targeting multiple properties: For simultaneous low-gap and low-energy targeting, cG-SchNet sampled significantly more unseen molecules from the low-energy domain than biased G-SchNet and the training set.The model used gap 4.0 eV and relative atomic energy −0.2 eV as conditioning values, while the comparison used 20k generated molecules.
Targeting multiple properties: The cG-SchNet low-gap candidates contained more double bonds and five- or six-membered rings, while avoiding strained three- or four-membered cycles.These patterns indicate aromatic and conjugated motifs; biased G-SchNet also produced less stable nitrogen-rich motifs.
Computational efficiency: Generation took 9 milliseconds per structure, compared with more than 15 minutes per structure for relaxation and property calculation.cG-SchNet proposed 5283 structures for relaxation, whereas exhaustive enumeration would require relaxing many more candidates.

III. DISCUSSION

cG-SchNet enables targeted generation of 3d molecules under combinations of chemical and structural conditions, including values unseen during training. Demonstrations include stable isomer discovery and accelerated exploration of sparsely represented electronic-property regions, while larger and more diverse systems remain future work.

III. DISCUSSION: cG-SchNet targets arbitrary combinations of multiple structural and chemical properties without target-specific biasing procedures.Explicit conditioning allows the model to learn from all available reference calculations and sample specific conditional distributions.
III. DISCUSSION: The model generalizes to unseen conditions and structures by capturing global and local molecular symmetries.The paper evaluates target property values not included in the training data.
III. DISCUSSION: Applying cG-SchNet to significantly larger systems, more diverse atom types, materials, and periodic systems requires further architectural and data adaptations.The discussion identifies atom-count scalability, limiting preceding-atom context, long-range interactions, training-data enrichment, and periodic boundary conditions as open issues.
III. DISCUSSION: cG-SchNet sampled particularly stable, low-energy C7O2H10 isomers and discovered motifs absent from QM9, including carboxylic acid groups.These results illustrate targeted exploration beyond the structures represented in the training database.
III. DISCUSSION: More than 800 low-energy molecules with HOMO-LUMO gaps smaller than 4.5 eV were sampled from a sparsely represented training-data domain.The model provides reasonable candidate structures and accelerates a process that could otherwise use exhaustive graph sampling followed by DFT evaluation.
III. DISCUSSION: The approach supports ML-driven, targeted exploration of chemical compound space and data-efficient improvement of chemical databases.The motivation includes the computational cost and unfavorable scaling of electronic-structure calculations.

A. Training Data

The training data and architecture encode target properties alongside partial 3d molecular structures. cG-SchNet uses property-specific embeddings, SchNet atom-wise features, and output networks that predict the next atom type and distances to preceding atoms.

A. Training Data: Each training run randomly samples 55k reference structures from QM9, which contains 133,885 molecules with up to nine heavy atoms.The data use carbon, nitrogen, oxygen, and fluorine, and 915 invalid molecules are removed from the training pool.
A. Training Data: The model input is a partial molecule of i −1 atoms, two auxiliary tokens, and k target properties Λ = (λ1, ..., λk).Atoms and tokens are represented by positions and types; the tokens are treated as ordinary neural-network inputs but do not affect molecule probability.
A. Training Data: SchNet extracts atom-wise features invariant to rotation and translation, using 128 features per atom and 9 interaction blocks.These features describe the partial molecular structure supplied to the conditional generator.
A. Training Data: Each target property is mapped into vector space by an embedding network specialized for scalar values, vector values, or atomic composition.Scalar properties use a Gaussian radial basis function expansion followed by an MLP, while vector-valued properties are directly processed by an MLP.
A. Training Data: Atomic composition is encoded by embedding the atom count and weighting learnable atom-type embeddings by target composition fractions before MLP processing.For hydrocarbons, the representation uses the hydrogen and carbon fractions nH and nC.
A. Training Data: An MLP aggregates the property feature vectors into combined conditional features y.The resulting conditional representation is supplied to the networks that generate the next atom.
A. Training Data: Given y and atom-wise features of the partial molecule, two output networks predict the next atom type and pairwise distance distributions.Type probabilities are obtained by softmax over atom types and averaging atom-wise predictions; distances are discretized into L bins with spacing ∆µ.

C. Sampling atom placement sequences for training

Training sequences are constrained with focus and origin tokens to reduce the factorial number of possible atom-placement orders. The resulting procedure grows molecules outward from the center of mass while following neighborhood relationships.

C. Sampling atom placement sequences for training: The number of atom-placement sequences grows factorially with molecule size, so the training procedure randomly samples one sequence per molecule in each epoch.Focus and origin tokens substantially reduce the number of possible sequences used by cG-SchNet.
C. Sampling atom placement sequences for training: At each step, a placed atom is selected as focus and the unplaced neighbor closest to the center of mass is chosen as the next atom.The focus token is moved to the selected atom before choosing the next placement.
C. Sampling atom placement sequences for training: When a focus atom has no unplaced neighbors, the type network predicts a stop marker before another focus is selected.Atoms marked finished cannot be selected as focus, and the sequence ends when all placed atoms are finished.
C. Sampling atom placement sequences for training: In the experiments, atoms sharing a bond define neighbors, although a radial cutoff such as 3 Å can also define neighborhood.Thus, bonding information is used for the reported sequences but is not required by the general neighborhood procedure.

D. Neural network training

Training uses mini-batches of sampled atom-placement sequences and minimizes cross-entropy losses for next-atom types and distances. ADAM optimization with learning-rate decay selects the model with the lowest validation error for generation.

D. Neural network training: Each mini-batch contains one randomly sampled atom-placement sequence for each of M molecules, including partial structures, target properties, and the next atom.The sequence elements provide the model inputs and ground-truth next type and position at every step.
D. Neural network training: For every atom placement, the model minimizes cross-entropy between predicted distributions and ground-truth next-atom type and position distributions.The next-type target is a one-hot encoding of Znext.
D. Neural network training: The distance loss averages cross-entropy over discretized distance distributions using Gaussian-expanded ground-truth distances.The Gaussian width is controlled by γ and is set to 10 ∆µ in the experiments.
D. Neural network training: The mini-batch loss averages type and distance losses over all atom-placement steps of all M molecules.The indicator δ is zero for stop-marker steps because those steps do not predict a position.
D. Neural network training: Training uses ADAM with initial learning rate η = 10^-4, halving after 10 epochs without validation improvement and stopping at η ≤ 10^-6.Mini-batches contain 5 molecules, and the model with lowest validation error is selected for generation.

E. Conditional generation of molecules

cG-SchNet generates 3d molecules conditionally by sequentially sampling atom types and positions, using a stop marker to finish atoms and terminate molecules.

Generation requires conditions covering all target properties used during training, such as atomic composition and relative atomic energy.
A molecule with n atoms requires 2n steps because each atom is placed and separately marked as finished.
At each step, cG-SchNet samples the next atom type and position from distributions conditioned on previously placed atoms and the specified properties.
The stop marker finishes the focused atom without sampling a position, after which generation selects another unfinished atom.
Candidate positions are obtained by aggregating distance probabilities near the focused atom and sampling from the resulting 3d distribution.
Generation terminates when all regular atoms are finished or when the model exceeds the 35-atom limit, in which case the molecule is invalid.

F. Checking validity and uniqueness of generated molecules

Generated molecules are converted into molecular graphs to assess chemical validity, while canonical SMILES representations identify duplicates, mirror images, and structures already present in the data.

Open Babel assigns bonds and bond orders to generated atom positions and types, enabling valence and connectivity checks.
A structure is invalid when any atom violates valence constraints or when the generated graph contains multiple disconnected components.
Canonical SMILES strings are used to determine whether generated molecules are duplicates and to match them against training and test structures.
Isomeric SMILES encode stereochemistry, while the C7O2H10 search additionally uses non-isomeric RDKit SMILES to identify novel stereoisomers.

G. Prediction of property values of generated molecules

Pretrained SchNet models predict key electronic and energetic properties of generated molecules, whose distributions are then examined alongside relaxed structures.

SchNet models predict the HOMO-LUMO gap, isotropic polarizability, and internal energy at zero Kelvin for generated molecules.
The reported prediction MAEs are 0.074 eV for the HOMO-LUMO gap, 0.124 Bohr3 for isotropic polarizability, and 0.012 eV for internal energy.
Predicted property values are used to plot distributions for the HOMO-LUMO gap, polarizability, and internal energy in the reported figures.
Generated molecules are relaxed in every experiment to assess how close they are to equilibrium structures.

X. SUPPLEMENTARY INFORMATION

The supplementary methods specify the spatial grid and sampling controls used for generation and define relative atomic energy as a composition-normalized stability measure.

Candidate positions lie on a 0.05 Å grid bounded by minimum and maximum distances from the focused atom.
For QM9 experiments, the neighboring-atom distance bounds are dmin = 0.9 Å and dmax = 1.7 Å.
The temperature parameter controls sampling randomness by smoothing the candidate-position distribution; the experiments use T=0.1.
The first atom uses a larger one-directional grid extending below 15 Å, with unsmoothed sampling at T=1.0.
Relative atomic energy subtracts the expected internal energy per atom for molecules with the same composition from a structure's internal energy per atom.
The expected composition-specific energy is predicted from atomic concentration using a linear regression model, while generated energies come from SchNet or ORCA depending on relaxation.

C. Calculation of fingerprints

The paper uses fingerprints to compare generated molecules with training structures and assess generated structures, conformations, and relaxation behavior. Supplementary analyses also report model settings and relaxation outcomes.

Fingerprint calculation: 1024-bit FP2 fingerprints capture linear fragments of up to seven atoms after converting 3d structures to SMILES.Fingerprints are calculated with Open Babel version 2.4.1.
Fingerprint-based comparison: Generated C7O2H10 molecules are compared with the most similar QM9 isomers using Tanimoto similarity of path-based fingerprints.The comparison focuses on novel, low-energy generated isomers and their closest QM9 counterparts.
Generated-structure analysis: The supplementary analyses examine carboxylic-acid-containing C7O2H10 isomers, multiple conformations, and structural changes before and after DFT relaxation.The conformational analysis samples 50 generated examples for each of five frequently generated isomers.
Model settings: cG-SchNet models use SchNet networks with 128 features, 9 interaction blocks, a 10 Å cutoff, and 25 radial-basis centers.Five models were trained with different target properties.
Relaxation evaluation: Relaxation results report validity, median RMSD before and after relaxation, and MAE in conditioned properties for 100 generated unseen molecules nearest to target values.Electronic-structure calculations used ORCA, with pre-optimization and final relaxation at specified density-functional levels.