Learning to design drug-like molecules in three-dimensional space using deep generative models

Yibo Li; Jianfeng Pei; Luhua Lai

Source-linked AI summary

Learning to design drug-like molecules in three-dimensional space using deep generative models

Yibo Li, Jianfeng Pei, Luhua Lai

arXiv:2104.08474v1Apr 17, 2021q-bio.QMcs.LG

TL;DR

Existing molecular generative models have mainly produced 2D structures, while direct generation of drug-like molecules with 3D structures remains limited. The paper introduces L-Net, an end-to-end graph generative model that directly outputs molecular topology and 3D coordinates, and reports chemically correct, conformationally valid, drug-like outputs, including potential ABL1 inhibitors when combined with MCTS.

Problem
Deep generative models have mainly generated topological structures, while direct generation of drug-like molecules with 3D structures remains limited.
Method
L-Net directly generates molecular topology and 3D structures end-to-end, using an architecture designed for drug-like molecules and evaluation metrics emphasizing 3D quality.
Results
L-Net generates chemically correct, conformationally valid, and drug-like molecules, and its combination with MCTS demonstrates potential for designing ABL1 inhibitors.
Takeaways & Limitations
The model provides a direct 3D generation approach for drug-like molecular design and a proof-of-concept application to structure-based ABL1 inhibitor design.
Takeaways & Limitations
L-Net is trained using 3D structures generated from RDKit rather than crystal structures.

Abstract

from arXiv · show

Recently, deep generative models for molecular graphs are gaining more and more attention in the field of de novo drug design. A variety of models have been developed to generate topological structures of drug-like molecules, but explorations in generating three-dimensional structures are still limited. Existing methods have either focused on low molecular weight compounds without considering drug-likeness or generate 3D structures indirectly using atom density maps. In this work, we introduce Ligand Neural Network (L-Net), a novel graph generative model for designing drug-like molecules with high-quality 3D structures. L-Net directly outputs the topological and 3D structure of molecules (including hydrogen atoms), without the need for additional atom placement or bond order inference algorithm. The architecture of L-Net is specifically optimized for drug-like molecules, and a set of metrics is assembled to comprehensively evaluate its performance. The results show that L-Net is capable of generating chemically correct, conformationally valid, and highly druglike molecules. Finally, to demonstrate its potential in structure-based molecular design, we combine L-Net with MCTS and test its ability to generate potential inhibitors targeting ABL1 kinase.

1 Introduction

Existing molecule generators have largely produced 2D structures, while direct generation of drug-like molecules with high-quality 3D structures remains limited. L-Net addresses this gap with an end-to-end 3D generative model and is reported to generate chemically correct, conformationally valid, drug-like molecules while supporting ABL1 inhibitor design.

Motivation: Most existing methods generate 2D molecular structures, whereas incorporating 3D information into deep generative models remains limited.3D information is relevant to docking and 3D-QSAR applications.
Limitations of prior work: Earlier 3D approaches focused on structurally simple or low-molecular-weight molecules, leaving their applicability to drug-like molecules unclear.Other approaches generate atom density maps and require multiple components to reconstruct final molecular structures.
Proposed approach: L-Net directly generates drug-like molecules with full 3D coordinates through an end-to-end iterative process that adds atoms and bonds step-by-step.The model avoids additional processing steps for placing atoms or inferring the final structure.
Proposed approach: The architecture is designed for conformational validity, uses rotation- and translation-covariant modeling, and incorporates chemical domain knowledge.The model explicitly targets drug-like molecule generation rather than only topological structure generation.
Results and application: L-Net generates chemically correct, conformationally valid, highly drug-like molecules and is combined with MCTS to design potential ABL1 inhibitors.The authors present this as an application to structure-based drug design.

2 Methods

L-Net represents molecule generation as iterative construction of a spanning tree, using node selection and tree expansion to determine graph structure, atom attributes, bonds, and 3D positions. A state encoder and policy network select actions, while training imitates expert trajectories from a drug-like ChEMBL subset.

Molecular representation: The model generates molecular graphs G = (V, E, A, B, X), where nodes and edges represent atoms and bonds, A and B label atom and bond types, and X contains 3D positions.The graph representation explicitly includes atom types, bond types, and three-dimensional coordinates.
Molecule generation process: L-Net builds each molecule by iteratively generating a spanning tree with depth-first node selection and tree expansion.The spanning-tree representation simplifies generation compared with general unconditional graphs.
Molecule generation process: At each iteration, tree expansion either appends new atoms to the focus atom or connects it to an existing atom, while node selection chooses the next suitable focus atom.Suitable focus atoms have unfilled valences; backtracking finds an ancestor when no eligible child remains.
Generation decisions: The generation decisions include connected atoms and bond types, appended atom counts and types, 3D locations, bond types, and atom rankings.Generation terminates when all atom valences are filled.
Network architecture: L-Net combines a state encoder that maps the intermediate graph to a continuous representation with a policy network that assigns probabilities to available actions.The model explicitly outputs bond types because inferring them from distances can be sensitive to errors.
Training and evaluation: The model is trained by imitating expert trajectories created for a drug-like subset of ChEMBL, and its performance is assessed with dedicated evaluation metrics.The metrics are intended to validate the model’s performance, including the quality of generated 3D structures.

2.2 The architecture of the state encoder

The state encoder maps molecular graphs to graph- and atom-level continuous representations using a U-net built from graph convolutions, pooling, and unpooling. A local coordinate system makes the 3D atom-placement policy rotationally covariant.

2.2 The architecture of the state encoder: L-Net encodes each molecular graph into graph-level and atom-level continuous representations with a U-net state encoder.The encoder first embeds atom and bond inputs, then processes them through convolutional, pooling, and unpooling layers.
2.2 The architecture of the state encoder: The encoder uses MPNN-style graph convolutions organized into DenseNet blocks, alongside molecularly tailored pooling and unpooling layers.The pooling and unpooling operations use a node-clustering method designed for molecular graphs.
2.3.1 Embeddings of atom and bond types: The encoder includes atom and bond type embeddings, temporal encodings for atom addition steps, and spatial features transformed into local coordinates.The spatial features are projected into vectors after transformation by the local-coordinate matrix.
2.3.1 Embeddings of atom and bond types: The local-coordinate strategy is presented as simpler to implement than specialized rotationally covariant architectures and is claimed as novel for small-molecule generation.The text contrasts it with tensor field networks and Cormorant, whose implementation difficulty and expressiveness limitations are noted.
2.3.1 Embeddings of atom and bond types: The model represents atom positions through a local coordinate system so its 3D placement policy transforms covariantly under rotations.The local policy is defined as q(x|G) = p(T(x)|T(G)), and the text states that q remains rotationally covariant.

2.4 Graph convolutional layers

The graph convolutional layers combine MPNN updates, attention-based aggregation, virtual bonds, and DenseNet blocks. Molecularly specific clustering enables graph U-net pooling and unpooling while compressing hydrogen information.

2.4 Graph convolutional layers: The graph-convolutional layers use MPNN-style updates with summation, maximization, and attention aggregation, plus virtual bonds to enlarge receptive fields.The layers are organized into multiple DenseNet blocks.
2.4 Graph convolutional layers: Reducing DenseNet depth or width hurts performance, so the standard configuration is recommended despite its greater computational burden.The standard configuration uses bottleneck size 94, growth rate 24, and block depths [2, 4, 6, 8, 6, 4, 2].
2.5 Pooling and unpooling operations in graph U-net: Because graphs lack canonical pooling and unpooling operations, the model uses a custom molecular clustering scheme and neural placement of cluster nodes.Cluster nodes begin at the geometric center of grouped nodes and receive a learned displacement.
2.5 Pooling and unpooling operations in graph U-net: The custom clustering scheme first collapses one-valence atoms, including many hydrogens, into neighboring atoms to reduce memory use while retaining hydrogen information.The text notes that almost half of the atoms in most molecules are hydrogen.
2.5 Pooling and unpooling operations in graph U-net: The second clustering level fragments molecules into ring assemblies and chains, while the final level collapses all nodes into one graph-level master node.These levels define the molecular hierarchy used by pooling and unpooling.

2.6 The policy network

The policy network encodes the current molecular state and predicts atom additions, connections, local geometry, coordinates, and generation order. It generates all atoms attached to a focused atom jointly, while SoftMADE improves modeling of low-dimensional positional distributions.

Append operation: The policy network predicts atom and bond types together with spherical coordinates for newly appended atoms.Each new atom is represented by its atom type, connecting bond type, and local coordinates (r, θ, φ).
Append operation: L-Net generates all atoms connected to the focused atom in one iteration, reducing repeated state encoding and potentially improving correlated neighboring-atom positions.The method contrasts with autoregressive 3D molecule generators that add atoms separately.
Drug-like priors: Prior chemical knowledge constrains generation through a maximum of three added atoms, valence masking, and explicit local geometries sp, sp2, and sp3.The model predicts local geometry before atom positions, which empirically improves learning of local geometry.
Distribution modeling: The model factorizes append decisions autoregressively across atom attributes and uses truncated Gaussian mixtures for radial and angular coordinates.The coordinate mixture counts are 15, 10, and 5 for the three generated atoms, respectively.
SoftMADE: SoftMADE adds noise to low-dimensional target position distributions so masked autoregressive density estimation can fit them more robustly.The tested configuration with cmax = 0.4 and α = 4 performed better than ordinary MADE and the lower-noise SoftMADE condition.

2.7 Data collection and preprocessing

The authors build a drug-like million-molecule dataset with generated 3D conformations and train L-Net through expert trajectories. Ring-first traversal and simulated coordinate errors improve generation quality and recovery from distribution mismatch.

Dataset construction: 3D conformations are initialized by distance geometry and optimized with the MMFF94s force field before dataset splitting.The resulting data use a 4/6 training split and 1/6 each for validation and testing.
Expert trajectories: Expert trajectories imitate depth-first generation paths, while ring-first traversal prioritizes closing rings and significantly improves generated-sample quality.The traversal scheme is modified because ordinary depth-first traversal works well for 2D models but performs poorly for this 3D generator.
Expert trajectories: Trajectory randomization by changing the starting position is evaluated alongside non-randomized trajectories as a training variant.

2.8 Model training

The model is trained in PyTorch with Adam for ten epochs using a batch size of 128 and a scheduled learning rate. Training takes approximately three days on one NVIDIA TITAN Xp.

Training setup: Training uses PyTorch and Adam, with default β1 and β2 values and an initial learning rate of 10^-3.The learning rate is decreased by 0.01 according to tested decay schedules.
Training setup: Learning-rate decay frequencies of every 50, 100, or 200 steps are experimentally compared.
Training setup: The model is trained with batch size 128 for 10 epochs on a single NVIDIA TITAN Xp, requiring around three days.

2.9 Hyperparameters

Hyperparameter selection balances architecture, data, and training choices against multiple evaluation metrics, while generation is accelerated through compilation, multiprocessing, and adaptive batching. The reported speed reaches 0.008 seconds per molecule, though it remains slower than SMILES-based samplers.

Hyperparameter selection: Hyperparameter optimization is limited because training is long, evaluation uses multiple metrics, and the best setting for one metric may not be best for another.The standard configuration targets 3D MMD through manual tuning and is proposed for future use.
Hyperparameter selection: The analyzed hyperparameters cover DenseNet depth and width, SoftMADE parameters, coordinate noise, and trajectory randomization.
Generation acceleration: Numba compilation provides more than 10 times speedup for relevant Python code, while multiprocessing overlaps CPU graph processing with GPU work.The CPU and GPU workloads are separated into asynchronous processes operating on different batches.
Generation acceleration: 0.008 seconds per molecule is achieved on an NVIDIA TITAN Xp after applying generation-acceleration techniques.The authors attribute the speed to combined optimization of CPU/GPU processing and generation scheduling.
Generation acceleration: The reported generation speed is slower than SMILES-based samplers because the workflow and network architecture are more complex.The authors identify quantization and pruning as possible future improvements.

2.11 Evaluation

The evaluation combines validity, uniqueness, molecular-property distributions, distributional metrics, local geometry, and conformational stability to assess 3D molecule generation.

Output validity and uniqueness: The evaluation measures chemical validity, structural uniqueness, molecular-property distributions, distribution discrepancy, local geometries, and conformational stability.Validity is based on RDKit sanitization, while uniqueness detects overtraining or mode collapse; geometry is assessed through bond lengths, angles, torsions, and RMSD after optimization.
Distribution of molecular properties: Molecular-property evaluation covers topological descriptors, 3D shape, and solvent-accessible surface areas.The reported properties include molecular weight, LogP, hydrogen-bond counts, rotatable bonds, QED, normalized PMI ratios, and polar and total SASA.
Distributional metrics: MMD compares generated and real distributions using Morgan fingerprints for topology and USRCAT fingerprints for 3D structure.USRCAT is chosen as an alignment-free alternative because shape alignment would be computationally infeasible for the MMD calculation.
Validity of local geometries: Local geometry is evaluated by comparing environment-conditioned bond lengths, bond angles, and torsion-angle distributions between generated and test molecules.Conformational stability is additionally quantified by heavy-atom RMSD before and after MMFF94s optimization, with ETKDG used for context.
Quality of generated conformers: RMSD comparisons with ETKDG provide context for generated-conformation quality but cannot establish which method is better because they solve different problems.The comparison is intended only to indicate the overall quality level of generated conformations.

design

The design combines L-Net with MCTS for structure-based molecular design, using L-Net as the rollout policy and binding-affinity optimization to explore ABL1 inhibitors.

Motivation: L-Net is combined with MCTS to demonstrate 3D structure-based molecular design, addressing a combination not previously reported for 3D generative models.Earlier work had combined MCTS with 2D generative models, whereas this design uses a 3D generative model.
MCTS integration: L-Net serves as the MCTS rollout policy, while MENTS supplies the tree policy and periodic exploitation samples from Q-value distributions.Local optimization is applied to each sample, and the process is parallelized at leaf and tree levels.
ABL1 case study: The case study targets the ABL1 tyrosine-protein kinase, focusing on its allosteric myristate-binding site because of potential selectivity advantages.The site is associated with chronic myelogenous leukemia, and the design uses smina affinity scores as the reward despite known concerns about this proxy.
Seeded design: A CClF2O− group from the active molecule asciminib is fixed as a seed, after which L-Net and MCTS generate and optimize the remaining structure.The seed is used because prior reports identify this functional group as essential for asciminib inhibition activity.

3 Results and Discussion

L-Net generates mostly valid, unique, structurally plausible, and conformationally stable molecules, while MCTS extends it toward affinity-guided ABL1 design with an affinity–drug-likeness trade-off.

Output validity and uniqueness: 94.3% maximum validity and near-100% uniqueness show that L-Net generates many chemically valid molecules without evident mode collapse.The best validity is reported for the slow-decay configuration, while uniqueness values generally approach 100%.
Molecular-property distributions: Generated topological properties broadly match the test distribution in their means, but are more dispersed and show a marked QED-distribution mismatch.The QED discrepancy is attributed to the hard 0.5 cutoff used during data selection.
Distributional metrics: SoftMADE and input noise improve topological MMD, while precision and recall reach 81.9% and 88.3% in one configuration.Another configuration reports 43.1% precision and 78.0% recall, and the results suggest stronger mode coverage than precision.
Local geometry: Generated molecules reproduce mean bond lengths and angles, adopt appropriate local geometries, and match real torsion distributions in 8 of 9 cases.The model correctly captures tetrahedral and planar geometries, although some distributional deviations remain.
Conformer quality: The best hyperparameter set achieves an average pre- versus post-optimization RMSD of 0.613 Å, below the approximately 0.8 Å observed for ETKDG.The authors interpret this as evidence that generated conformations may lie closer to MMFF94s local minima, while cautioning that the methods solve different problems.
Structure-based design: MCTS-guided L-Net generates molecules with high predicted affinity faster than pure virtual screening, but higher affinity is accompanied by lower drug-likeness.Despite this trade-off, a significant portion of high-affinity molecules retain QED > 0.5, providing candidate choices.

4 Conclusion

L-Net is an autoregressive 3D graph generative model designed to generate drug-like molecules directly, including their topology and 3D structures. It produces chemically correct, conformationally valid, druglike molecules and can be combined with MCTS for structure-based design, while its training data impose important limitations.

Model contribution: L-Net directly generates drug-like molecules with both topological and 3D structures, without additional atom placement or bond order inference.The authors describe this as the first application of autoregressive 3D graph generative models to generating drug-like molecules with 3D structures.
Overall results: The model generates chemically correct, conformationally valid, and druglike molecules.These properties are presented as the main overall outcome of the model.
Model design: The model’s performance reflects techniques for rotational equivariance, atom-position fitting, valence and hybridization modeling, noise correction, and ring-first traversal.Hierarchical graph clustering also reduces memory requirements and enables hydrogen atoms during generation.
Structure-based design: L-Net combines with reinforcement learning methods such as MCTS to support objective-oriented molecule generation in 3D space, including docking objectives.The authors demonstrate this capability by testing potential ABL1 inhibitors.
Structure-based design: With MCTS, L-Net generated molecules showing similar interaction modes and predicted binding affinities within the ABL1 pocket as known inhibitors.The result is presented as evidence of potential application in structure-based drug discovery.
Limitations: L-Net is trained on RDKit-generated 3D structures rather than crystallographic data, which are theoretically more accurate but more limited for drug-like molecules.The authors suggest pretraining on large artificial data followed by crystallographic fine-tuning as a possible solution.