Source-linked AI summary
Deep Reinforcement Learning for De-Novo Drug Design
Mariya Popova, Olexandr Isayev, Alexander Tropsha
TL;DR
De novo molecular design must search an enormous chemical space without exhaustive screening or reliable global optimization. ReLeaSE combines generative and predictive neural networks with reinforcement learning to generate targeted chemical libraries, producing 95% valid structures and libraries biased toward desired properties.
Problem
The enormous space of synthetically feasible drug-like molecules prevents exhaustive screening, while local optimization can converge on practical optima or restrict chemical-space exploration.
Method
ReLeaSE jointly trains generative and predictive neural networks with reinforcement learning to generate chemically feasible SMILES molecules optimized for target properties.
Results
95% of generated structures were valid, and the method produced chemical libraries biased toward targeted physical and biological properties, including melting temperature and bioactivity.
Takeaways & Limitations
ReLeaSE supports generation of novel targeted chemical libraries optimized for desired physical, chemical, or bioactivity properties.
Takeaways & Limitations
The authors caution that synthetic accessibility is not well-defined in the current SAS formulation, and optimizing it could reduce molecular novelty.
Abstract
from arXiv · showhide
We propose a novel computational strategy for de novo design of molecules with desired properties termed ReLeaSE (Reinforcement Learning for Structural Evolution). Based on deep and reinforcement learning approaches, ReLeaSE integrates two deep neural networks - generative and predictive - that are trained separately but employed jointly to generate novel targeted chemical libraries. ReLeaSE employs simple representation of molecules by their SMILES strings only. Generative models are trained with stack-augmented memory network to produce chemically feasible SMILES strings, and predictive models are derived to forecast the desired properties of the de novo generated compounds. In the first phase of the method, generative and predictive models are trained separately with a supervised learning algorithm. In the second phase, both models are trained jointly with the reinforcement learning approach to bias the generation of new chemical structures towards those with the desired physical and/or biological properties. In the proof-of-concept study, we have employed the ReLeaSE method to design chemical libraries with a bias toward structural complexity or biased toward compounds with either maximal, minimal, or specific range of physical properties such as melting point or hydrophobicity, as well as to develop novel putative inhibitors of JAK2. The approach proposed herein can find a general use for generating targeted chemical libraries of novel compounds optimized for either a single desired property or multiple properties.
Introduction
ReLeaSE is a deep reinforcement-learning method for de novo molecular design that combines SMILES-based generative and predictive neural networks. It addresses chemical-library search and feasibility challenges while optimizing user-specified physical, chemical, or bioactivity properties.
- Motivation: ReLeaSE targets de novo compound generation because exhaustive virtual-library sampling is impractical and local optimization can converge to practical optima.
- Method: The method combines generative and predictive deep neural networks in a two-stage workflow: separate supervised training followed by joint reinforcement learning.
- Representation: Both models use molecules represented only by SMILES strings, integrating generation and property prediction into one reinforcement-learning workflow.
- Method: The generative model produces chemically feasible molecules, while the predictive model estimates user-specified molecular properties to guide optimization.
- Validation: 95% of generated structures were valid, chemically sensible molecules according to the ChemAxon structure checker.
- Predictive performance: For LogP prediction, five-fold cross-validation produced R2ext of 0.91 and RMSE = 0.53, compared with R2ext = 0.90 and RMSE = 0.57 for Random Forest with DRAGON7 descriptors.
Discussion
ReLeaSE combines generative and predictive deep neural networks with reinforcement learning to design novel compound libraries optimized for desired properties. The discussion emphasizes its descriptor-free workflow, flexible optimization strategies, broad proof-of-principle applications, and tradeoffs involving synthetic accessibility.
- Method: ReLeaSE combines generative and predictive deep neural networks, trained separately by supervised learning and jointly through reinforcement learning, to design compounds with desired properties.The workflow uses molecular SMILES representations and integrates both models with an RL step.
- Comparison with prior work: Prior reinforcement-learning molecular design work lacked evidence that predicted properties were optimized, while another RNN-based study reported only ~30-35% valid molecules.These comparisons frame ReLeaSE's claimed innovation in demonstrating property optimization and chemically feasible generation.
- Distinctive features: Unlike descriptor-based approaches, ReLeaSE uses a descriptor-free predictive model that forms a coherent workflow with molecular generation and evaluates predominantly novel structures.The method contrasts with prior work using numerical molecular descriptors and traditional QSAR-based virtual screening.
- Applications: Flexible reward functions support minimizing, maximizing, or targeting a property range across physical properties, biological activity, and chemical substructure bias.The proof-of-principle studies also produced a dataset exceeding 1M novel compounds.
- Limitations: Optimizing synthetic accessibility with reinforcement learning could substantially reduce molecular novelty and bias generation toward low-SAS training substructures.The discussion also questions whether current SAS scores adequately define synthetic accessibility and notes their practical independence from property optimization.