Source-linked AI summary
The Synthesizability of Molecules Proposed by Generative Models
Wenhao Gao, Connor W. Coley
TL;DR
Generative models can propose molecular structures that are challenging or infeasible to synthesize. This paper analyzes their synthesizability and finds that goal-directed methods risk unsynthesizable top suggestions, while synthesizability heuristics improve tractability at the expense of the main objective.
Problem
Generative models may propose molecular structures that are challenging or infeasible to synthesize.
Method
The paper analyzes the synthesizability of molecules generated by de novo generative algorithms.
Results
Goal-directed generation methods have a significant risk of proposing unsynthesizable structures as their top suggestions, making objective-function-only benchmarking insufficient.
Takeaways & Limitations
Heuristic biasing almost always improves the synthesizability of generated candidates but necessarily detracts from the main objective function.
Takeaways & Limitations
Improving synthesizability through heuristic biasing necessarily detracts from the main objective function.
Abstract
from arXiv · showhide
The discovery of functional molecules is an expensive and time-consuming process, exemplified by the rising costs of small molecule therapeutic discovery. One class of techniques of growing interest for early-stage drug discovery is de novo molecular generation and optimization, catalyzed by the development of new deep learning approaches. These techniques can suggest novel molecular structures intended to maximize a multi-objective function, e.g., suitability as a therapeutic against a particular target, without relying on brute-force exploration of a chemical space. However, the utility of these approaches is stymied by ignorance of synthesizability. To highlight the severity of this issue, we use a data-driven computer-aided synthesis planning program to quantify how often molecules proposed by state-of-the-art generative models cannot be readily synthesized. Our analysis demonstrates that there are several tasks for which these models generate unrealistic molecular structures despite performing well on popular quantitative benchmarks. Synthetic complexity heuristics can successfully bias generation toward synthetically-tractable chemical space, although doing so necessarily detracts from the primary objective. This analysis suggests that to improve the utility of these models in real discovery workflows, new algorithm development is warranted.
Introduction
Generative molecular design can explore chemical space more efficiently than brute-force screening, but its practical utility is limited when proposed molecules are difficult or infeasible to synthesize. This study uses ASKCOS to analyze the synthesizability of molecules generated by de novo algorithms across distribution-learning and goal-directed tasks.
- Motivation: Chemical space may contain exceeding 1060 theoretically possible, pharmacologically relevant small molecules, making brute-force virtual screening computationally intractable.Existing virtual libraries of hundreds of millions or billions of molecules cover only a tiny fraction of this space.
- Generative molecular design: Generative models can avoid enumerating every candidate and bias molecular generation toward desired properties using Bayesian optimization, genetic algorithms, or tree search.Once trained, these models generate multiple molecular structures at linear computational cost and can explore beyond their initial training pool.
- Synthetic feasibility: Generative models may propose molecular structures that are challenging or infeasible to synthesize, obstructing their usefulness in realistic discovery workflows.Manufacturing is required to validate and apply a proposed molecule, even when computational property predictions are accurate.
- Study approach: The study analyzes generative-model synthesizability with the open-source ASKCOS retrosynthesis tool, which can capture nonlinear structure–synthesizability relationships and recommend actionable pathways.Retrosynthetic analysis can identify building blocks and the number of synthetic steps, unlike simpler structure-based proxies.
Results
ASKCOS predictions were consistent with expected synthesizability across standard chemical libraries and supported benchmarking molecular-generation outputs. Distribution-learning models largely inherited training-set synthesizability, whereas goal-directed optimization often required heuristic biasing to produce feasible candidates, reducing objective performance.
- Validation of ASKCOS: MOSES had the highest synthesizability rate at 89.8%, whereas ZINC, ChEMBL, and GDB17 reached 60.8%, 68.3%, and 3.5%, respectively.Predicted reaction steps correlated with expert scores, supporting ASKCOS as a benchmark for molecular-generation synthesizability.
- Heuristic synthesizability scores: SA Score, SMILES length, and SCScore all showed decreasing synthesizability with increasing heuristic values, with AUCs of 0.87, 0.69, and 0.61.None perfectly distinguished synthesizable from unsynthesizable compounds, but SA Score performed best and structural complexity could differ from synthetic complexity.
- Distribution learning: Distribution-learning models produced synthesizable fractions comparable to their training sets, without improving synthesizability relative to those sets.Training on MOSES rather than ChEMBL therefore provided an effective prior bias, while no generation method was particularly superior.
- Goal-directed optimization: 30.2% of compounds were synthesizable for hard objectives without biasing when trained on ChEMBL, versus 32.7% with MOSES; some tasks had few or no feasible top-100 candidates.Genetic algorithms were especially vulnerable to producing unsynthesizable top candidates, making post hoc filtering unreliable in some cases.
Discussion of other approaches
Several approaches could improve synthesizability-aware molecular generation, but explicit retrosynthetic evaluation is computationally expensive. Generation constrained by synthetic rules may explore chemical space efficiently, though its validity depends on reaction-prediction accuracy and decreases with longer synthetic routes.
- CASP-based evaluation: Biasing generation with a full CASP tool can evaluate synthesizability directly, but computational expense is its main disadvantage.The approach may spend up to one minute evaluating each molecule to reduce false negatives.
- Computational cost: Genetic-algorithm optimization could require millions or at least hundreds of thousands of calls to the CASP oracle.It may start from a pool of up to millions of structures (∼106), then score hundreds of child compounds (∼103) across hundreds of iterations (∼103).
- Constrained generation: Embedding synthesizability constraints directly into generation restricts search to molecules producible from available building blocks.Examples include MoleculeChef, which generates reactants and predicts forward reactions, and ChemBO, which searches a directed synthetic graph.
- Constrained generation: Forward-direction methods are subject to CASP caveats because their validity depends on reaction-prediction accuracy, and allowing more synthetic steps lowers the chance that each reaction proceeds as predicted.The authors expect a similar success rate to virtual libraries, with anecdotal 85% successful delivery for compounds enumerated with a single synthetic step.
Conclusion
Distribution-learning methods can generate synthesizable molecules when they learn the training distribution well, whereas goal-directed methods risk proposing unsynthesizable top candidates. Heuristic biasing generally improves synthesizability but detracts from the primary objective, and better synthesis-aware methods are needed for practical discovery.
- Distribution-learning methods generate synthesizable molecules at frequencies similar to their training sets when they learn the chemical distribution well.
- Goal-directed methods, especially SMILES GA and Graph GA, have a significant risk of proposing unsynthesizable structures among their top suggestions.In some cases, enough high-performing synthesizable molecules appear in the top 100 for post hoc filtering to be viable.
- Benchmarking generative methods solely by objective-function value is insufficient because proposed molecules can be chemically absurd.
- Biasing generation by training-set synthesizability works for distribution learning but has no noticeable effect on goal-directed optimization tasks.Modifying the objective with SA Score can outperform post hoc filtering for some tasks, while heuristic biasing almost always improves candidate synthesizability but detracts from the main objective.
- Despite limitations in CASP tools, ASKCOS enables a meaningful analysis of generated-molecule synthesizability, while improved synthesis-aware algorithms could increase practical utility.Suggested directions include better post hoc filtering, new synthesizability heuristics, reinforcement learning with CASP-oracle biasing, and algorithms constrained by chemical-reactivity predictions.
Methods
The study evaluates molecular synthesizability with ASKCOS, a data-driven retrosynthetic planning framework, under explicit search and starting-material constraints. It tests multiple molecular-generation methods and benchmark objectives, including heuristic biasing that modifies the main objective to penalize undesired properties.
- Retrosynthetic analysis: ASKCOS applies learned retrosynthetic transformations, reaction-condition identification, and reaction-success prediction using models trained on millions of USPTO and Reaxys reactions.Its retrosynthetic search recursively applies reaction templates encoded as SMARTS patterns and uses an upper confidence bound tree search.
- Generation methods: The experiments compare random sampling, best-from-data screening, LSTM, VAE, AAE, SMILES GA, and Graph GA molecular-generation approaches.The models include SMILES-based neural architectures and genetic algorithms operating on SMILES tokens or molecular graphs.
- Optimization objectives: Goal-directed optimization uses Brown et al.’s objective-function suite, divided into trivial and hard tasks, with hard tasks expected to distinguish methods more strongly.The objectives include QED, CNS MPO, several drug-specific MPO tasks, SMARTS, Scaffold Hop, and Decorator Hop, but remain relatively simple structural heuristics.
Supporting Information · The Synthesizability of Molecules Proposed by Generative Models
This supporting-information block identifies the paper’s authors and their institutional affiliations. It concerns “The Synthesizability of Molecules Proposed by Generative Models.”
- The Synthesizability of Molecules Proposed by Generative Models: The paper is authored by Wenhao Gao and Connor W. Coley.
- The Synthesizability of Molecules Proposed by Generative Models: The authors are affiliated with the Department of Chemical Engineering at MIT in Cambridge, Massachusetts.
- The Synthesizability of Molecules Proposed by Generative Models: The listed MIT affiliation is located in Cambridge, MA 02139.
- The Synthesizability of Molecules Proposed by Generative Models: The affiliations include the Broad Institute of Harvard and MIT in Cambridge.
- The Synthesizability of Molecules Proposed by Generative Models: The affiliations include the Department of Chemical and Biomolecular Engineering at Johns Hopkins University.
- The Synthesizability of Molecules Proposed by Generative Models: The Johns Hopkins affiliation is located in Baltimore, MD 21218.
Additional Results
Additional analyses show that explicit synthetic-route planning distinguishes compounds that heuristic scores miss, while heuristic biasing improves synthesizability across difficult optimization settings at a cost to objective performance. Synthesizability is less problematic for trivial tasks, except primarily for Pioglitazone MPO.
- Explicit route planning: ASKCOS distinguishes structurally similar compounds with different synthetic accessibility by finding a route for ribavirin but none for its harder analogue.SA Score and SCScore do not reflect the analogue’s greater difficulty, whereas explicit route planning does.
- Heuristic classification: On the “All but GDB” compound set, SA Score outperforms the SMILES heuristic, which outperforms SCScore, and all three outperform random guessing for ASKCOS-based classification.The receiver operating characteristic analysis uses area under the curve to quantify these trends.
- Heuristic biasing: Heuristic biasing changes both synthesizability and objective function values, with SA Score or SCScore applied after training on ChEMBL or MOSES.The supplementary analysis reports these changes across generative methods and objective functions.
- Hard optimization tasks: For many “hard” optimization tasks, ASKCOS cannot identify routes to a large fraction of top-100 generated molecules, particularly from Graph GA and SMILES GA methods.The analysis compares no biasing with SA Score and SCScore heuristic biasing using ChEMBL or MOSES for initial training.
- Trivial optimization tasks: Synthesizability is primarily an issue for the Pioglitazone MPO task among the “trivial” optimization tasks.The comparison includes no biasing and SA Score or SCScore biasing with ChEMBL or MOSES initial training.