PaperlayerView exact revision on arXiv

Source-linked AI summary

The Synthesizability of Molecules Proposed by Generative Models

Wenhao Gao, Connor W. Coley

arXiv:2002.07007v1q-bio.QMcs.LGstat.ML

TL;DR

Generative molecular design is limited by uncertainty about whether proposed structures can be synthesized, so this paper evaluates them with a full computer-aided synthesis-planning tool. It finds that goal-directed models risk proposing unsynthesizable top candidates, particularly with SMILES-based approaches.

  • Problem

    Evidence is limited on whether molecular structures proposed by generative models are challenging or infeasible to synthesize.

    Source: Introduction · p. 3

  • Method

    The paper evaluates synthesizability using a full computer-aided synthesis-planning tool rather than a proxy score.

    Source: Discussion of other approaches · p. 13

  • Results

    Goal-directed methods risk proposing unsynthesizable top suggestions, particularly with SMILES, and objective-function values alone are insufficient for benchmarking.

    Source: Conclusion · pp. 14–15

  • Takeaways & Limitations

    Objective-function value alone is insufficient for benchmarking generative methods, so synthesizability also warrants evaluation.

    Source: Conclusion · p. 15

    Heuristic biasing almost always improves synthesizability but necessarily detracts from the main objective function.

    Source: Conclusion · p. 15

Abstract

from arXiv · show

The discovery of functional molecules is an expensive and time-consuming process, exemplified by the rising costs of small molecule therapeutic discovery. One class of techniques of growing interest for early-stage drug discovery is de novo molecular generation and optimization, catalyzed by the development of new deep learning approaches. These techniques can suggest novel molecular structures intended to maximize a multi-objective function, e.g., suitability as a therapeutic against a particular target, without relying on brute-force exploration of a chemical space. However, the utility of these approaches is stymied by ignorance of synthesizability. To highlight the severity of this issue, we use a data-driven computer-aided synthesis planning program to quantify how often molecules proposed by state-of-the-art generative models cannot be readily synthesized. Our analysis demonstrates that there are several tasks for which these models generate unrealistic molecular structures despite performing well on popular quantitative benchmarks. Synthetic complexity heuristics can successfully bias generation toward synthetically-tractable chemical space, although doing so necessarily detracts from the primary objective. This analysis suggests that to improve the utility of these models in real discovery workflows, new algorithm development is warranted.

Introduction

This section examines the synthesizability of molecules proposed by de novo generative models across distribution-learning and goal-directed tasks, using ASKCOS for assessment. It focuses on post hoc filtering, training-set differences, and heuristic biasing as approaches to the problem.

Introduction

  • More than 10^60 pharmacologically relevant small molecules make brute-force screening computationally intractable.Even libraries containing billions of candidates represent only a tiny fraction of this space.Source: Introduction · p. 2
  • Generative models can bias candidate generation toward desired functionality and produce novel structures beyond a limited starting pool.Unlike virtual screening, they need not enumerate every candidate and may offer preferential intellectual-property positions.Source: Introduction · p. 3
  • Generated molecules may be challenging or infeasible to synthesize, including nonsensical structures arising as models explore beyond their training data.Unlike virtual-screening libraries built from characterized reaction templates, de novo generation does not ensure ready synthesizability.Source: Introduction · pp. 3–4
  • Structure-based synthesizability scores can miss route differences between similar molecules, while expert assessments are labor-intensive, hard to replicate, and unscalable.A single functional-group transposition can change synthetic routes because of reaction selectivity or building-block availability.Source: Introduction · p. 4
  • Retrosynthetic planning can capture nonlinear structure–synthesizability relationships and recommend actionable routes, including building blocks and step counts.The authors present this route-based analysis as a form of interpretability for why a molecule is believed to be synthesizable.Source: Introduction · pp. 4–5
  • The study analyzes molecules from distribution-learning and goal-directed generation with the open-source retrosynthesis tool ASKCOS.It distinguishes unoptimized molecules from goal-directed molecules optimized against a scoring function.Source: Introduction · p. 5
  • A heuristic synthesizability estimator provides a computationally inexpensive scalar measure of synthesizability.Source: Introduction · p. 5
  • The study examines post hoc filtering, training-set differences, and heuristic biasing as three major approaches to synthesizability.Source: Introduction · p. 5

Results

The Results section examines synthesizability predictions across common datasets and generative models, including how training data and optimization bias relate to feasibility and objective values. It also considers a filter-first workflow, using SA-score biasing only if needed.

Synthesizability of common databases according to ASKCOS

  • ASKCOS’s predicted synthesizability rates were 89.8% for MOSES, 60.8% for ZINC, 68.3% for ChEMBL, and 3.5% for GDB17, and its step counts correlated with expert scores.MOSES had the highest rate and GDB17 the lowest; the authors judged the trends and expert-score correlation consistent with expectations, supporting ASKCOS as a benchmark.Source: Synthesizability of common databases… · p. 7

Agreement between synthesizability heuristics and ASKCOS

  • For synthesizable and unsynthesizable compounds, SA Score, SCScore, and SMILES string length showed decreasing synthesizability trends as heuristic scores increased, with AUCs of 0.87, 0.69, and 0.61, respectively; none distinguished them perfectly.Source: Agreement between synthesizability… · p. 7

Synthesizability of unoptimized generated molecules

  • Distribution-learning methods generated molecules with synthesizability comparable to their training sets, and no method improved on its training set.The difference between MOSES- and ChEMBL-trained results suggests that training on a more synthesizable dataset can bias generation toward more feasible molecules.Source: Synthesizability of unoptimized… · p. 9
  • No distribution-learning method was notably superior, and post hoc filtering remained viable because relatively few generated molecules failed the synthesizability check.These findings concern synthesizability only, not novelty, uniqueness, or diversity.Source: Synthesizability of unoptimized… · p. 9

Synthesizability of optimized generated molecules

  • 80.2% with SA Score bias and 55.4% with SCScore bias, up from 30.2% for hard objectives; some top-candidate objective values fell by over 0.2.Without bias, several tasks—particularly those using genetic algorithms—had few or no synthesizable candidates among the top 100, so post hoc filtering could leave none.Source: Synthesizability of optimized… · pp. 11–12, Table 1 · p. 12
  • In some cases, the top-1 synthesizable candidate’s objective increased even as the top-1 candidate’s objective decreased.The comparison is between the top candidates after post hoc filtering.Source: Synthesizability of optimized… · p. 12
  • For just 1–10 synthesizable candidates, first optimize without bias and filter; if they underperform unsynthesizable leaders, repeat optimization with the SA score.Source: Synthesizability of optimized… · p. 12

Discussion of other approaches

The discussion weighs full CASP scoring and synthesis-constrained generation as routes to synthesizable optimization, highlighting oracle cost and reaction-prediction reliability as constraints.

Discussion of other approaches

  • Evaluating each molecule with a full CASP tool can take up to one minute, reducing false negatives but adding computational expense.ASKCOS finds pathways for some molecules in a few seconds.Source: Discussion of other approaches · p. 13
  • Genetic optimization could require millions or at least hundreds of thousands of CASP-oracle calls across its pool, iterations, and child-compound scoring.The described setup selects synthesizable high-scoring initial compounds from a pool of up to millions, then scores hundreds of children over hundreds of iterations.Source: Discussion of other approaches · p. 13
  • MolDQN reports 200k function calls; reinforcement-learning methods may require hundreds of thousands or millions of iterations, while existing methods require ≥5 thousand evaluations for one task versus Korovina et al.’s 100.Source: Discussion of other approaches · p. 13
  • As reinforcement learning becomes more sample-efficient and CASP tools become faster, explicit retrosynthetic planning during optimization may become computationally viable.The discussion presents this as a prospective strategy for reducing oracle calls, not as a demonstrated result.Source: Discussion of other approaches · p. 13
  • Search can be constrained to molecules producible from available building blocks; Vinkers iteratively optimizes a growing structure by selecting blocks to react with it [35], while MoleculeChef generates a bag of reactants and uses forward-reaction prediction [36].Source: Discussion of other approaches · pp. 13–14
  • 85% successful delivery is the anecdotal rate expected for single-step virtual libraries; greater synthetic depth lowers reliability because forward-reaction predictions may fail.The prediction engine may use hand-coded or algorithmically inferred rules, and these approaches’ validity depends on its accuracy.Source: Discussion of other approaches · p. 14

Conclusion

Using ASKCOS to assess de novo generative models, the paper finds that synthesizability varies by generation approach and that heuristic bias can improve tractability while compromising the primary objective.

  • The study evaluates common compound libraries and model proposals, with and without heuristic bias, using ASKCOS; despite its imperfections, the tool enables meaningful synthesizability analysis.Source: Conclusion · pp. 14–15
  • Distribution-learning methods that learn the training-set chemical distribution appear to generate synthesizable molecules at a frequency similar to that of their training set.Source: Conclusion · p. 14
  • Goal-directed methods risk unsynthesizable top suggestions, especially with SMILES GA or Graph GA; sometimes top-100 filtering works, but absurd candidates make objective-only benchmarking insufficient.Source: Conclusion · pp. 14–15
  • Training-set synthesizability bias works for distribution learning but has no noticeable effect on goal-directed optimization; SA Score improves candidates over post hoc filtering on some tasks, while heuristic bias usually reduces the primary objective.Source: Conclusion · p. 15
  • ASKCOS pathway identification is neither necessary nor sufficient for synthesizability, and predicted forward-synthesis reactions are not guaranteed to work experimentally; CASP tools remain imperfect.Source: Conclusion · p. 15
  • CASP predictions may miss chemical reactivity or be overly optimistic, including about selectivity, and pathway discovery depends on the buyable-chemical database and retrosynthesis settings.Source: Conclusion · p. 15
  • The authors propose improving CASP-based filtering, developing synthesizability heuristics, sampling CASP oracles for reinforcement-learning bias, and designing algorithms constrained by predicted chemical reactivity.Source: Conclusion · p. 15

Methods

The methods use ASKCOS to assess synthesizability across molecules produced or selected by baseline and learned algorithms on optimization tasks, while excluding less meaningful benchmarks and examining dataset- and heuristic-based ways to steer generation toward synthesizable compounds.

ASKCOS

  • ASKCOS models trained on USPTO and Reaxys reactions recursively apply SMARTS reaction templates, with expansion parallelized by upper confidence bound tree search [4,39].The framework also supports programmatic and graphical interfaces for processing compounds without human intervention.Source: ASKCOS · p. 16
  • The search required starting materials to be commercially available according to a 2018 database of molecules from eMolecules or Sigma Aldrich, priced no greater than $100 per gram—a very strict drug-discovery limit.Source: ASKCOS · p. 16

Compound databases

  • The study also sampled GDB1730’s 800,000-compound Lead-like Set from a database enumerating 166.4 billion molecules with up to 17 heavy atoms.The Lead-like Set has molar masses of 100–350 g/mol and CLogP of 1–3, and excludes 3- or 4-membered rings.Source: Compound databases · p. 18

Molecular generation algorithms

  • Generation methods ranged from random sampling and best-from-data screening to LSTM, VAE, and AAE neural models, plus SMILES- and graph-based genetic algorithms.The LSTM predicts the next SMILES character and can be hill-climbed; the VAE learns a SMILES-to-latent-space mapping, while the AAE uses adversarial rather than KL regularization.Source: Molecular generation algorithms · pp. 18–19

Objective functions for optimization

  • Optimization used Brown et al.’s benchmark functions [27], dividing tasks into trivial objectives that most methods solve well and harder tasks with greater method-dependent variation.The objectives are structural heuristics; trivial examples include QED, CNS MPO, and Pioglitazone MPO, while hard examples include drug MPO and scaffold or decorator hops.Source: Objective functions for optimization · pp. 19–20
  • The study excludes commercial-drug similarity and hard-task isomer benchmarks because the authors consider them less meaningful for drug discovery.Source: Objective functions for optimization · p. 20

Biasing techniques for molecular generation

  • Training-set biasing uses databases with different synthesizability rates—ChEMBL (68.3% as tested) and MOSES (89.8% as tested)—for optimized or unoptimized generation.Source: Biasing techniques for molecular… · p. 20
  • Heuristic biasing applies a 0-to-1 multiplier to a pre-normalized objective using modified Gaussian and sigmoid functions; 30 TPE iterations maximize synthesizable-suggestion fraction times average objective, for optimized generation only.Source: Biasing techniques for molecular… · p. 21
  • SA Score [31] estimates synthesizability using fragment contributions, treating rarer PubChem fragments as evidence of lower synthesizability.Source: Biasing techniques for molecular… · p. 21
  • SCScore [20] is a neural-network synthetic-complexity score trained on Reaxys reaction data and designed to cover products, intermediates, and simpler building blocks.Source: Biasing techniques for molecular… · p. 21
  • SMILES length serves as a simple difficulty heuristic: it tracks heavy-atom count and increases with formal charges, ring closures, and defined stereochemistry.Source: Biasing techniques for molecular… · pp. 21–22

Additional Results

Supplementary results show that heuristic scores can miss reactivity-driven differences in synthetic accessibility, while ASKCOS distinguishes them through route planning. They also document synthesizability problems in some optimization tasks and cases where heuristic biasing yields synthesizable candidates.

Suitability of heuristic functions for estimating synthesizability

  • Although ribavirin and its analogue are structurally similar, ribose reactivity makes the analogue harder to access; SA Score and SCScore miss this difference, whereas ASKCOS finds a route only for ribavirin.The figure attributes ribavirin’s favored substitution to ribose’s inherent reactivity.Source: Figure S1 · p. 30, Figure S2 · p. 31
  • For classifying the “All but GDB” set as synthesizable or unsynthesizable according to ASKCOS, SA Score outperforms the SMILES heuristic, which outperforms SCScore; all three beat random guessing.Source: Figure S3 · p. 32

Full results of heuristics biasing in goal-directed generation

  • For hard optimization tasks, ASKCOS cannot find routes for many generated molecules, particularly those from Graph GA and SMILES GA methods.The supplementary results compare unbiasing with SA Score and SCScore biasing, using ChEMBL or MOSES for initial training.Source: Figure S6 · p. 35
  • For trivial optimization tasks, synthesizability is primarily an issue for the Pioglitazone MPO task.The figures show top-100 synthesizable fractions with no biasing, SA Score biasing, and SCScore biasing, using ChEMBL or MOSES for initial training.Source: Figure S8 · p. 36

Successful cases of heuristic biasing in goal-directed generation

  • Among candidates with no synthesizable structure in the unbiasing top 100, many SMILES GA and Graph GA structures are nonsensical despite high objective values; a separate set shows improved top-1 synthesizable objective values with SMILES LSTM.Source: Figure S10 · p. 39, Figure S11 · p. 40, Figure S13 · p. 42

Agreement between ASKCOS results and synthesizability heuristics

  • Supplementary figures compare ASKCOS’s first-pathway length with expert chemist scores and with SA Score, SCScore, and SMILES length.A length of 0 indicates a readily purchasable compound, while 11 indicates that no pathway was found with the fixed expansion settings.Source: Figure S14 · p. 43, Figure S15 · p. 44, Figure S16 · p. 45
  • Additional figures compare Sheridan et al.’s meanComplexity with SA Score, SCScore, and SMILES string length, coloring compounds by ASKCOS-assessed synthesizability.Source: Figure S18 · p. 47, Figure S19 · p. 47, Figure S20 · p. 48

Related papers

Loading 2002.07007v1…