PaperlayerView exact revision on arXiv

Source-linked AI summary

ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?

Mostapha Benhenda

arXiv:1708.08227v3stat.MLcs.AIcs.LG

TL;DR

Drug-discovery generative models need evaluation beyond visual inspection, including whether their outputs retain the diversity of natural molecules. The paper quantifies internal chemical diversity and compares RL and ORGAN, finding that neither matches natural diversity, though ORGAN does somewhat better than RL.

  • Problem

    Generative-model research often relies on visual inspection or metrics less relevant to drug discovery, leaving internal chemical diversity in need of rigorous evaluation.

    Source: Introduction · p. 1

  • Method

    The paper introduces an internal-diversity metric and challenges models to match natural diversity among molecules satisfying a desired property, illustrating the comparison with RL and ORGAN.

    Source: Introduction · p. 1

  • Results

    Both RL and ORGAN fail to match natural chemical diversity for desired molecules, although ORGAN is slightly better than RL.

    Source: Conclusion and future work · p. 4

  • Takeaways & Limitations

    The proposed challenge is to build a nontrivial generative model whose property-satisfying outputs reach the diversity found in nature for the same kind of molecules.

    Source: Introduction · p. 1

    ORGAN's discriminator quickly becomes perfect, and it might overfit the 15k-sample training set; drug-discovery datasets rarely exceed 10k molecules.

    Source: The perfect discriminator problem · pp. 4–5

Abstract

from arXiv · show

Generating molecules with desired chemical properties is important for drug discovery. The use of generative neural networks is promising for this task. However, from visual inspection, it often appears that generated samples lack diversity. In this paper, we quantify this internal chemical diversity, and we raise the following challenge: can a nontrivial AI model reproduce natural chemical diversity for desired molecules? To illustrate this question, we consider two generative models: a Reinforcement Learning model and the recently introduced ORGAN. Both fail at this challenge. We hope this challenge will stimulate research in this direction.

Introduction

AI generative models aim to produce promising drug candidates, but evaluation often relies on visual inspection or metrics less relevant to drug discovery. The paper quantifies internal chemical diversity, proposes comparing it with natural diversity for property-satisfying molecules, and illustrates the challenge with RL and ORGAN.

Introduction

  • Generated samples are often evaluated visually or with metrics that may not reflect the drug-discovery process, while chemical diversity matters because candidates can fail later in development.The paper argues that rigorous evaluation of internal chemical diversity is particularly needed.Source: Introduction · p. 1
  • Visual accounts disagree: prior work described RL as simplistic and ORGAN as less repetitive, but the authors' inspection judged ORGAN as repetitive and simplistic as RL.These assessments are based on visual inspection and are therefore subjective.Source: Introduction · p. 1
  • The paper introduces a metric for internal chemical diversity and asks whether a nontrivial model can match natural diversity among molecules satisfying a nontrivial chemical property.The challenge compares outputs with natural molecules of the same kind.Source: Introduction · p. 1
  • The challenge is illustrated by comparing RL and ORGAN on desired chemical properties, including activity against the dopamine D2 receptor, the main receptor for antipsychotic drugs.Source: Introduction · p. 1

2. Druglikeness

The paper's druglikeness measure averages four components and differs from the QED definition; experiments using QED did not change the authors' conclusions.

2. Druglikeness

  • This druglikeness definition differs from QED, an index of physicochemical properties related to oral drug action.Source: §2 · pp. 1–2
  • Here, druglikeness is the arithmetic mean of solubility, novelty, synthesizability, and conciseness.Novelty is scored as 1 outside the training set, 0.3 for a valid training-set SMILES, and 0 for an invalid SMILES; the other components are normalized as described in the passage.Source: §2 · p. 2
  • Experiments using the QED property did not affect the authors' conclusions.Source: §2 · p. 2

The metric of internal chemical diversity

The paper measures diversity within property-satisfying molecule sets using average pairwise Tanimoto distance, then frames the comparison against natural molecules as distinct from training-to-generation relative diversity.

The metric of internal chemical diversity

Internal diversity

  • A sufficiently large uniform sample from a sufficiently large molecule set has the same internal diversity, supporting estimation from a large generated sample.The passage attributes this property to the law of large numbers.Source: Internal diversity · p. 2

External diversity

  • External diversity compares the training set with a large generated sample, whereas the paper's central comparison uses internal diversity for generated and natural molecules satisfying the property of interest.Source: External diversity · p. 2

Generative Models

The paper describes reinforcement learning for generating SMILES with property-based rewards, then presents ORGAN as an adversarial extension that adds a discriminator-based reward component. ORGAN’s λ parameter interpolates between RL and SeqGAN.

Reinforcement Learning

  • For dopamine D2 activity, the reward uses a predictive model’s probability that a generated sequence is D2-active; for druglikeness, it uses the sequence’s druglikeness score.The paper notes that its D2 reward differs slightly from the reward in Olivecrona et al. 2017.Source: Reinforcement Learning · pp. 2–3
  • The RL generator treats each character as an action and the existing SMILES prefix as its state, choosing the next character according to its policy.The generator maximizes expected long-term reward; for incomplete sequences, it estimates Q values through Monte Carlo rollouts under the generator policy.Source: Reinforcement Learning · p. 3

Objective-Reinforced Generative Adversarial Network (ORGAN)

  • ORGAN adds a discriminator that distinguishes training-data SMILES from generated SMILES, and combines its signal with the objective reward.The discriminator’s output is the probability that a SMILES sequence belongs to the training data.Source: Objective-Reinforced Generative… · p. 3
  • The ORGAN parameter λ controls the blend: λ = 0 recovers RL, while λ = 1 gives SeqGAN.The paper describes the generator and discriminator as trained adversarially.Source: Objective-Reinforced Generative… · p. 3

Experiments

The experiments evaluate reinforcement learning and ORGAN outputs against reference molecule sets on D2 activity and druglikeness. Across both properties, selecting molecules that meet the target leaves diversity below the corresponding natural or training-set reference.

Experiments

Dopamine D2 activity

  • 2 orders of magnitude: ORGAN-0.04 diversity for Pa > 0.8 at 30 epochs exceeds RL 30, but remains one order of magnitude below DRD2; RL diversity also drops sharply at Pa > 0.8.The table reports validity, average activity and diversity, the fraction with Pa > 0.8, and diversity within that thresholded subset.Source: Dopamine D2 activity · p. 3, Table 1 · p. 4
  • ORGAN-0.5 has only 6 samples with Pa > 0.8 among 32k after 60 epochs, so its thresholded internal diversity is indetectable.The authors report high diversity overall but no learning of the D2 objective in this case.Source: Dopamine D2 activity · pp. 3–4
  • The intermediate λ cases resemble either λ = 0.04 or λ = 0.5, and the authors cannot locate the tipping point between slow learning and training that never takes off.Source: Dopamine D2 activity · p. 4

Druglikeness

  • 2 orders of magnitude lower than ZINC diversity when L > 0.8: ORGAN improves over RL to detectable diversity, but both models fail to generate diverse molecules for druglikeness.ORGAN diversity also remains 3 orders of magnitude below ZINC’s total diversity.Source: Druglikeness · p. 4

Conclusion and future work

Both RL and ORGAN fall short of natural chemical diversity, though ORGAN is slightly better; the authors identify discriminator perfection and objective imbalance as directions for improving ORGAN training.

Conclusion and future work

  • Both RL and ORGAN fail to match natural chemical diversity for desired molecules, although ORGAN is slightly better than RL.The authors identify two training problems for future work: a perfect discriminator and imbalance between objectives.Source: Conclusion and future work · p. 4

The perfect discriminator problem

  • In ORGAN training, the discriminator quickly becomes perfect, yet its average probability for generated samples to belong to training data remains above zero and below 0.5.That probability is transmitted to the generator through the reward function.Source: The perfect discriminator problem · p. 4
  • Even SeqGAN with λ = 1 cannot fool the discriminator, indicating generator weakness and inability to reproduce a plain druglike dataset like ZINC.The authors propose properly training SeqGAN as a first step toward improving ORGAN.Source: The perfect discriminator problem · p. 4
  • Possible discriminator remedies include a larger generator, replacing discriminator loss with another function, and one-sided label smoothing.CramerGAN is given as an example of an alternative loss.Source: The perfect discriminator problem · pp. 4–5
  • The discriminator might overfit training data; the authors used 15k samples, while drug-discovery datasets rarely exceed 10k molecules.They suggest larger training data could help, but also point to low-data predictive neural networks as relevant.Source: The perfect discriminator problem · p. 5

Imbalance in multi-objective RL

  • Future work includes using multiple adversarially trained discriminators and penalizing new molecules too similar to generated molecules already meeting desired properties.The authors also note that internal-diversity definitions should exclude trivial solutions such as reproducing the training set.Source: The perfect discriminator problem · p. 5, Imbalance in multi-objective RL · p. 5
  • The reward-function objectives can be imbalanced, and a weighted sum can make the agent focus on the easiest objective while ignoring harder ones.Relative objective difficulty can change over time; D2 activity probability initially grows exponentially, then grows less as it approaches zero.Source: Imbalance in multi-objective RL · p. 5
  • Adaptive, potentially nonlinear weights may focus generation on harder objectives, but the min(x, y) form did not work in these experiments because of the perfect discriminator problem.The relative weights must be determined automatically rather than by guesswork.Source: Imbalance in multi-objective RL · p. 5

Related papers

Loading 1708.08227v3…