PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring Chemical Space with Score-based Out-of-distribution Generation

Seul Lee, Jaehyeong Jo, Sung Ju Hwang

arXiv:2206.07632v3q-bio.BMcs.LGphysics.chem-ph

TL;DR

Molecular generative models often produce molecules resembling their training data, limiting exploration of novel chemical space. MOOD combines OOD-controlled score-based diffusion with property-predictor guidance. It outperforms existing methods in multi-objective molecule optimization, including novel molecules with binding affinity above the training set's top 0.01%.

  • Problem

    Existing molecular generative models tend to produce molecules resembling their training data, while drug discovery seeks novel molecules satisfying target properties.

    Source: §1 · p. 1

  • Method

    MOOD controls out-of-distribution generation in reverse-time score-based diffusion and uses property-predictor gradients to guide sampling toward target properties.

    Source: §1 · p. 2

  • Results

    MOOD outperforms existing methods across five protein targets, generating novel molecules with higher binding affinity than the training set's top 0.01%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MOOD's optimization generates novel molecules with high docking scores while satisfying drug-likeness and synthesizability conditions.

    Source: §1 · p. 2

    MOOD does not account for properties such as yield, synthesis cost, or toxicity, and it generates a small portion of unrealistic molecules.

    Source: A. Limitations and Potential Societal… · p. 12

Abstract

from arXiv · show

A well-known limitation of existing molecular generative models is that the generated molecules highly resemble those in the training set. To generate truly novel molecules that may have even better properties for de novo drug discovery, more powerful exploration in the chemical space is necessary. To this end, we propose Molecular Out-Of-distribution Diffusion(MOOD), a score-based diffusion scheme that incorporates out-of-distribution (OOD) control in the generative stochastic differential equation (SDE) with simple control of a hyperparameter, thus requires no additional costs. Since some novel molecules may not meet the basic requirements of real-world drugs, MOOD performs conditional generation by utilizing the gradients from a property predictor that guides the reverse-time diffusion process to high-scoring regions according to target properties such as protein-ligand interactions, drug-likeness, and synthesizability. This allows MOOD to search for novel and meaningful molecules rather than generating unseen yet trivial ones. We experimentally validate that MOOD is able to explore the chemical space beyond the training distribution, generating molecules that outscore ones found with existing methods, and even the top 0.01% of the original training pool. Our code is available at https://github.com/SeulLee05/MOOD.

1. Introduction

MOOD targets a central limitation of molecular generation: training-distribution methods often produce near-training analogs, while real-world discovery requires novel molecules meeting multiple property constraints. It combines OOD-controlled diffusion with property guidance and outperforms existing methods on molecule optimization.

1. Introduction

  • MOOD addresses the infeasible breadth of chemical space, where examining every possible molecule for target properties is not practical.Its generation framework aims to find novel candidates meeting specified constraints.Source: §1 · pp. 1–2
  • Distributional models often generate molecules resembling training examples, limiting exploration beyond the training distribution.Novel core structures can be needed for further optimization or to avoid already patented scaffolds.Source: §1 · p. 1
  • Common optimization targets such as penalized logP and QED may not identify useful drugs, and docking scores alone can favor heavy-molecule false positives.Real-world discovery also seeks molecules that satisfy requirements such as drug-likeness and synthesizability.Source: §1 · pp. 1–2
  • OOD-controlled reverse-time diffusion adjusts deviation from the training distribution, while property-predictor gradients guide sampling toward high-scoring molecules.MOOD combines exploration beyond known molecules with conditional optimization without additional computational costs.Source: §1 · p. 2
  • Higher binding affinity than the training set's top 0.01%: MOOD generated novel molecules and outperformed existing methods while meeting QED and SA conditions.The experiments covered five protein targets.Source: §1 · p. 2, Figure 1 · p. 2

2. Related Work

Prior molecule-generation approaches span several model families, but distributional learning and fragment vocabularies tend to limit exploration beyond known molecules. MOOD applies score-based diffusion to conditional molecule optimization and OOD generation.

2. Related Work

  • Score-based models had been applied to graph generation, conformation generation, and 3D molecule generation, but targeted de novo optimization poses a distinct challenge.The challenge is finding novel molecules that satisfy specific constraints in vast chemical space.Source: §2 · pp. 2–3
  • Directly adapting conditional score-based methods from image tasks to molecule optimization is challenging because node-edge dependencies determine molecular validity and properties.MOOD combines conditional reverse-time diffusion with property-predictor guidance toward low-density, high-property regions.Source: §2 · p. 3
  • Distributional-learning and fragment-vocabulary methods commonly explore within known data, while exploration-promoting reinforcement learning is computationally expensive and fragment-limited.MOOD is presented as generating molecules beyond the training distribution without high computational costs or a fragment vocabulary.Source: §2 · p. 3

3. Molecule Optimization with Score-based Out-of-distribution Generation

MOOD seeks molecules that differ from the training distribution while satisfying desired chemical properties. It combines OOD-controlled reverse diffusion with property-guided sampling.

3. Molecule Optimization with Score-based Out-of-distribution Generation

  • MOOD targets molecules novel relative to training data while seeking desired chemical properties through OOD-controlled, property-guided diffusion.The framework introduces OOD-controlled diffusion and then property-guided sampling.Source: §3 · p. 3, §3.2 · p. 5

3.1. Score-based out-of-distribution generation

  • On a Gaussian mixture, increasing λ generated a wide scope of OOD samples, while λ = 0 (GDSS) generated samples from the data distribution.The OOD control uses λ to adjust deviation toward lower-density regions without additional computation.Source: §3.1 · p. 4

3.2. Molecule property optimization

  • MOOD adds property-predictor guidance to OOD-controlled reverse diffusion, targeting molecules in low-density regions with high desired-property scores.The method uses reverse-time SDEs to model node-edge relationships, and sets guidance scales according to a predefined ratio of score and property-gradient magnitudes.Source: §3.2 · p. 5

4. Experiments

The experiments test whether MOOD can control exploration beyond the training distribution while optimizing for drug-relevant properties. Across molecule-generation and property-optimization tasks, results support controllable OOD generation and improved discovery of novel chemical optima.

4.1. Novel molecule generation

  • Increasing λ moves generated molecules farther from the training distribution and increases FCD, NSPDK MMD, and novelty, making OOD deviation controllable.The unconstrained generation experiment generates 3,000 molecules while varying λ.Source: §4.1 · p. 5, Figure 3 · p. 6

4.2. Property optimization

  • The optimization objective combines normalized docking score, QED, and normalized synthetic accessibility, and MOOD is evaluated on five protein targets with λ = 0.04.Evaluation includes novel hit ratio and novel top 5% docking score, which jointly assess novelty and multiple properties.Source: §4.2 · p. 6
  • MOOD significantly outperforms all baselines on most target proteins, while showing competitive results with FREED and FREED-QS on fa7.The reported comparisons use novel hit ratio and novel top 5% docking score.Source: §4.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
  • UMAP and distributional-distance results show MOOD samples extending beyond ZINC250k more than baseline samples, with broad chemical-space coverage in diversity and novelty.The reported distances are FCD and NSPDK MMD.Source: §4.2 · pp. 6–7, Figure 4 · p. 8
  • MOOD found parp1 hit molecules with higher binding affinity than the top 0.01% of ZINC250k molecules, while retaining low similarity to ZINC250k.Figure 6 depicts the novel hits and top-0.01% training-set molecules.Source: §4.2 · p. 7, Figure 6 · p. 8

4.3. Ablation study

  • Comparisons among MOOD variants and GDSS indicate that both OOD control and property-predictor guidance contribute to finding better chemical optima.The ablations compare MOOD with variants lacking either OOD control or property prediction.Source: §4.3 · p. 7
  • Trained on the lower-property half of ZINC250k, L-MOOD produces a higher top 5% docking score than the original dataset despite never seeing its higher-property half.The comparison uses molecules satisfying QED > 0.5 and SA < 5.Source: §4.3 · p. 8, Figure 7 · p. 8

5. Conclusion

MOOD is a score-based generative model designed to generate novel molecules with desired properties beyond the training distribution. Its OOD-controlled reverse diffusion and property-gradient guidance are validated on multi-objective tasks, where it outperforms existing methods in explorability and property optimization.

  • MOOD is a score-based generative model for generating novel molecules with desired chemical properties outside the training distribution.The conclusion names the approach Molecular Out-Of-distribution Diffusion.Source: §5 · p. 8
  • Its OOD-controlled reverse-time diffusion controls sample OOD-ness without additional costs, while property-predictor gradients guide sampling toward high-property regions.Source: §5 · p. 8
  • On multi-objective tasks analogous to real-world scenarios, MOOD outperforms existing molecular generation methods in explorability and property optimization performance.The conclusion presents this as evidence of its potential for de novo drug discovery.Source: §5 · p. 8

A. Limitations and Potential Societal Impacts

The supplement identifies omitted real-world properties as a limitation, derives the OOD-controlled reverse process, and distinguishes MOOD's control from a prior low-density image sampler.

A. Limitations and Potential Societal Impacts

  • MOOD does not consider yield, synthesis cost, or toxicity, which the authors plan to add to its property-optimization framework.The authors also note that it generates a small portion of unrealistic molecules, which could be filtered using chemical-knowledge algorithms.Source: A. Limitations and Potential Societal… · p. 12

B. Deriving the OOD-controlled Diffusion Process

  • The derivation models p_t(y_o = λ|G_t) as proportional to the negative exponent of p_t(G_t), then uses a G_t-independent normalization constant to recover the proposed OOD-controlled diffusion process.The resulting process corresponds to Eq. (6) in the main paper.Source: B. Deriving the OOD-controlled… · p. 12

C. Additional Remark on Related Work

D.1. Toy experiment

The supplement describes the toy-experiment setup, molecule-visualization and novelty procedures, and implementation choices for property optimization.

D.1. Toy experiment

  • The toy experiment uses a bivariate Gaussian mixture and generates 213 samples with a PC sampler at SNR 0.05 and scale coefficient 0.8.The score network has 20 residual linear layers and hidden dimension 512, uses VPSDE with β_min = 0.01 and β_max = 0.05, and trains for 5,000 epochs with batch size 2,048.Source: §D.1 · p. 12

D.2. UMAP visualization

  • The UMAP visualization combines ChemNet activations for 5,000 ZINC250k molecules and 3,000 molecules each from REINVENT, FREED-QS, MOOD without OOD control, and MOOD.The generated sets target parp1.Source: §D.2 · p. 12

D.3. Novel molecule generation

  • Novelty is the fraction of valid molecules whose nearest training molecule has Tanimoto similarity below 0.4, using radius-2, 1,024-bit Morgan fingerprints.Molecules are represented as graphs; preprocessing kekulizes structures, removes hydrogens, and applies valency correction.Source: §D.3 · p. 13

D.4. Property optimization

  • Property optimization evaluates five target proteins selected for high AUROC when AutoDock Vina approximates DUD-E ligand binding affinities.The targets are parp1, fa7, 5ht1b, braf, and jak2.Source: §D.4 · p. 13
  • Time-dependent magnitude ratios schedule the property-guidance scaling coefficients during diffusion, with r1,0 and r2,0 denoting their final values at t = 0.The supplement reports protein-specific r1,0 settings and that r2,0 = 0 performs reasonably well across targets.Source: §D.4 · pp. 13–14
  • Novel hit ratio requires unique hits with similarity below 0.4 to training molecules, docking score above the active-molecule median, QED above 0.5, and SA below 5.The novel top 5% docking score averages docking scores among the top 5% of unique generated molecules satisfying QED > 0.5, SA < 5, and similarity < 0.4.Source: §D.4 · pp. 13–14
  • Baseline implementations use differing setups, including two-cycle active learning for HierVAE at twice the docking-computation cost of other methods.The supplement also specifies baseline datasets, rewards, training choices, and oracle-call limits.Source: §D.4 · p. 14
  • For the 3D-generation comparison, GDSS and MOOD are trained on 96.77% of CrossDocked2020's original training set, using glmu as the target protein.The hit threshold is the test-set reference ligand's docking score.Source: §D.4 · p. 14

D.5. Computing resources

The supplement lists the GPUs used and identifies the scopes and reporting conventions of three results tables.

D.5. Computing resources

  • Experiments ran on TITAN RTX, GeForce RTX 2080 Ti, or GeForce RTX 3090 GPUs.Source: §D.5 · p. 14
  • Table 4 reports novel-molecule generation results on QM9.Source: Table 4 · p. 15
  • Table 5 reports novel hit ratio percentages as means and standard deviations over five runs.Source: Table 5 · p. 15
  • Table 6 reports novel top 5% docking scores in kcal/mol as means and standard deviations over five runs.Source: Table 6 · p. 15

E. Additional Experimental Results

The supplement adds a QM9 novel-molecule-generation experiment to assess whether OOD control can regulate generated samples' OOD-ness.

E. Additional Experimental Results

  • The QM9 experiment reports FCD, NSPDK MMD, and novelty to assess control over generated samples' OOD-ness.Source: §E.1 · p. 15

E.2. Property optimization

MOOD outperforms existing methods on property optimization across most target proteins, with a larger performance gap under the 0.3 novelty condition. The results also show that balancing OOD control against property guidance matters for finding novel, high-quality molecules.

E.2. Property optimization

  • MOOD outperforms baselines on most target proteins, with a larger performance gap under the harsher 0.3 novelty condition.The comparison covers novel hit ratio and novel top 5% docking score.Source: §E.2 · pp. 15–16, Table 7 · p. 16, Table 8 · p. 16
  • MOOD reports novelty, diversity, uniqueness, hit ratio, and top 5% docking score; high novelty alone is uninformative when molecules lack quality or meaningful binding affinity.MORLD and LIMO often fail QED and SA constraints or lack high binding affinity; HierVAE has very low validity, limiting the information in its FCD and NSPDK MMD values.Source: §E.2 · p. 17, Table 9 · p. 17, Table 10 · p. 17
  • Increasing the latent sampling variance in LIMO does not produce novel, high-quality drugs and instead degrades performance.The authors explain that this sampling distribution does not match the standard normal targeted by the ELBO objective.Source: §E.2 · p. 19, Table 15 · p. 19
  • λ = 0.04 yields better novel hit ratio and novel top 5% docking score than λ = 0.03 or 0.05.The authors attribute the difference to the need to balance OOD control and property gradients because molecules farther from the training distribution tend to be lower quality.Source: §E.2 · p. 19, Table 16 · p. 19
  • MOOD’s OOD control uses λ without additional computation, and its sampling time is about the same as random-generation baselines.The authors report that MOOD largely outperforms methods requiring on-the-fly oracle calls in generation time.Source: §E.2 · p. 19, Table 17 · p. 20
  • The appendix also reports Dockstring benchmark F2 task results, averaged across three runs.The supplied passages do not state the F2 outcome.Source: §E.2 · p. 19, Table 18 · p. 20

E.3. Additional molecule samples

MOOD’s generated hits do not share large substructures with their most similar training molecules, unlike baseline samples. The figures also show hit-molecule similarities, docking scores, and binding poses, including hits reported to exceed the top 0.01% of ZINC250k molecules in binding affinity.

E.3. Additional molecule samples

  • Even against their maximally similar training molecules, MOOD hits do not share large substructures, unlike baseline-generated molecules.Figure 8 pairs generated hits with their nearest-similarity ZINC250k molecules and shows each hit’s similarity and docking score.Source: §E.3 · p. 20, Figure 8 · p. 21
  • MOOD’s novel hits are reported to have higher binding affinity than the top 0.01% of ZINC250k molecules.Figure 9 visualizes the binding poses of these hits.Source: §E.3 · p. 20, Figure 9 · p. 21

Related papers

Loading 2206.07632v3…