PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion Models in $\textit{De Novo}$ Drug Design

Amira Alakhdar, Barnabas Poczos, Newell Washburn

arXiv:2406.08511v1physics.chem-phcs.LG

TL;DR

Three-dimensional molecular diffusion models have advanced, but the literature lacks a review comparing their recent implementations, evaluations, and performance. This review synthesizes those approaches and applications; it concludes that challenges remain in data availability, performance, and relevance to applications.

  • Problem

    Recent diffusion models for 3D molecular generation lacked a review comparing their performance, evaluation methods, and implementation details.

    Source: §1 · p. 2

  • Method

    The review synthesizes technical implementations of diffusion models for 3D molecular generation and their applications in drug discovery.

    Source: §1 · p. 2

  • Results

    The review concludes that data availability, model performance, and relevance to applications remain challenges for diffusion models in chemical applications.

    Source: §11 · p. 22

  • Takeaways & Limitations

    The review identifies drug discovery as a platform for further development of diffusion models.

    Source: §11 · p. 22

    Available training data remain limited and omit important properties such as pharmacokinetics and toxicity.

    Source: §10 · p. 21

Abstract

from arXiv · show

Diffusion models have emerged as powerful tools for molecular generation, particularly in the context of 3D molecular structures. Inspired by non-equilibrium statistical physics, these models can generate 3D molecular structures with specific properties or requirements crucial to drug discovery. Diffusion models were particularly successful at learning 3D molecular geometries' complex probability distributions and their corresponding chemical and physical properties through forward and reverse diffusion processes. This review focuses on the technical implementation of diffusion models tailored for 3D molecular generation. It compares the performance, evaluation methods, and implementation details of various diffusion models used for molecular generation tasks. We cover strategies for atom and bond representation, architectures of reverse diffusion denoising networks, and challenges associated with generating stable 3D molecular structures. This review also explores the applications of diffusion models in $\textit{de novo}$ drug design and related areas of computational chemistry, such as structure-based drug design, including target-specific molecular generation, molecular docking, and molecular dynamics of protein-ligand complexes. We also cover conditional generation on physical properties, conformation generation, and fragment-based drug design. By summarizing the state-of-the-art diffusion models for 3D molecular generation, this review sheds light on their role in advancing drug discovery as well as their current limitations.

1 Introduction

Three-dimensional molecular generation remains challenging because of molecular geometry and equivariance requirements, despite established string-based generation. This review addresses a gap in recent technical comparisons of diffusion models for 3D molecular generation and drug-design applications.

1 Introduction

  • 3D molecular generation lags string-based generation because molecular shapes are complex and models must satisfy E(3) and SE(3) equivariance requirements.The review links 3D geometry to molecular electrochemical properties and interactions with biological target pockets.Source: §1 · p. 1
  • Recent diffusion-model advances in 3D molecular generation were not covered by earlier reviews focused on models through May 2022.A related survey covering chemistry applications through April 2023 also addressed antibody, protein, and material design.Source: §1 · p. 2
  • The review compares diffusion-model implementation, evaluation, and performance, including atom and bond representations, reverse-diffusion architectures, and applications in drug discovery.It examines strategies for learning atoms and bonds together during reverse diffusion to avoid unstable molecules with inconsistent structures.Source: §1 · p. 2

2 Diffusion Models

The review describes DDPMs, score-based generative models, and score SDEs as diffusion formulations used to generate molecular data. It outlines their forward perturbation and learned or score-based reverse processes.

2 Diffusion Models

  • Diffusion models add noise to data and reverse that process to generate samples; DDPMs are the most popular formulation for molecular generation, though SGMs and score SDEs are also used.The three formulations address challenges including efficient sampling, likelihood estimation, and structured or discrete data.Source: §2 · p. 2

2.1 Denoising Diffusion Probabilistic Models (DDPMs)

  • DDPMs use a forward Markov chain to transform data into Gaussian noise and a learned reverse chain to recover data by denoising.The forward process uses preset β_t values; reverse transitions have neural-network-parameterized means and variances, and sampling iterates from Gaussian noise.Source: §2.1 · pp. 2–3, Figure 2 · p. 4

2.2 Score-Based Generative Models (SGMs)

  • SGMs perturb data with increasing Gaussian noise levels and train a noise-conditioned network to estimate the score of noisy data distributions.Sampling begins with Gaussian noise and can use annealed Langevin dynamics or other methods.Source: §2.2 · p. 4

2.3 Stochastic Differential Equations (Score SDEs)

  • Score SDEs extend diffusion to continuous time, using stochastic differential equations for perturbation and generation and score estimation for denoising.DDPMs and SGMs are described as time-discretized versions of the SDE; reverse-time sampling can use numerical solvers or predictor-corrector methods.Source: §2.3 · pp. 4–5

3 Molecular Representations

The section covers SMILES and graph representations used in diffusion-based molecular generation. It distinguishes 2D and 3D graph features and points to a catalog of commonly used datasets.

3 Molecular Representations

  • Diffusion models have been reported using SMILES, 2D graphs, and 3D graphs; DIFFUMOL combines diffusion and a Transformer to generate SMILES molecules with specified scaffolds and properties.SMILES represents molecular structure as a one-dimensional string.Source: §3 · p. 5
  • In 2D molecular graphs, atoms are node features and bonds are edges; 3D graphs also encode atom positions and may represent edges explicitly or through interatomic distances.Atom and bond types are encoded using one-hot embeddings.Source: §3 · p. 5
  • Table 1 summarizes datasets commonly used in molecular generation with diffusion models.Source: Table 1 · p. 6

4 Essential Requirements for Diffusion Models in Molecular Graph Generation

Molecular graph generation requires models to respect geometric transformations and node-order invariance while learning discrete atom and bond features. Generated molecules must also reflect chemical structures and their underlying data distributions.

4 Essential Requirements for Diffusion Models in Molecular Graph Generation

  • E(3)-equivariant models are invariant to translations, rotations, and reflections, whereas SE(3)-equivariant models are sensitive to chirality.Graph-based models can meet geometric requirements using GNNs and centering molecules at the center of mass.Source: §4 · p. 6
  • Graph generation must be permutation invariant, so reordering atoms, bonds, or adjacency-matrix rows and columns leaves the represented molecular graph unchanged.The passage says diffusion models overcame this issue, surpassing autoregressive models that depend on generated-node sequence.Source: §4 · p. 6
  • Gaussian-noise diffusion is challenging for molecular graphs because atom-type and bond-type features are discrete.The review discusses different forward and reverse diffusion methods for molecular graphs in later sections.Source: §4 · p. 6
  • A faithful generator must capture distributions of atom and bond types, scaffolds, functional groups, 3D conformations, and stereoisomers while producing chemically stable structures.Fidelity includes graph connectivity, valency, ionic charges, stable rings, and tolerable angle and torsional strains, which vary with the target distribution.Source: §4 · pp. 6–7

5 Forward Diffusion Process (Discrete vs Continuous)

Molecular diffusion handles discrete atom and bond categories through continuous or categorical noise, sometimes combined with continuous coordinate diffusion. The section also describes latent-space diffusion, which encodes molecules before diffusion and decodes sampled latents afterward.

5 Forward Diffusion Process (Discrete vs Continuous)

  • EDM applies continuous Gaussian diffusion to one-hot atom and bond features, tuning noise so reverse sampling recovers the original active category.MolDiff instead adds a none-type absorbing category and perturbs bonds before atom types and coordinates.Source: §5 · p. 7
  • Continuous diffusion supports classifier-free guidance, uncertainty modeling, and faster sampling with advanced ODE solvers, while discrete graph diffusion has been associated with lower MMD.The cited comparison frames lower MMD as indicating distributions closer to the original data distribution.Source: §5 · p. 7
  • MolDiff’s two-stage forward process first perturbs bonds, then atom types and coordinates toward a none-type absorbing category.The absorbing category is the added type toward which atom or bond types are gradually perturbed.Source: §5 · p. 7
  • DiGress applies categorical transition matrices to atom and bond types, while MiDi combines discrete graph diffusion with Gaussian-noised 3D coordinates and an adaptive schedule.MiDi’s schedule is tuned to predict bond types and coordinates before atom types during denoising.Source: §5 · pp. 7–8

5.1 Latent Graph Diffusion

  • GEOLDM encodes molecular structures into equivariant latent features before stable diffusion, whereas 3M-Diffusion encodes molecular graphs into text-aligned latents for description-conditioned generation.Generated latent samples are decoded back to molecular structures.Source: §5.1 · p. 8, Figure 3 · p. 8

6 Reverse Diffusion Denoising Neural Networks Architectures

Reverse diffusion uses denoising networks to remove forward-added noise, with transformers, GNNs, CNNs, or combinations serving as common architectures. The GNN examples span equivariant, shape-conditioned, force-field-feature, and transition-metal-complex generation approaches.

6.1 Graph Neural Networks (GNNs)

  • GNNs process molecular graphs through node-to-neighbor message passing, and variants are used in models including GDSS, EGNN-based methods, GCPNET, ShapeMol, and 3M-Diffusion.3M-Diffusion uses GINs to encode discrete molecular graphs into a continuous latent space.Source: §6.1 · pp. 8–9

6.1.1 EGNNs

  • EDM’s EGNN infers covalent bonds from coordinates, and this implicit representation is linked to atom-bond inconsistency, especially for larger drug-like molecules.DiffBridges improved results over EDM on QM9, but inconsistent atom types and bonds remained.Source: §6.1.1 · p. 9
  • MolDiff addresses EGNN atom-bond inconsistency with staged bond and atom diffusion, while HierDiff uses fragment-level hierarchical generation to avoid the issue in local environments.MolDiff generated a higher percentage of valid molecules than EDM, but did not report stability or atom valencies; HierDiff and GEOLDM embed structures into smaller domains using EGNNs.Source: §6.1.1 · pp. 9–10

6.1.2 GCPNET of the GCDM

  • GCDM uses GCPNET’s geometry-complete, chirality-aware message passing, but it did not improve 3D stability for larger GEOM-DRUGs molecules.GCPNET layers embed node and edge features into scalar and vector-valued values before geometry-complete message passing.Source: §6.1.2 · p. 10

6.1.3 ShapeMol

  • ShapeMol conditions generation on molecular shape, using an invariant GNN for atom features and an equivariant GNN for atomic coordinates.Source: §6.1.3 · p. 10

6.1.4 MACE of the SiMGen Model

  • SiMGen combines MACE force-field features from the million-molecule SPICE dataset with time-varying local similarity kernels for arbitrary-size generation and conditional generation.The model uses a score-based diffusion formulation without altering the model for conditional generation.Source: §6.1.4 · p. 10

6.1.5 LigandDiff

  • LigandDiff denoises transition-metal complexes with MPNNs and embeds molecules with GVPs, conditioning on context such as heavy-atom count and metal identity.Source: §6.1.5 · p. 10

6.2 Convolutional Neural Networks (CNN)

The section describes CNN and transformer denoisers for molecular generation. Transformers are used to capture long-range dependencies in molecular graphs, and message-passing transformer models are reported as successful relative to early EGNN-based models.

6.2.1 SchNet of the MDM Model

  • MDM adapts SchNet embeddings and uses separate local-edge and global-edge score networks for covalent bonding and van der Waals interactions, improving GEOM-DRUGs 3D stability over EDM.Source: §6.2.1 · pp. 10–11

6.2.2 3D U-Net of the VoxMol model

  • VoxMol represents atom types as Gaussian-like densities on 3D voxel grids and uses a 3D U-Net denoiser, achieving notable GEOM-DRUGs molecular stability.Generated voxel grids are converted back into molecules using peak detection.Source: §6.2.2 · p. 11

6.3.1 Diffusion Graph Transformer (DGT) of the JODO Model

  • JODO’s diffusion graph transformer jointly models node and edge dependencies, using self-conditioning from prior-step adjacency matrices and achieving successful conditional and joint unconditional 2D/3D design.Source: §6.3.1 · p. 11

6.3.2 Dual Track Transformer Network (DTN) of the GFMDiff Model

  • GFMDiff’s DTN uses atom, pairwise-distance, and triple-wise-angle features to predict atom types, valencies, and coordinates; its geometric-facilitated loss supports favorable molecular and atomic stability.Source: §6.3.2 · p. 11

6.3.3 Molecule Unified Transformer (MUformer) of the MUDiff Model

  • MUDiff’s MUformer combines 2D structure and 3D geometry channels, generating valid and stable 3D geometries and supporting property prediction on QM9.Source: §6.3.3 · pp. 11–12

6.3.4 DiGress

  • DiGress uses discrete graph denoising with atom and bond features augmented by structural, valency, and molecular-weight features.Source: §6.3.4 · p. 12

6.3.5 MiDi

  • MiDi jointly predicts 2D and 3D molecular graphs, allowing charged atoms and predicting coordinates and bonds before atom types and formal charges.Its denoiser integrates relaxedEGNN layers and pools edge, node, pairwise, and global features into node representations.Source: §6.3.5 · p. 12

6.4 Hybrid Architectures

Hybrid denoisers combine GNN message passing and transformer attention, while pretrained graph networks can condition diffusion on molecular shape.

6.4 Hybrid Architectures

  • CDGS combines local GINE message passing with global transformer attention, while Diff-Shape uses pretrained graph conditioning on ligand shapes, including high-docking-score poses.Diff-Shape combines pretrained GrCN with pretrained unconditioned MiDi.Source: §6.4 · p. 12

7 Molecular Generation for Structure-Based Drug Design (SBDD)

For structure-based drug design, diffusion models condition 3D molecular generation on protein pockets and can also guide generation through interaction, property, and ligand prompts. The review describes methods for full-molecule and fragment-level design, with reported gains on some binding or drug-likeness measures and remaining limitations on others.

7 Molecular Generation for Structure-Based Drug Design (SBDD)

  • DiffSBDD supports fixed-pocket conditioning, joint ligand–pocket generation, and inpainting for partial-molecule or fragment generation while preserving the rest of a structure.The review connects these approaches to lead optimization, scaffold hopping, fragment growing, and linker design.Source: §7 · p. 12
  • Although DiffSBDD and TargetDiff achieved better Vina Scores than cited non-diffusion models, the review notes that SAS and QED performance still needs improvement.Source: §7 · p. 12
  • DecompDiff’s pocket prior enhanced binding versus TargetDiff, while DECOMPOPT improved QED and SAS while maintaining binding affinity through molecular-arm conditioning.Source: §7 · p. 14
  • KGDiff achieved an average Vina Score 46.2% higher than TargetDiff on CrossDocked2020.Source: §7 · p. 14
  • BindDM adaptively extracts binding subcomplexes during reverse diffusion, while IPDiff uses a pretrained interaction prior with prior-shifting or prior-conditioning strategies.Source: §7 · p. 14
  • PROMPTDIFF steers generation with ligand prompts and protein–molecule interaction embeddings, using self-promoting or exemplar-prompting approaches.Source: §7 · p. 14
  • MOOD uses property-predictor gradients to guide reverse diffusion and reports high Novel top 5% docking scores for generated compounds across five protein targets.PILOT instead combines pocket conditioning and synthetic-accessibility guidance, using importance sampling at inference.Source: §7 · p. 14
  • VoxBind uses Langevin MCMC sampling followed by a conditional denoiser, while MolSnapper guides generation with 3D pharmacophores for target interactions.Source: §7 · p. 14

8 Other Applications

The review covers diffusion applications beyond molecular generation, including fragment and linker design, conformer sampling, docking, and molecular dynamics. Reported results vary by task and benchmark.

8.1 Fragment-Based Drug Design and Linker Design

  • DiffLinker connects disconnected fragments and can predict attachment sites and linker size, while FragDiff and AutoFragDiff generate pocket-conditioned fragments.Source: §8.1 · p. 15
  • SILVR conditions reverse diffusion on noisy reference fragments derived from hits screened against a protein binding site.Source: §8.1 · p. 15

8.2 Conformations Generation

  • Geodiff outperformed DGSM on GEOM-DRUGS matching and coverage scores; Torsional Diffusion operates on torsional angles, and SDEGen reported high coverage and matching scores on QM9 and GEOM-DRUGs.Source: §8.2 · pp. 15–16

8.3 Molecular Docking

  • DiffDock retrieved 38.2% of PDBBind ligand poses below 2 Å RMSD and 63.2% below 5 Å, while NeuralPLexer reached up to 78% below 2 Å on PDBBind2020.The review also notes that PoseBusters found physically invalid poses for deep-learning tools including DiffDock; its assessment covered DiffDock, not later models.Source: §8.3 · pp. 16–17
  • PocketCFDM reached 23.9% accuracy at a 2 Å RMSD cutoff on PDBBind, below DiffDock, but performed better on non-bond interactions, steric clashes, and inference speed.Source: §8.3 · p. 17

8.4 Molecular Dynamics

  • DiffMD outperformed EGNN baselines on MD17 and C7O2H10 isomers, while a denoising force field improved protein simulations for systems up to 56 amino acids.Source: §8.4 · p. 17

9 Evaluation Metrics

This section organizes molecular-generation evaluation around chemical validity, distributional similarity, drug-likeness, conditional-property accuracy, docking, and conformer quality. The appropriate metrics depend on the generation task and representation.

9.1 Evaluation Metrics for Molecular Generation

  • Validity, novelty, uniqueness, atom stability, molecule stability, and connectivity assess distinct aspects of generated-molecule quality, from chemical correctness to dataset novelty and graph connectedness.Validity is often used for 2D graphs and SMILES and evaluated with RDKit; stability metrics are usually used for 3D generation.Source: §9.1 · pp. 17, 19

9.2 Similarity Metrics

  • MMD and Jensen-Shannon divergence compare generated and training distributions of bond lengths, bond angles, and dihedral angles.MMD measures distance through embeddings in a reproducing kernel Hilbert space, while JS divergence uses Kullback-Leibler divergence.Source: §9.2 · p. 19

9.3 Drug-Likeness Metrics

  • Drug-likeness metrics include logP for lipophilicity, QED for likelihood of becoming a drug, SAS for synthetic feasibility, and Lipinski-rule satisfaction.SAS combines fragment scores and a complexity penalty; Lipinski summarizes satisfaction of the rule of five.Source: §9.3 · p. 19

9.4 Evaluation Metrics for Propriety-Based Conditional Generation

  • For property-conditioned generation, MAE between requested and predicted properties evaluates accuracy, often using a classifier trained on half of QM9 while the model trains on the other half.Common conditioning properties include polarizability, dipole moment, HOMO and LUMO energies, their difference, and heat capacity.Source: §9.4 · pp. 19–20

9.5 Evaluation Metrics for Structure-Based Generation

  • Structure-based generation uses Vina scores and energies to estimate binding affinity, high-affinity percentage against a reference molecule, and fingerprint-based pairwise dissimilarity for diversity.Vina Min adds local minimization before scoring, while Vina Dock re-docks molecules before scoring.Source: §9.5 · p. 20

9.6 Evaluation Metrics for Conformations Generation

  • Conformer evaluation uses aligned RMSD and derives coverage and matching metrics for precision and recall across generated and reference conformer sets.Coverage counts structures within an RMSD threshold; matching or average minimum RMSD measures nearest-neighbor RMSD, with lower MAT scores indicating more realistic conformations.Source: §9.6 · pp. 20–21

10 Current Limitations

The review identifies limitations involving chirality, practical relevance and interpretability, data, benchmarking, computational cost, and target-aware molecular generation.

10 Current Limitations

  • Most diffusion models are E(3)-equivariant and insensitive to chirality; the review says this has been considered in GCDM and GCPNET reverse networks.Chirality should also be considered when designing new models or improvements.Source: §10 · p. 21
  • Molecular-generation metrics do not guarantee real-world utility, and black-box models make it difficult to trace generated molecules or identify chemical-space biases.SAS and Vina scores are estimates, not guarantees of library availability, synthesizability, or cost efficiency.Source: §10 · p. 21
  • Limited chemical data can omit pharmacokinetics and toxicity and bias models toward known chemical space, especially when similarity metrics such as MMD are rewarded.The review notes synthetic data generation for pharmacokinetic properties, toxicity, and hERG as one response in some applications.Source: §10 · p. 21
  • There is no unified 3D-generation benchmark, and models differ in how they define stability and stable atoms.MiDi permits specified ionized valencies, whereas JODO and EDM require formal charge zero; QM9 conditional-property evaluation also relies on an EGNN predictor with error relative to DFT.Source: §10 · p. 21
  • Training and sampling costs are high for complex molecular data and increase with molecule size.The review notes that new architectures and training strategies may improve training and sampling.Source: §10 · p. 21
  • In target-aware generation, DiffSBDD and TargetDiff produced highly strained molecules with high steric clashes against protein pockets in the PoseCheck study.The study suggested penalizing steric clashes.Source: §10 · p. 21

11 Conclusions

The review concludes that diffusion models have potential across chemical applications, while data availability, model performance, and application relevance remain challenges.

11 Conclusions

  • Diffusion models have early successes across chemical applications, but further work must address data availability, model performance, and relevance to specific applications.The review identifies drug discovery as a platform for further diffusion-model development, given its large datasets and decades of research.Source: §11 · p. 22

Related papers

Loading 2406.08511v1…