PaperlayerView exact revision on arXiv

Source-linked AI summary

A Systematic Evaluation of Molecule Generation Models for De Novo Drug Design: From Benchmarks to Practical Insights

Xinrui Xu, Xueer Wang, Dan Luo, Sisi Yuan, Xuan Lin

arXiv:2609.10099v1cs.LG

TL;DR

Existing reviews often treat molecule-generation model families or application scenarios separately, limiting integration across the generation workflow. This review synthesizes 82 models across five frameworks, benchmarks their reported performance, and connects computational evaluation with experimentally validated cases and practical limitations.

  • Problem

    Existing reviews provide limited integration across representations, generative frameworks, pocket-conditioned strategies, and benchmark interpretation.

  • Method

    The review organizes 82 molecule-generation models across five frameworks, summarizes representations and training objectives, compares benchmark results, and examines experimentally validated case studies.

  • Results

    The comparison shows that model performance varies across representations, datasets, task settings, and evaluation protocols, with improvements in one metric not necessarily translating into consistent gains elsewhere.

  • Takeaways & Limitations

    Reliable interpretation requires standardized benchmarks, interaction-aware and receptor-flexible generation, synthesis- and property-aware constraints, and closer integration with experimental validation.

  • Takeaways & Limitations

    Reported results remain difficult to compare because studies use heterogeneous benchmark settings and simplified receptor and interaction modeling.

Abstract

from arXiv · show

Molecule generation has emerged as a powerful computational tool for de novo drug design, enabling the exploration of chemical space beyond the limits of conventional virtual screening. The field has progressed rapidly, driven by advances in molecular representations, generative architectures, and target-aware modeling strategies. However, existing reviews typically address specific model families or application scenarios in isolation, rather than offering an integrated perspective on how these components collectively form a coherent generation workflow. In this review, we present a comprehensive evaluation of molecule generation models for de novo drug design, covering 82 methods across five deep generative frameworks, including recurrent neural network (RNN)- and Transformer-based models, variational autoencoders (VAEs), generative adversarial networks (GANs), flow-based models, and diffusion models. We first summarize widely used benchmarks and molecular representations, and then examine the methodological principles underlying both general and pocket-conditioned generation. A central contribution of this work is a systematic synthesis and comparative analysis of reported performance across commonly used benchmarks and evaluation metrics. We also summarize representative experimentally validated case studies. Looking ahead, we discuss future directions in standardized 3D data, interaction-aware generation, receptor flexibility, and multi-objective molecular design, with the aim of improving the reliability and experimental relevance of molecule generation. All collected benchmark resources, evaluation metrics, and model references are provided in a publicly accessible repository at https://github.com/JacklinGroup/molecule-generation-review.

1 Introduction

This review integrates molecule-generation representations, model families, benchmarks, experimental validation, and future directions into a unified de novo drug-design workflow. It covers 82 models across five generative frameworks and compares reported performance and practical utility.

  • Motivation: De novo drug design constructs molecules from scratch to explore chemical space beyond predefined compound libraries and close analogues.Molecule generation supports learning molecular distributions and proposing chemically valid, diverse structures for later evaluation and prioritization.
  • Workflow: The review organizes molecule generation as a workflow from curated molecular and target-structure data through representations, generative models, and molecular outputs.Representations include sequences, graphs, 3D structures, and protein-environment encodings such as grids, geometric graphs, and latent embeddings.
  • Model taxonomy: The review provides a taxonomy of 82 representative models across five generative frameworks, offering a structured view of the expanding field.The frameworks include RNN/Transformer-based, VAE-based, GAN-based, flow-based, and diffusion-based models.
  • Benchmark synthesis: It systematically compiles and compares reported results across common benchmarks and metrics, including docking protocols for pocket-conditioned generation.The review examines how performance and practical utility should be evaluated and interpreted across benchmarks.
  • Experimental validation: Eight representative case studies connect computationally generated molecules with synthesis and experimental testing across diverse targets and diseases.The cases are analyzed by generative strategy, experimental validation, and lessons for translating generated molecules into validated candidates.
  • Resources and future directions: The review provides an organized, continuously updated repository of benchmark datasets, original papers, code, evaluation resources, and model references.It also discusses limitations and future directions toward more reliable and decision-oriented molecule generation.

2 Benchmarking and Evaluation of Molecule Generation

Benchmarking molecule generation requires datasets, evaluation frameworks, representations, and splitting strategies that capture chemical, geometric, conformational, and structure-based performance. The reviewed resources support comparisons across general generation, optimization, 3D quality, and pocket-conditioned tasks.

  • Benchmark datasets: Benchmark datasets span ligand-only corpora, quantum-chemical data, conformer ensembles, protein–ligand complexes, structural repositories, and drug-centered databases.These resources determine available molecular and biological information, generation tasks, metrics, and generalization settings.
  • Benchmark datasets: QM9 contains approximately 134,000 small neutral organic molecules with structures and computed geometric, energetic, electronic, and thermodynamic properties.It is used for molecular property prediction and benchmarking two- and three-dimensional generation, including validity and geometric quality.
  • Benchmark datasets: CrossDocked2020 contains approximately 22.5 million docked ligand poses across structurally related binding sites for pocket-conditioned generation and docking evaluation.Filtered subsets separate training and test pockets using protein-similarity criteria to assess generalization to unseen or dissimilar pockets.
  • Evaluation benchmarks: GuacaMol standardizes distribution-learning and goal-directed evaluation using validity, uniqueness, novelty, KL divergence, FCD, and objective scores.It includes five distribution-learning benchmarks and 20 goal-directed benchmarks, with baseline methods evaluated for each group.
  • Evaluation design: Dataset splitting strategies include random, scaffold-based, molecule-level, and protein-level splits, each defining a distinct generalization setting.Protein-level splitting is especially relevant when evaluating pocket-conditioned generation across binding sites.
  • Evaluation benchmarks: PoseBusters evaluates protein–ligand poses through chemical consistency, intramolecular geometry, and intermolecular interaction checks.A pose passing all applicable checks is classified as PB-valid.

3 Representation

Molecular representation determines what chemical and interaction information generative models preserve, shaping their architectures, learning objectives, and chemical constraints. Representations range from 1D strings and 2D graphs to 3D geometries and multi-level protein-pocket encodings.

  • Representation choices influence validity constraints, topology, stereochemistry, conformational behavior, and protein–ligand interaction modeling.These choices determine which structural and interaction cues are available to the generative model.
  • 1D representations: SMILES encodes molecular graphs as compact linear strings suited to sequence models but vulnerable to syntax errors and atom-ordering effects.
  • 2D representations: 2D graph representations encode atoms as nodes and bonds as edges with chemical, stereochemical, and connectivity features.Graph neural networks aggregate information across neighboring atoms to learn local and global molecular features.
  • 3D representations: 3D representations support conformation-aware generation, joint topology–geometry modeling, and protein–ligand interaction modeling through E(3)- or SE(3)-equivariant architectures.
  • Protein-pocket representations: Protein information can be represented through sequences, residue relationships, or explicit three-dimensional pocket structures, with sequence-based encodings reflecting spatial chemistry only indirectly.

4 Deep Generative Frameworks for Molecule Generation

Deep molecule-generation frameworks learn molecular distributions using distinct representations, objectives, and sampling mechanisms. The section spans autoregressive sequence models, VAEs, GANs, and representative molecular-generation pipelines across sequence, graph, and 3D forms.

  • Deep generative models provide reusable foundations for autoregressive sequence modeling, latent-space sampling, distribution matching, invertible transformation, and iterative denoising.
  • RNN/Transformer-based models: RNNs and Transformers autoregressively model tokenized molecular sequences, generating molecular strings that are subsequently parsed into molecular graphs.
  • RNN/Transformer-based models: RNNs recurrently update hidden states, whereas Transformers use causal self-attention to access preceding tokens and model long-range dependencies.
  • VAE-based models: VAEs encode 1D, 2D, or 3D molecular representations into a regularized Gaussian latent space and decode samples into novel structures.
  • VAE-based models: VAE variants impose representation-specific structures, including formal grammars for SMILES, direct graph decoding, and equivariant generation of linker graphs with three-dimensional coordinates.
  • GAN-based models: GAN pipelines generate molecular representations from latent noise and use a discriminator to distinguish generated samples from real molecules.

4.3 GAN-based Models

The section contrasts GAN-based adversarial generation with flow-based and diffusion-based approaches. GANs learn through generator–discriminator competition, flows use transformations or probability updates, and diffusion models generate molecules through iterative denoising.

  • GAN-based models: GANs formulate molecule generation as adversarial learning between a generator and discriminator.
  • GAN-based models: A GAN generator maps latent noise, optionally combined with property information, into molecular graphs, SMILES sequences, or scaffold-conditioned representations.
  • GAN-based models: GAN-based models extend adversarial learning across sequence, graph, and property-oriented optimization settings.
  • Flow-based models: Flow-matching models learn continuous or discrete transport processes without requiring every neural-network block to be explicitly invertible.
  • Flow-based models: Normalizing flows map molecular structures to Gaussian latent variables through invertible transformations and generate structures through the inverse mapping.
  • Diffusion-based models: Diffusion models progressively transform molecules into noise and reverse the process through learned denoising to generate samples.
  • Diffusion-based models: Molecular diffusion models can jointly generate graph topology and chemical types and can also support property-oriented and three-dimensional generation.

4.6 Methodological Trade-offs Across Generative Paradigms

Generative paradigms differ in architectural assumptions and generation mechanisms, creating trade-offs among sampling efficiency, representational flexibility, and related modeling properties. VAEs, GANs, and normalizing flows generally support efficient sampling after training.

  • Generative paradigms differ in sampling efficiency, representational flexibility, likelihood tractability, and their ability to accommodate complex molecular topology.
  • Sampling speed versus expressiveness: VAEs and GANs typically generate molecules through a single decoding or forward pass, enabling fast sampling after training.
  • Sampling speed versus expressiveness: Normalizing flows support efficient generation through invertible mappings, although continuous-flow variants introduce different computational considerations.

4.7 Training Objectives

Molecule-generation objectives differ across model paradigms but generally optimize fidelity to molecular data, distribution alignment, and structural or physical consistency. These objectives may operate on molecules alone or incorporate protein-pocket conditioning.

  • Training objectives generally regulate reconstruction fidelity, distribution alignment, and structural or physical consistency.The relative contributions of distribution-alignment and consistency terms are controlled by non-negative weighting hyperparameters.
  • Autoregressive RNN- and Transformer-based models use token-level negative log-likelihood to predict each molecular token from preceding context.Conditional sequence generation extends the objective by incorporating properties, scaffolds, targets, or other conditions.
  • Variational autoencoders minimize the negative evidence lower bound to jointly enforce reconstruction fidelity and latent-space regularization.This objective is used in latent-variable and graph-based molecule-generation models.
  • Generative adversarial models train generators to match the empirical molecular distribution through implicit adversarial learning.MolGAN uses adversarial training to promote chemical realism in generated molecular graphs.
  • Flow-based models have no universal objective; continuous normalizing flows minimize negative log-likelihood, enabling exact likelihood evaluation and reversible generation.Other flow formulations, including discrete flows, flow matching, and Bayesian flow networks, use different objectives.
  • Diffusion models minimize denoising or score-matching objectives to learn the reverse process.EDM predicts joint noise on atom coordinates and features, whereas TargetDiff combines coordinate MSE with atom-type KL-divergence loss.

4.8 Pocket-Conditioned Molecule Generation

Pocket-conditioned generation incorporates the structural and chemical context of a target binding pocket during molecule construction. Methods differ in their pocket representations, ligand-construction strategies, guidance signals, and use of external structural feedback.

  • Pocket conditioning imposes shape, residue, steric, and chemical constraints so generated ligands are compatible with a target binding environment.The review distinguishes direct pocket conditioning from goal-directed optimization using external objectives or post-training strategies.
  • Pocket–ligand interfaces align protein and ligand features in a shared 3D frame while exposing local pocket information during generation.Interfaces guide atom placement, bond formation, fragment growth, or full-ligand refinement and should preserve spatial symmetries.
  • Voxel-based conditioning makes pocket shape and steric exclusion explicit, but its spatial precision depends on grid resolution.VoxBind represents occupied and unoccupied spatial regions around binding sites with voxelized protein representations.
  • Chemically structured interfaces use scaffolds, arms, functional groups, motifs, or other intermediate units to capture localized interaction patterns.DecompDiff assigns different priors to scaffold and arm components, while related methods use functional-group- or motif-aware construction.
  • Flow-based pocket conditioning learns continuous transformations from simple priors to pocket-conditioned ligand distributions.PAFlow combines flow matching with interaction guidance and atom-number prediction to adapt generation to pocket geometry.
  • Pocket-conditioned models include direct atom-level generation, intermediate-unit construction, and guided sampling or refinement.Direct methods jointly model coordinates, atom types, and connectivity; guidance signals act during generation rather than only during post hoc evaluation.
  • Docking and co-folding provide alternative structure-based information through external scoring, reward signals, or joint protein–ligand complex modeling.These strategies differ from direct pocket conditioning by incorporating structural information through optimization loops or joint structural models.

5 Comparative Performance Analysis

The review evaluates molecule-generation models using complementary chemical, geometric, conformational, and target-specific metrics across standard and pocket-conditioned benchmarks. Reported results show strong basic validity but protocol-sensitive trade-offs among binding, drug-likeness, synthetic accessibility, and diversity, while experimental validation remains mostly preclinical.

  • 5.1 Evaluation Metrics: Evaluation spans intrinsic chemical quality, 3D geometric and conformational fidelity, and target-specific binding performance.Chemical quality includes validity, novelty, uniqueness, and diversity; geometric metrics assess spatial agreement, while binding metrics assess pocket fit and interactions.
  • 5.1.2 Geometric and Conformational Fidelity: RMSD measures aligned spatial deviation from a reference, with lower values indicating closer generated poses or conformers.The calculation uses matched atomic coordinates after optimal rigid-body rotation and translation.
  • 5.1.2 Geometric and Conformational Fidelity: COV measures reference-conformer coverage, whereas MAT measures closest-conformer RMSD; higher COV and lower MAT indicate better conformational agreement.Both metrics compare generated and reference conformer sets using an RMSD threshold or nearest-match distance.
  • 5.2 Performance Comparison: Many models achieve high validity on QM9 and ZINC, but near-saturated validity, novelty, and uniqueness values make small differences difficult to interpret.Clearer weaknesses include lower uniqueness or novelty for MolGAN, JT-VAE, and DiffLinker on ZINC, and reduced validity for EDM and MiDi on GEOM-Drugs.
  • 5.2 Performance Comparison: PAFlow reports the lowest Vina Score, Vina Min, and Vina Dock values (-8.31, -8.79, and -9.46) and the highest High Affinity ratio (80.80%).VoxBind leads QED (0.57) and normalized SA (0.70), while PocketFlow leads diversity (0.87), illustrating trade-offs across objectives.
  • 5.3 Practical Implications: Docking comparisons are sensitive to receptor preparation, pocket definition, search settings, sample size, and post-processing, limiting interpretation of small absolute differences.More favorable docking scores may not imply experimentally viable molecules, especially when scoring functions reward unrealistic geometries or steric clashes.
  • 5.3 Practical Implications: Experimental validation has advanced beyond computational scoring but remains largely early preclinical: seven of eight cases include prospective in vitro testing, four include X-ray characterization, and none reached human clinical evaluation.The RI-962 case provides the most extensive pipeline, extending through cellular, in vivo pharmacokinetic, tolerability, and efficacy studies.

6 Future Directions

Future progress depends on improving data quality, task definition, and methodological consistency alongside model architectures. The review emphasizes standardized 3D data, interaction-aware generation, receptor flexibility, and evaluation protocols that better reflect binding context and experimental relevance.

  • Data and Benchmarks: Future datasets should represent receptor conformational heterogeneity, curated binding annotations, consistent data splits, and binding-relevant evaluation protocols.Richer 3D corpora could combine crystal structures, computed conformers, and field-based descriptors.
  • Methodological Development: Generation pipelines should align representation, generation, docking, scoring, and filtering under shared geometric assumptions.Interaction-aware constraints can incorporate pharmacophore, shape, electrostatic, and other protein–ligand information.
  • Receptor Flexibility: Modeling receptor flexibility is a future direction because treating receptors as fixed simplifies pocket-conditioned generation but limits ligand-induced interaction modeling.The supplied passage introduces this limitation while discussing more realistic receptor representations.

7 Conclusions

The review systematically evaluates molecule generation models and concludes that performance depends on representations, datasets, task settings, and evaluation protocols. Reliable assessment must jointly consider chemical validity, structural plausibility, target relevance, and evaluation consistency while addressing persistent benchmarking and modeling limitations.

  • The review organizes molecule generation models by mechanism and comparatively analyzes reported results across chemical, geometric, conformational, and target-specific metrics.The frameworks include RNN/Transformer-based, VAE-based, GAN-based, flow-based, and diffusion-based models.
  • Performance varies across representations, datasets, task settings, and evaluation protocols, so improvement in one metric does not ensure broader molecular-quality gains.
  • Reliable assessment requires jointly considering chemical validity, structural plausibility, target relevance, and evaluation consistency.
  • Existing studies remain constrained by heterogeneous benchmarks, simplified receptor and interaction modeling, and limited comparability across reported results.
  • Future work should prioritize standardized benchmarks, consistent reporting, receptor-flexible and interaction-aware generation, synthesis- and property-aware constraints, interpretability, and reproducible cross-dataset evaluation.
  • Closer integration with medicinal chemistry, synthesis, and experimental validation is identified as important for translating methodological advances into practical drug-design applications.

Data and Software Availability

The review’s findings were compiled from cited publications and publicly available resources, with benchmark resources, metrics, model references, and available code repositories released online.

  • The review collected supporting data from cited publications and publicly available resources.
  • No dedicated software was developed for the review.
  • Compiled benchmark resources, evaluation metrics, model references, and available code repositories are accessible in a public GitHub repository.
Loading 2609.10099v1…