PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Diffusion Models on Graphs: Methods and Applications

Chengyi Liu, Wenqi Fan, Yunqing Liu, Jiatong Li, Hang Li, Hui Liu, Jiliang Tang, Qing Li

arXiv:2302.02591v3cs.LGcs.AIcs.SI

TL;DR

Graph generation seeks to learn graph distributions and generate new graphs, but faces structural challenges and lacks a systematic overview of graph diffusion. This survey synthesizes diffusion methods, applications, and future directions, concluding with a comprehensive overview of the field and its promising research directions.

  • Problem

    Graph generation seeks to learn graph distributions and produce new graphs, but discrete structure, complex dependencies, node-order invariance, and a lack of systematic overview complicate the field.

    Source: §1 · pp. 1–2

  • Method

    The survey reviews graph diffusion paradigms and representative methods, summarizes molecule-generation and protein-modeling applications, and discusses future directions.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    The survey provides an overview of graph diffusion models, applications, and promising future research directions.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The synthesis identifies promising future research directions for generative diffusion models on graphs.

    Source: §6 · p. 8

    Prior graph generators face limitations: VAEs require graph matching for permutation invariance, GANs risk mode collapse, and flows constrain large-scale node-edge dependencies.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

Diffusion models, as a novel generative paradigm, have achieved remarkable success in various image generation tasks such as image inpainting, image-to-text translation, and video generation. Graph generation is a crucial computational task on graphs with numerous real-world applications. It aims to learn the distribution of given graphs and then generate new graphs. Given the great success of diffusion models in image generation, increasing efforts have been made to leverage these techniques to advance graph generation in recent years. In this paper, we first provide a comprehensive overview of generative diffusion models on graphs, In particular, we review representative algorithms for three variants of graph diffusion models, i.e., Score Matching with Langevin Dynamics (SMLD), Denoising Diffusion Probabilistic Model (DDPM), and Score-based Generative Model (SGM). Then, we summarize the major applications of generative diffusion models on graphs with a specific focus on molecule and protein modeling. Finally, we discuss promising directions in generative diffusion models on graph-structured data. For this survey, we also created a GitHub project website by collecting the supporting resources for generative diffusion models on graphs, at the link: https://github.com/ChengyiLIU-cs/Generative-Diffusion-Models-on-Graphs

1 Introduction

Graph generation learns graph distributions but must contend with discrete structure, complex dependencies, and node-order invariance. This survey reviews graph diffusion methods, their applications, and future directions.

1 Introduction

  • Graph generation is challenged by discrete structure, complex dependencies among nodes and edges, and up to N! equivalent adjacency matrices for an N-node graph.These properties complicate gradient-based end-to-end training, graph generation, and permutation invariance.Source: §1 · p. 1
  • VAEs, GANs, and normalizing flows have advanced graph generation, but face graph-matching costs, mode collapse, or architectural limits on structural learning.The cited examples include VAE posterior estimation and permutation invariance, GAN discriminator training, and flow constraints.Source: §1 · pp. 1–2
  • Diffusion models perturb data with noise, then learn a reverse process to recover it, and have succeeded across image, text-to-image, and molecular graph tasks.The passage describes the models as Markov chains trained with variational inference.Source: §1 · p. 2
  • The survey reviews graph diffusion techniques, molecule generation and protein modeling, and future research directions, addressing a gap in surveys focused on image diffusion.The authors describe it as the first survey of this developing area to their knowledge.Source: §1 · p. 2

2 Preliminaries

This section introduces graph-generation background and three diffusion paradigms, then describes their core processes: SMLD, DDPM, and continuous-time SGM.

2 Preliminaries

  • Graph-generation preliminaries introduce a graph representation and use Figure 1 to illustrate deep generative models on graphs.The representation is G=(X,A), with A as the adjacency matrix and X as node features.Source: §2 · p. 2, Figure 1 · p. 2

2.1 Deep Generative Models on Graphs

  • VAEs encode graphs into latent variables and decode samples, GANs use generator-discriminator training, and normalizing flows map samples through invertible functions.The section notes graph matching for VAE permutation invariance, GAN mode collapse, and flow limits on large-scale node-edge dependencies.Source: §2.1 · pp. 2–3

2.2 Diffusion Models

  • SMLD and DDPM learn reverse processes using score matching and nonequilibrium thermodynamics, respectively, while SGM extends discrete diffusion to continuous time with SDEs.The section presents these as the three main diffusion paradigms.Source: §2.2 · p. 3

Score Matching with Langevin Dynamics (SMLD)

  • SMLD adds Gaussian noise at increasing scales, trains a noise-conditional score network, and uses annealed Langevin dynamics to generate samples by gradual denoising.The noise scheme is described as supporting score matching across low-density regions.Source: Score Matching with Langevin Dynamics… · p. 3

Denoising Diffusion Probabilistic Model (DDPM)

  • DDPM uses a predefined forward-noising chain and a learned reverse Gaussian chain, trained to optimize a variational upper bound on negative log-likelihood.The forward equations allow sampling a noisy latent at an arbitrary step conditioned on the original input.Source: Denoising Diffusion Probabilistic… · p. 3

Score-based Generative Model (SGM)

  • SGM approximates the score in a continuous-time diffusion process, unifies SMLD and DDPM, and uses a probability-flow ODE for adaptive-interval sampling.The ODE has the same marginal density as the SDE and reduces score-function evaluations.Source: Score-based Generative Model (SGM) · p. 3

3 Generative Diffusion Models on Graphs

The section organizes graph diffusion methods into SMLD, DDPM, and SGM, describing how each adapts noise processes and denoising to graph structures.

3.1 SMLD on Graphs

  • EDP-GNN models symmetric adjacency matrices under multiple Gaussian noise scales and uses annealed Langevin dynamics to generate undirected graphs; ConfGF score-matches interatomic distances to generate molecular conformations.EDP-GNN uses a multi-channel GNN and noise-conditioned MLP, while ConfGF maps atomic coordinates to distances before sampling.Source: §3.1 · p. 4

3.2 DDPM on Graphs

  • DiGress uses discrete transitions for categorical node and edge attributes, with cross-entropy training and graph descriptors such as spectral features guiding generation.Its Markov formulation permits sampling directly at arbitrary time steps; discrete noise accelerates sampling.Source: §3.2 · p. 4
  • Other DDPM variants extend graph generation through equivariant networks, energy guidance, interatomic-force modeling, and autoregressive row-by-row adjacency prediction.EDMs handle continuous coordinates and categorical features; EEGSDE adds a property-prediction guide, MDM models interatomic forces, and GRAPHARM generates one adjacency row per reverse step.Source: §3.2 · p. 4

3.3 SGM on Graphs

  • SGM methods include GraphGDP’s variance-preserving SDE with position-enhanced score estimation and GDSS’s joint continuous-time diffusion over nodes and edges.GDSS uses Monte Carlo estimation and score-based MCMC correction; GSDM instead diffuses the top-k largest adjacency eigenvalues to reduce computation while retaining graph information.Source: §3.3 · pp. 4–5

4 Applications

The applications review covers molecular conformation generation and docking, protein generation, and protein–ligand structure prediction, emphasizing diffusion methods tailored to each task.

4 Applications

  • Figure 2 illustrates graph diffusion for molecules and proteins as forward noise addition followed by reverse refinement from a Gaussian-initialized graph.The forward process ultimately destroys molecular or protein structure; reverse Markov kernels gradually refine it.Source: Figure 2 · p. 5

4.1 Molecule Modeling

  • Molecular graphs represent atoms as nodes and bonds as edges, supporting downstream tasks including drug discovery, computational chemistry, and materials science.The section distinguishes molecule conformation generation from molecular docking.Source: §4.1 · p. 5

Molecule Conformation Generation

  • Molecular conformation methods use diffusion over coordinates, interatomic distances, torsion angles, or energy-guided geometric representations.Examples include GeoDiff, DGSM, Torsional Diffusion, EDMs, EEGSDE, MDM, and DiffBridges, with MDM modeling spacing-dependent forces and DiffBridges incorporating physical priors.Source: Molecule Conformation Generation · pp. 5–6

Molecular Docking

  • Docking methods generate molecules for protein binding pockets, including nonautoregressive, fragment-by-fragment, and pose-prediction approaches.DiffBP conditions generation on the protein target; FragDiff builds a molecule from pocket-centered fragments, while DiffLink completes supplied fragments and can condition on the pocket.Source: Molecular Docking · p. 6

4.2 Protein Modeling

  • Protein modeling targets protein structure generation and prediction, relevant to understanding protein function and interactions and designing proteins with specified properties.The section notes prior sequence-based protein representations and emerging diffusion applications.Source: §4.2 · pp. 6–7, Protein Generation · p. 7

Protein Generation

  • Protein-generation methods span structural and sequence pretraining, conditional backbone generation, folding-inspired denoising, motif scaffolding, and antibody CDR design.SiamDiff predicts noise across related structural and sequence views; ProteinSGM and ProSSDG generate conditioned structures, while DiffFold, SMCDiff, and DiffAntigen address folding, motifs, and antibody regions.Source: Protein Generation · p. 7

Protein-ligand Complex Structure Prediction

  • Protein–ligand structure methods predict complex structures by sampling 3D configurations from learned distributions or generating varied structures with binding poses.NeuralPLexer combines multi-scale induced bias with diffusion; DiffEE uses a pretrained protein language model in an end-to-end diffusion generator.Source: Protein-ligand Complex Structure… · p. 7

5 Future Challenges and Opportunities

The paper identifies open directions in graph diffusion spanning discrete data, conditional generation, trustworthiness, evaluation, and applications beyond molecules and proteins.

Discrete Nature of Graphs

  • Graph diffusion lacks a universal, widely recognized approach to handling graphs’ discrete nature while adapting methods often developed for continuous spaces.Existing work has used discrete probability distributions or bridges between continuous and discrete spaces.Source: §5 · p. 7

Conditional Generation for Graph Diffusion Models

  • Conditional graph generation remains a direction for guiding molecules and proteins toward specified properties using constraints or auxiliary knowledge.The paper identifies knowledge graphs and visual or textual information as possible conditions, with knowledge graphs potentially supporting desired generation and diversity.Source: §5 · pp. 7–8

Trustworthiness for Graph Diffusion Models

  • Trustworthy graph generation must address safety and robustness, explainability, fairness, and privacy, especially for safety-critical uses such as drug discovery.The paper notes vulnerability to adversarial attacks and difficulty explaining complex graph diffusion architectures.Source: §5 · p. 8

Evaluation Metrics

  • Graph-generation evaluation remains challenging because graph-statistics metrics may be unreliable, while validity and diversity also matter across applications.The paper calls for quantitative measures of generated-graph quality.Source: §5 · p. 8

Graph Diffusion Applications

  • Applications beyond molecule and protein generation remain relatively unexplored, including recommendation, anomaly detection, and causal graph generation.The paper describes these as prospective opportunities for graph diffusion models.Source: §5 · p. 8

6 Conclusion

The conclusion presents the survey as a systematic overview of graph diffusion models, their applications, and future directions.

6 Conclusion

  • The paper addresses the lack of a systematic overview and discussion of state-of-the-art diffusion models on graphs.Source: §6 · p. 8
  • Its coverage includes representative graph diffusion models and applications.Source: §6 · p. 8
  • The paper discusses promising future research directions for generative diffusion models on graph-structured data.Source: §6 · p. 8

Related papers

Loading 2302.02591v3…