PaperlayerView exact revision on arXiv

Source-linked AI summary

Hierarchical Generation of Molecular Graphs using Structural Motifs

Wengong Jin, Regina Barzilay, Tommi Jaakkola

arXiv:2002.03230v2cs.LGstat.ML

TL;DR

Existing molecule-generation methods rely on small building blocks whose performance degrades for larger molecules. This paper introduces a hierarchical encoder-decoder using larger motifs and reports superior performance across molecule-generation and graph-translation tasks.

  • Problem

    Atom- and small-substructure-based graph generators degrade on large molecules, while prior substructure assembly becomes combinatorially challenging for large components.

  • Method

    A hierarchical encoder-decoder represents molecules from atoms to motifs and autoregressively adds motifs while resolving their attachments to the emerging graph.

  • Results

    The model outperforms prior atom- and substructure-based methods across small-molecule and polymer domains, achieving 20% absolute improvement in polymer reconstruction accuracy and 6.3× faster decoding than prior substructure-based methods.

  • Takeaways & Limitations

    Structural motifs serve as effective building blocks for hierarchical molecular graph generation across polymer generation and molecular property optimization tasks.

  • Takeaways & Limitations

    Prior substructure assembly cannot be directly extended to motifs because its enumeration is exponential in substructure size and often fails on rings larger than 10 atoms.

Abstract

from arXiv · show

Graph generation techniques are increasingly being adopted for drug discovery. Previous graph generation approaches have utilized relatively small molecular building blocks such as atoms or simple cycles, limiting their effectiveness to smaller molecules. Indeed, as we demonstrate, their performance degrades significantly for larger molecules. In this paper, we propose a new hierarchical graph encoder-decoder that employs significantly larger and more flexible graph motifs as basic building blocks. Our encoder produces a multi-resolution representation for each molecule in a fine-to-coarse fashion, from atoms to connected motifs. Each level integrates the encoding of constituents below with the graph at that level. Our autoregressive coarse-to-fine decoder adds one motif at a time, interleaving the decision of selecting a new motif with the process of resolving its attachments to the emerging molecule. We evaluate our model on multiple molecule generation tasks, including polymers, and show that our model significantly outperforms previous state-of-the-art baselines.

1. Introduction

Existing molecule-generation methods use atoms or small substructures and degrade on larger molecules. The paper proposes motif-based hierarchical generation, achieving stronger reconstruction and generation results across molecule sizes and tasks.

  • Motivation: Atom-by-atom and small-substructure methods perform well for small molecules but degrade significantly on larger molecules such as polymers.The paper attributes this pattern to the many generation steps required and the resulting gradient challenges.
  • Motivation: Large molecules contain repeated structural motifs, motivating their use as larger building blocks for reconstruction and generation.The authors hypothesize that explicit motifs can improve accuracy by reducing the burden of assembling large molecular graphs.
  • Approach: The proposed model uses a hierarchical encoder-decoder that extracts motifs and generates molecules coarse-to-fine through autoregressive motif selection and attachment decisions.The encoder proceeds fine-to-coarse, while the decoder predicts a new motif, its attachment configuration, and its contact points.
  • Results: 20% absolute improvement in reconstruction accuracy was achieved on polymer generation over the best baselines, alongside state-of-the-art results across evaluated metrics.The model was also evaluated on graph translation tasks for molecular property optimization.
  • Results: For polymers, the motif-based model maintains high reconstruction accuracy for large molecules, whereas atom-based CG-VAE fails above 80 atoms.Figure 1 compares reconstruction accuracy across polymer sizes measured by number of atoms.

2. Background and Motivation

Prior atom- and small-substructure-based methods require many decoding steps and face combinatorial difficulties on large molecules. The paper addresses this with a hierarchical representation and efficient motif-based assembly.

  • Background and Motivation: Polymer datasets contain thousands of molecules with more than 80 atoms, exposing reconstruction failures in existing variational autoencoders.The tested models often fail to reconstruct molecules from their latent embeddings.
  • Background and Motivation: Existing autoregressive approaches generate molecules either atom by atom or from small substructures restricted mainly to rings and bonds.Substructure-based methods often use components no larger than six atoms.
  • Background and Motivation: Small building blocks require many decoding steps, making large-molecule generation error-prone despite the presence of larger structural motifs.Generating motif by motif can significantly reduce the number of decoding steps.
  • Background and Motivation: Previous substructure assembly has exponential time complexity in substructure size and often fails on rings larger than 10 atoms.Its assumptions about single cycles or bonds prevent direct extension to larger, flexible motifs.
  • Proposed Architecture: The proposed hierarchical encoder-decoder supports arbitrary motif types and combines coarse motif information with fine-grained atom connectivity.This design is intended to assemble motifs efficiently without combinatorial explosion.
  • Motif Extraction: Motifs are extracted by breaking selected bridge bonds, retaining frequently occurring connected fragments, and decomposing unselected fragments into rings and bonds.The motif vocabulary is constructed from fragments occurring more than 100 times in the training set.
  • Hierarchical Graph Encoder: The hierarchical graph encoder represents atoms, attachment configurations, and motifs across connected layers.Figure 2 shows atom-to-motif links and attachment nodes representing motif configurations at neighbor intersections.

3. Hierarchical Graph Generation

The model represents molecules hierarchically from atoms through attachments to motifs, then decodes them by selecting motifs and resolving their connections step by step.

  • The graph probability is modeled over structural motifs and their attachment configurations, conditioned on a latent representation.
  • Each step predicts the next motif, its attachment configuration, and the atom-pair connections linking it to the current graph.
  • The hierarchical graph has motif, attachment, and atom layers that encode coarse connectivity, fine-grained motif intersections, and molecular atom connectivity.
  • The encoder uses three message-passing networks, passing information upward from atom representations through attachment representations to motif representations.
  • The decoder maintains frontier motifs in a depth-first stack and conditions each generation step on the latent representation and the current partial hierarchical graph.
  • Attachment prediction reduces candidate space, while the three autoregressive decisions condition subsequent motif generation on earlier attachment outcomes.
  • The same architecture extends to graph-to-graph translation for modifying molecular compounds toward improved biochemical properties.

4. Experiments

The experiments evaluate hierarchical motif-based generation on polymers and graph translation, showing strong reconstruction, distributional, translation, and efficiency results. Ablations indicate that large motifs and hierarchical components contribute to performance.

  • Experimental Setup: The model is evaluated on polymer generative modeling and graph-to-graph translation for molecular property optimization.The polymer dataset contains 86K polymers, and motif extraction produced 436 motifs.
  • Evaluation Metrics: The evaluation covers chemical validity, uniqueness, diversity, property statistics, structural statistics, and reconstruction accuracy.Diversity uses Tanimoto distance over Morgan fingerprints, while property statistics use Frechet distances between generated and test distributions.
  • Polymer Generative Modeling: 79.9% vs 58.5% reconstruction accuracy: HierVAE significantly outperforms previous methods on polymer generation.It also achieves state-of-the-art logP, molecular-weight Frechet distance, and structural-similarity results.
  • Polymer Generative Modeling: Replacing large motifs with rings and bonds smaller than ten atoms significantly worsens performance despite retaining the same hierarchical architecture.This ablation isolates the contribution of larger structural motifs.
  • Graph-to-Graph Translation: 76.9% versus 59.9% QED translation accuracy: HierG2G surpasses JTNN and achieves state-of-the-art results across four translation tasks.HierG2G also reaches 0.564 versus 0.480 diversity on logP and runs 6.3 times faster than JTNN during decoding.
  • Ablation Studies: Removing the top motif layer reduces translation accuracy by 0.8% and 2.4%, while removing the attachment layer causes significant degradation on both datasets.The authors attribute the latter degradation to losing motif information and requiring the model to infer motif construction.

5. Related Work

Prior molecular graph-generation methods use strings, adjacency matrices, atoms, or simple substructures, while related hierarchical graph methods mainly target prediction tasks. This work distinguishes itself by using multiresolution hierarchies for efficient graph generation.

  • Graph Generation: Earlier methods generate molecules from SMILES strings, adjacency matrices, atoms, or simple substructures such as rings and bonds.These approaches span sequence-based, graph-based, and substructure-based generation strategies.
  • Graph Generation: The proposed model differs from blockwise graph generation by encoding input graphs across multiple hierarchies and learning multiresolution representations.Its hierarchy combines coarse motif information with fine-grained atom connectivity for generation.
  • Graph Generation: Jin et al. generate molecules with substructure junction trees, but combinatorial attachment enumeration limits them to relatively simple rings and bonds.Their method cannot scale directly to larger, flexible motifs.
  • Graph Encoders: Related hierarchical graph encoders construct graph hierarchies for regression or classification, whereas this work constructs its hierarchy for efficient graph generation.The distinction concerns the downstream purpose of the hierarchy rather than hierarchy construction alone.

6. Conclusion

The paper develops a hierarchical encoder-decoder that generates molecular graphs using structural motifs as building blocks and outperforms prior atom- and substructure-based methods across small-molecule and polymer domains.

  • The model uses structural motifs as building blocks in a hierarchical encoder-decoder for molecular graph generation.
  • The experimental results show improvements over prior atom- and substructure-based methods in both small-molecule and polymer domains.

A. Motif Construction

Motifs are extracted by disconnecting chemically valid bridge bonds, selecting frequently occurring fragments, and decomposing unselected fragments into rings and bonds.

  • Motif extraction decomposes a molecular graph into disconnected fragments by breaking bridge bonds that preserve chemical validity.
  • Bridge bonds are identified when both endpoints have degree at least 2 and either endpoint belongs to a ring.
  • After detaching bridge bonds, the molecule becomes a set of disconnected subgraphs G1, · · · , GN.
  • A disconnected subgraph becomes a motif when its training-set occurrence exceeds 100; otherwise, it is decomposed into rings and bonds for the motif vocabulary.

B. Network Architecture

The architecture encodes molecular graphs hierarchically and decodes them autoregressively, resolving graph expansion, atom or motif choice, and attachment bonds in successive decisions.

  • The message passing network uses edge-directional messages, with each edge carrying separate messages from u to v and from v to u.
  • AtomG2G encodes molecules solely at the atom level and decodes them atom by atom in breadth-first order using a frontier-node queue.
  • At each decoding step, AtomG2G predicts whether to expand the selected frontier node, creates and types a new atom, then predicts its bonds to frontier nodes autoregressively.
  • The current graph is encoded with an LSTM message passing network, and its graph representation is formed by summing atom representations.
  • Bond prediction is sequential: each new bond updates the representation of the new atom through one-step message passing over its currently predicted neighbors.

C.1. Polymer Generation

The polymer-generation experiments use a dataset-specific motif vocabulary and compare hierarchical models with baselines and restricted small-motif variants using established molecular-generation metrics.

  • The polymer dataset and motif vocabulary sizes are reported for the experimental datasets.
  • HierVAE uses hidden dimension 400 and latent dimension |z| = 20, while all models use KL regularization weight λKL = 0.1 and approximately 5M parameters.
  • The experiments include an atom-based decoder and one- and two-layer encoder ablations in graph translation tasks.
  • The small-motif baseline retains the hierarchical architecture but restricts generated motifs to single rings or bonds.
  • Evaluation metrics use the implementation from Polykovskiy et al. (2018), and model samples are presented in Figure 10.

C.2. Graph-to-Graph Translation

The graph-to-graph translation experiments use datasets downloaded from Jin et al. (2019), with dataset-specific motif vocabularies and matched model configurations. Ablations modify decoder granularity and the number of encoder-decoder hierarchies, while sampled polymers are shown separately.

  • Data: The graph translation datasets were downloaded from the link provided in Jin et al. (2019), with dataset and motif vocabulary sizes listed in Table 4.
  • Hyperparameters: HierG2G uses hidden and embedding dimensions of 270 and 200, latent dimension |z| = 8, KL weight λKL = 0.3, and T = 20 message-passing iterations.
  • Hyperparameters: AtomG2G uses 400-dimensional hidden and embedding layers so the two models have roughly the same number of parameters, with λKL = 0.3 and T = 20.
  • Ablation Study: The ablation study replaces the HierG2G decoder with AtomG2G’s atom-based decoder while retaining the hierarchical encoder and adding atom and motif vectors to decoder attention.
  • Ablation Study: Further ablations reduce the number of hierarchies in both encoder and decoder message-passing networks, using two-layer and one-layer variants with adjusted dimensions.
  • Samples: Figure 10 presents sampled polymers generated by HierVAE.
Loading 2002.03230v2…