Source-linked AI summary
MoFlow: An Invertible Flow Model for Generating Molecular Graphs
Chengxi Zang, Fei Wang
TL;DR
Generating chemically valid molecular graphs with desired properties is difficult because molecular structures combine complex combinatorial constraints. MoFlow learns invertible mappings between graphs and latent representations, generating bonds first and atoms conditionally before validity correction. Across generation, reconstruction, visualization, and optimization tasks, it achieves the paper’s reported state-of-the-art performance, including 100% training-data reconstruction and chemically valid prior samples.
Problem
Generating novel molecular graphs with desired properties is challenging because drug-like chemical space is enormous and graphs must satisfy multi-type structural and bond-valence constraints.
Method
MoFlow uses invertible flow mappings, generating bonds with a Glow variant, atoms conditionally on bonds with a graph conditional flow, and applying post-hoc validity correction.
Results
MoFlow achieves state-of-the-art performance across molecular generation, reconstruction, and optimization, with 100% reconstruction and 100% chemically-valid prior samples.
Takeaways & Limitations
MoFlow supports efficient one-shot molecular graph embedding and generation while exploring chemical space with a chemical-validity guarantee.
Takeaways & Limitations
Constrained optimization assumes a similarity threshold δ and a step budget K for retaining optimized molecules.
Abstract
from arXiv · showhide
Generating molecular graphs with desired chemical properties driven by deep graph generative models provides a very promising way to accelerate drug discovery process. Such graph generative models usually consist of two steps: learning latent representations and generation of molecular graphs. However, to generate novel and chemically-valid molecular graphs from latent representations is very challenging because of the chemical constraints and combinatorial complexity of molecular graphs. In this paper, we propose MoFlow, a flow-based graph generative model to learn invertible mappings between molecular graphs and their latent representations. To generate molecular graphs, our MoFlow first generates bonds (edges) through a Glow based model, then generates atoms (nodes) given bonds by a novel graph conditional flow, and finally assembles them into a chemically valid molecular graph with a posthoc validity correction. Our MoFlow has merits including exact and tractable likelihood training, efficient one-pass embedding and generation, chemical validity guarantees, 100\% reconstruction of training data, and good generalization ability. We validate our model by four tasks: molecular graph generation and reconstruction, visualization of the continuous latent space, property optimization, and constrained property optimization. Our MoFlow achieves state-of-the-art performance, which implies its potential efficiency and effectiveness to explore large chemical space for drug discovery.
1 INTRODUCTION
MoFlow addresses the difficulty of generating chemically valid molecular graphs by using an invertible flow framework that combines efficient one-shot generation with validity correction. Experiments evaluate generation, reconstruction, latent-space visualization, and property optimization, reporting strong reconstruction, validity, novelty, and optimization results.
- Motivation: Drug-like chemical space is vast, while generating graphs with multi-type nodes, multi-type edges, and bond-valence constraints remains combinatorially difficult.The paper motivates graph generation as a way to explore chemical space for drug discovery.
- Approach: MoFlow introduces a normalizing-flow graph generator designed to invertibly map molecular graphs to latent representations and back.The paper positions flow-based modeling as distinct from VAE-, GAN-, and autoregressive-based approaches.
- Approach: MoFlow generates bonds with a Glow variant, generates atoms conditionally on bonds with a graph conditional flow, and assembles them using validity correction.The architecture explicitly models bond and atom structure before enforcing chemical validity.
- Results: 100% reconstruction rate demonstrates that MoFlow memorizes the input training data exactly.This result is reported for the paper’s memorization evaluation.
- Results: 100% chemically-valid molecules are generated when sampling from prior distributions, while validity without correction reaches 96% and 82% on two datasets.GraphAF obtains 67% and 68% validity-without-check scores on the corresponding datasets.
- Evaluation: The paper evaluates MoFlow across molecular generation, reconstruction, visualization, property optimization, and constrained property optimization against multiple generative baselines.The reported comparisons include VAE-based, autoregressive, and flow-based models.
2 RELATED WORK
Related work includes VAE-, GAN-, autoregressive-, and flow-based molecular generators. MoFlow is presented as combining one-shot invertible generation with a chemical-validity guarantee, unlike earlier one-shot flow models.
- Existing frameworks: Prior molecular generators use VAE, GAN, or autoregressive frameworks for SMILES codes or molecular graphs.These approaches differ in how they represent and generate molecular structures.
- Existing frameworks: Autoregressive models generate molecules sequentially and apply validity checks during atom or bond addition.Examples include sequential SMILES generation and sequential graph construction.
- Flow-based models: Flow-based models can potentially memorize and reconstruct training data while generating novel, valid, and unique molecules.The related-work discussion identifies these properties as motivations for exploring normalizing flows.
- Flow-based models: Earlier one-shot flow models GraphNVP and GRF lack chemical-validity guarantees and show limited performance on valid and novel molecule generation.GraphAF is described as an autoregressive flow model with sequential validity checking.
- MoFlow: MoFlow combines a Glow-based bond model, a graph conditional atom flow, and post-hoc validity correction for one-shot molecular graph generation.The paper attributes its reported performance and efficiency to holistic chemical-structure modeling.
3 MODEL PRELIMINARY
Normalizing flows learn invertible transformations between complex data and an easily modeled latent distribution. Glow supplies architectural components for expressive, tractable, and numerically stable flow transformations used in MoFlow.
- Flow framework: A flow model learns invertible transformations between high-dimensional data X and latent variables Z with the same dimensionality.The latent distribution is chosen to be easy to model, such as an isotropic Gaussian with independent dimensions.
- Flow framework: Sampling reverses the learned mapping: a latent sample is drawn first and transformed back into the data space.This reverse operation enables one-pass generation from latent representations.
- Flow framework: Change-of-variables likelihood evaluation makes flow training exact and tractable when the Jacobian determinant is efficiently computable.The paper states that the learned transformation is an invertible deep neural network.
- Glow components: Affine coupling layers preserve invertibility while using scale and transformation functions to provide expressive mappings and efficient Jacobian computation.The coupling construction splits the input dimensions into two partitions.
- Glow components: Glow uses invertible 1 × 1 convolutions to learn channel partitions before affine coupling transformations.The convolution has a learnable kernel and is followed by a fixed channel split.
- Glow components: Actnorm normalizes each channel with learnable scale and bias to improve numerical stability.The parameters are initialized from batch channel statistics.
- MoFlow representation: MoFlow represents a molecule as an atom feature matrix and a bond adjacency tensor, then maps both to latent variables with a spherical Gaussian distribution.Figure 1 shows conditional atom encoding given bonds and separate bond encoding.
4 PROPOSED MOFLOW MODEL
This section introduces MoFlow as a roadmap for learning and generating molecular graphs. The model is presented as combining an atom representation with a bond representation within an overall flow-based framework.
- Proposed MoFlow model: The proposed-model section first defines the problem and then introduces the Molecular Flow model using Figure 1 as its roadmap.The passage frames the section as covering MoFlow’s formulation and architecture.
- Proposed MoFlow model: MoFlow treats molecular graphs as structured objects whose atom and bond components are modeled within the proposed framework.The supplied passage identifies the section’s focus but does not provide further architectural detail.
- Proposed MoFlow model: The section organizes the presentation around defining the molecular-graph problem and explaining MoFlow in detail.Figure 1 is used as a roadmap for the section.
4.1 Problem Definition: Learning a Probability Model of Molecular Graphs
MoFlow models molecular graphs as atom matrices paired with multi-type bond tensors, decomposing their distribution into bonds and atoms conditioned on bonds. This decomposition supports exact maximum-likelihood learning and property-guided generation.
- Molecules are represented as pairs of one-hot atom matrices and symmetric multi-type bond tensors, forming graphs with typed nodes and edges.
- The molecular-graph distribution is decomposed into a bond distribution and an atom distribution conditioned on bonds.
- Exact maximum-likelihood estimation trains the model parameters through the change-of-variable framework.
- MoFlow combines a bond flow with a graph conditional atom flow and learns a latent-to-property mapping to guide optimized graph generation.
4.2 Graph Conditional Flow for Atoms
The graph conditional flow learns an invertible, dimension-preserving mapping from atom features to latent variables while holding the bond graph fixed. Graph convolutions parameterize coupling transformations, with normalization and alternating partitions supporting stacked layers.
- B-Conditional Flow and Graph Conditional Flow: The atom flow transforms an atom matrix conditioned on a fixed bond tensor into an isotropic-Gaussian latent variable using a conditional change-of-variable formulation.
- B-Conditional Flow and Graph Conditional Flow: A graph conditional flow is a B-conditional, invertible, dimension-kept mapping for node features given a typed adjacency tensor.
- Graph coupling layer: Each graph coupling layer splits the atom matrix by rows and uses graph-based scale and transformation functions to map one partition while preserving the other.
- Graph coupling layer: Relational graph convolutions use normalized adjacency matrices across bond channels, while the fixed bond graph allows graph normalization to be computed only once.
- Graph coupling layer: Sigmoid scaling improves numerical stability when multiple flow layers are cascaded, and stacked layers alternate atom partitions to transform previously unchanged components.
- Actnorm for 2-dimensional matrix: The architecture also includes a two-dimensional activation-normalization layer that normalizes each atom-feature row across batches.
4.3 Glow for Bonds
The bond flow learns an invertible mapping from adjacency tensors to Gaussian latent variables and reverses that mapping for generation. It uses a Glow variant with affine coupling, invertible convolutions, activation normalization, and spatial squeezing.
- The bond flow maps bond tensors invertibly to latent variables modeled by an isotropic Gaussian and generates tensors by reversing the mapping.
- Affine coupling splits bond tensors along channels and transforms one part using scale and translation functions while retaining invertibility.
- Sigmoid scaling replaces exponential scaling to improve numerical stability when many affine coupling layers are stacked.
- The bond-flow architecture uses invertible 1 × 1 convolutions, actnorm, spatial squeezing, and a reverse unsqueeze operation.
4.4 Validity Correction
MoFlow addresses chemical invalidity after assembling generated atoms and bonds by checking valency constraints and iteratively reducing excessive bond orders. The correction seeks minimal changes while preserving the largest connected component.
- Molecular graphs can become chemically invalid when generated atom matrices and bond tensors violate per-atom valency constraints.
- The valency definition accounts for formal charge, allowing charged N, S, and O atoms to support additional bonds.
- Post-hoc correction checks valency, returns the largest connected component when valid, and otherwise identifies invalid atoms for repair.
- For an over-valent atom, the procedure lowers the highest-order bond by one order and repeats the check until constraints are satisfied.
4.5 Inference and Generation
MoFlow provides exact likelihood training, one-pass inference and generation, invertibility, and a chemical validity guarantee for molecular graphs.
- MoFlow uses exact likelihood estimation and training for molecular graph inference and generation.
- One-pass inference embeds molecular graphs into latent representations, while invertible one-pass generation decodes them back into graphs.
- MoFlow includes a chemical validity guarantee during molecular graph generation.
5 EXPERIMENTS
Experiments evaluate MoFlow on generation, reconstruction, latent-space visualization, property optimization, and constrained optimization, using QM9 and ZINC250K. Across these tasks, MoFlow reconstructs all training molecules, generates chemically valid and diverse molecules, and supports smooth latent-space optimization with strong property and similarity outcomes.
- Experiments: MoFlow is evaluated on molecular generation and reconstruction, latent-space visualization, property optimization, and constrained property optimization.
- Datasets: QM9 contains 133,885 molecules with at most 9 atoms, while ZINC250K contains 249,455 drug-like molecules with at most 38 atoms.
- Generation and Reconstruction: 100% reconstruction rate demonstrates that MoFlow reconstructs all input molecules from their latent representations.
- Generation and Reconstruction: 100% validity and higher N.U.V. scores show that MoFlow generates chemically valid molecules while improving novelty and uniqueness relative to compared models.Its N.U.V. scores are reported as 2 and 3 times those of GraphNVP and GRF, respectively.
- Visualizing Continuous Latent Space: MoFlow’s latent space is smooth, with interpolations changing molecular graphs only slightly and latent Euclidean distance corresponding to chemical similarity.The authors identify this continuous latent space as the basis for property and constrained optimization.
- Property Optimization: MoFlow finds more novel molecules with the best QED score of 0.948 than the baselines.
- Constrained Property Optimization: MoFlow finds the most similar new molecules while achieving very good penalized logP improvement in constrained property optimization.It achieves higher similarity and property improvement than JT-VAE, and the best similarity with the second-best property improvement against GCPN and GraphAF.
6 CONCLUSION
MoFlow is presented as a one-shot invertible flow-based model for molecular graph generation with a validity guarantee. It achieves state-of-the-art performance across molecular generation, reconstruction, and optimization tasks.
- 6 CONCLUSION: MoFlow uses invertible mappings to generate molecular graphs in one shot while providing a chemical validity guarantee.Its pipeline combines a Glow variant for bonds, a graph conditional flow for atoms, and post-hoc validity correction.
- 6 CONCLUSION: MoFlow achieves state-of-the-art performance on molecular generation, reconstruction, and optimization.The paper also reports that MoFlow achieves 100% reconstruction of training data.
- 6 CONCLUSION: The authors identify combining sequential and one-shot generation as future work for producing chemically feasible molecular graphs.Codes and datasets are open-sourced for the model.