Source-linked AI summary
GraphNVP: An Invertible Flow Model for Generating Molecular Graphs
Kaushalya Madhawa, Katushiko Ishiguro, Kosuke Nakago, Motoki Abe
TL;DR
Molecular graph generation requires models that can represent graph-structured data for discovering molecules with desirable properties. GraphNVP addresses this with an invertible flow that separately generates adjacency structure and node attributes, achieving valid, nearly non-duplicated molecules and property-directed latent-space search.
Problem
Generating molecules with desirable properties is crucial in computational drug discovery, while prior graph models mainly use VAE or GAN latent representations.
Method
GraphNVP uses two invertible latent representations and reversible flows to generate an adjacency tensor first, then node attributes, with dequantization for discrete graph components.
Results
Almost 100% uniqueness ratio shows that GraphNVP generates molecular graphs with almost no duplicates, while its latent space supports generating molecules with desired chemical properties.
Takeaways & Limitations
GraphNVP generates valid molecules with high uniqueness and provides a latent space for searching for molecules that maximize a desired chemical property.
Takeaways & Limitations
The current masking choice makes GraphNVP dependent on node permutations, and permutation-invariant alternatives produce dramatically worse performance because molecular graphs are sparse.
Abstract
from arXiv · showhide
We propose GraphNVP, the first invertible, normalizing flow-based molecular graph generation model. We decompose the generation of a graph into two steps: generation of (i) an adjacency tensor and (ii) node attributes. This decomposition yields the exact likelihood maximization on graph-structured data, combined with two novel reversible flows. We empirically demonstrate that our model efficiently generates valid molecular graphs with almost no duplicated molecules. In addition, we observe that the learned latent space can be used to generate molecules with desired chemical properties.
1 Introduction
GraphNVP introduces an invertible flow model for molecular graph generation that separately models graph structure and node attributes. The model precisely maximizes likelihood and generates valid molecules with nearly no duplication while supporting property-directed generation.
- GraphNVP decomposes molecular graph generation into adjacency-tensor generation followed by node-attribute generation.It uses two latent representations and generates node attributes according to the generated graph structure.
- Invertible flows enable precise likelihood maximization, which the authors motivate as important because small atom substitutions can strongly affect drug molecules.
- GraphNVP uses reversible flows so new graph samples can be generated by feeding latent vectors through the model in reverse order.
- Almost 100% uniqueness ratio indicates that GraphNVP-generated molecular graphs contain almost no duplicates.The experiments compare GraphNVP with existing graph-generation models on two molecular datasets.
- The learned latent space supports generating molecular graphs with desired chemical properties without encoded domain-expert knowledge.
2 Related Work
Existing molecular graph generators differ in how they learn data distributions and how they construct graphs. GraphNVP introduces one-shot molecular graph generation using invertible flow, addressing prior flow-based models’ inability to generate graph structures.
- Distribution learning: Existing models commonly learn molecular graph distributions with variational autoencoders or implicitly through generative adversarial networks.Some models also use reinforcement learning to ease direct optimization difficulties.
- Generation process: Graph generation is primarily either sequential, adding nodes and edges step by step, or one-shot, generating the molecular graph in a single step.The one-shot strategy resembles commonly used image-generation models.
- GraphNVP: GraphNVP is presented as the first invertible-flow molecular graph generator with a one-shot generation strategy.It combines invertible flow with graph generation rather than using a sequential construction process.
- Graph-based flows: Earlier graph-based flow transformations modeled node assignments for classification but could not learn adjacency-tensor representations or generate graph structures.GraphNVP addresses this limitation by introducing two latent representations for graph generation.
3 GraphNVP: Flow-based graph generation model
GraphNVP models molecular graphs with an invertible flow that separately transforms adjacency structure and node features into latent representations. Dequantization, reversible coupling layers, and reverse-order generation enable likelihood-based training and graph reconstruction, while node-axis masking improves performance but sacrifices permutation invariance.
- Graph representation: GraphNVP represents each molecular graph as an adjacency tensor A and feature matrix X, covering bond and atom types.The adjacency tensor has dimensions N × N × R, while the feature matrix has dimensions N × M.
- Graph representation: Uniform noise dequantizes discrete adjacency and feature components before applying the continuous change-of-variable likelihood.The scaling hyperparameter satisfies 0 < c < 1, with c = 0.9 used experimentally; floor operations recover the original discrete inputs.
- Coupling layers: Two reversible affine coupling-layer families transform adjacency and node features into separate latent representations zA and zX.Adjacency coupling uses MLPs, whereas node-feature coupling conditions its scale and translation functions on the adjacency tensor through relational graph neural networks.
- Masking patterns: Each coupling layer updates one node-axis slice, requiring at least N layers with different masks for both adjacency and feature transformations.Experiments found node-axis masking performed best among the considered masking patterns.
- Masking patterns: Node-axis masking makes the transformations non-invariant to node permutations, while alternative permutation-invariant masks performed dramatically worse because molecular graphs are sparse.The authors identify permutation dependence as a limitation and future-work direction.
- Generative process: Graph generation reverses the trained flow: sampled zA first produces an adjacency tensor, which then conditions generation of node features from zX.Adjacency coupling is inverted first because node-feature coupling requires a valid adjacency tensor as input; node-wise argmax discretizes generated features.
4 Experiments
Experiments evaluate GraphNVP on QM9 and ZINC-250k using graph representations, likelihood-based generation, latent-space smoothness, and QED-guided optimization. GraphNVP achieves high uniqueness and exact reconstruction, while its property-optimization validity depends on the query molecule and linear regression may be improved by Bayesian optimization.
- Experimental Setup: QM9 contains 134k molecules and ZINC-250k contains 250k drug-like molecules, with maximum sizes of 9 and 38 non-hydrogen atoms, respectively.Molecules are kekulized, hydrogens are removed, and the resulting graphs use single, double, and triple bonds.
- Experimental Setup: Each molecule is represented by an adjacency tensor and feature matrix, with virtual nodes and virtual bonds providing fixed dimensions and sparse-graph connectivity.The adjacency tensor has four bond channels: single, double, triple, and virtual bonds.
- Numerical Evaluation: 1,000 latent vectors are sampled from a temperature-truncated normal distribution and decoded into molecular graphs for validity, novelty, uniqueness, and reconstruction evaluation.The reported metrics measure valid molecules, novel valid molecules, unique valid molecules, and reconstruction accuracy.
- Numerical Evaluation: 100% reconstruction accuracy is guaranteed by the invertible flow construction, while GraphNVP attains high uniqueness despite some baselines having higher QM9 validity and many duplicates.GraphNVP also generates many valid molecules without explicit chemical-knowledge procedures such as valency checks.
- Numerical Evaluation: GraphNVP’s high uniqueness is attributed in the discussion to avoiding the small number of unique molecules associated with MolGAN’s mode collapse.The paper contrasts VAE smoothing with invertible flows, whose encodings cannot omit minor graph variations.
- Smoothness of the Learned Latent Space: Neighboring points in randomly selected orthogonal latent-space grids correspond to molecules with minor variations, indicating smooth latent spaces for QM9 and ZINC.Empty grid cells indicate invalid generated molecules, and red circled molecules mark visualization centers rather than latent-space origins.
- Property-Targeted Molecule Optimization: QED-guided interpolation in latent space successfully searches for molecules similar to a query molecule with improved drug-likeness, although valid-molecule counts vary by query.A linear regressor learns the direction of increasing QED; the authors suggest extensive Bayesian optimization may yield better results.
5 Conclusion
The conclusion presents GraphNVP as an invertible flow-based molecular graph generator that combines high uniqueness with guaranteed reconstruction and supports property-directed molecular search. It also identifies permutation invariance and the causes of high uniqueness as open problems.
- 5 Conclusion: GraphNVP is presented as the first invertible flow-based model for generating molecular graphs.
- 5 Conclusion: GraphNVP generates valid molecules with high uniqueness and guaranteed reconstruction ability.
- 5 Conclusion: Its learned latent space supports searching for molecules similar to a given molecule while maximizing a desired chemical property.
- 5 Conclusion: The paper identifies improving permutation invariance and explaining GraphNVP’s high uniqueness ratio as important open problems.
A Network Architecture details
The architecture uses dataset-dependent stacks of coupling layers, affine transformations parameterized by MLPs, and node-feature couplings based on node assignments and adjacency information.
- A Network Architecture details: QM9 uses 27 adjacency coupling and 36 node feature coupling layers, while ZINC uses 38 coupling layers matching its maximum molecule size.
- A Network Architecture details: An MLP models both scale and translation in each adjacency coupling layer.
- A Network Architecture details: Node feature coupling layers use both node assignments and adjacency information.
- A Network Architecture details: The final layers of each RelGCN and MLP are zero-initialized so affine transformations initially implement the identity function.
B Training Details
Training uses Adam with default settings, dataset-specific minibatch sizes, batch normalization, fixed train/test splits, and 200 epochs without early stopping.
- B Training Details: Training uses the train/test splits from prior work, runs for 200 epochs without early stopping, and evaluates the final epoch snapshot.Implementations use Chainer-Chemistry and RDKit.
C Effect of temperature
Lower sampling temperatures increase molecular validity but reduce uniqueness, revealing a validity–diversity trade-off in graph generation.
- Lower temperatures produce more valid molecules but fewer unique molecules.Figure 6 averages results across five experiments.
D Additional Visualizations
The latent space supports chemical property optimization by interpolating a QM9 molecule along an axis associated with logP.
- A QM9 molecule is selected randomly, then its latent vector is interpolated along the axis maximizing water-octanol partition coefficient (logP).