Source-linked AI summary
Convolutional Networks on Graphs for Learning Molecular Fingerprints
David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Timothy Hirzel, Alán Aspuru-Guzik, Ryan P. Adams
TL;DR
Molecular prediction pipelines typically convert arbitrary-sized molecules into fixed fingerprints before learning. This paper replaces that step with a differentiable graph convolutional network, whose learned fingerprints match or beat standard fingerprints across solubility, drug efficacy, and photovoltaic-efficiency tasks.
Problem
Molecules have arbitrary size and shape, while prevailing prediction pipelines rely on fixed-dimensional fingerprints before applying standard machine-learning models.
Method
A differentiable graph convolutional network replaces fixed molecular fingerprints by applying shared local filters to atom neighborhoods followed by global pooling.
Results
Neural graph fingerprints matched or beat circular fingerprints on solubility, drug efficacy, and organic photovoltaic efficiency datasets.
Takeaways & Limitations
The approach enables end-to-end optimization of interpretable molecular features for diverse prediction tasks.
Takeaways & Limitations
Information propagation across the graph is limited by network depth, which may restrict the approach beyond the small organic molecules studied.
Abstract
from arXiv · showhide
We introduce a convolutional neural network that operates directly on graphs. These networks allow end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. The architecture we present generalizes standard molecular feature extraction methods based on circular fingerprints. We show that these data-driven features are more interpretable, and have better predictive performance on a variety of tasks.
1 Introduction
The paper replaces fixed molecular fingerprint computation with a differentiable graph neural network that handles molecules of arbitrary size and shape. Neural graph fingerprints are presented as task-adaptive, potentially more predictive, parsimonious, and interpretable than fixed fingerprints.
- Problem: Neural graph fingerprints address the arbitrary size and shape of molecules that fixed-size machine-learning pipelines cannot directly handle.Existing pipelines commonly compute fixed-dimensional vectors with off-the-shelf fingerprint software before prediction.
- Method: The method replaces fixed-dimensional molecular fingerprints with a differentiable neural network operating on molecular graphs, where atoms are vertices and bonds are edges.Shared local filters process each atom and its neighborhood, followed by global pooling.
- Predictive performance: Neural graph fingerprints match or beat standard fingerprints on solubility, drug efficacy, and organic photovoltaic efficiency datasets.Their data-adaptive features are optimized for the task at hand.
- Parsimony: Differentiable fingerprints can encode only relevant features, reducing downstream computation and regularization requirements compared with extremely large fixed fingerprints.One cited fixed fingerprint used 43,000 dimensions after removing rarely occurring features.
- Interpretability: Neural graph fingerprint features can activate for similar but distinct molecular fragments, making the representation more meaningful than standard fingerprints.Standard fingerprints encode every possible fragment completely distinctly and do not represent similarity between fragments.
2 Circular fingerprints
Circular fingerprints, including state-of-the-art ECFPs, encode molecular substructures invariantly to atom relabeling. They construct features through fixed local hashing and global pooling, paralleling convolutional-network computation.
- Circular fingerprints: ECFPs are the state of the art in molecular fingerprints and refine the Morgan algorithm to encode molecular substructures invariantly to atom relabeling.Circular fingerprints are designed to represent which substructures occur in a molecule.
- Circular fingerprints: Each layer applies a fixed hash to concatenated features from the previous neighborhood layer, then maps hashed features to active indices in a binary fingerprint vector.Ignoring collisions, each index denotes the presence of a particular substructure.
- Circular fingerprints: Circular fingerprints apply the same local operation throughout a graph and combine information through global pooling, analogously to convolutional networks.
3 Creating a differentiable fingerprint
The authors create a differentiable generalization of circular fingerprints by replacing their discrete operations with neural-network analogs. Neural graph fingerprints retain circular fingerprints as a limiting case with large random weights while supporting smooth, permutation-invariant processing.
- Differentiable design: Neural graph fingerprints replace each discrete circular-fingerprint operation with a differentiable analog.The design starts from circular fingerprints as a known-good configuration.
- Hashing: A neural-network layer replaces hashing, allowing activations to remain similar when local molecular structures differ in unimportant ways.Circular-fingerprint hashing combines atom and neighboring-substructure information so fragment changes activate different fingerprint indices.
- Indexing: Softmax replaces circular-fingerprint indexing, whose pooling-like operation maps arbitrary-sized graphs to fixed-sized vectors.Circular fingerprints set one fingerprint bit per node at an index determined by its hashed feature vector.
- Canonicalization: The architecture addresses neighborhood ordering through sorting, permutation-based transforms, or permutation-invariant functions such as summation.Circular fingerprints are invariant to the ordering of atoms in each neighborhood.
- Relationship to circular fingerprints: Circular fingerprints are a special case of neural graph fingerprints with large random weights, as tanh approaches step functions and softmax approaches one-hot argmax.These limiting behaviors reproduce hashing and indexing operations, respectively.
4 Experiments
Experiments show that neural graph fingerprints with large random weights resemble circular fingerprints, while learned fingerprints provide interpretable molecular features and match or improve predictive accuracy across tasks.
- Random-weight experiments: r = 0.823 correlation links pairwise distances from circular and neural fingerprints on solubility molecules.The fingerprints had length 2048, and binary ECFP pairs sometimes had exactly zero overlap.
- Random-weight experiments: Neural graph fingerprints with fixed large random weights closely match circular-fingerprint predictive performance, whereas fixed small random weights perform substantially better.Both methods with large random weights follow similar performance curves on solubility prediction, unlike small random weights.
- Interpretability: Neural graph fingerprint features can respond to structural variations, making them more interpretable than circular features and permitting shorter feature vectors.Each circular feature corresponds to one fragment and radius except for accidental collisions, while neural features can activate on related structures.
- Interpretability: Solubility-predictive features automatically identify hydrophilic R-OH groups, while insolubility-predictive features are activated by distinct molecular fragments.The fingerprint network was trained as input to a linear model predicting solubility.
- Interpretability: Toxicity-predictive features identify sulfur attached to aromatic rings and fused aromatic rings across two toxicity datasets.The latter correspond to polycyclic aromatic hydrocarbons, described as well-known carcinogens.
- Predictive performance: Across solubility, drug efficacy, and organic photovoltaic efficiency, neural graph fingerprints matched or beat circular fingerprints, with neural-network prediction heads typically outperforming linear layers.The comparison used linear and one-hidden-layer neural-network predictors, optimizing differentiable parameters jointly.
5 Limitations
The neural fingerprint architecture has higher practical computational cost, limited per-layer expressiveness, and depth-limited information propagation. It also does not yet distinguish stereoisomers without additional bookkeeping, leaving several extensions for future work.
- Computational cost: Neural fingerprints retain circular fingerprints’ asymptotic complexity but add matrix-multiplication costs of O(RNFL + RNF^2).For depth R, fingerprint length L, molecule size N, and F features per layer, the molecular convolutional net has this stated cost.
- Limited computation at each layer: The paper uses a single neural-network layer between message-passing steps, while deeper nonlinearities or LSTM-style information transfer remain possible extensions.The authors describe their chosen architecture as the simplest feasible one and suggest multiple nonlinearities or adapted Long Short-Term Memory architectures.
- Limited information propagation across the graph: Although local message passing scales well for low-degree organic molecules, graph-wide information propagation is limited by network depth and may require depth N^2 in the worst case.Hierarchical clustering could reduce this to log(N) layers, but a tree-structured network would need to learn how to parse molecules.
- Inability to distinguish stereoisomers: Neural fingerprints currently cannot distinguish stereoisomers, including enantiomers and cis/trans isomers, without special bookkeeping.The authors note that sensitivity to stereoisomers could be added, but leave it for future work.
6 Related work
The paper relates its graph networks to differentiable computational architectures, neural graph fingerprints, and unrolled inference. It distinguishes its method from prior graph and QSAR models through graph-dependent inputs, simple gradient-based optimization, circular-fingerprint generalization, and neural QSAR pipelines.
- The work is similar in spirit to neural Turing machines because it makes parts of a discrete computational architecture differentiable for gradient-based optimization.
- Modern molecular QSAR commonly combines circular fingerprints with fully connected neural networks or other regression methods, including multitask models and ensembles.The cited studies use circular fingerprints with neural networks, Gaussian processes, and random forests; one multitask neural network used depth-2 fingerprints.
- Unlike the closest neural graph-fingerprint approach, which removes cycles, roots each resulting tree at every atom, and sums recursive representations, this method operates directly on graphs.A graph with N nodes yields N possible roots in the prior approach.
- Standard convolutional networks use fixed computational graphs, whereas this paper addresses inputs where each training example is a different graph.Prior graph convolutions considered fixed graph structure with varying vertex features, while this work handles graph structure that varies across inputs.
- Compared with prior input-dependent graph networks, the method uses simple gradient-based optimization, generalizes circular fingerprints, and fits neural-network-based QSAR pipelines.Prior work used message passing to equilibrium and applied graph networks to molecular mutagenesis and web-page ranking.
- Viewed as an unrolled inference procedure, the neural fingerprint resembles an unrolled message-passing algorithm on the original graph.This perspective parameterizes inference steps and trains a neural network to approximate exact inference using few iterations.
7 Conclusion
The work generalizes hand-crafted molecular features into differentiable neural fingerprints that can be optimized end-to-end for diverse tasks. It also motivates applying data-driven features to virtual screening, drug design, and materials design.
- 7 Conclusion: Differentiable feature-pipeline operations enable scalable end-to-end optimization of neural molecular fingerprints with standard neural-network training methods.The approach generalizes existing hand-crafted molecular features and optimizes them for diverse tasks.
- 7 Conclusion: The new molecular fingerprints demonstrated interpretability and predictive performance.These results establish the fingerprints as data-driven alternatives to hand-crafted molecular features.
- 7 Conclusion: Applying data-driven features to virtual screening, drug design, and materials design is identified as a natural next step.Data-driven features have already replaced hand-crafted features in speech recognition, machine vision, and natural-language processing.