Source-linked AI summary
Few-Shot Graph Learning for Molecular Property Prediction
Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, Nitesh V. Chawla
TL;DR
Molecular property prediction often has too few labeled examples for new properties, limiting conventional deep learning methods. The paper proposes Meta-MGNN, combining pretrained molecular GNN representations with meta-learning, self-supervision, and task-aware weighting. On two public multi-property datasets, it outperforms state-of-the-art methods and its components are individually validated.
Problem
Few-shot molecular property prediction addresses the limited laboratory data available for individual molecular properties.
Method
Meta-MGNN combines pretrained molecular GNN representations, meta-learning, bond reconstruction, atom type prediction, and self-attentive task weights.
Results
Meta-MGNN outperforms state-of-the-art methods on two public multi-task datasets, with each model component’s effectiveness also verified.
Takeaways & Limitations
The approach transfers information across molecular-property tasks while adapting to new properties from few-shot data.
Abstract
from arXiv · showhide
The recent success of graph neural networks has significantly boosted molecular property prediction, advancing activities such as drug discovery. The existing deep neural network methods usually require large training dataset for each property, impairing their performances in cases (especially for new molecular properties) with a limited amount of experimental data, which are common in real situations. To this end, we propose Meta-MGNN, a novel model for few-shot molecular property prediction. Meta-MGNN applies molecular graph neural network to learn molecular representation and builds a meta-learning framework for model optimization. To exploit unlabeled molecular information and address task heterogeneity of different molecular properties, Meta-MGNN further incorporates molecular structure, attribute based self-supervised modules and self-attentive task weights into the former framework, strengthening the whole learning model. Extensive experiments on two public multi-property datasets demonstrate that Meta-MGNN outperforms a variety of state-of-the-art methods.
1 INTRODUCTION
Molecular property prediction is important for virtual screening, but deep learning methods often struggle when properties have few labeled molecules. Meta-MGNN addresses this few-shot setting with graph representation learning, meta-learning, self-supervision, and task-aware weighting.
- Motivation: More than half of MoleculeNet properties are shared by fewer than 100 molecules, creating a few-shot data challenge for current approaches.The limited availability of labeled molecules is common across several benchmark datasets.
- Motivation: Virtual screening filters molecules likely to fail early, reducing the number requiring extensive biological investigation.This motivates improving the effectiveness of molecular property prediction for candidate selection.
- Challenges: Molecular graphs contain heterogeneous atoms and bond types, while sequence-based SMILES methods may not capture bond information well.The paper identifies bond attributes as a challenge for molecular representation learning.
- Approach: Meta-MGNN combines a pretrained molecular GNN, meta-learning, bond reconstruction, atom type prediction, and self-attentive task weights.The components target molecular representation learning, few-shot adaptation, unlabeled information, and differing task importance.
- Evaluation: Experiments on two public datasets report superior performance over state-of-the-art methods and verify the effectiveness of individual model components.The evaluation covers both overall comparisons and component effectiveness.
2 RELATED WORK
Prior work applies GNNs and sequence models to molecular representation learning, while few-shot learning commonly uses metric-based or gradient-based approaches. Molecular property prediction methods are likewise distinguished by graph or SMILES inputs.
- Graph Neural Network: GNNs model graph-structured data through neighborhood aggregation and have been applied to molecular property prediction.Their broader applications include recommendation, behavior modeling, and anomaly detection.
- Molecular Property Prediction: Molecular property prediction methods use either molecular graphs with atom and bond edges or SMILES sequence representations.The review organizes methods according to their molecular input type.
- Molecular Property Prediction: Graph-based approaches include hierarchical GNNs, pretrained GNNs, and models that fuse graph and sequence information.These methods target molecular representations at different structural levels or combine complementary inputs.
- Few-shot Learning: Few-shot learning approaches are commonly categorized as metric-based learning or gradient-based learning.Metric-based methods learn comparisons between few examples, whereas gradient-based methods adapt model parameters.
3 PRELIMINARY
The paper formulates molecular property prediction and few-shot adaptation over molecular graphs, then uses a pretrained GNN to encode graph structure and molecular attributes for downstream prediction.
- Problem Definition: A molecular graph represents atoms as nodes and chemical bonds as edges, with a mapping from molecular graphs to property labels.The formal representation is G = (V, E), where V contains nodes and E contains edges.
- Problem Definition: Few-shot molecular property prediction seeks a classifier that rapidly adapts to unseen properties using only a few labeled molecular examples.The task distinguishes training properties from new test properties with few-shot samples.
- Meta-learning Setup: Meta-learning trains across property-prediction tasks using support sets for adaptation and query sets for evaluation.The setup follows MAML and uses k samples for task-specific training before testing on remaining task data.
- Molecular Graph Neural Network: GNNs iteratively aggregate neighboring node and edge information to learn representations for molecular graphs.The model uses graph structure together with node and edge features.
- Molecular Graph Neural Network: Node attributes include atom number and chirality, while edge attributes include bond type and bond direction.These attributes initialize node and edge representations before graph propagation.
- Molecular Graph Neural Network: The aggregation function combines representations from a node’s neighbors and incident edges at each GNN layer.The paper uses GIN as the aggregation architecture after describing alternatives such as GCN and GAT.
- Molecular Graph Neural Network: The graph-level molecular representation is obtained by averaging final-layer node embeddings and can then feed a classifier.Pretraining is used to obtain better parameter initialization for downstream tasks with small datasets.
4 META-MGNN
Meta-MGNN combines graph-based molecular representation learning with MAML-style meta-learning, self-supervised molecular objectives, and self-attentive task weighting for few-shot property prediction.
- Meta-MGNN framework: Meta-MGNN uses a GNN with pre-training to fuse heterogeneous molecular graph information into molecular embeddings.
- Meta-learning framework: The MAML-based framework adapts to each task using support data, then optimizes task-agnostic parameters using query losses across training tasks.At meta-test time, new-task parameters are updated with one or a small number of gradient-descent steps before query evaluation.
- Self-supervised learning: Meta-MGNN supplements sparse supervised signals with self-supervised bond reconstruction and atom-type prediction objectives based on molecular structure and attributes.Bond reconstruction samples existing and non-existing edges, while atom-type prediction uses neighboring-node context.
- Loss function: The molecular property prediction objective applies an MLP to graph-level representation h and uses cross-entropy loss between predicted and ground-truth labels.
- Task-aware attention: Self-attentive task weights measure the differing importance of property-prediction tasks during meta-learner optimization.Task embeddings are computed by averaging molecular embeddings for each task.
5 EXPERIMENTS
Experiments on Tox21 and Sider evaluate few-shot molecular property prediction across datasets, baselines, model variants, and embedding visualizations. Meta-MGNN achieves the strongest overall performance, while ablations support contributions from pre-training, self-supervision, and task-aware attention.
- Datasets: Tox21 contains 7,831 instances across 12 binary tasks, while Sider contains 1,427 instances across 27 binary tasks.Each molecular property is treated as an independent few-shot classification task.
- Experimental Settings: Evaluation uses ROC-AUC with 3 Tox21 and 6 Sider test tasks under 2-way 1-shot and 5-shot settings.The study compares Meta-MGNN with multiple graph and few-shot baseline models.
- Overall Performance: Meta-MGNN outperforms all baseline models on both Tox21 and Sider and has the best performance for every reported task.PreGNN and EGNN are the strongest baselines on average, but their performance is unstable across tasks.
- Overall Performance: +1.80% and +1.87% are Meta-MGNN’s average improvements over the best baseline on Sider for 1-shot and 5-shot learning, respectively.On Tox21, the corresponding improvements are +1.04% and +0.84%.
- Performance on Different Datasets: Sider shows larger gains than Tox21, consistent with the paper’s explanation that more tasks provide more knowledge for few-shot transfer.The paper also reports better overall performance on Tox21 for all models, attributing this to its larger dataset size.
- Ablation Study: Ablations show that pre-training, combined pre-training and meta-learning, bond reconstruction, atom-type prediction, and task-aware attention each improve model performance.M2 performs worst, M6 is best among the three self-supervised variants, and M8 performs best in most cases.
- Embedding Analysis: t-SNE visualizations indicate that Meta-MGNN better discriminates positive and negative SR-MMP molecules than PreGNN and MAML.Blue points denote negative labels and orange points denote positive labels.
6 CONCLUSIONS
The paper proposes Meta-MGNN for few-shot molecular property prediction, combining graph representation learning, meta-learning, self-supervision, and self-attentive task weights. Experiments on two public multi-task datasets report that the model outperforms state-of-the-art methods, while component effectiveness is also verified.
- Meta-MGNN addresses molecular property prediction as a few-shot learning problem to exploit information across properties when individual laboratory data are limited.
- Meta-MGNN uses a pre-trained molecular graph neural network, meta-learning for rapid adaptation, self-supervision, and self-attentive task weights.
- Experiments on two public multi-task datasets show that Meta-MGNN outperforms state-of-the-art methods, with its model components also contributing benefits.
- The study suggests future work on better task-embedding formulations and fusing graph and sequence models for molecular embeddings.