Source-linked AI summary
MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph
Mengying Sun, Jing Xing, Huijun Wang, Bin Chen, Jiayu Zhou
TL;DR
Biomedical GNNs need pretraining because task-specific labels are scarce and costly, while graph contrastive learning may use unsuitable augmentations and overlook dataset-wide structure. MoCL injects local and global molecular knowledge through semantics-preserving augmentation and a double contrast objective, and the paper reports state-of-the-art performance across molecular datasets under linear and semi-supervised settings. Its effectiveness varies with dataset and task properties, with toxicity prediction benefiting less when cellular environment matters.
Problem
Biomedical GNN labels are scarce and costly, while general graph augmentations can alter molecular semantics and contrastive learning can neglect global dataset structure.
Method
MoCL uses domain knowledge for semantics-preserving substructure substitution, global graph-similarity encoding, and a double contrast objective.
Results
MoCL achieves state-of-the-art performance across various molecular datasets under linear and semi-supervised evaluation settings.
Takeaways & Limitations
Both local- and global-level domain knowledge improve the quality of learned molecular representations.
Takeaways & Limitations
Effectiveness depends on dataset and task properties, with toxicity prediction benefiting less because it also depends on cellular environment.
Abstract
from arXiv · showhide
Recent years have seen a rapid growth of utilizing graph neural networks (GNNs) in the biomedical domain for tackling drug-related problems. However, like any other deep architectures, GNNs are data hungry. While requiring labels in real world is often expensive, pretraining GNNs in an unsupervised manner has been actively explored. Among them, graph contrastive learning, by maximizing the mutual information between paired graph augmentations, has been shown to be effective on various downstream tasks. However, the current graph contrastive learning framework has two limitations. First, the augmentations are designed for general graphs and thus may not be suitable or powerful enough for certain domains. Second, the contrastive scheme only learns representations that are invariant to local perturbations and thus does not consider the global structure of the dataset, which may also be useful for downstream tasks. Therefore, in this paper, we study graph contrastive learning in the context of biomedical domain, where molecular graphs are present. We propose a novel framework called MoCL, which utilizes domain knowledge at both local- and global-level to assist representation learning. The local-level domain knowledge guides the augmentation process such that variation is introduced without changing graph semantics. The global-level knowledge encodes the similarity information between graphs in the entire dataset and helps to learn representations with richer semantics. The entire model is learned through a double contrast objective. We evaluate MoCL on various molecular datasets under both linear and semi-supervised settings and results show that MoCL achieves state-of-the-art performance.
1 INTRODUCTION
GNN pretraining addresses the scarcity and expense of biomedical labels, but graph contrastive learning faces domain-specific augmentation and global-structure challenges. MoCL incorporates local and global molecular knowledge through domain-aware augmentation and a double contrast objective.
- GNNs perform strongly on graph tasks and are increasingly used for drug-related biomedical problems.
- Because biomedical labels are limited, expensive, and time-consuming to obtain, unsupervised GNN pretraining has been actively explored.
- Graph contrastive learning learns representations invariant to transformations by maximizing mutual information between augmented views.
- General graph augmentations may harm molecular semantics, while standard contrastive methods often neglect similarities across the entire graph dataset.
- MoCL uses local domain knowledge for substructure substitution and global similarity information in a double contrast objective.Substructure substitution introduces variation while avoiding large changes to molecular properties.
- The paper contributes domain-aware molecular augmentation, global-structure encoding, theoretical links to triplet loss, and evaluations across molecular datasets.
2 RELATED WORK
Prior graph self-supervised learning designs pretext tasks for unlabeled data, including node- and graph-level prediction, reconstruction, and context-based objectives.
- Graph self-supervised methods learn representations from unlabeled data by designing pretext tasks for nodes or graphs.
- Node-level tasks include clustering, graph partition, graph completion, contextual-position prediction, and generative reconstruction of attributes and edges.
- Graph-level methods predict neighborhood context or node attributes before applying the pretrained GNN to downstream tasks.
3 METHOD
MoCL learns unlabeled molecular graph representations by combining domain-aware local augmentations with global similarity information. Its double contrastive framework preserves molecular semantics during variation while encoding relationships among similar graphs.
- 3.1 Problem Definition: MoCL maps an unlabeled molecular graph G = (V, E) with node and edge features to a vector representation for downstream tasks.The encoder produces h = f(X, A, E), and the learned representations can be used directly or via finetuning.
- 3.2 Contrastive Learning Framework: Two domain-informed augmented views and the original view are encoded and projected for local and global contrastive learning.Local contrast maximizes mutual information between augmented views, while global contrast uses similar graphs identified from domain knowledge.
- 3.2 Contrastive Learning Framework: The local objective contrasts each augmented view against the other, treating corresponding views as positive pairs and other batch pairs as negatives.The similarity function and temperature scale the contrastive objective, which uses both contrast directions.
- 3.3 Local-level Domain Knowledge: Substructure substitution replaces valid molecular substructures with bioisosteres that have similar physical or chemical properties.MoCL compiles 218 bioisostere rules, adds 12 carbon-group rules, and uses 230 rules in total to generate molecule variants.
- 3.4 Global-level Domain Knowledge: Global similarity is derived from molecular fingerprints or drug-target interaction profiles using the Tanimoto coefficient, where larger values indicate greater structural similarity.MoCL uses this information either as direct supervision or to define similar-graph neighborhoods for a global contrastive objective.
- 3.4 Global-level Domain Knowledge: The full MoCL objective combines local and global losses, with λ controlling their relative emphasis.The framework computes the local loss and global loss before optimizing their combined objective.
4 EXPERIMENT
MoCL is evaluated on seven molecular datasets using linear and semi-supervised protocols, testing local and global domain-knowledge components. Results show stronger performance from domain-aware augmentation and global similarity information, while performance varies with augmentation strength, hyperparameters, and evaluation setting.
- Experimental Setup: The study compares fixed pretrained representations with linear classifiers and encoder initialization followed by full fine-tuning.The linear protocol freezes representations, whereas the semi-supervised protocol fine-tunes all layers using a small label fraction.
- Experimental Setup: Experiments use seven benchmark molecular datasets spanning binding affinity, bioassay response, toxicity, and adverse-reaction tasks.Models are evaluated with linear and semi-supervised protocols.
- Local-level domain knowledge: MoCL-DK representations match training-from-scratch accuracy on bace, bbbp, and sider, and exceed it on clintox and mutag under linear evaluation.Figure 3 compares augmentation combinations using five-run averaged performance differences.
- Local-level domain knowledge: Combining MoCL-DK with attribute masking usually performs best, while increasing MoCL-DK strength typically improves performance before causing a decline; MoCL-DK3 is often strongest.The trend is unclear between protocols for clintox and toxcast.
- Global-level domain knowledge: Global similarity information generally improves augmentation schemes, with larger gains for MoCL-DK1 and MoCL-DK3; global contrastive loss outperforms least-square supervision.The comparison uses median gains across all seven datasets and evaluates both global-loss variants.
- Hyperparameter sensitivity: A relatively small but nonminimal neighbor size and a moderately large global-loss weight produce the best results across datasets.The reported sensitivity analysis uses bbbp as the illustrated example and notes a similar pattern elsewhere.
5 CONCLUSION
MoCL uses multi-level domain knowledge to assist contrastive representation learning on molecular graphs. Local knowledge enables a new augmentation scheme, while global knowledge incorporates dataset structure into learning.
- MoCL uses local- and global-level domain knowledge to improve contrastive representation learning on molecular graphs.Local knowledge enables a new augmentation scheme, while global knowledge incorporates global data structure into learning.
Implementation Details
The implementation uses dataset-dependent semi-supervised sampling and neighbor clustering, alongside general augmentation operations that modify node and edge information in specified ways. Proposed augmentations are also examined through their generated distributions.
- Semi-supervised sampling selects around 100 molecule labels from each dataset.
- Neighbor size is dataset-dependent, keeping the number of clusters between 5 and 30.
- MoCL-DK uses shared values across datasets for global-loss weight and augmentation time.
- General augmentations operate on both node and edge features, with dropping, perturbing, subgraph sampling, and masking producing specified changes.Drop removes nodes and edges; edge perturbation permutes edge information; subgraph extraction subsamples nodes and features; masking masks attributes.
- Figure 7 shows the distribution of augmentations generated by the proposed augmentation rules on the BACE dataset.
Proof of Lemma 1
The lemma proof specializes the representation objective to an identity projection, inner-product similarity, one nearest neighbor, and λ = 1 before applying a first-order Taylor expansion.
- The proof assumes the projection head is identity, so z = h, and defines similarity as the inner product z_i^Tz_j.
- It considers the 1-nearest neighbor of each graph in the batch to represent global structure information.
- With λ = 1, the objective L_i is transformed into an equivalent expression before first-order Taylor expansion is applied.