Source-linked AI summary
Neighbor Contrastive Learning on Learnable Graph Augmentation
Xiao Shen, Dewang Sun, Shirui Pan, Xi Zhou, Laurence T. Yang
TL;DR
GCL seeks representations from graphs with limited labels, but existing methods use dataset-sensitive handcrafted augmentations and contrastive objectives that treat neighbors as negatives. NCLA learns adaptive graph views and embeddings end-to-end while using network topology to define multiple positives, and it reports consistently superior performance under extremely scarce labels.
Problem
Existing GCL methods rely on dataset-sensitive handcrafted augmentations and contrastive losses that treat neighboring nodes as negatives despite graph homophily.
Method
NCLA uses multi-head GAT to learn adaptive graph augmentations and a neighbor contrastive loss that defines multiple positives from network topology, jointly with embeddings.
Results
NCLA consistently outperforms state-of-the-art GCL methods and can exceed supervised GNNs for semi-supervised node classification with scarce labels.
Takeaways & Limitations
Topology-aware positives and end-to-end learnable augmentation provide NCLA's supported approach to label-scarce node classification across graph datasets.
Takeaways & Limitations
Studying graphs beyond homophily is outside the scope of this work.
Abstract
from arXiv · showhide
Recent years, graph contrastive learning (GCL), which aims to learn representations from unlabeled graphs, has made great progress. However, the existing GCL methods mostly adopt human-designed graph augmentations, which are sensitive to various graph datasets. In addition, the contrastive losses originally developed in computer vision have been directly applied to graph data, where the neighboring nodes are regarded as negatives and consequently pushed far apart from the anchor. However, this is contradictory with the homophily assumption of networks that connected nodes often belong to the same class and should be close to each other. In this work, we propose an end-to-end automatic GCL method, named NCLA to apply neighbor contrastive learning on learnable graph augmentation. Several graph augmented views with adaptive topology are automatically learned by the multi-head graph attention mechanism, which can be compatible with various graph datasets without prior domain knowledge. In addition, a neighbor contrastive loss is devised to allow multiple positives per anchor by taking network topology as the supervised signals. Both augmentations and embeddings are learned end-to-end in the proposed NCLA. Extensive experiments on the benchmark datasets demonstrate that NCLA yields the state-of-the-art node classification performance on self-supervised GCL and even exceeds the supervised ones, when the labels are extremely limited. Our code is released at https://github.com/shenxiaocam/NCLA.
Introduction
NCLA addresses dataset-sensitive graph augmentations and topology-blind contrastive losses with learnable augmentation and neighbor-aware positives. It learns both augmentations and embeddings end-to-end and reports strong node-classification performance with scarce labels.
- Motivation: GNN training often depends on abundant labels, motivating graph contrastive learning for unlabeled graph representation learning.GCL is presented as a self-supervised approach that reduces reliance on manual labels.
- Motivation: Handcrafted graph augmentations can damage task-relevant topology and must be selected per dataset because no universal augmentation suits diverse graphs.The paper identifies edge dropping and other random perturbations as potentially harmful and dependent on prior domain knowledge or trial and error.
- Motivation: InfoNCE and NT-Xent treat only the same node across views as positive and neighboring nodes as negatives, conflicting with graph homophily.This topology-blind treatment pushes connected nodes apart even though GNNs commonly assume connected nodes have similar labels.
- NCLA: NCLA uses multi-head GAT as a learnable augmentation function, with each head producing an augmented view that avoids improper topology modification and adapts across datasets.The method retains the original nodes and edges while learning adaptive edge weights.
- NCLA: Its neighbor contrastive loss uses network topology to define multiple positives per anchor, addressing the topology mismatch in standard node-node GCL.Neighbors provide additional positive relationships rather than being uniformly treated as negatives.
- Results: NCLA learns graph augmentation jointly with embeddings end-to-end and consistently outperforms state-of-the-art GCL methods, sometimes surpassing supervised GNNs under scarce labels.The reported gains are strongest when only 1–4 labeled nodes per class are available.
Related Work
Prior GCL methods rely on handcrafted or removed augmentations and commonly use node-node or node-graph contrastive modes. NCLA instead learns adaptive attention-based views and introduces topology-supervised neighbor contrastive learning.
- Graph Augmentation: Existing GCL frameworks generate discrepant views through handcrafted operations such as node dropping, edge perturbation, attribute masking, subgraphs, or diffusion.These strategies are sensitive to graph datasets, motivating approaches that remove augmentation or learn it automatically.
- Graph Augmentation: NCLA uses multi-head graph attention to learn adaptive augmentation across datasets while preserving the original graph topology.The learned views retain the same nodes and edges but assign different adaptive edge weights.
- Contrastive Modes: GCL includes node-graph and node-node contrastive modes, with node-node methods comparing representations across graph views.The paper positions NCLA within node-node GCL.
- Contrastive Objective: NCLA's neighbor contrastive loss allows multiple positives per anchor by using network topology as supervision instead of observed class labels.This distinguishes it from NT-Xent and supervised contrastive learning.
Methodology
NCLA jointly learns multiple graph views and node embeddings end-to-end, using multi-head attention for adaptive topology and neighbor-aware contrastive supervision. It treats same-node and neighboring embeddings as positives while distinguishing non-neighbors, avoiding manual augmentation choices and protecting sensitive topology.
- Learnable Graph Augmentation: NCLA uses multi-head GAT to generate K learnable augmented views with adaptive topology, assigning each head a distinct augmentation function.Each view learns adaptive edge coefficients from connected-node features.
- Learnable Graph Augmentation: The encoder aggregates neighbor embeddings using adaptive edge coefficients, applies ELU, and concatenates embeddings from all views into the output representation.The embeddings are learned without downstream labels.
- Learnable Graph Augmentation: NCLA learns augmentation and embeddings jointly end-to-end, avoiding the two-stage optimization and manual dataset-specific choices used by standard GCL.This improves flexibility and removes dependence on prior domain knowledge.
- Learnable Graph Augmentation: Each augmented view has distinct graph-augmentation and embedding parameters, enhancing diversity compared with shared-weight encoders.The view-specific parameters are φ^(k) and W^(k).
- Neighbor Contrastive Learning: NCLA defines multiple positives per anchor from the same node across views and neighboring nodes within and across views, rather than treating neighbors as negatives.An anchor has 2|N_i| + 1 positive pairs, while non-neighbors serve as negatives.
- Neighbor Contrastive Learning: The neighbor contrastive loss increases agreement with same-node and neighbor embeddings while separating non-neighbors within and across views.For more than two views, a randomly selected pivot view is contrasted with each other view.
- Complexity: NCLA has overall time complexity O((NFF′ + N^2F′)K) when |E| ≪ N^2, with K typically very small in practice.The complexity combines multi-head view generation and neighbor contrastive learning.
Experiments
NCLA is evaluated on five benchmark datasets against supervised, semi-supervised, and self-supervised baselines. It performs especially strongly with extremely scarce labels, while ablations and sensitivity analyses examine its loss components and hyperparameters.
- Experimental Setup: Experiments use five benchmark datasets and compare NCLA with 11 state-of-the-art supervised, semi-supervised, and self-supervised methods.The datasets include Cora, Citeseer, Pubmed, Coauthor-CS, and Amazon-Photo.
- Node Classification Results: NCLA consistently outperforms state-of-the-art GCL baselines on all datasets with 1, 2, 3, or 4 labeled nodes per class.With 20 labeled nodes per class, it achieves the best or second-best results, comparable to previous state-of-the-art methods.
- Ablation Study: The neighbor contrastive loss achieves the highest accuracy among its variants across all five datasets.Removing either intra-view or inter-view neighbors from the positives leads to worse results; gains over InfoNCE and NT-Xent are significant on Cora, Citeseer, and Coauthor-CS.
- Hyperparameter Analysis: K=2 augmented views yield the best performance on both Cora and Amazon Photo, while K=6 yields the worst results.The analysis suggests generating 2 or 4 views because more views increase complexity; performance also varies with embedding dimension and temperature.
Conclusion
NCLA addresses limited exploration of learnable graph augmentation and graph-suitable contrastive losses through an end-to-end design. It learns adaptive views and uses network topology to define multiple positives while preserving the original graph structure.
- Conclusion: NCLA uses multi-head GAT to learn adaptive graph augmentations compatible with various graph datasets.Each view preserves the original nodes and edges while learning different adaptive edge weights and view-specific parameters.
- Conclusion: NCLA’s neighbor contrastive loss defines multiple positives per anchor using network topology instead of applying computer-vision contrastive losses directly.Positive pairs include the same node across views and neighboring nodes within and across views.