Source-linked AI summary
Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning
Sheng Wan, Shirui Pan, Jian Yang, Chen Gong
TL;DR
Graph-based SSL must transfer labels from few labeled examples to many unlabeled nodes despite limited supervision. CG3 addresses this gap with semi-supervised contrastive and graph generative losses that use data similarities, class information, graph topology, and features. Experiments on public datasets report effective node classification across different tasks and scarce-label settings.
Problem
Graph-based SSL lacks sufficient supervision because only a small number of examples are labeled, limiting existing methods' effectiveness.
Method
CG3 combines a semi-supervised contrastive loss for data similarities and class information with a graph generative loss relating graph topology to input features.
Results
Experiments on various public datasets show that CG3 outperforms other state-of-the-art approaches under different small label rates across three datasets.
Takeaways & Limitations
Using labeled and unlabeled data together with graph structure provides enriched supervision for learning discriminative node representations and classification.
Takeaways & Limitations
The graph generative component assumes that edges are conditionally independent and that an edge probability depends only on the representations of its endpoint nodes.
Abstract
from arXiv · showhide
Graph-based Semi-Supervised Learning (SSL) aims to transfer the labels of a handful of labeled data to the remaining massive unlabeled data via a graph. As one of the most popular graph-based SSL approaches, the recently proposed Graph Convolutional Networks (GCNs) have gained remarkable progress by combining the sound expressiveness of neural networks with graph structure. Nevertheless, the existing graph-based methods do not directly address the core problem of SSL, i.e., the shortage of supervision, and thus their performances are still very limited. To accommodate this issue, a novel GCN-based SSL algorithm is presented in this paper to enrich the supervision signals by utilizing both data similarities and graph structure. Firstly, by designing a semi-supervised contrastive loss, improved node representations can be generated via maximizing the agreement between different views of the same data or the data from the same class. Therefore, the rich unlabeled data and the scarce yet valuable labeled data can jointly provide abundant supervision information for learning discriminative node representations, which helps improve the subsequent classification result. Secondly, the underlying determinative relationship between the data features and input graph topology is extracted as supplementary supervision signals for SSL via using a graph generative loss related to the input features. Intensive experimental results on a variety of real-world datasets firmly verify the effectiveness of our algorithm compared with other state-of-the-art methods.
Introduction
Graph-based SSL represents labeled and unlabeled examples as graph nodes and transfers labels from the scarce labeled set to many unlabeled nodes. The paper targets the resulting supervision shortage by combining graph-based learning with a semi-supervised contrastive loss that uses data similarities and class information.
- Graph-based SSL transfers labels from a handful of labeled nodes to the remaining massive unlabeled nodes through graph relationships.
- Existing graph-based SSL methods do not directly address the shortage of supervision, making stable GCN training difficult when labeled data are limited.
- The paper develops a GCN-based transductive SSL algorithm that extracts supervision information from the available data themselves.
- The semi-supervised contrastive loss combines similarities among unlabeled examples with scarce class information from labeled examples for node representation learning.
Related Work
Graph-based SSL methods have evolved from neighborhood-similarity regularization and embedding techniques toward graph convolutional networks. Contrastive learning has also expanded into graph representation learning, but standard contrastive methods do not incorporate scarce labeled data directly into SSL.
- Early graph-based SSL methods assume nearby nodes are likely to share labels and use embeddings, spectral kernels, random walks, or Planetoid to exploit this structure.
- Later approaches improve these methods with additional graph-based SSL techniques, while graph convolutional networks extend CNNs to graph-structured data.
- Contrastive learning trains encoders by distinguishing representations that capture statistical dependencies of interest from those that do not.
- Graph representation methods apply contrastive learning by contrasting node and graph encodings, different graph structures, or node representations across graph views.
- Standard contrastive learning does not directly fit SSL because it fails to incorporate scarce but valuable labeled data.
Problem Description
The graph-based SSL problem consists of learning from a dataset containing labeled and unlabeled examples, with the labels of the small labeled subset represented alongside features and graph structure. Typically, the number of labeled examples is much smaller than the number of unlabeled examples.
- The dataset contains n = l + u examples, with the first l labeled and the remaining u unlabeled.
- The labeled subset is typically much smaller than the unlabeled subset, expressed as l ≪u.
- The feature matrix X ∈R^n×d stores each example as a d-dimensional row vector, while Y ∈R^n×c encodes class membership across c classes.
- The dataset is represented by a graph G = ⟨V, E⟩ whose nodes contain the examples and whose edges represent relations among them.
Method
CG3 builds local and global graph views, then combines semi-supervised contrastive learning with graph generative modeling to enrich supervision for node classification.
- Multi-View Establishment for Graph Convolutions: CG3 constructs local and global node representations using parallel GCN and hierarchical GCN views.The local view uses GCN neighborhood aggregation, while HGCN coarsens and refines the graph to capture information from local to global scales.
- Multi-View Establishment for Graph Convolutions: The two views are contrasted so their complementary local and global information can be encoded simultaneously.The hierarchical view serves as an augmented view of the localized view without directly changing the graph topology.
- Semi-Supervised Contrastive Learning: The semi-supervised contrastive loss combines unsupervised cross-view similarity with supervised same-class and different-class pair construction.Unlabeled data contribute similarity-based supervision, while labeled pairs are positive when they share a class and negative otherwise.
- Model Training: The model integrates the two view outputs and applies cross-entropy to the originally labeled nodes for classification training.The integrated output is O = λφ1Hφ1 + (1 − λφ1)Hφ2, with 0 < λφ1 < 1.
- Semi-Supervised Contrastive Learning: Minimizing the combined contrastive loss lets labeled and unlabeled data provide additional supervision for discriminative representation learning.The resulting representations are intended to improve the subsequent classification result.
- Graph Generative Loss: A graph generative loss models the relationship between node representations and graph topology using edge probabilities conditioned on the two views.Edges are treated as conditionally independent binary variables, with their probability depending on the representations of the incident nodes and both views.
Experimental Results
CG3 is evaluated across six benchmark datasets and against diverse graph-based and contrastive baselines. It consistently performs strongly, especially when labeled data are scarce, while ablations and embeddings support contributions from its contrastive and graph-generative components.
- Benchmark evaluation: CG3 is evaluated on six benchmark datasets against traditional, graph-convolutional, attention-based, and contrastive-learning baselines.The datasets include Cora, CiteSeer, PubMed, Amazon Computers, Amazon Photo, and Coauthor CS.
- Benchmark evaluation: CG3 achieves top-level performance across all six datasets and consistently surpasses the other contrastive methods.The authors attribute this to jointly exploiting supervision from data similarities and graph structure through local and global graph views.
- Scarce supervision: CG3 outperforms competing approaches across different small label rates on Cora, CiteSeer, and PubMed.The evaluated label rates include 0.5%, 1%, 2%, and 3% for Cora and CiteSeer, and 0.03%, 0.05%, and 0.1% for PubMed.
- Scarce supervision: GCN performance significantly declines with very limited labels, whereas GCN-based contrastive models generally achieve better results with few labeled examples.The paper gives Cora at a 0.5% label rate as an example of GCN degradation.
- Ablation study: Removing either the contrastive or graph-generative loss decreases classification accuracy, with the contrastive loss raising CiteSeer accuracy by nearly 4%.The ablation study covers Cora, CiteSeer, and PubMed.
- Embedding analysis: CG3 produces more coherent Cora embedding clusters than GCN and HGCN in the t-SNE visualization.The model combines local and global information through two parallel graph-convolution views and contrastive operation between them.
Conclusion
CG3 enriches supervision for transductive semi-supervised node classification by combining semi-supervised contrastive learning with graph-generation signals.
- CG3 uses scarce class information and data similarities to provide abundant supervision for discriminative representation learning.
- The method further leverages the underlying relationship between input graph topology and data features to enrich supervision signals.
- Experiments on various public datasets illustrate CG3's effectiveness across different node classification tasks.