Source-linked AI summary
Graph Few-shot Learning via Knowledge Transfer
Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh V. Chawla, Zhenhui Li
TL;DR
Semi-supervised node classification remains difficult when labels are scarce and shallow GNNs have limited receptive fields. GFL transfers structural knowledge from auxiliary graphs through a transferable metric space with node and prototype embeddings. Experiments on four graph datasets demonstrate its effectiveness.
Problem
Semi-supervised node classification is challenging because shallow GNNs have restricted receptive fields and perform less satisfactorily when labeled nodes are especially scarce.
Method
GFL learns a transferable metric space with node and class-prototype embedding functions, transferring node-level and graph-level structures from auxiliary graphs to a target graph.
Results
GFL-mean and GFL-att achieve the best performance among the compared transfer and few-shot methods on four datasets.
Takeaways & Limitations
GFL provides a framework for improving semi-supervised node classification on a new target graph by transferring knowledge learned from auxiliary graphs.
Takeaways & Limitations
The graph few-shot learning setting assumes graphs are sampled from a task distribution and provide labeled support nodes for adaptation.
Abstract
from arXiv · showhide
Towards the challenging problem of semi-supervised node classification, there have been extensive studies. As a frontier, Graph Neural Networks (GNNs) have aroused great interest recently, which update the representation of each node by aggregating information of its neighbors. However, most GNNs have shallow layers with a limited receptive field and may not achieve satisfactory performance especially when the number of labeled nodes is quite small. To address this challenge, we innovatively propose a graph few-shot learning (GFL) algorithm that incorporates prior knowledge learned from auxiliary graphs to improve classification accuracy on the target graph. Specifically, a transferable metric space characterized by a node embedding and a graph-specific prototype embedding function is shared between auxiliary graphs and the target, facilitating the transfer of structural knowledge. Extensive experiments and ablation studies on four real-world graph datasets demonstrate the effectiveness of our proposed model.
1 Introduction
Semi-supervised node classification is difficult when labels are scarce and shallow GNNs cannot capture global graph information. GFL transfers node-level and graph-level structural knowledge from auxiliary graphs through a transferable metric space.
- Semi-supervised node classification is challenging because annotations are costly and available for only a small fraction of nodes.
- Shallow GNNs aggregate neighbor information but have restricted receptive fields, limiting their ability to characterize global information with few labels.
- GFL uses a transferable metric space where node labels are predicted from the nearest class prototype.
- GFL learns node representations with a GNN-based graph autoencoder and constructs relational structures among examples belonging to the same class.
- GFL transfers node-level embedding functions and graph-level representations from auxiliary graphs to the target graph.
- Experiments on four node classification tasks empirically demonstrate GFL's effectiveness.
2 Related Work
Prior work includes GNNs that exploit graph structure and few-shot methods developed for non-graph data. Metric-based few-shot learning is limited by the absence of explicit interactions in independently distributed data.
- Graph Neural Networks: GNN research includes non-spectral and spectral methods for exploiting structures underlying graphs.
- Few-shot Learning: Few-shot learning methods commonly use gradient-based parameter initialization or metric-based representations.
- Few-shot Learning: Traditional metric-based few-shot methods target independently and identically distributed data without explicit interactions.
3 Preliminaries
The paper formulates graph few-shot learning over graphs represented by adjacency and feature matrices, using GNN embeddings and prototype-based prediction. Meta-training learns an embedding from historical graphs for use on a new graph with few labeled support nodes.
- Graph Neural Network: A graph is represented by an adjacency matrix A and node feature matrix X, which initialize GNN-based node embeddings.
- Graph Neural Network: GNN message passing updates node embeddings through a function M using learnable layer weights.
- The Graph Few-Shot Learning Problem: Graph few-shot learning samples graphs from a task distribution and provides each graph with labeled support and query node sets.
- The Graph Few-Shot Learning Problem: For each class, prototypical networks define a prototype from support examples and classify query nodes using embedding similarity.
- The Graph Few-Shot Learning Problem: Meta-training optimizes the embedding function over historical graphs, then applies it to a new graph with few support nodes.
4 Methodology
GFL transfers graph-structured knowledge through node-level prototypes and graph-level representations, while auxiliary graph reconstruction stabilizes node embedding learning.
- Graph Few-shot Learning: GFL adapts knowledge from existing graphs to a target graph by modeling relational structure at both node and graph levels.Node-level prototypes capture relations among same-class support nodes, while graph-level representations tailor transferable knowledge to each graph.
- Graph Structured Prototype: For each class, GFL constructs a relational graph from support-node similarities and uses a prototype GNN to learn its prototype representation.Similarities may use common neighbors or inverse topological distance; thresholding reduces the influence of outlier nodes.
- Hierarchical Graph Representation Gate: Hierarchical graph representations alternate node assignment and representation fusion across levels to encode whole-graph structure.Assignment GNNs produce soft assignments between levels, while fusion GNNs compute higher-level adjacency and feature matrices.
- Hierarchical Graph Representation Gate: Mean pooling or attention aggregates level representations into a graph-specific representation used to gate the globally shared prototype-GNN parameters.The gate maps the graph representation into the parameter space and applies element-wise multiplication to adapt prototype-GNN parameters.
- Auxiliary Graph Reconstruction: A graph autoencoder adds reconstruction loss to improve training stability and node-representation quality alongside the matching objective.Training samples graph batches, support and query sets, embeddings, reconstruction errors, prototypes, and matching losses before updating model parameters.
5 Experiments
The experiments evaluate GFL on four graph datasets under 10-shot node classification using graph-based, representation-learning, and transfer/few-shot baselines.
- Experimental Setup: Experiments address baseline performance, component effectiveness, and whether GFL learns better class representations.The evaluation uses four datasets representing collaboration, Reddit, citation, and biomedical citation graphs.
- Dataset Description: The four tasks predict author research domains, Reddit post communities, paper categories, and PubMed diabetes class labels.Collaboration and citation data come from AMiner, Reddit graphs connect posts sharing commenters, and PubMed graphs come from the PubMed database.
- Experimental Setup: Each graph provides N labeled nodes per class as support, with remaining nodes forming the query set for evaluation.The node embedding uses a two-layer GCN with 32 neurons in each layer.
- Baseline Methods: Baselines include graph-based semi-supervised methods, graph representation-learning methods, and transfer/few-shot methods.The listed graph-based and representation-learning baselines include LP, Planetoid, DeepWalk, node2vec, and Non-transfer-GCN.
5.1 Results
GFL and its variants outperform competing graph-based and few-shot methods across four datasets, while ablations show that its graph-level and reconstruction components contribute to performance.
- GFL-mean and GFL-att achieve the best performance among transfer and few-shot methods on all four datasets.The comparison uses averaged accuracy with 95% confidence intervals for 10-shot node classification.
- Transfer and few-shot methods significantly improve performance over graph-based semi-supervised and graph representation learning methods.
- The ablation study evaluates GFL’s graph structured prototype, hierarchical graph representation gate, and auxiliary graph reconstruction components.
- Removing the hierarchical graph representation gate produces inferior results, supporting the contribution of graph-level representation.
- Removing the decoder GNN reduces performance because graph reconstruction refines node representations and enhances training stability.
5.2 Sensitivity Analysis
Sensitivity analyses examine support-set size, prototype-construction choices, similarity functions, and distance functions. GFL remains robust across these settings and achieves the best performance across support-set sizes.
- Support Set Size: GFL achieves the best performance across all examined support-set sizes, including scenarios with small support sets.The comparison includes Protonet and MAML.
- Support Set Size: When support sets are small, Protonet performs worse than MAML, potentially because equal-weight averaging makes its prototype sensitive to outliers.
- Threshold µ: The best threshold for relational structure construction varies across datasets, while threshold effectiveness demonstrates robustness to outliers.
- Similarity Functions: GFL is not very sensitive to the similarity function used to construct the graph prototype’s relational structure.The evaluated functions include Jaccard Index, Adamic-Adar, PageRank, and Top-k Common Neighbors.
- Distance Function: Replacing inner-product distance with cosine distance produces similar results, indicating low sensitivity to the distance function.
5.3 Analysis of Learned Representation
The learned embeddings are visualized with t-SNE for positive samples and 1,000 negative samples per class. Compared with Protonet, GFL better distinguishes positive and negative samples.
- GFL better distinguishes positive and negative samples than Protonet in the collaboration-data embedding visualization.For each class, t-SNE visualizes positive samples and 1,000 negative samples using the node embedding fθ.
6 Conclusion
The paper introduces GFL to transfer knowledge from auxiliary graphs to improve semi-supervised node classification on a new target graph. GFL combines local node-level and global graph-level knowledge in a transferable metric space, with effectiveness demonstrated on four datasets.
- GFL transfers knowledge learned from auxiliary graphs to improve semi-supervised node classification on a new target graph.
- GFL learns a transferable metric space characterized by node and prototype embedding functions while integrating local node-level and global graph-level knowledge.
- Empirical results demonstrate GFL’s effectiveness on four node classification datasets.
A Additional Hyperparameter Settings
The appendix specifies additional optimization and graph-construction settings for GFL, including a similarity-based relational graph among few-shot labeled nodes.
- A Additional Hyperparameter Settings: The learning rate is set to 0.01, the hierarchical graph representation dimension hi to 32, and the reconstruction loss weight γ to 1.0.These settings define the optimization scale, representation size, and reconstruction-term weighting.
- A Additional Hyperparameter Settings: For each class, GFL constructs a relational graph of few-shot labeled nodes using pairwise counts of k-hop common neighbors with k=3.The construction begins by counting shared three-hop neighbors between node pairs.
- A Additional Hyperparameter Settings: The common-neighbor similarity matrix is further smoothed with a sigmoid function before constructing the relational graph.This smoothing is applied after computing pairwise similarity scores.
B Detailed Descriptions of Baselines
The baselines span graph-based semi-supervised learning, graph representation learning, and transfer or few-shot methods using gradient-based or metric-based strategies.
- Graph-based semi-supervised methods: Label Propagation is included as a traditional graph-based semi-supervised learning baseline.The graph-based semi-supervised category also includes Planetoid, which uses graph embeddings in transductive form.
- Graph representation learning methods: Deepwalk and node2vec learn unsupervised node embeddings, which are used with node features or for downstream classification.Deepwalk concatenates learned embeddings with node features before multiclass classification, using few-shot node labels for classifier training.
- Graph representation learning methods: Non-transfer-GCN trains a GCN on each meta-testing network without transferring knowledge from meta-training networks.This provides a non-transfer graph neural network comparison.
- Transfer/Few-shot methods: AGF learns GCN parameter initialization from meta-training graphs and fine-tunes it on each meta-testing graph.AGF feeds meta-training graphs one by one and uses the same hyperparameters as GFL.
- Transfer/Few-shot methods: K-NN learns a GCN embedding function from all meta-training graphs, then classifies query nodes using support-node representations and k-nearest neighbors.The learned GCN produces representations for both support and query nodes before k-NN classification.
- Transfer/Few-shot methods: Matching Network, MAML, and Prototypical Network represent metric-based, gradient-based, and prototype-based few-shot baselines, respectively.Matching Network matches query representations to support nodes; MAML adapts a learned initialization with a few gradient steps; Prototypical Network uses class-wise mean pooling.