PaperlayerView exact revision on arXiv

Source-linked AI summary

GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphs

Binghui Wang, Ang Li, Hai Li, Yiran Chen

arXiv:2012.04187v1cs.LGstat.ML

TL;DR

Graph-based semi-supervised node classification is difficult to centralize because collecting complete graphs and labels is costly and may threaten privacy, while existing FL methods struggle with non-IID data, new label domains, and unlabeled nodes. GraphFL uses MAML-inspired federated methods and self-training to address these issues, significantly outperforming conventional FL and improving further when unlabeled data are leveraged.

  • Problem

    Centralized GraphSSC requires costly graph and label collection, while existing FL methods struggle with non-IID data, new label domains, and unlabeled nodes.

  • Method

    GraphFL combines two MAML-inspired federated methods for non-IID data and new label domains with self-training for unlabeled graph data.

  • Results

    GraphFL significantly outperforms conventional FL for non-IID graph data and new label domains, while self-training further improves performance.

  • Takeaways & Limitations

    Federated GraphSSC can address heterogeneous clients, changing label domains, and unlabeled nodes within one GraphFL framework.

  • Takeaways & Limitations

    Each client is assumed to have only a few labeled nodes and the server learns from aggregated local parameters without accessing client graphs.

Abstract

from arXiv · show

Graph-based semi-supervised node classification (GraphSSC) has wide applications, ranging from networking and security to data mining and machine learning, etc. However, existing centralized GraphSSC methods are impractical to solve many real-world graph-based problems, as collecting the entire graph and labeling a reasonable number of labels is time-consuming and costly, and data privacy may be also violated. Federated learning (FL) is an emerging learning paradigm that enables collaborative learning among multiple clients, which can mitigate the issue of label scarcity and protect data privacy as well. Therefore, performing GraphSSC under the FL setting is a promising solution to solve real-world graph-based problems. However, existing FL methods 1) perform poorly when data across clients are non-IID, 2) cannot handle data with new label domains, and 3) cannot leverage unlabeled data, while all these issues naturally happen in real-world graph-based problems. To address the above issues, we propose the first FL framework, namely GraphFL, for semi-supervised node classification on graphs. Our framework is motivated by meta-learning methods. Specifically, we propose two GraphFL methods to respectively address the non-IID issue in graph data and handle the tasks with new label domains. Furthermore, we design a self-training method to leverage unlabeled graph data. We adopt representative graph neural networks as GraphSSC methods and evaluate GraphFL on multiple graph datasets. Experimental results demonstrate that GraphFL significantly outperforms the compared FL baseline and GraphFL with self-training can obtain better performance.

1 Introduction

GraphFL introduces a federated framework for semi-supervised node classification that targets non-IID graph data, new label domains, and unlabeled nodes. MAML-inspired methods and self-training outperform the compared FL baseline on multiple graph datasets.

  • GraphFL is presented as the first federated semi-supervised node classification framework for graphs.
  • GraphFL addresses non-IID client data, testing nodes with new label domains, and unlabeled nodes through two methods and self-training.
  • GraphFL significantly outperforms the compared FL baseline when labeled nodes across clients are highly non-IID.
  • GraphFL shows better ability than FL to handle testing nodes with new label domains, while self-training further improves performance.

2 Related Work

Graph-based semi-supervised node classification is studied across networking, security, data mining, and machine learning. Representative approaches include conventional propagation-based methods and graph neural networks such as GCN and SGC.

  • Graph-based semi-supervised node classification has applications across networking, security, data mining, and machine learning.
  • Conventional approaches include label propagation, iterative classification, manifold regularization, and belief propagation.
  • GCN and SGC are representative GNNs for graph-based semi-supervised node classification.GCN uses learnable first-order spectral filters and nonlinear activation, whereas SGC removes intermediate nonlinearities and is more computationally efficient.

3 Problem Definition and Background

The problem setting distributes partially labeled client graphs and uses server aggregation to predict testing-node labels without accessing client graphs. MAML provides a two-level adaptation framework that motivates GraphFL’s federated design.

  • 3.1 Problem definition: Each client owns a graph with node features, labels, and a small labeled subset, while a server aggregates local parameters without accessing client graphs.
  • 3.1 Problem definition: The goal is to predict testing-node labels using global parameters aggregated from client models, even when testing and training label domains differ.
  • 3.1 Problem definition: The federated GraphSSC design aims to address non-IID graph data, new label domains, and unlabeled client nodes.
  • 3.2 Model-agnostic meta learning (MAML): MAML learns a task-independent initialization that adapts to tasks through inner optimization on support sets and meta-optimization over query-set losses.
  • 3.2 Model-agnostic meta learning (MAML): GraphFL maps MAML tasks to clients and its task-independent initialization to a server-learned global model.

4 The Proposed GraphFL Framework

GraphFL is an FL framework for federated GraphSSC that uses meta-learning to address non-IID graph data and new label domains, while self-training incorporates unlabeled nodes.

  • 4 The Proposed GraphFL Framework: GraphFL combines two meta-learning-based FL methods for non-IID graph data and testing nodes with new label domains, plus self-training for unlabeled nodes.The framework reformulates MAML in FL to learn a shared initialization that adapts across clients or new label domains.
  • 4.1 GraphFL for federated GraphSSC with non-IID graph data: For non-IID graphs, GraphFL first uses MAML-style client losses and server updates, then applies FL updates to improve generalization across clients.The method’s two stages learn a global model through meta-learning and subsequently use existing FL methods for further updating.
  • 4.2 GraphFL for federated GraphSSC with new label domains: For new label domains, GraphFL learns a shared global initialization that adapts with a few labeled nodes before predicting testing nodes from those domains.The global model is updated through a few gradient steps using labeled nodes from the new label domains.
  • 4.3 Leveraging unlabeled nodes via self-training: Self-training selects unlabeled nodes with the most confident local predictions, assigns pseudo labels, and adds them to each client’s training set.GraphFL is then trained on these augmented client datasets.

5.1 Experimental Setup

The experiments use representative graph neural networks and compare GraphFL with individual and federated learning across benchmark graph datasets under non-IID and new-label-domain settings.

  • 5.1 Experimental Setup: The evaluation covers four benchmark datasets: Cora, Citeseer, Coauthor CS, and Amazon2M.These include citation, coauthorship, and product-purchase graphs with node features and labels.
  • 5.1 Experimental Setup: Experiments use GCN and SGC as GraphSSC models and compare GraphFL with individual learning and federated learning.Cluster-GCN is used for Amazon2M because of its large size.
  • 5.1 Experimental Setup: Non-IID experiments sample 80 nodes per class and distribute labeled nodes across 50 clients, leaving each client only 9 to 70 labeled nodes.This setup makes labeled nodes across clients potentially highly non-IID.
  • 5.1 Experimental Setup: New-label-domain experiments train clients on nodes from one class subset and test on nodes from the remaining classes.The number of held-out classes varies by dataset.

5.2 Node classification results with non-IID graph data

GraphFL improves federated node classification under non-IID graph data, with the largest advantages when client participation or graph overlap is low.

  • Experimental setup: The experiments compare GCN and SGC accuracy against labeled nodes, participating-client fraction, and graph-overlap fraction.Figures 1–4 cover these settings for federated GraphSSC.
  • 5.2.1 Impact of the number of labeled nodes per class.: ∼20% accuracy gap separates GraphFL from FL with 10 labeled nodes per class, where non-IID conditions are more serious.GraphFL consistently outperforms FL, while individual learning performs worst because it uses only each client’s labels.
  • 5.2.2 Impact of the fraction of participating clients per episode.: GraphFL outperforms FL across datasets, and its advantage is larger when fewer clients participate per episode.Both methods improve as more clients participate because they can leverage more labeled nodes.
  • 5.2.3 Impact of the fraction of overlapped nodes between client graphs.: GraphFL consistently outperforms FL as client-graph overlap varies, with its largest gain when graphs have no overlapping nodes.Increasing overlap improves both methods; no overlap corresponds to the most non-IID graph data.

5.3 Node classification results with new label domains

GraphFL handles federated node classification with new label domains better than FL, including when only limited labeled data are available.

  • New label domains: At least 10% higher accuracy distinguishes GraphFL from FL in almost all new-label-domain cases.The comparison covers GCN and SGC on four graph datasets across labeled-node counts.
  • New label domains: GraphFL’s performance gain over FL + transfer learning becomes larger as the number of labeled nodes per class increases.The results indicate that transfer learning is not sufficiently effective for these federated new-label-domain tasks.

5.4 Node classification results with self-training

Self-training enhances GraphFL by adding confidently pseudo-labeled unlabeled nodes, but excessive pseudo-labeling can reduce accuracy.

  • Self-training: Accuracy improves when GraphFL trains with pseudo-labeled nodes, demonstrating that selected unlabeled nodes benefit federated training.The method selects unlabeled nodes with the most confident predictions and adds them to each client’s training set.
  • Self-training: Accuracy first increases and then decreases as pseudo-labeled nodes per class increase.Larger selections eventually include more incorrectly predicted labels, offsetting the benefit of additional pseudo-labels.
  • Self-training: Figure 5 evaluates GCN and SGC GraphFL accuracy against the number of pseudo-labeled nodes per class.The evaluation examines self-training on the four datasets.

5.5 Summary

GraphFL improves federated semi-supervised node classification for non-IID graph data and new label domains, with further gains from self-training.

  • Summary: GraphFL significantly outperforms conventional FL when handling non-IID graph data and testing nodes with new label domains.This is the paper’s summary comparison across the two principal federated GraphSSC challenges.
  • Summary: GraphFL can be further enhanced by leveraging unlabeled nodes through self-training.Self-training adds pseudo-labeled nodes to the federated training process.
  • Summary: More training nodes, participating clients, client-graph size, or leveraged unlabeled nodes can improve federated graph node-classification performance.These factors are summarized as performance-enhancing experimental conditions.

6 Conclusion

GraphFL is presented as the first federated framework for semi-supervised node classification on graphs, addressing non-IID data, new label domains, and unlabeled nodes. MAML-inspired methods and self-training improve federated graph classification across multiple datasets.

  • GraphFL addresses three federated graph challenges: non-IID client data, differing training and testing label domains, and substantial unlabeled data.
  • Two MAML-inspired GraphFL methods target non-IID data and new label domains, while self-training addresses unlabeled nodes.
  • Across multiple graph datasets and two representative graph neural networks, GraphFL outperforms compared baselines, with self-training obtaining better performance.
Loading 2012.04187v1…