PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Graph Neural Networks: Overview, Techniques and Challenges

Rui Liu, Pengwei Xing, Zichao Deng, Anran Li, Cuntai Guan, Han Yu

arXiv:2202.07256v2cs.DCcs.LG

TL;DR

FedGNN research must reconcile graph learning with private, distributed, and heterogeneous client data while using relationships among clients. This paper surveys the field with a two-dimensional taxonomy, reviews its techniques and challenges, and outlines directions toward more trustworthy systems.

  • Problem

    A comprehensive and insightful survey of FedGNNs was lacking despite rapid development of this field and challenges involving private, isolated, and non-IID graph data.

  • Method

    The paper synthesizes FedGNN research using a 2D taxonomy of FL–GNN integration and heterogeneity across FL clients, while reviewing methods, limitations, benchmarks, and applications.

  • Results

    The survey provides an organized view of FedGNN techniques, challenges, limitations, applications, datasets, evaluation metrics, and partition methods.

  • Takeaways & Limitations

    The paper identifies future directions toward FedGNNs that are more robust, explainable, efficient, fair, inductive, and comprehensive.

Abstract

from arXiv · show

With its capability to deal with graph data, which is widely found in practical applications, graph neural networks (GNNs) have attracted significant research attention in recent years. As societies become increasingly concerned with the need for data privacy protection, GNNs face the need to adapt to this new normal. Besides, as clients in Federated Learning (FL) may have relationships, more powerful tools are required to utilize such implicit information to boost performance. This has led to the rapid development of the emerging research field of federated graph neural networks (FedGNNs). This promising interdisciplinary field is highly challenging for interested researchers to grasp. The lack of an insightful survey on this topic further exacerbates the entry difficulty. In this paper, we bridge this gap by offering a comprehensive survey of this emerging field. We propose a 2-dimensional taxonomy of the FedGNNs literature: 1) the main taxonomy provides a clear perspective on the integration of GNNs and FL by analyzing how GNNs enhance FL training as well as how FL assists GNNs training, and 2) the auxiliary taxonomy provides a view on how FedGNNs deal with heterogeneity across FL clients. Through discussions of key ideas, challenges, and limitations of existing works, we envision future research directions that can help build more robust, explainable, efficient, fair, inductive, and comprehensive FedGNNs.

I. INTRODUCTION

FedGNNs emerged to address private, distributed, and heterogeneous graph data while exploiting relationships among federated clients. This survey organizes the field with a two-dimensional taxonomy and reviews methods, challenges, limitations, benchmarks, and future directions.

  • Motivation: Privacy concerns and data isolation prevent multiple graph-data owners from freely sharing the training data needed by GNNs.Client data may also be non-IID in graph structures or node-feature distributions.
  • Motivation: Federated learning keeps local data private while allowing clients to benefit from one another through shared model parameters or embedding features.Client relationships can form a graph containing useful information such as similarities and trust, but exploiting it remains challenging.
  • Research gap: FedGNNs research has developed rapidly, but existing early categorizations and the broader literature lacked a comprehensive survey for new researchers.Earlier works grouped settings by whether clients contain multiple graphs, sub-graphs, or one node.
  • Contributions: The survey proposes a 2D taxonomy covering how FL and GNNs integrate and how FedGNNs handle heterogeneity across clients.The authors also highlight challenges, methods, and potential limitations for each category.
  • Scope and outlook: The paper reviews datasets, evaluation metrics, and partition methods, then proposes seven future directions for more trustworthy FedGNNs.The envisioned systems are more robust, explainable, efficient, fair, inductive, and comprehensive.

A. Terminology

GNNs aggregate graph-neighborhood information for node embeddings, while FL trains across private client datasets through server aggregation. The taxonomy organizes FedGNNs by their integration of GNNs and FL and by client heterogeneity.

  • GNN terminology: GNNs perform feature embedding and inference from an adjacency matrix and node features, updating node embeddings through neighborhood aggregation.The aggregation function may use mean, weighted average, or max/min pooling.
  • FL terminology: In federated learning, multiple data owners train collaboratively without exchanging raw data, using horizontal or vertical feature-and-sample overlap settings.Clients are coordinated by a central server when sensitive local data are involved.
  • FL terminology: FL aggregation updates server model parameters from client-uploaded local parameters, for example through FedAvg weighted by client sample counts.The paper distinguishes this operation from GNN aggregation.
  • Main taxonomy: The main taxonomy separates GNN-assisted FL, which uses GNN techniques for structured-client FL training, from FL-assisted GNNs, which trains global GNNs over isolated graph-data silos.The two branches are further divided by server presence and node-ID overlap, respectively.
  • Auxiliary taxonomy: The auxiliary taxonomy groups clients by shared node IDs, shared network structure with different nodes, or different network structures, using different intermediate information for aggregation.For clients with the same nodes, node embeddings are uploaded to the server.

III. GNN-ASSISTED FEDERATED LEARNING

GNN-assisted FL uses relationships among clients to improve federated training, either through server-side or client-side GNNs. Server-side approaches train GNNs on inter-client information to facilitate aggregation through bi-level or sequential optimization.

  • GNN-Assisted Federated Learning: GNNs exploit inter-client relationships to address non-IID data distributions and can model heterogeneous neural network architectures.Client relationships may reflect similar data distributions, while neural networks can be represented as graphs when local architectures differ.
  • Centralized FedGNNs: Centralized FedGNNs coordinate clients through a central server, with GNN training performed either on the server or by clients.Clients’ local data need not be graph data; the location of GNN training depends on where the inter-client graph is stored.
  • Server-Side GNNs Training: Server-side GNN training treats uploaded local model parameters as node features and uses the inter-client graph to facilitate FL aggregation.The server collects client parameters, trains a GNN, and sends updated parameters back to clients; the graph may be predefined or learned with self-attention.
  • Server-Side GNNs Training: Bi-level methods jointly train local models and the server GNN using separate local-task and GNN-training objectives.Big-Fed uses an unsupervised contrastive loss, whereas SFL uses a supervised loss with graph-smoothness regularization.
  • Server-Side GNNs Training: Sequential methods train local models first and then train a server GNN to fuse or refine local estimates using client similarities.The server-side objective can include data reconstruction error and graph regularization.

2) Client-Side GNNs Training:

Client-side GNNs train graph models within clients while retaining the general FL procedure, addressing data or model heterogeneity. In decentralized settings, clients aggregate through neighbors using weighted updates or graph regularization.

  • Client-Side GNNs Training: Client-side GNN training addresses data-distribution and model heterogeneity within the general FL workflow.Clients train GNNs locally, upload model weights, receive aggregated weights, and continue training in subsequent rounds.
  • Data Distribution Heterogeneity: For data heterogeneity, clients use a shared inter-client graph to obtain global knowledge and combine global and local embeddings.FedCG combines the local model embedding and global embedding with a trainable weight.
  • Model Heterogeneity: HAFL-GHN represents each client’s neural architecture as a graph, with layers as vertices and computational flows as edges.A GNN-based graph hypernetwork processes this architecture representation to address heterogeneous local models.
  • Decentralized FedGNNs: Decentralized FedGNNs lack a central server, making decentralized model aggregation a key challenge.Clients communicate with neighbors directly, and existing approaches aggregate through weighted neighborhood summation or graph regularization.
  • Weighted Summation of FL Model Parameters: Weighted-summation methods update each client’s model by aggregating neighboring clients’ parameters according to the graph topology.Clients generally aggregate models from their 1-hop neighborhoods once per round, with encryption used in some formulations.
  • Decentralized FL Aggregation: Decentralized aggregation variants include stochastic gradient tracking and multi-server architectures to improve convergence or communication efficiency.These designs target faster convergence and large-scale graphs, respectively.
  • Graph Regularization on FL Model Parameters: Graph Laplacian regularization encourages neighboring clients’ model parameters to be similar while addressing non-IID data.The regularization uses neighboring-client weights and a balance parameter in each client’s local objective.

C. Summary

The survey summarizes GNN-assisted FL methods by their advantages and disadvantages, including centralized and decentralized settings. Centralized methods offer flexible aggregation for non-IID clients, while decentralized methods enable serverless personalized learning but introduce privacy and communication concerns.

  • Centralized FedGNNs: Centralized FedGNNs use a server-side GNN or locally stored inter-client graph to improve aggregation and address client non-IIDness.Server-side training offers flexible aggregation, while local graph storage can also relieve non-IID effects.
  • Centralized FedGNNs: Centralized approaches face convergence, scalability, graph-accuracy, and potential privacy challenges.Two objective functions complicate convergence analysis, large inter-client graphs increase server training cost, and imprecise or locally stored graphs create risks.
  • Decentralized FedGNNs: Decentralized FedGNNs address client non-IIDness without a central server by direct neighbor communication, producing personalized local models.Different client neighborhoods lead to different aggregated models.
  • Decentralized FedGNNs: Decentralized methods incur privacy leakage, communication, security, and continual-learning limitations.Direct model-weight sharing can leak information, highly central clients may be vulnerable to attacks, and models require retraining when new neighbors join.

IV. FL-ASSISTED GRAPH NEURAL NETWORKS

FL-assisted GNNs use federated learning to train GNNs across isolated graph-data silos while addressing non-IID distributions and additional challenges such as temporal dynamics, scale, architecture selection, privacy, and aggregation.

  • FL algorithms assist GNN training when graph data are isolated across clients and clients need privacy-preserving global model training.
  • Non-IID Problem in Graph Data: Non-IID graph data arise from heterogeneous graph structures or node-feature distributions across clients, motivating model-based and data-based personalization approaches.
  • Distributed Spatial-Temporal Graph Data Embedding: Spatial-temporal methods differ in whether they ignore temporal relationships, use dynamic GNNs, or learn evolving graphs with missing time points.
  • Distributed Large-Scale Graph Data Embedding: FedGraph reduces large-scale training costs by using server-side reinforcement learning to refine client sampling strategies and GNN parameters.
  • Distributed Neural Architecture Search: HAFL-GHN searches for suitable GCN architectures through federated evolutionary optimization, a GCN SuperNet, and weight sharing.
  • FL Aggregation Improvement: Aggregation improvements include round-aware or trainable attention weights, alternating FedAvg with model exchange, client clustering, and encryption-based privacy protection.

2) With Missing Edges:

With missing cross-client edges, FedGNNs amend local graphs or exchange protected representations to improve distributed graph learning while addressing privacy, non-IID data, and communication costs.

  • Missing cross-client edges may connect different node IDs or aligned nodes, and reconstructing them can improve graph representations and mitigate non-IID data.
  • Local Graph Amendment: Local graph amendment uses server-, client-, or third-party edge and node generators before client-side GNN training and server-side FL aggregation.
  • Local Graph Amendment: FASTGNN broadcasts Gaussian-randomly generated edges, while FedGL creates a global pseudo graph from uploaded node embeddings for local graph repair.
  • Local Graph Amendment: FedSage+ predicts missing neighborhood nodes from held-out local edges before reconstructing missing edges with a client-side feature generator.
  • Local Graph Amendment: Privacy-preserving alternatives use differential privacy, private set intersection, encryption, secure aggregation, or a third-party server for graph expansion.
  • Local Graph Amendment: Communication-focused methods exchange sampled neighbor embeddings or averaged one- and two-hop features to balance convergence, runtime, and graph repair.
  • Knowledge Graph Completion: Knowledge graph completion shares or translates aligned embeddings across clients, with relation-only uploads, secure aggregation, and regularization addressing privacy and non-IID concerns.

B. Vertical FedGNNs

Vertical FedGNNs address clients with completely overlapping node IDs but different feature spaces, using federated learning to combine their graph information for global GNN training.

  • Vertical FedGNNs assume clients share node IDs but hold different feature spaces, with graph data potentially distributed across clients.

1) Clients with Incomplete Graph Data:

Incomplete-graph vertical FedGNNs coordinate clients that separately own node features, graph topology, or labels, using privacy-preserving information exchange to train global GNNs.

  • Clients with Incomplete Graph Data: Clients may separately own node features, graph topology, and node labels, making privacy-preserving collaboration a central challenge.
  • Clients with Incomplete Graph Data: SGNN conceals graph structure with a DTW-based similarity matrix and protects node features through one-hot encoding before global node-classification training.
  • Clients with Incomplete Graph Data: FedSGC supports two clients without a central server by using additively homomorphic encryption for sensitive information and GNN parameter updates.
  • Summary: Table II summarizes the advantages and disadvantages of different FL-assisted GNN scenarios.
  • Clients with Incomplete Graph Data: Complete-graph vertical settings concatenate different node-feature types, while FedVGCN and VFGNN exchange or integrate protected intermediate representations.
  • Summary: Existing incomplete-graph approaches generally support only two or three clients, remain vulnerable to malicious attacks, and cover limited basic GNN models.

V. THE AUXILIARY TAXONOMY

The auxiliary taxonomy organizes FedGNNs by increasing heterogeneity across clients, from shared nodes to different network structures. The section also situates these methods within applications, benchmarks, datasets, and evaluation design.

  • Auxiliary taxonomy: FedGNNs are divided into clients with the same nodes, different nodes but the same network structure, and different network structures.These categories represent increasing levels of heterogeneity across local data and models.
  • Same nodes: Clients sharing nodes aggregate feature embeddings for the same nodes, while model weights may also be exchanged.This setting includes vertical FedGNNs and some horizontal FedGNNs with overlapping nodes.
  • Different nodes, same structure: When clients have different nodes but the same architecture, node embeddings cannot be aggregated, so aggregation relies on parameters whose size is independent of graph topology.This category includes most GNN-Assisted FL and horizontal FedGNNs works.
  • Different structures: Different client architectures create the most difficult setting because clients also differ in node-feature dimensionality.The cited work converts each local architecture into a graph, representing layers as vertices and layer types as node features.
  • Applications and benchmarking: FedGNNs research uses citation, social-network, and chemical-property datasets, with applications including knowledge-graph completion, recommendation, computer vision, and healthcare.The section emphasizes benchmarking and discusses applications involving graph-structured information or relationships among clients.

B. Evaluation Metrics

FedGNN evaluation uses task-specific metrics and experimental designs spanning centralized, horizontal, vertical, and FL-Assisted GNN settings. Because real cross-silo graph datasets are scarce, most studies partition public datasets to simulate federation.

  • Evaluation Metrics: Classification tasks use accuracy, F1-score, and ROC-AUC, while regression tasks use metrics suited to predicted-value errors or graph differences.The cited examples include node, graph, and image classification, as well as embedding regression and graph learning.
  • Evaluation Metrics: Embedding regression tasks use MAE, MSE, RMSE, and MAPE to measure distance between predictions and ground truth.These metrics are reported for applications such as recommendation systems and traffic-flow prediction.
  • Evaluation Metrics: Graph learning can additionally be evaluated by Frobenius distance between estimated graphs and ground-truth graphs.Brain connectivity estimation is given as an example.
  • Evaluation Metrics: Link prediction uses Mean Rank, MRR, and Hits@N to evaluate knowledge-graph completion.These metrics assess ranking quality and the proportion of correct entities among the top N ranked entities.
  • Experimental design: Most FedGNN studies simulate distributed settings by partitioning public datasets because real cross-silo graph datasets are scarce.The partitioning discussion covers IID and non-IID distributions, inter-client graphs, horizontal settings, and vertical settings.
  • Experimental design: Non-IID client construction includes imbalanced, clustered, and label-distribution-skew partitions, while inter-client graphs may be natural or similarity-based.These choices are used across FL-Assisted and GNN-Assisted settings.

D. FedGNNs Platforms

FedGNN platforms provide reusable infrastructure for models, datasets, partitioning, and aggregation. The paper presents these tools alongside unresolved challenges involving security, scale, explainability, and realistic evaluation.

  • D. FedGNNs Platforms: FedGraphNN supports 3 GNN models and 2 FL aggregation methods and includes 36 graph datasets partitioned into distributed silos.It is presented as an open-source benchmarking platform for FedGNNs.
  • D. FedGNNs Platforms: FederatedScope-GNN is an event-driven framework with ModelZoo and DataZoo components for reusable GNN models and data resources.Examples of supported models include GCN, GAT, and GraphSage.
  • Open challenges: Existing FedGNNs are generally studied with small-scale distributed datasets, leaving communication efficiency insufficiently addressed for large-scale graphs.The paper identifies efficient FedGNNs as a future research direction.
  • Open challenges: FedGNNs face unresolved challenges in defending against malicious attacks, explaining complex training processes, supporting fairness, and enabling continual training for new clients.The discussion also calls for comprehensive frameworks and realistic cross-silo graph datasets.
  • Open challenges: Existing FedGNNs use only a limited set of GNN models, motivating frameworks that support more algorithms and strategies.The paper names GCN, GAT, and GraphSage as examples of the currently used models.
Loading 2202.07256v2…