Source-linked AI summary
FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation
Chuhan Wu, Fangzhao Wu, Yang Cao, Yongfeng Huang, Xing Xie
TL;DR
Existing GNN recommenders rely on centralized user-item graphs, although interaction data is privacy-sensitive and local-only training is limited by sparse user data. FedGNN jointly trains GNN models from decentralized clients using gradient aggregation, local differential privacy, pseudo interacted items, and privacy-preserving graph expansion. On six benchmark datasets, it achieves competitive performance with centralized methods while protecting user privacy.
Problem
Existing GNN-based recommendation requires centralized user-item graphs, while decentralized local data is sparse and high-order interactions are difficult to exploit without privacy leakage.
Method
FedGNN locally trains GNNs on device-level graphs, aggregates client gradients, applies local differential privacy and pseudo interacted items, and expands graphs through privacy-preserving embedding exchange.
Results
FedGNN achieves competitive performance with centralized methods on six benchmark datasets while effectively protecting user privacy.
Takeaways & Limitations
Federated GNN training can exploit high-order user-item interactions from decentralized data while keeping recommendation performance competitive and protecting user privacy.
Takeaways & Limitations
A privacy budget that is too small can reduce model-gradient accuracy, so privacy and performance hyperparameters must be balanced.
Abstract
from arXiv · showhide
Graph neural network (GNN) is widely used for recommendation to model high-order interactions between users and items. Existing GNN-based recommendation methods rely on centralized storage of user-item graphs and centralized model learning. However, user data is privacy-sensitive, and the centralized storage of user-item graphs may arouse privacy concerns and risk. In this paper, we propose a federated framework for privacy-preserving GNN-based recommendation, which can collectively train GNN models from decentralized user data and meanwhile exploit high-order user-item interaction information with privacy well protected. In our method, we locally train GNN model in each user client based on the user-item graph inferred from the local user-item interaction data. Each client uploads the local gradients of GNN to a server for aggregation, which are further sent to user clients for updating local GNN models. Since local gradients may contain private information, we apply local differential privacy techniques to the local gradients to protect user privacy. In addition, in order to protect the items that users have interactions with, we propose to incorporate randomly sampled items as pseudo interacted items for anonymity. To incorporate high-order user-item interactions, we propose a user-item graph expansion method that can find neighboring users with co-interacted items and exchange their embeddings for expanding the local user-item graphs in a privacy-preserving way. Extensive experiments on six benchmark datasets validate that our approach can achieve competitive results with existing centralized GNN-based recommendation methods and meanwhile effectively protect user privacy.
1 INTRODUCTION
FedGNN addresses the difficulty of learning accurate GNN recommenders from decentralized, privacy-sensitive interaction data while preserving high-order user-item information. It combines federated training with privacy mechanisms and graph expansion, achieving competitive results on six benchmark datasets.
- Motivation: Centralized GNN recommenders model high-order user-item interactions but require centralized storage of the entire user-item graph.This centralization creates privacy concerns for user-item interaction data.
- Motivation: Local-only training is insufficient because most users have too little interaction data to learn accurate GNN models.A coordinated framework is therefore needed to learn a unified global model from decentralized clients.
- Motivation: Decentralized data also makes privacy-preserving high-order interaction modeling difficult because local models and interaction items may reveal private information.Users’ local data directly contains first-order interactions, while interaction items cannot be freely exchanged.
- FedGNN Framework: FedGNN locally trains GNN models on user-item graphs inferred from device data, aggregates client gradients centrally, and distributes updates for local training.The framework collectively learns from decentralized user data without requiring a global user-item graph.
- Privacy and Graph Expansion: FedGNN protects gradients with local differential privacy, hides interacted items through pseudo-item sampling, and expands local graphs to exploit high-order interactions.These mechanisms jointly target gradient privacy, item anonymity, and decentralized graph connectivity.
- Evaluation: Six benchmark datasets show that FedGNN achieves competitive results with centralized GNN-based recommendation methods while protecting user privacy.The reported evaluation covers both recommendation performance and privacy protection.
2 RELATED WORK
Related work covers GNN-based recommendation and federated learning. GNN methods model high-order relations, whereas federated learning keeps data local and aggregates model updates, reducing privacy-leakage risk.
- GNN-based Recommendation: GNN-based recommendation methods model high-order user-item relations using graph representations and graph neural architectures.Examples include graph convolutional matrix completion and PinSage.
- Federated Learning: Federated learning collectively trains models from decentralized data while keeping user data on local devices.Devices compute local model updates, which a central server aggregates and redistributes iteratively.
- Comparison: Table 1 compares methods by their ability to model high-order user-item interactions and protect privacy under centralized or decentralized storage.“Cen.” denotes centralized storage, while “Local” denotes decentralized storage.
- Federated Learning: Because raw data stays on devices and model updates generally contain less private information, federated learning can reduce privacy-leakage risk.This privacy property motivates applying federated learning to personalized recommendation.
3 METHODOLOGY
FedGNN coordinates decentralized clients to train GNN recommendation models while protecting interaction data, gradients, ratings, and interacted-item identities. Its local subgraphs are expanded with privacy-preserving neighboring-user information to incorporate high-order interactions.
- FedGNN Framework: FedGNN uses a central server and user clients that locally construct subgraphs, learn embeddings and GNN models, and exchange aggregated gradients for parameter updates.The server aggregates local gradients into g and sends the unified gradient back to clients for iterative updates.
- FedGNN Framework: Each local subgraph connects a user to interacted items and neighboring users identified through co-interacted items.Neighbor embeddings are initially excluded for T epochs, then incorporated while neighboring-user embeddings remain fixed.
- FedGNN Framework: A GNN produces hidden user and item representations, whose predicted ratings are compared with locally stored ratings to compute client-side losses and gradients.The framework can use GCN, GGNN, or GAT models, and uploads model and embedding gradients for aggregation.
- Privacy-Preserving Model Update: Direct gradient uploads can reveal interacted items through non-zero embedding gradients and expose user preferences through GNN and rating-predictor gradients.These leakage risks motivate privacy-preserving model updates rather than transmitting raw interaction histories.
- Privacy-Preserving Model Update: FedGNN clips local gradients and adds Laplacian noise before uploading them, while pseudo-interacted items receive statistically matched random gradients to hide real interactions.The number of pseudo-interacted items must exceed the number of real interacted items, and the server aggregates only protected gradients and corresponding embeddings.
- Privacy-Preserving User-Item Graph Expansion: Privacy-preserving graph expansion finds anonymous neighboring users and extends local user-item graphs so decentralized clients can exploit high-order interactions.The approach addresses the difficulty of deriving high-order relations when the global user-item graph is unavailable.
4 EXPERIMENTS
Experiments evaluate FedGNN on six benchmark datasets against centralized and privacy-preserving recommendation methods, then analyze architecture, privacy, and pseudo-item hyperparameters. FedGNN achieves competitive recommendation performance while examining trade-offs among high-order information, privacy protection, and communication cost.
- Dataset and Experimental Settings: Experiments use six benchmark recommendation datasets: ML-100K, ML-1M, ML-10M, Flixster, Douban, and YahooMusic.The MovieLens variants are drawn from the 100K, 1M, and 10M datasets.
- Performance Evaluation: The comparison includes centralized recommendation methods, federated privacy-preserving methods, and PMF as a first-order baseline.Table 3 reports performance in terms of RMSE.
- Performance Evaluation: High-order-information methods outperform first-order PMF, while FedGNN achieves comparable or better performance than centralized GC-MC and NGCF and the best performance among privacy-preserving methods.The reported comparison attributes the advantage of high-order methods to enhanced user and item representation learning.
- Model Effectiveness: FedGNN is compatible with GGNN, GCN, and GAT, with GAT slightly outperforming its GCN- and GGNN-based variants.Variants using high-order information perform better than those without it.
- Hyperparameter Analysis: A clipping threshold of 0.1 offers better privacy protection than 0.2 without much performance sacrifice, whereas a threshold of 0.05 substantially increases prediction error.Model performance declines as Laplacian noise strength increases; λ=0.2 is described as a moderate balance.
- Hyperparameter Analysis: Increasing pseudo interacted items can mitigate random-gradient effects after aggregation, but communication cost grows proportionally with M; the paper selects M=1,000.M=0 gives the best performance but does not protect interaction histories, while excessively large M makes communication heavy.
5 CONCLUSION
FedGNN combines federated GNN training with privacy protections and high-order interaction modeling for decentralized recommendation. Experiments show competitive performance against centralized methods while protecting user privacy.
- FedGNN locally trains GNN models on user-item graphs inferred from decentralized interaction data, then aggregates client gradients through a server.The aggregated gradients are returned to clients for local model updates.
- Local differential privacy protects gradients, while pseudo interacted item sampling protects embeddings of items users interacted with.
- Privacy-preserving graph expansion finds neighboring users with co-interacted items and exchanges embeddings to incorporate high-order interactions.
- ML-100K, 1M, and 10M have rating densities of 0.0630, 0.0447, and 0.0134, respectively.
- Six benchmark datasets show competitive performance with methods based on centralized user-item interaction storage while effectively protecting user privacy.