PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Social Recommendation with Graph Neural Network

Zhiwei Liu, Liangwei Yang, Ziwei Fan, Hao Peng, Philip S. Yu

arXiv:2111.10778v1cs.IRcs.LG

TL;DR

Centralized social and interaction data creates privacy concerns, while federated social recommendation must handle heterogeneous relations and personalized client data. FeSoG uses local relational GNNs and pseudo-labeling with item sampling, and it improves recommendation performance across three datasets while protecting privacy. The paper identifies communication cost and random pseudo-item sampling as future limitations.

  • Problem

    Existing social recommendation methods centralize social links and user-item interactions, creating privacy concerns and motivating federated social recommendation.

  • Method

    FeSoG uses local relational attention and aggregation, on-device personalized embedding inference, and pseudo-labeling with item sampling in federated training.

  • Results

    FeSoG outperforms state-of-the-art federated recommender systems across all datasets, with average relative improvements of 2.99% in RMSE and 4.03% in MAE over FedGNN.

  • Takeaways & Limitations

    FeSoG provides a federated framework that jointly addresses relation heterogeneity, personalization, and privacy protection in social recommendation.

  • Takeaways & Limitations

    Training requires numerous communication rounds, motivating efforts to reduce communication time or increase bandwidth for large-scale clients.

Abstract

from arXiv · show

Recommender systems have become prosperous nowadays, designed to predict users' potential interests in items by learning embeddings. Recent developments of the Graph Neural Networks~(GNNs) also provide recommender systems with powerful backbones to learn embeddings from a user-item graph. However, only leveraging the user-item interactions suffers from the cold-start issue due to the difficulty in data collection. Hence, current endeavors propose fusing social information with user-item interactions to alleviate it, which is the social recommendation problem. Existing work employs GNNs to aggregate both social links and user-item interactions simultaneously. However, they all require centralized storage of the social links and item interactions of users, which leads to privacy concerns. Additionally, according to strict privacy protection under General Data Protection Regulation, centralized data storage may not be feasible in the future, urging a decentralized framework of social recommendation. To this end, we devise a novel framework \textbf{Fe}drated \textbf{So}cial recommendation with \textbf{G}raph neural network (FeSoG). Firstly, FeSoG adopts relational attention and aggregation to handle heterogeneity. Secondly, FeSoG infers user embeddings using local data to retain personalization. Last but not least, the proposed model employs pseudo-labeling techniques with item sampling to protect the privacy and enhance training. Extensive experiments on three real-world datasets justify the effectiveness of FeSoG in completing social recommendation and privacy protection. We are the first work proposing a federated learning framework for social recommendation to the best of our knowledge.

1 INTRODUCTION

Social recommendation combines social links with user-item interactions to address cold-start limitations, but existing approaches centralize privacy-sensitive data. FeSoG introduces a federated framework addressing heterogeneity, personalization, and privacy protection.

  • Social links can supplement sparse user-item histories because socially connected users are assumed to share similar item interests.
  • Existing social recommendation methods require centralized storage of social networks and users’ item transaction histories, creating privacy risks under GDPR constraints.
  • Federated social recommendation must jointly handle heterogeneous user-user and user-item relations, personalized non-IID client data, and privacy protection.
  • FeSoG uses relational attention, local graph neural networks, personalized on-device training, and pseudo-labeling with item sampling.
  • FeSoG achieves up to 5.26% improvement in RMSE and 5.46% improvement in MAE across three datasets compared with other baselines.
  • The paper reports FeSoG as the first federated learning framework for social recommendation and evaluates it on three publicly available datasets with ablation studies.

2 RELATED WORK

Related work covers GNN-based recommendation, social recommendation, and federated recommender systems. Prior methods establish graph-based embedding learning and privacy-preserving recommendation, while motivating federated social recommendation.

  • 2.1 Graph Neural Network for Recommendation: GNN recommender systems aggregate graph neighbors to learn user or item embeddings for rating, collaborative filtering, and basket recommendation.
  • 2.2 Social Recommendation: Social recommendation uses social links to relieve data sparsity and cold-start problems, through matrix factorization or graph neural network approaches.
  • 2.3 Federated Learning for Recommender System: Federated learning distributes data across clients rather than storing it on a server, supporting privacy-preserving machine learning.
  • 2.3 Federated Learning for Recommender System: Federated recommender systems train user embeddings locally while servers retain item embeddings, avoiding transfer of users’ interactions.
  • 2.3 Federated Learning for Recommender System: Model updates can still contain enough information to uncover raw data, motivating differential privacy for federated recommender systems.

3 PRELIMINARY

The paper formulates federated social recommendation as predicting unobserved ratings from decentralized local rating and social data. Each client stores a local graph, and collaborative training must address heterogeneous graph structure without raw-data access.

  • 3.1 Definitions: Social recommendation predicts users’ ratings for items from both social interactions and user-item interactions.
  • 3.1 Definitions: A client is a local device storing one user’s rating data and social data, while the server coordinates training without exchanging raw client data.
  • 3.1 Definitions: An FSRS predicts unobserved client ratings from partially observed local rating and social data without accessing raw data on clients.
  • 3.2 Formulation: Each client’s local graph contains first-order user and item neighbors, with rated-item edges carrying ratings and user-user edges representing social interactions.
  • 3.2 Formulation: The formal problem is to collaboratively predict an unobserved edge’s attribute value without accessing raw data from any local graph.
  • 3.2 Formulation: The task is treated as link prediction requiring graph embeddings that preserve structural information while handling heterogeneous local graphs.

4 PROPOSED FRAMEWORK

FeSoG comprises an embeddings layer, local graph neural networks, and a gradient protector. Its local GNN component learns embeddings and predictions through heterogeneous attention, relational aggregation, and rating prediction layers.

  • The FeSoG framework contains three modules: an embeddings layer, local graph neural networks, and a gradient protector.

4.1 Embeddings

FeSoG maintains server-side user and item embedding tables, while clients use local interaction records to infer embeddings and update them through aggregated gradients.

  • Server-maintained user and item embedding tables represent the graph nodes, with embedding dimension d.Clients request these tables before local processing.
  • Local embeddings and a local GNN model are learned from each client’s interaction data to infer user embeddings and predict item scores.
  • Client-derived embedding updates are uploaded as gradients and aggregated by the server.
  • Each client uses user and item IDs from local interaction records to obtain the corresponding embeddings.

4.2 Local Graph Neural Network

FeSoG’s local GNN combines heterogeneous social and item relations through attention and relational aggregation, then predicts ratings and computes local losses for federated updates.

  • Local GNN architecture: The local GNN comprises heterogeneous graph attention, relational graph aggregation, and rating prediction layers.
  • Graph attention: Attention weights are computed from mapped and concatenated embeddings through an attention layer, nonlinear activation, and softmax normalization.
  • Relational aggregation: Separate attention mechanisms learn weights for user-user and user-item neighbors before relational aggregation.User and item neighbors use distinct mapping matrices and attention parameters.
  • Relational aggregation: Three relation-aware weights combine the center user embedding with aggregated user-neighbor and item-neighbor embeddings.The weights correspond to hidden user neighbors, hidden item neighbors, and the center node itself.
  • Prediction: The resulting local user embedding supports prediction while preserving client-specific user representations.
  • Prediction: Predicted ratings use the dot product between inferred user and item embeddings, and local ratings optimize the prediction with RMSE.
  • Federated training: Each client samples pseudo-items alongside local data before calculating loss and gradients, with gradients uploaded after LDP operation.
  • Federated training: Local losses generate gradients for server aggregation, but direct gradient uploads can expose user-item interaction data.

4.3 Privacy Protection

FeSoG protects local interaction data during gradient upload with dynamic Local Differential Privacy and pseudo-item labelling. Pseudo-items also add rating information that can reduce leakage and improve robustness.

  • Privacy Protection: FeSoG combines dynamic Local Differential Privacy with pseudo-item labelling to protect local user-item interaction data during gradient uploads.The two techniques are presented as the privacy-protection module.
  • Privacy Protection: Directly uploading optimized gradients can expose users’ rating information, while non-rated items otherwise produce zero embedding gradients.Prior work motivates encryption and pseudo-interacted item sampling as alternatives for reducing these risks.
  • Local Differential Privacy: Dynamic noise clips local gradients and scales Laplacian noise using the gradient magnitude rather than applying one constant noise strength.The paper argues that parameter gradients have different magnitudes during training, making constant noise inappropriate.
  • Pseudo-Item Labelling: The client samples items outside a user’s neighbor items, predicts their ratings locally, and rounds those predictions into pseudo-ratings.The sampled pseudo-items are denoted as ˜T(u), and their pseudo-ratings enter the modified loss.
  • Pseudo-Item Labelling: The modified loss uses true and pseudo items, combining ground-truth and pseudo-item information while adding randomness that improves local-model robustness.The difference between predicted and rounded pseudo-ratings contributes to the gradients.

4.4 Optimization

FeSoG trains local graph neural networks on client data and uses a server to aggregate privatized client gradients. Repeated communication rounds update shared parameters and embeddings while retaining locally inferred embeddings for prediction.

  • Optimization: FeSoG’s optimization alternates server communication with client-side updates until convergence.The server sends parameters, collects client gradients, aggregates them, and applies gradient descent across multiple rounds.
  • Server Update: The server aggregates client gradients as weighted averages based on each client’s real and pseudo interactions.The aggregated model, item, and user gradients are then used for parameter updates.
  • Server Update: The trainable parameter set includes model parameters, item embeddings, and user embeddings, which are jointly updated by gradient descent.The learning rate η controls the update, and the process repeats over communication rounds.
  • Algorithm: Algorithm 1 takes training hyperparameters and clients’ local graphs as input, and outputs shared parameters plus local client embeddings.The algorithm returns privatized gradients and interaction counts from each client during training.
  • Client Update: Each client downloads current parameters, infers local user embeddings, samples pseudo-items, and computes privatized gradients.ClientUpdate performs local inference and pseudo-item processing before returning gradients and interaction counts.

5 EXPERIMENTS

The experiments evaluate FeSoG on three social recommendation datasets against centralized and federated baselines using MAE and RMSE. FeSoG is reported to outperform federated recommender systems across all datasets, with additional analyses of hyperparameters and components.

  • Research Questions: Experiments address whether FeSoG outperforms existing methods, how hyperparameters affect it, and whether its components are necessary.These questions define RQ1, RQ2, and RQ3.
  • Datasets: The evaluation uses Ciao, Epinions, and Filmtrust, which contain item ratings and trust links between users.Ciao and Epinions use ratings from 1 to 5, while Filmtrust uses a 1-to-8 scale.
  • Baselines: The baselines include matrix-factorization social recommenders, GNN-based social recommenders, and federated learning frameworks.Centralized baselines do not protect user privacy, while federated baselines do not fuse local social and rating information.
  • Evaluation: MAE and RMSE measure prediction performance, with smaller values indicating better test performance; evaluation is conducted locally on devices.The server does not access the local privacy-sensitive data during evaluation.
  • Overall Comparison: 2.99% and 4.03% relative improvements are reported for FeSoG over FedGNN on average in RMSE and MAE, respectively, across the datasets.The paper also reports that FeSoG significantly outperforms SOTA federated recommender systems in all datasets.
  • Analysis: The experiments vary user batch size, embedding size, learning rate, pseudo-item count, and Local Differential Privacy parameters.These analyses correspond to the hyperparameter-impact study and complement the component ablation analysis.

embedding size

FeSoG is evaluated across batch size, pseudo-item count, embedding size, learning rate, privacy parameters, and component ablations. Results show stronger and more robust performance than federated baselines, while computational cost, noise, overfitting, and privacy create practical trade-offs.

  • user batch size: Larger user batches improve FeSoG’s performance across datasets by providing more accurate global information estimates.The trade-off is higher computational cost and longer convergence time when aggregating more users per step.
  • number of pseudo items: FeSoG remains better than FedGNN as pseudo-item counts vary, although additional pseudo items can increase error through added noise.For Ciao, performance does not deteriorate much below 100 pseudo items, indicating a sampling-performance trade-off.
  • embedding size: FeSoG consistently outperforms FedGNN across all embedding sizes and datasets.FedGNN also outperforms FedMF, supporting GNN-based graph embedding for federated recommendation.
  • embedding size: Suitable embedding sizes are dataset-dependent, with the best performance generally at d = 16 or d = 32.Small sizes such as d = 4 limit representation ability, whereas large sizes such as d = 64 may overfit limited data.
  • learning rate: FeSoG converges smoothly, but the preferred learning rate differs by dataset: η = 0.1 for Epinions and η = 0.05 for Ciao and Filmtrust.Learning-rate choices affect convergence speed and therefore communication requirements.
  • privacy parameters: Increasing δ improves performance at fixed λ, whereas increasing λ worsens performance because stronger Laplace noise overwhelms data-derived gradients.The privacy module therefore requires balancing gradient clipping and noise strength.
  • ablation study: Ablations show that separate GAT layers and relational vectors improve performance, while removing pseudo items eliminates protection against inferring true interacted items.The sharing-GAT variant is worse than FeSoG, with the largest reported relative differences on Ciao.

6 CONCLUSION AND FUTURE WORK

FeSoG is a federated framework for social recommendation that combines local graph modeling, personalization, relational aggregation, pseudo-item labeling, and dynamic LDP. Experiments report improved performance over federated baselines, while future work targets adaptive sampling, richer decentralized graphs, and communication efficiency.

  • Conclusion: FeSoG decentralizes storage and collaboratively trains social recommendation models from clients’ local privacy data.Its components address heterogeneity, personalization, and communication privacy through relational aggregation, local embeddings, pseudo-item labeling, and dynamic LDP.
  • Conclusion: Experiments report that FeSoG significantly outperforms state-of-the-art federated learning frameworks for social recommendation.The paper also reports that ablation studies support the necessity of its designed components.
  • Future Work: Future work includes adaptive pseudo-item sampling, higher-order local graphs through privacy-preserving peer-to-peer communication, and more efficient communication.The motivation includes reducing sampling noise, extending graph information, and addressing numerous communication rounds.
Loading 2111.10778v1…