PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Federated Recommendation Systems

Zehua Sun, Yonghui Xu, Yong Liu, Wei He, Lanju Kong, Fangzhao Wu, Yali Jiang, Lizhen Cui

arXiv:2301.00767v2cs.IRcs.AIcs.CRcs.LG

TL;DR

Federated recommendation systems face challenges across privacy, security, heterogeneity, and communication costs, while surveying literature has gaps. This survey reviews existing approaches and concludes that FedRS remains a promising field requiring further development.

  • Problem

    Earlier surveys leave gaps in comprehensive coverage of federated recommendation systems and their challenges.

    Source: C. Our Contribution · p. 2, §IX · p. 12

  • Method

    The survey reviews FedRS definitions, communication architectures, privacy, security, heterogeneity, communication costs, applications, datasets, and future directions.

    Source: C. Our Contribution · p. 2

  • Results

    The survey compares existing designs and solutions and identifies prospective research directions for federated recommendation systems.

    Source: §IX · p. 12

  • Takeaways & Limitations

    FedRS is a promising field with potential opportunities that requires more effort to develop.

    Source: §IX · p. 12

Abstract

from arXiv · show

Federated learning has recently been applied to recommendation systems to protect user privacy. In federated learning settings, recommendation systems can train recommendation models only collecting the intermediate parameters instead of the real user data, which greatly enhances the user privacy. Beside, federated recommendation systems enable to collaborate with other data platforms to improve recommended model performance while meeting the regulation and privacy constraints. However, federated recommendation systems faces many new challenges such as privacy, security, heterogeneity and communication costs. While significant research has been conducted in these areas, gaps in the surveying literature still exist. In this survey, we-(1) summarize some common privacy mechanisms used in federated recommendation systems and discuss the advantages and limitations of each mechanism; (2) review some robust aggregation strategies and several novel attacks against security; (3) summarize some approaches to address heterogeneity and communication costs problems; (4)introduce some open source platforms that can be used to build federated recommendation systems; (5) present some prospective research directions in the future. This survey can guide researchers and practitioners understand the research progress in these areas.

I. INTRODUCTION

The introduction presents federated recommendation as collaborative model training that exchanges intermediate parameters rather than users’ real data. It outlines privacy, security, heterogeneity, and communication-cost challenges that remain.

I. INTRODUCTION

  • Centralized collection creates privacy risks, while privacy concerns and regulation can also restrict cross-platform data integration and leave recommendation systems with insufficient data.The introduction cites GDPR as an example of restrictions on collecting and sharing user data.Source: §I · p. 1
  • Federated recommendation systems (FedRS) train collaboratively by exchanging intermediate parameters, such as model parameters or gradients, instead of users’ real data.The paper defines FedRS as applying federated learning to recommendation systems.Source: §I · p. 1

A. Challenges

  • FedRS still faces privacy leakage from uploaded parameters and auxiliary information, as well as participant attacks that can promote selected products or harm competitors.The paper notes that poisoning defenses are needed to protect recommendation fairness and performance.Source: A. Challenges · p. 1
  • System, statistical, and privacy heterogeneity can create stragglers, limit personalization from a unified model, and make uniform privacy budgets inefficient.Client devices vary in resources, client data is often Non-IID, and users have different privacy needs.Source: A. Challenges · pp. 1–2
  • Repeated model exchanges and large intermediate parameters, along with item data needed for local recommendations, create communication costs that constrain large-scale FedRS.The paper notes that real-world systems may involve millions of parameters to communicate.Source: A. Challenges · p. 2

B. Related Surveys

The paper addresses a gap in surveys that treat recommendation systems and federated learning separately, and it organizes FedRS through definitions, architectures, and contributions across key challenges. It contrasts centralized client-server coordination with peer-to-peer exchange.

B. Related Surveys

  • Earlier surveys generally cover recommendation systems or federated learning separately; few focus on FedRS, and the prior FedRS survey did not comprehensively address its main challenges.Those challenges include privacy, security, heterogeneity, and communication costs.Source: B. Related Surveys · p. 2

C. Our Contribution

  • The survey contributes an overview of FedRS definitions, communication architectures, and categories, a synthesis of research areas, public datasets and applications, and prospective directions.The research areas include privacy, security, heterogeneity, and communication costs.Source: C. Our Contribution · p. 2

A. Definition

  • FedRS is defined as privacy-preserving recommendation in which participants collaboratively train a model by exchanging intermediate parameters rather than real data.The ideal performance is close to that of centralized training, within a small positive difference δ.Source: A. Definition · p. 2

B. Communication Architecture

  • Client-server training uses a trusted central server to initialize and aggregate models, while peer-to-peer training has participants broadcast updates to neighbors and aggregate locally.The central server creates a single point of failure and privacy concerns; peer-to-peer avoids those server-associated issues but increases client communication and computation overhead.Source: B. Communication Architecture · p. 3

C. Categorization

FedRS is categorized by participant type and recommendation-model family. The two participant settings and three model families differ in their data, training, and system constraints.

1) Participant Type:

  • Cross-device FedRS uses many mobile devices with small local datasets for personal recommendations, but device limits and dropouts constrain complex training and create efficiency and straggler challenges.The paper identifies power and network status as reasons devices may leave training.Source: 1) Participant Type · p. 3
  • Cross-platform FedRS lets a relatively small number of data platforms collaborate without directly sharing user data, but raises fair-incentive and trusted-server challenges.A peer-to-peer architecture may be a suitable choice when a trusted server is difficult to find.Source: 1) Participant Type · p. 3

2) Recommendation Model:

  • Matrix-factorization FedRS keeps user factors on clients and uploads item factors or their gradients for aggregation; it captures tastes from interactions but remains limited by sparsity and cold start.Matrix factorization is described as a common model in FedRS.Source: 2) Recommendation Model · pp. 3–4
  • Deep-learning FedRS includes frameworks that aggregate masked or perturbed model parameters, but large models impose substantial client computation and communication overhead.Examples include federated neural collaborative filtering, graph neural networks, and multi-view recommendation.Source: 2) Recommendation Model · p. 4
  • Meta-learning FedRS aggregates client parameters into an initialization that clients fine-tune locally for personalization, but privacy leakage can still occur during initialization learning.The paper contrasts this approach with a unified global model for handling non-IID and highly personalized client data.Source: 2) Recommendation Model · p. 4

III. PRIVACY OF FEDERATED RECOMMENDATION SYSTEMS

Federated recommendation systems keep raw user data local, but uploaded parameters can still reveal interactions and ratings. The section reviews pseudo items and homomorphic encryption as privacy mechanisms, each with distinct protections and costs.

III. PRIVACY OF FEDERATED RECOMMENDATION SYSTEMS

  • Uploaded gradients can reveal users’ interacted items or ratings, motivating additional privacy mechanisms beyond keeping raw data local [32].The section reviews pseudo items, homomorphic encryption, secret sharing, and differential privacy.Source: §III · p. 4

A. Pseudo Items

  • Pseudo items conceal interaction behavior by adding sampled unrated items’ gradients, but leave rated-item gradients unchanged, so servers may still infer ratings.FedRec uses hybrid filling for virtual ratings; FedRec++ [40] uses denoising clients to eliminate gradient noise introduced by that strategy.Source: A. Pseudo Items · p. 4

B. Homomorphic Encryption

  • Homomorphic encryption can also support privacy-preserving integration of information from other participants, including anonymous neighbors and learned parameters from friends [32] [44].The review also identifies keeping secret keys from servers or malicious participants as a serious challenge.Source: B. Homomorphic Encryption · p. 5
  • Homomorphic encryption protects ratings while preserving recommendation accuracy, but FedMF incurs serious computation overhead and assumes participants keep secret keys from the server.FedMF encrypts item-embedding gradients for server-side aggregation; CLFM-VFL applies encryption to user hidden vectors in vertical federated learning.Source: B. Homomorphic Encryption · p. 5

C. Secret Sharing

Secret sharing protects ratings while retaining accuracy at lower computation cost than homomorphic encryption, but raises communication costs. Local differential privacy avoids heavy computation and communication overhead, while added noise can reduce recommendation performance.

C. Secret Sharing

  • Secret sharing splits item-matrix gradients into random pieces distributed among participants, protecting ratings and interactions from the server but potentially exposing rated items to recipients [30].Combining secret sharing with pseudo items is proposed to strengthen privacy.Source: C. Secret Sharing · p. 5
  • Secret-sharing FedRS preserves accuracy with lower computation costs than homomorphic encryption, but exchanging pieces greatly increases communication costs.Source: C. Secret Sharing · p. 5

D. Local Differential Privacy

  • Differential privacy can perturb parameters before server aggregation, but comparing item-embedding changes may still reveal which items a user rated.Local differential privacy is presented as a perturbation mechanism suited to reducing the computation or communication costs of encryption-based approaches.Source: D. Local Differential Privacy · pp. 5–6
  • FedGNN combines pseudo items to protect interaction behavior with clipped, Laplace-perturbed gradients to protect ratings; dynamic noise addresses varying gradient magnitudes [32].Local differential privacy avoids heavy computation and communication overhead, but its added noise inevitably affects recommendation performance.Source: D. Local Differential Privacy · p. 6

E. Comparison

Privacy mechanisms differ in what they protect and in their accuracy, computation, and communication trade-offs. Federated recommendation systems also remain vulnerable to poisoning attacks, including targeted methods that can operate with little or no prior knowledge.

E. Comparison

  • Pseudo items protect interaction behavior, while homomorphic encryption, secret sharing, and differential privacy protect ratings; encryption preserves accuracy but costs computation or communication, while noise adds low-cost performance impact.Homomorphic encryption can also support privacy-preserving integration of other participants’ data.Source: E. Comparison · p. 6

IV. SECURITY OF FEDERATED RECOMMENDATION SYSTEMS

  • Despite FedRS’s locally distributed data offering stronger security than traditional recommendation systems, recent studies show attackers can still conduct poisoning attacks with limited prior knowledge [52] [53]; these attacks are targeted or untargeted.Source: §IV · p. 6, A. Poisoning Attacks · p. 6

1) Target Poisoning Attacks:

  • PipAttack aligns target items with popular items in embedding space and constrains modified gradients to remain close to normal ones; FedRecAttack [52] uses public interactions to generate poisoned gradients.Source: 1) Target Poisoning Attacks · p. 6
  • Random approximation and hard user mining enable attacks without prior knowledge, and hard user mining can remain effective with an extremely small proportion of malicious users [53].Earlier approaches rely on popularity information or public interactions, limiting their generality when that knowledge is absent.Source: 1) Target Poisoning Attacks · p. 6

2) Untarget Poisoning Attacks:

The section describes FedAttack’s untargeted poisoning strategy and surveys defenses based on robust aggregation and anomaly detection. It details robust aggregators while noting that several can reduce recommendation performance and may be circumvented by attacks designed to resemble normal users.

2) Untarget Poisoning Attacks:

  • FedAttack infers user interests, then uses well-matched items as negative samples and poorly matched items as positives, allowing malicious clients to damage recommendation performance even under defense.The attack modifies training samples rather than making malicious clients’ behavior obviously different from normal clients.Source: 2) Untarget Poisoning Attacks · pp. 6–7

B. Defense Methods

  • Poisoning defenses include robust aggregation, which seeks convergence with up to 50% malicious participants, and anomaly detection, which identifies and filters poisoned updates.Robust aggregation selects statistically robust parameter values instead of taking their mean.Source: B. Defense Methods · p. 7, 1) Robust Aggregation · p. 7

1) Robust Aggregation:

  • Median aggregates each parameter’s median, while Trimmed-Mean removes the β smallest and largest values before averaging the remaining updates.Trimmed-Mean is described as reducing the effect of outliers.Source: 1) Robust Aggregation · p. 7
  • Krum selects the local model closest to others, Multi-Krum aggregates several such models, Bulyan combines Krum with Trimmed-Mean, and Norm-Bounding clips updates.A-FRS instead uses gradient-based Krum to filter malicious clients in momentum-based FedRS.Source: 1) Robust Aggregation · p. 7
  • Most surveyed robust aggregators can degrade FedRS performance, while PipAttack and FedAttack use constraints that approximate normal-user patterns and circumvent defenses.The survey identifies these limitations as making defense more difficult.Source: 1) Robust Aggregation · p. 7

2) Anomaly Detection:

The section presents anomaly detection and approaches to system and statistical heterogeneity in federated recommendation. It describes resource-aware communication and personalization strategies, alongside costs or difficulties associated with some methods.

2) Anomaly Detection:

  • FSAD extracts four gradient-based features and trains a semi-supervised Bayes classifier to detect and filter poisoned gradients, but varied user interests make anomaly detection harder in FedRS.The survey notes that users’ uploaded parameters can differ substantially because their interests vary.Source: 2) Anomaly Detection · p. 8

V. HETEROGENEITY OF FEDERATED RECOMMENDATION SYSTEMS

  • FedRS heterogeneity includes system, statistical, and privacy differences; system variation in device resources can slow training and reduce global recommendation accuracy.Clients may become stragglers or drop out, motivating asynchronous communication and resource-based client selection.Source: §V · p. 8, A. System Heterogeneity · p. 8

A. System Heterogeneity

  • FedSA aggregates models by arrival order, FedAsync weights updates by staleness, and FedCS estimates client task times to select participants.These approaches address straggling and resource variation through asynchronous communication or client selection.Source: A. System Heterogeneity · p. 8

B. Statistical Heterogeneity

  • For non-IID client data, FedMeta uses MAML for rapid adaptation but incurs second-order-gradient and data-splitting costs, while Reptile avoids data splitting and uses approximate first-order derivatives.Clustering-based methods personalize models for groups of homogeneous clients, though clustering and model combination increase computation costs.Source: B. Statistical Heterogeneity · p. 9

C. Privacy Heterogeneity

The section describes methods for heterogeneous privacy needs and communication-cost reduction in federated recommendation. These methods include differentiated privacy handling and selective model updates, with trade-offs involving accuracy, privacy concerns, or client computation.

C. Privacy Heterogeneity

  • FedeRank lets users control the proportion of interacted-item updates shared, while HPFL aggregates public components directly and keeps user and item representations local for privacy-constrained components.HPFL aggregates local drafts for privacy information components without aligning the representations.Source: C. Privacy Heterogeneity · p. 9

VI. COMMUNICATION COSTS OF FEDERATED RECOMMENDATION SYSTEMS

  • Large models require millions of communicated parameters, motivating importance-based updating, model compression, active sampling, and one-shot learning to reduce FedRS communication costs.Resource-limited clients may struggle with the communication burden in large-scale industrial scenarios.Source: §VI · p. 9

A. Importance-based Model Updating

  • PPRSF’s recall layer narrows the items clients update and communicate, reducing communication and local computation, but requires public user information that raises privacy concerns.The framework uses a four-layer structure: recall, ranking, re-ranking, and service.Source: A. Importance-based Model Updating · pp. 9–10
  • Efficient-FedRec separates a small user model from a large news model and uses secure aggregation for news representations, while FCF-BTS reduces model payload by 90% on highly sparse datasets at a 4%–8% accuracy loss.FCF-BTS selects a smaller-payload part of the global model on the server side, avoiding additional client computation.Source: A. Importance-based Model Updating · p. 10
  • An all-MLP network replaces Transformer self-attention with a Fourier sub-layer and prunes non-contributing model components; importance-based updating can reduce costs but also recommendation performance.The all-MLP approach is reported to reduce communication and computation costs and accelerate convergence.Source: A. Importance-based Model Updating · p. 10

B. Model Compression

The section reviews model compression and client sampling as ways to reduce federated recommendation communication. It reports compression and communication-round reductions while noting computation trade-offs and cold-start constraints.

B. Model Compression

  • 12.83× larger compression ratio while maintaining recommendation performance is reported for JointRec, which combines low-rank matrix factorization and 8-bit probabilistic quantization.Structured and sketched updates are also described as reducing communication costs by two orders of magnitude; compression shifts some burden to client computation.Source: B. Model Compression · p. 10

C. Client Sampling

  • 94% fewer communication rounds than FedAvg is reported for FedFast, which clusters users and propagates local updates within clusters.FedFast faces a cold-start problem and must retrain to support new users and items.Source: C. Client Sampling · p. 10

D. One Shot Federated Learning

The section describes one-shot communication reduction, federated recommendation applications, and public benchmarks.

D. One Shot Federated Learning

  • FedSPLIT achieves similar RMSE to multi-round communication with one post-initial communication pair, but is unsuitable when participants are individual users.It aggregates models through knowledge distillation.Source: D. One Shot Federated Learning · p. 10

A. Applications

  • The healthcare platform improved precision by 9.71% and F1 score by 4% and was deployed in both organizations and online operation.It complements common user data from mobile network operators with healthcare-provider data.Source: A. Applications · p. 11
  • Applications span online services, healthcare, advertising, e-commerce, and POI recommendation, using cross-platform or sensitive data without centralized storage.Examples include a deployed online-content recommendation system and an advertisement CTR method integrating multi-platform behavior.Source: A. Applications · p. 11

B. Public Benchmark Datasets

  • Public benchmarks cover ratings and social data (MovieLens, FilmTrust, Epinions), POI check-ins (Foursquare), news impressions (MIND), music listening (LastFM), and book ratings (Book-Crossing).The described datasets vary in domain, scale, and available user information.Source: B. Public Benchmark Datasets · p. 11

VIII. FUTURE DIRECTIONS

The survey identifies future directions spanning decentralized architectures, incentives, system design, cold start, and security. It concludes with a synthesis of FedRS research and prospective directions.

VIII. FUTURE DIRECTIONS

  • Current client-server FedRS faces single-point-of-failure and privacy issues; peer-to-peer communication and swarm learning are proposed decentralized alternatives.In the described gossip-based framework, participants send copies of the global recommendation model to random online neighbors.Source: §VIII · pp. 11–12
  • Incentive mechanisms should measure client contributions fairly and efficiently to encourage data sharing and participation, especially across organizations.Global-model performance depends on the quantity and quality of data participants provide.Source: §VIII · p. 12
  • Local recall and ranking can impose enormous client communication, computation, and memory costs; server-side recall exposes recalled items to the server.The alternative described places recall on the server and ranking on the client, with encrypted or noised user embeddings sent for recall.Source: §VIII · p. 12
  • Cold start remains challenging because locally stored user data makes auxiliary information, such as social relationships, harder to integrate while preserving privacy.Source: §VIII · p. 12
  • Existing robust aggregation and detection methods often perform poorly in FedRS, and the survey reports no effective defense that maintains recommendation accuracy.Krum, Median, and Trimmed-mean can degrade recommendation performance; some attacks mimic normal-user patterns and complicate detection.Source: §VIII · p. 12

IX. CONCLUSION

  • The survey compares existing designs across privacy, security, heterogeneity, and communication costs, and presents prospective research directions.Source: §IX · p. 12
Loading 2301.00767v2…