PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Neural Collaborative Filtering

Vasileios Perifanis, Pavlos S. Efraimidis

arXiv:2106.04405v2cs.IRcs.CRcs.LG

TL;DR

Federated recommendation must protect user interactions while handling update inconsistencies caused by latent-factor models. The paper develops FedNCF with tailored aggregation and privacy-preserving SecAvg, achieving recommendation quality comparable to centralized NCF and faster convergence for its proposed aggregation. A multi-round participant-selection attack can nevertheless reveal interacted items.

  • Problem

    Federated learning avoids transmitting raw user data, but plain client updates can leak preferences and standard FedAvg is inconsistent with sparse latent-factor updates.

  • Method

    FedNCF adapts NCF to federated next-item prediction, uses MF-FedAvg for latent-factor aggregation, and integrates SecAvg secure aggregation against honest-but-curious entities.

  • Results

    FedNCF achieves recommendation quality comparable to centralized NCF, while MF-FedAvg provides faster convergence and higher recommendation quality than standard FedAvg.

  • Takeaways & Limitations

    The experiments support the feasibility of privacy-preserving federated collaborative filtering with neural recommendation models and tailored aggregation.

  • Takeaways & Limitations

    In multiple rounds, repeatedly selecting the same client group and excluding one participant can let the coordination server infer that participant’s interacted items.

Abstract

from arXiv · show

In this work, we present a federated version of the state-of-the-art Neural Collaborative Filtering (NCF) approach for item recommendations. The system, named FedNCF, enables learning without requiring users to disclose or transmit their raw data. Data localization preserves data privacy and complies with regulations such as the GDPR. Although federated learning enables model training without local data dissemination, the transmission of raw clients' updates raises additional privacy issues. To address this challenge, we incorporate a privacy-preserving aggregation method that satisfies the security requirements against an honest but curious entity. We argue theoretically and experimentally that existing aggregation algorithms are inconsistent with latent factor model updates. We propose an enhancement by decomposing the aggregation step into matrix factorization and neural network-based averaging. Experimental validation shows that FedNCF achieves comparable recommendation quality to the original NCF system, while our proposed aggregation leads to faster convergence compared to existing methods. We investigate the effectiveness of the federated recommender system and evaluate the privacy-preserving mechanism in terms of computational cost.

1. Introduction

The paper adapts Neural Collaborative Filtering to federated recommendation while keeping raw user data local. It addresses privacy risks in transmitted updates and inconsistency between standard aggregation and latent-factor updates.

  • Motivation: Federated learning keeps user profiles and interactions local, transmitting model updates instead of raw data to address privacy and regulatory constraints.Centralized collection can conflict with regulations such as the GDPR.
  • Aggregation: Standard FedAvg can produce inconsistent updates in collaborative filtering because users update only item vectors observed locally.The paper proposes MF-FedAvg to handle latent-factor updates.
  • Aggregation: MF-FedAvg experimentally produces faster convergence and higher recommendation quality than standard FedAvg.The evaluation compares aggregation methods using recommendation quality and convergence behavior.
  • Privacy: SecAvg is integrated to protect updates from honest-but-curious entities, while the paper also discusses secure aggregation and k-anonymity.The privacy mechanism is intended to address risks from transmitting updates in plain format.
  • Proposed approach: FedNCF adapts NCF to federated next-item prediction and combines matrix factorization with neural architectures.The work presents this as an analysis of NCF in the federated setting.
  • Evaluation: Across four real-world datasets, FedNCF achieves recommendation quality comparable to centralized NCF without high communication and computational overhead.The experiments validate recommendation quality against the centralized NCF system.

2. Related work

The related work reviews collaborative filtering, matrix factorization, neural collaborative filtering, federated learning, aggregation, and privacy-preserving approaches. It positions FedNCF around synchronous federated aggregation tailored to latent-factor updates and efficient secure computation.

  • Collaborative filtering and matrix factorization: Collaborative filtering recommends previously unseen items from user behavior, while matrix factorization represents interactions through latent factors and inner products.The interaction matrix is decomposed into latent representations.
  • Neural collaborative filtering: Neural collaborative filtering extends matrix factorization by learning interaction functions with GMF, MLP, and their NeuMF fusion.NeuMF combines linear and nonlinear components for recommendation prediction.
  • Federated learning: Federated learning trains models across decentralized devices without sharing user data, while FedAvg aggregates locally computed updates at a coordination server.FedAvg weights updates using participants’ local training-instance counts.
  • Privacy-preserving federated learning: Plain transmission of federated updates can leak participant information, motivating secure multiparty computation, homomorphic encryption, and differential privacy.These approaches differ in how they hide updates and affect computational cost or model utility.
  • Prior privacy-preserving methods: Earlier privacy-preserving matrix-factorization approaches used garbled circuits or homomorphic encryption, but reported execution and communication costs can be substantial.One cited protocol required about 1.5 hours and almost 40 GB per iteration for 4096 interaction tuples.
  • Positioning FedNCF: FedNCF uses synchronous federated learning and an efficient SMC aggregation scheme instead of the asynchronous update procedures used in several prior federated recommender systems.The paper frames this design as avoiding staleness while limiting computational overhead relative to homomorphic encryption.

3. Federated neural collaborative filtering

FedNCF adapts neural collaborative filtering to federated recommendation while addressing both inconsistent latent-factor updates and privacy risks from plain update transmission. MF-SecAvg decomposes aggregation into matrix-factorization and neural-network steps, masks updates, and supports k-anonymity against honest-but-curious adversaries, subject to a participant-selection attack.

  • Federated collaborative filtering generates recommendations from users’ past preferences while minimizing disclosure of interacted items.
  • FedAvg is inconsistent with matrix-factorization updates because users modify only item vectors observed locally, causing stale or low-quality aggregated parameters.Unobserved item vectors are still included in FedAvg, leaving them close to their previous global values.
  • Plain update transmission can expose user preferences, while SecAvg alone remains inconsistent with weighted latent-factor aggregation.The issue arises because item-vector updates reveal interacted items and simple secure masking does not correct the aggregation mismatch.
  • MF-SecAvg decomposes aggregation into item-profile averaging by item-specific updater counts and weighted FedAvg for neural-network weights, integrated with SecAvg.The protocol masks updates while separately handling matrix-factorization and neural-network parameters.
  • MF-SecAvg masks participant updates so the server learns aggregate values and updater counts, but not which users interacted with particular items.Masked parameters preserve the aggregated output while hiding individual updates and preferences.
  • FedNCF with MF-SecAvg ensures k-anonymity against an honest-but-curious adversary during an aggregation round.The proof relies on blind calculation of update sums and item-specific participant counts.
  • An honest-but-curious server can break this guarantee by repeatedly selecting nearly identical participant groups and excluding one user to infer that user’s interacted items.The attack compares item updater counts across successive rounds; synthetic negative feedback may blur the inference but does not remove the described exposure.

4. Experiments

The experiments evaluate FedNCF across recommendation datasets, aggregation functions, heterogeneity, robustness, and computational cost. They include dataset statistics and analyses of how federated conditions affect training.

  • The evaluation examines recommendation quality, distributed learning, computational cost, heterogeneity, robustness, and MF-SecAvg overhead.

4.1. Evaluation settings

The evaluation uses four established recommender-system datasets and a leave-one-out ranking protocol. Recommendation quality is measured by HR@10 and NDCG@10 averaged across users.

  • Four datasets are evaluated: MovieLens 100K, MovieLens 1M, Lastfm 2K, and Foursquare New York.Users with fewer than 5 interactions are excluded.
  • Client observation counts vary substantially, creating quantity heterogeneity that can affect quality and training time.
  • MovieLens ratings are converted to implicit feedback, while Lastfm 2K and Foursquare New York already contain implicit interactions.
  • Leave-one-out evaluation holds out each user’s last interaction and pairs it with 100 randomly sampled uninteracted items.
  • HR and NDCG are reported at K = 10, with both metrics averaged across users.

4.2. Implementation details

The implementation fixes common model and optimization settings across experiments, including the latent dimension, neural hidden layers, loss, negative sampling, and optimizer.

  • All models use latent dimension D = 12 and hidden layers h = {48, 24, 12, 6}.
  • Training optimizes BCE loss with four sampled negative instances per positive instance using Adam.

4.3. Aggregation function impact

The aggregation experiment compares MF-FedAvg with FedAvg and SimpleAvg on recommendation quality and convergence. MF-FedAvg better matches matrix-factorization updates and converges faster than FedAvg.

  • MF-FedAvg decomposes aggregation into matrix-factorization and neural-network-based averaging, whereas SimpleAvg performs non-weighted averaging.
  • The comparison reports average HR@10 over 5 experiments and uses plain weight transmission without privacy-preserving aggregation.
  • MF-FedAvg outperforms FedAvg, which outperforms SimpleAvg, in recommendation quality.
  • 25% higher HR is observed for MF-FedAvg than FedAvg at the end of training iterations.
  • MF-FedAvg converges faster than FedAvg across the four evaluated datasets.

4.4. Data federation impact

FedNCF’s federated models approach centralized NCF quality but differ substantially in convergence behavior. FedNeuMF benefits from combining early MLP convergence with GMF’s later refinement, whereas FedMLP fails to converge.

  • Centralized NCF models define the upper bound for FedNCF recommendation quality after 400 training epochs.
  • In Foursquare NY, FedNeuMF and FedMLP overfit from global rounds 180 and 60, respectively, while FedGMF has not peaked after 400 rounds.
  • FedMLP and FedNeuMF improve early federated convergence compared with FedGMF.
  • FedNeuMF and FedGMF achieve similar final recommendation quality, while FedMLP fails to converge.
  • FedNeuMF trails centralized NeuMF by 8%, 12%, 16% and 3% in HR across the four datasets, yet federated models retain acceptable quality.

4.5. Efficiency comparison

GMF is substantially more efficient than the neural alternatives in federated recommendation, requiring less computation and communication while remaining close to traditional MF in cost.

  • Client computation is measured with FLOPs, while transferred parameter size measures communication overhead.
  • GMF requires 89 and 90 times less average computational cost than FedMLP and FedNeuMF, respectively.
  • GMF requires almost half as many transmitted parameters as the corresponding NeuMF model.
  • GMF’s FLOPs and parameter-transmission requirements are almost equivalent to traditional MF.
  • GMF adds only 13 float values, or 0.1 KB, compared with traditional MF in the considered setting.

4.6. Heterogeneity impact

Training time is driven mainly by the amount of local data rather than system heterogeneity. For GMF, computation grows linearly with local observations and remains under one second in the tested worst case.

  • GMF training time increases with local dataset size, while system heterogeneity does not heavily affect computation time.
  • With 10,000 observations on one CPU, training completes in less than a second; 16 CPU cores make it 10% faster.
  • GMF computation time increases linearly with the number of local observations.
  • The experiment varies resources across 1, 2, 4, 8 and 16 CPU cores and local datasets from 100 to 10,000 instances.

4.7. Number of participants impact

FedNCF remains robust when fewer users participate in training. Removing users causes only modest recommendation-quality declines in both centralized and federated settings.

  • The experiment removes randomly selected users at participation fractions of 0.9, 0.8, 0.7 and 0.5, repeating training five times.
  • Removing users decreases recommendation quality by 3–7% centrally and 1–6% federatedly.
  • As the number of participating users increases, recommendation quality also increases.
  • The finalized centralized and federated models remain robust to the number of participating users.

4.8. Secure aggregation impact

The study measures MF-SecAvg’s computational overhead across the smallest and largest NCF models. Its additional cost is reported as negligible, including 93ms on the largest dataset with NeuMF.

  • The experiment evaluates MF-SecAvg using FedGMF and FedNeuMF, representing the smallest and largest NCF models, respectively.The comparison targets computational overhead introduced by secure aggregation.
  • The overhead measurement concerns one aggregation round for each participant on a 4.0GHz, 8-core CPU.
  • 93ms of additional computation is required for NeuMF on the largest dataset, Foursquare NY.
  • MF-SecAvg has an imperceptible impact on computation overhead and provides computational and communication efficiency while preserving participant privacy.

5. Conclusion

The paper presents FedNCF with privacy-preserving aggregation and finds federated recommendation feasible while retaining local data. It identifies privacy guarantees across multiple rounds as an open issue.

  • FedNCF adapts Neural Collaborative Filtering to federated recommendation and uses privacy-preserving aggregation for latent-factor updates.
  • The experiments demonstrate the feasibility of federated learning for recommender systems and support high-quality models from local data contributions.
  • Secure aggregation protects participants in a single round, but privacy guarantees across multiple rounds remain an open issue.
Loading 2106.04405v2…