LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix Factorization

Honglei Zhang; Fangyuan Luo; Jun Wu; Xiangnan He; Yidong Li

Source-linked AI summary

LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix Factorization

Honglei Zhang, Fangyuan Luo, Jun Wu, Xiangnan He, Yidong Li

arXiv:2206.11743v2Jun 23, 2022cs.IRcs.CR

TL;DR

Federated recommendation must accommodate large item collections on resource-constrained devices while limiting privacy risks from real-valued updates. LightFR uses learning to hash and federated discrete optimization to train binary representations, and experiments on four real-world datasets report improvements in accuracy, inference efficiency, and data privacy over state-of-the-art FRS methods.

Problem
Existing federated recommenders use continuous inner-product representations that become costly at large item scales, while real-valued gradient transmission may leak private information.
Method
LightFR combines learning to hash with federated matrix factorization and discrete optimization to train binary user and item representations across clients and server.
Results
Experiments on four real-world datasets show that LightFR outperforms several state-of-the-art FRS methods in recommendation accuracy, inference efficiency, and data privacy.
Takeaways & Limitations
Binary representations provide LightFR with fast online inference and economical memory and communication consumption in federated settings.
Takeaways & Limitations
The current framework focuses on cold-start cases with a few interactions; entirely interaction-free users or items require side information and warm-up techniques beyond its scope.

Abstract

from arXiv · show

Federated recommender system (FRS), which enables many local devices to train a shared model jointly without transmitting local raw data, has become a prevalent recommendation paradigm with privacy-preserving advantages. However, previous work on FRS performs similarity search via inner product in continuous embedding space, which causes an efficiency bottleneck when the scale of items is extremely large. We argue that such a scheme in federated settings ignores the limited capacities in resource-constrained user devices (i.e., storage space, computational overhead, and communication bandwidth), and makes it harder to be deployed in large-scale recommender systems. Besides, it has been shown that transmitting local gradients in real-valued form between server and clients may leak users' private information. To this end, we propose a lightweight federated recommendation framework with privacy-preserving matrix factorization, LightFR, that is able to generate high-quality binary codes by exploiting learning to hash technique under federated settings, and thus enjoys both fast online inference and economic memory consumption. Moreover, we devise an efficient federated discrete optimization algorithm to collaboratively train model parameters between the server and clients, which can effectively prevent real-valued gradient attacks from malicious parties. Through extensive experiments on four real-world datasets, we show that our LightFR model outperforms several state-of-the-art FRS methods in terms of recommendation accuracy, inference efficiency and data privacy.

1 INTRODUCTION

Federated recommendation preserves local-data privacy but remains difficult to deploy at scale because continuous representations impose storage, communication, inference, and privacy costs. LightFR addresses these constraints with binary representations and federated discrete optimization.

FRS challenges: Existing federated recommender systems transmit large item representations, creating substantial storage and communication costs for resource-constrained clients.These costs grow with the number of items and can be unaffordable for cost-conscious devices.
FRS challenges: Dense embedding methods retain costly client-side inference and may expose private information through transmitted real-valued gradients.Meta-learning and distillation reduce some overheads but still require dense forward propagation and real-valued gradient transmission.
LightFR: LightFR uses learning to hash to produce compact binary user and item representations in federated settings.Binary codes reduce memory and communication requirements while replacing inner-product search with bit operations in Hamming space.
LightFR: LightFR introduces federated discrete optimization to collaboratively train binary representations on clients and the server.The method is designed to embed user preferences in Hamming space while reducing resource use and preserving privacy.
Contributions: The paper evaluates LightFR on four datasets and reports advantages over state-of-the-art federated recommender systems in effectiveness, efficiency, and privacy.The claimed advantages include storage/communication efficiency, inference efficiency, and privacy preservation.

2 RELATED WORK

The related work covers matrix factorization, hashing, and federated learning as foundations for efficient and privacy-preserving recommendation. It positions LightFR against methods that reduce model payloads but retain dense representations or costly parameter exchange.

2.1 Matrix Factorization: Matrix factorization maps users and items into a shared latent space, estimating observed ratings through inner products and optimizing a regularized squared loss.User and item latent matrices can be learned with gradient descent or alternating least squares.
2.2 Learning to Hash: Learning to hash indexes data with compact binary codes so similar objects have small Hamming distances and dissimilar objects have large distances.The literature includes locality-sensitive hashing and learned hash codes, including direct binary-code optimization.
2.3 Federated Recommender System: Federated learning enables clients to train on private data while a server aggregates local models or gradients into a global model.Federated recommender systems apply this paradigm to sensitive user attributes and interaction histories.
2.3 Federated Recommender System: Meta-learning methods reduce client payloads, but federated recommendation still faces efficiency and privacy challenges from storing and transmitting model parameters.The comparison framework considers memory, inference, communication, and privacy dimensions.
2.3 Federated Recommender System: The paper directs quantitative analysis of these efficiency and privacy dimensions to its experimental comparison in Fig. 3.Table 1 summarizes the corresponding properties across federated recommender methods.

3 THE PROPOSED LIGHTFR FRAMEWORK

LightFR replaces continuous federated matrix factorization with binary user and item codes in Hamming space. Its federated discrete optimization alternates local user-code updates with server aggregation of item-code gradients while keeping interaction data local.

3 THE PROPOSED LIGHTFR FRAMEWORK: The server initializes and distributes the item binary matrix, while clients update private user vectors from local interaction data and return item gradients for aggregation.The latest global item matrix is redistributed for the next optimization round.
3.2 The LightFR Model: LightFR represents users and items with binary codes and measures their similarity in Hamming space instead of continuous Euclidean embeddings.The Hamming similarity compares matching bits and supports efficient storage and similarity search.
3.3 Federated Discrete Optimization: The optimization alternates client-side minimization of each private user vector with item-vector minimization driven by client gradients and server aggregation.User updates use local data only, and the balanced constraint makes the discrete problem generally NP-hard, motivating DCD-based updates.

9 end

The client-update procedure repeatedly updates private user bits and computes item-matrix gradients from local interactions before returning those gradients to the server.

ClientUpdate: Each client updates its private user binary vector bit by bit and computes gradients for the item binary matrix over local observations.The returned gradients are used in global aggregation, and the outer training loop repeats until convergence.

25 end

LightFR extends its discrete federated design to cold-start users and items and analyzes storage, communication, inference, and privacy properties. Its cold-start procedure assumes at least a few interactions, while entirely interaction-free cases require side information outside the paper’s scope.

9 Cold-start Scenario: LightFR handles new users and items by learning temporary binary codes online and retraining the full model offline when possible.The stated cold-start setting allows a few interactions but excludes cases with no interactions.
9 Cold-start Scenario: New users update their binary vectors locally without retraining existing user codes, while new-item gradients are calculated across clients and aggregated at the server.Both procedures preserve the federated separation of local interactions and global item updates.
Beyond-accuracy analysis: The analysis states that Hamming-space communication consumes one-eighth of the Euclidean-space requirement under the specified formulas.The comparison assumes 64-bit floating-point Euclidean representations.
Beyond-accuracy analysis: Hamming similarity enables faster inference than Euclidean search, whose stated top-k complexity is O(mf + klogk).The paper notes that bit operations can even support constant-time scanning.
Privacy Preserving: LightFR exchanges binary rather than real-valued representations, and its irreversible sign operation makes private ratings difficult to infer from uploaded information.The paper concludes that malicious attackers cannot infer sensitive local ratings to some extent.

4 EXPERIMENTS

The experiments introduce an evaluation of LightFR across multiple aspects after describing the experimental settings. The supplied passage does not report specific datasets, baselines, metrics, or outcomes.

4 EXPERIMENTS: The experimental section first specifies the settings and then presents results and analysis evaluating LightFR from multiple aspects.No specific experimental values or comparisons are stated in the supplied passage.

4.1 Experimental Settings

The experiments use four public datasets, standard ranking metrics, and centralized and federated matrix-factorization benchmarks. Model dimensions and implementation settings are specified to support resource-saving and fair comparison.

Datasets: Four public datasets—MovieLens-1M, Filmtrust, Douban-Movie, and Ciao—cover varied scales and interaction densities.MovieLens-1M contains about 1 million ratings, while Filmtrust has the fewest interactions and Douban-Movie has the most ratings per user.
Evaluation Metrics: HR@k and NDCG@k evaluate item-ranking performance by measuring test-item retrieval and ranking quality at cutoff k.Both metrics emphasize the first retrieved items in the ranked list.
Baselines: Comparisons include centralized Matrix Factorization methods and federated Matrix Factorization approaches.The centralized methods do not protect user privacy, while existing federated methods generally use inner-product similarity search in Euclidean space.
Implementation Details: Real-valued MF methods use 32-dimensional embeddings, whereas hashing-based models use 64 dimensions to compare performance while saving resources.The stated effectiveness threshold for LightFR is the length of its binary codes.
Implementation Details: Experiments specify 50 global rounds, one local epoch, and a 0.6 selected-client ratio, alongside baseline-specific hyperparameters.The settings also include FedMF, FedRec, MetaMF, and PrivRec parameters for reproducible comparison.

4.2 Experimental Results

Across four datasets, LightFR delivers competitive recommendation quality while reducing client-side resource costs through binary representations and federated discrete optimization. Ablations and sensitivity analyses support the value of collaborative discrete training and characterize effects of code length, trade-off weighting, and client participation.

4.2.1 Overall Performance: Federated baselines generally slightly underperform centralized matrix-factorization models, while PrivRec is superior to other federated baselines in most cases.PrivRec’s advantage is particularly evident on Filmtrust and Ciao, whereas MetaMF is optimal on Douban-Movie.
4.2.1 Overall Performance: LightFR achieves recommendation performance comparable to real-valued federated models in most cases while using less inference time, memory, and bandwidth.Its binary-code design targets deployment on resource-constrained federated clients.
4.2.1 Overall Performance: LightFR uses about 3.1% of FedMF storage, 8.1% of MetaMF storage, and 8% of FCF, FedRec, and PrivRec storage.The reduction is attributed to storing binary rather than real-valued item embeddings and avoiding additional encryption or prediction-module parameters.
4.2.2 Ablation Study: The full LightFR model outperforms random and direct median-quantization variants, while gradient aggregation is marginally better than direct parameter aggregation.These results support collaborative discrete optimization between the server and local clients.
4.2.3 Sensitivity Analysis: Increasing binary code length from 8 to 64 improves performance across all four datasets, although gains slow as dimensionality increases.The authors therefore favor longer binary representations when client storage permits.
4.2.3 Sensitivity Analysis: Recommendation accuracy initially increases with λ and then declines at larger values, while varying λ causes only small overall fluctuations.As the selected-client ratio p rises, HR@10 and NDCG@10 generally improve before plateauing beyond dataset-specific thresholds.

5 CONCLUSION

LightFR combines learning to hash with federated matrix factorization to reduce resource demands and protect transmitted parameters. Experiments report improved recommendation accuracy, resource savings, and data privacy, while future work targets richer side information and advanced user-modeling algorithms.

5 CONCLUSION: LightFR provides fast online inference with lower memory and communication consumption than conventional federated matrix factorization approaches.The framework is designed to reduce resource consumption in federated settings through binary representations.
5 CONCLUSION: LightFR significantly outperforms state-of-the-art federated recommender methods in recommendation accuracy, resource savings, and data privacy.The authors report this conclusion from their overall comparison experiments.
5 CONCLUSION: Learning to hash produces binary user and item representations, addressing memory, communication, computation, and parameter-privacy challenges on resource-constrained clients.The framework uses federated discrete optimization to collaboratively produce binary representations on clients and the server.
5 CONCLUSION: Future work may enhance binary representations with side information and extend federated discrete representation learning beyond matrix factorization to factorization machines and graph neural networks.These directions are presented as ways to obtain more accurate, efficient, compact, and informative representations.