Source-linked AI summary
Secure Federated Matrix Factorization
Di Chai, Leye Wang, Kai Chen, Qiang Yang
TL;DR
The paper addresses privacy leakage in federated matrix factorization, where uploaded gradients can reveal users’ raw preference data. FedMF combines user-level distributed matrix factorization with homomorphic encryption; experiments report acceptable time efficiency when the number of items is small, while identifying efficiency and information-leakage challenges.
Problem
Matrix factorization in federated learning must protect users’ raw preferences and latent features, because gradients and learned representations can reveal sensitive attributes.
Method
FedMF has users compute gradients from local ratings, upload them instead of raw data, and protect the gradients with homomorphic encryption.
Results
FedMF has the same accuracy as matrix factorization on users’ raw data, and real-world experiments show acceptable time efficiency when the number of items is small.
Takeaways & Limitations
FedMF provides a secure federated matrix factorization framework that is feasible on real movie-rating data under the reported experimental setting.
Takeaways & Limitations
Efficiency remains constrained because time consumption increases linearly with the number of items and about 95% of runtime is spent on ciphertext-based server updates.
Abstract
from arXiv · showhide
To protect user privacy and meet law regulations, federated (machine) learning is obtaining vast interests in recent years. The key principle of federated learning is training a machine learning model without needing to know each user's personal raw private data. In this paper, we propose a secure matrix factorization framework under the federated learning setting, called FedMF. First, we design a user-level distributed matrix factorization framework where the model can be learned when each user only uploads the gradient information (instead of the raw preference data) to the server. While gradient information seems secure, we prove that it could still leak users' raw data. To this end, we enhance the distributed matrix factorization framework with homomorphic encryption. We implement the prototype of FedMF and test it with a real movie rating dataset. Results verify the feasibility of FedMF. We also discuss the challenges for applying FedMF in practice for future research.
1 Introduction
FedMF addresses privacy risks in federated matrix factorization, where traditional methods either trade predictive power for privacy or require complicated third-party encryption services. It distributes gradient computation to users and protects gradients with homomorphic encryption.
- Matrix factorization remains under-investigated in federated learning despite its use in recommendation and environmental monitoring.
- Traditional matrix factorization can expose raw preferences and learned latent vectors that reveal sensitive attributes.
- Obfuscation-based privacy methods can reduce the predictive power of learned latent feature vectors, creating a privacy–performance trade-off.
- Encryption-based methods can preserve predictive power but commonly require a difficult-to-find third-party crypto-service provider and risk collusion.
- FedMF has users compute gradients locally, optionally encrypt them homomorphically, and upload them instead of raw preference data.
- The prototype was tested on a real movie rating dataset, and the reported results verify FedMF’s feasibility.
2 Preliminaries
The paper uses horizontal federated learning for users whose rating data share a feature space but differ in samples, with security against an honest-but-curious server. Additive homomorphic encryption permits operations on encrypted data.
- Horizontal Federated Learning: Horizontal federated learning applies when contributors share the same feature space but have different samples.
- Horizontal Federated Learning: FedMF is categorized as horizontal federated learning because users’ rating information has a common feature space while users represent different samples.
- Horizontal Federated Learning: The system assumes honest users and is designed to protect data against an honest-but-curious server.
- Additively Homomorphic Encryption: Homomorphic encryption allows third parties to operate on encrypted data without decrypting it beforehand.
- Additively Homomorphic Encryption: For additive homomorphic encryption, ciphertext operations produce an encrypted sum that can later be decrypted to recover the plaintext addition.
3 User-level Distributed Matrix Factorization
User-level distributed matrix factorization keeps ratings on user devices while splitting iterative optimization between local user updates and server-side item-profile updates. Users download the latest item profiles, compute gradients locally, and return those gradients for aggregation.
- Stochastic gradient descent is used to optimize the matrix factorization objective through iterative updates.
- Matrix factorization represents users and items with profile matrices whose inner products predict user ratings.
- The server initializes item profiles, users initialize user profiles, and training continues until the profiles converge.
- Training stops when the relevant user or item gradients become smaller than a threshold ε.
- Server Update: The server receives user gradients and updates the item profiles, while user and server computations together form the distributed optimization process.
- User Update: Each user downloads the latest item-profile matrix, performs local updates using private ratings, and sends computed gradients to the server.
4 Gradients Leak Information
The paper shows that a server can infer a user's rating information from gradients uploaded in two continuous update steps, despite users keeping raw ratings locally.
- Plaintext gradients uploaded during distributed matrix factorization can reveal users’ original rating data to the server.The framework keeps ratings on user devices, but the server can deduce them from uploaded gradients.
- The distributed update equations separate user-side and server-side computations, with gradients representing information uploaded for item-profile updates.The gradient G is uploaded from user i to the server, and the derivation uses successive update equations to recover ratings.
- The analysis considers user and item latent vectors as D-dimensional and examines element-wise updates across consecutive time steps.It analyzes the k-th entry of a user vector and the correlation between successive user-vector states.
- Knowing a user’s gradients from two continuous steps provides enough information to infer the user’s rating information.The paper proposes solving an intermediate scalar equation numerically, for example with Newton’s method, before computing the rating.
5 FedMF: Federated Matrix Factorization
FedMF combines user-level distributed matrix factorization with homomorphic encryption so users can train collaboratively while sending encrypted gradients to the server. The paper claims ciphertext-only communication protects privacy under ciphertext indistinguishability, while preserving accuracy relative to distributed matrix factorization.
- FedMF encodes gradients with homomorphic encryption so the server can perform updates without reversing the encoding process.The server receives ciphertexts and performs the matrix-factorization update on encrypted gradients.
- FedMF assumes an honest-but-curious server, honest users, and user privacy protected against the server.The framework includes two participant types: the server and the users.
- Users generate the keys, share the public key with all participants, and keep the secret key among users rather than exposing it to the server.Secure TLS/SSL channels are established for key and ciphertext communication.
- The protocol initializes item profiles at the server and user profiles locally, then iterates encrypted item-profile updates and user-side gradient computation until convergence.Users download the latest encrypted item profiles, compute gradients locally, encrypt them, and return ciphertexts to the server.
- FedMF is claimed to be accuracy equivalent to user-level distributed matrix factorization because encryption does not change the parameter-updating process.The equivalence claim applies when the homomorphic-encryption component is removed from the update comparison.
6 Prototype and Evaluation
The prototype uses Paillier-based homomorphic encryption with floating-point and negative-number extensions, and evaluates FullText and PartText on MovieLens. FedMF preserves original MF computation values, while PartText is substantially faster and server updates dominate runtime.
- Prototype Implementation: Paillier encryption instantiates the FedMF prototype, with encrypted operations extended to floating-point and negative values.Floating-point values use an integer representation and shared base exponent; negative values are recovered using a maximum-number threshold and modular arithmetic.
- Prototype Implementation: FullText uploads gradients for every item, assigning zero to unrated items, whereas PartText uses a different user-uploading setting for sparse feedback.The passage states that both settings follow FedMF’s overall steps but differ at upload time.
- Evaluation Setup: MovieLens evaluation uses 100K ratings from 610 users on 9724 movies, with 1024-bit Paillier keys and 100-dimensional user and item profiles.Experiments run on a 5.0GHz 6-core CPU with 32GB RAM using Python and gmpy acceleration.
- Performance: FedMF outputs the same user and item profiles as original MF because distributed computing and homomorphic encryption do not affect computation values.The experiments therefore focus mainly on computation time.
- Performance: Nearly 20 times faster, PartText improves iteration efficiency over FullText, although it leaks some information.For both settings, iteration time roughly increases linearly with the number of items.
- Performance: ~95% of one iteration’s time is spent on server updates, identifying server computation as the main runtime bottleneck.The authors suggest that increasing server computing power or lowering ciphertext-operation complexity could significantly improve efficiency.
7 Conclusion and Future Work
FedMF combines homomorphic encryption with federated matrix factorization to protect against an honest-but-curious server while preserving the accuracy of matrix factorization on raw data. Experiments show acceptable efficiency for small item sets, but scaling, upload-policy, and threat-model challenges remain.
- Conclusion: FedMF is secure against an honest-but-curious server and matches matrix factorization accuracy on users’ raw data.The framework first establishes that plaintext gradients can leak ratings, then protects the distributed model with homomorphic encryption.
- Conclusion: FedMF’s time efficiency is acceptable when the number of items is small, but time consumption increases linearly with item count.This identifies item-set size as a practical boundary for the current prototype.
- Future Work: About 95% of system time is spent on server updates performed over ciphertext, making more efficient homomorphic encryption a key improvement target.The authors expect better ciphertext-operation efficiency to improve overall performance.
- Future Work: PartText is more efficient than FullText, but reveals which items a user rated and may still expose sensitive information.The authors suggest uploading more than the rated-item gradients but fewer than all-item gradients as a possible compromise.
- Future Work: The current security definition assumes honest participants and an honest-but-curious server, leaving malicious participants and collusion as open challenges.Future work considers stronger definitions that include participant misbehavior and collusion with the server.