Source-linked AI summary
Practical Secure Aggregation for Federated Learning on User-Held Data
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth
TL;DR
Federated Learning can train neural networks from sensitive user-held data without collecting raw messages, but individual model updates may still reveal private information. The paper designs a communication-efficient Secure Aggregation protocol for high-dimensional gradients, combining dropout tolerance with protections against specified server behaviors and reporting low communication expansion at large scales.
Problem
Federated Learning needs to aggregate mobile users’ model updates without exposing individual gradients that may reveal sensitive training information.
Method
The paper designs a communication-efficient Secure Aggregation protocol for high-dimensional user vectors, using staged masking, secret sharing, and server-mediated key agreement.
Results
The protocol tolerates up to 1/3 of users failing to complete the protocol and achieves 1.73× and 1.98× communication expansion at the reported large scales.
Takeaways & Limitations
Secure Aggregation can protect user model gradients while supporting high-dimensional Federated Learning updates under mobile-device participation constraints.
Takeaways & Limitations
The paper does not analyze security against arbitrarily malicious servers or colluding users and defers formal analysis to the full version.
Abstract
from arXiv · showhide
Secure Aggregation protocols allow a collection of mutually distrust parties, each holding a private value, to collaboratively compute the sum of those values without revealing the values themselves. We consider training a deep neural network in the Federated Learning model, using distributed stochastic gradient descent across user-held training data on mobile devices, wherein Secure Aggregation protects each user's model gradient. We design a novel, communication-efficient Secure Aggregation protocol for high-dimensional data that tolerates up to 1/3 users failing to complete the protocol. For 16-bit input values, our protocol offers 1.73x communication expansion for $2^{10}$ users and $2^{20}$-dimensional vectors, and 1.98x expansion for $2^{14}$ users and $2^{24}$ dimensional vectors.
1 Introduction
The paper develops a communication-efficient Secure Aggregation protocol for federated deep-learning gradients on private mobile-device data, tolerating up to one-third user failures.
- The protocol protects each user’s model gradient while enabling distributed stochastic gradient descent across user-held training data.
- It tolerates up to 1/3 of users failing to complete the protocol.
- 1.73× communication expansion supports 2^10 users and 2^20-dimensional vectors with 16-bit inputs.
- 1.98× communication expansion supports 2^14 users and 2^24-dimensional vectors with 16-bit inputs.
2 Secure Aggregation for Federated Learning
Federated Learning keeps sensitive training data on users’ mobile devices while sending processed updates, but Secure Aggregation is needed to prevent individual gradients from revealing private information and to handle mobile-system constraints.
- Federated Learning keeps each user’s private text-message database on-device and trains a shared model from processed, minimally scoped, ephemeral updates.
- A user’s gradient update can reveal recently typed words because vocabulary parameters may have non-zero gradients only for encoded words.
- The server needs only summed updates rather than inspecting individual user updates.
- Secure Aggregation lets the server learn that one or more users wrote a word without learning which users did.
- Mobile participation is unpredictable because devices have sporadic power and network access, requiring robustness to user dropouts.
- High-dimensional updates can impose substantial costs on users with metered plans, while devices lack direct authenticated communication channels.
3 A Practical Secure Aggregation Protocol
The protocol evolves masking and secret sharing into a practical Secure Aggregation design that protects private vectors, recovers from dropouts, and reduces communication for high-dimensional inputs. Its security and robustness depend on threat-model-specific thresholds and assumptions about participants and communication.
- Model and requirements: Each user holds a private k-dimensional vector, and the server should learn only the aggregate under defined threat models.The threat models vary from an honest-but-curious server to servers that can misreport dropouts or access some users’ memory.
- Protocol 0: Masking with One-Time Pads: Pairwise one-time-pad perturbations cancel when all users complete the protocol, hiding individual inputs from the server.Users exchange matched perturbations, add them to their vectors, and the server sums the perturbed values.
- Protocol 1: Dropped User Recovery: Secret sharing enables recovery when users drop out, provided at least t users complete the protocol and t > n/2.The server reconstructs perturbations associated with dropped users, allowing the remaining users’ masks to cancel.
- Protocol 2: Double-Masking: Double masking preserves security when a server reconstructs a user’s pairwise perturbations by separately protecting the input with a random b_u mask.For Threat Model T1, security holds when t > n/2; stronger threat models require higher thresholds, and arbitrary malicious or colluding parties are deferred.
- Protocol 3: Efficient Secret Exchange: O(kn^2) communication makes Protocol 2 impractical for high-dimensional data, motivating a refinement that exchanges scalar seeds and expands them with a PRG.Key agreement reduces the need to publish shares for every pairwise vector secret while retaining essentially the same security in the considered threat models.
4 Related work
Secure aggregation work addresses efficient operation and malicious-user settings, but early user aborts can force expensive protocol restarts. The paper also situates its approach relative to DC-nets and secret-sharing implementation details.
- DC-nets provide anonymity through pairwise blinding, extending secure aggregation to untraceably learn each user’s input.
- Existing protocols may need to restart from scratch when even one user aborts too early, creating substantial expense.
- Recent research has examined communication efficiency and operation in the presence of malicious users.
- Aligned (t, n)-Shamir shares can be reconstructed in O(t^2 + nt) by caching Lagrange coefficients.
- Setting R = n(R_U − 1) + 1 ensures no overflow.