PaperlayerView exact revision on arXiv

Source-linked AI summary

LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning

Jinhyun So, Chaoyang He, Chien-Sheng Yang, Songze Li, Qian Yu, Ramy E. Ali, Basak Guler, Salman Avestimehr

arXiv:2109.14236v3cs.LGcs.CRcs.DCcs.ITstat.ML

TL;DR

Secure aggregation in federated learning must protect individual models while tolerating user dropouts, but existing seed-reconstruction protocols become costly as dropouts increase. LightSecAgg instead reconstructs surviving users’ aggregate mask in one shot, preserving privacy and dropout resilience while reducing overhead; experiments report 8.5×–12.7× speedups over SecAgg and 2.9×–4.4× over SecAgg+.

  • Problem

    Secure aggregation must protect users’ individual models during global aggregation and remain robust to likely dropouts, while existing pairwise seed-reconstruction protocols face scaling bottlenecks.

  • Method

    LightSecAgg encodes users’ random masks so the server can reconstruct the surviving users’ aggregate mask in one shot instead of reconstructing dropped users’ seeds.

  • Results

    8.5×–12.7× speedup over SecAgg and 2.9×–4.4× over SecAgg+ was achieved across considered FL training tasks in realistic bandwidth settings.

  • Takeaways & Limitations

    LightSecAgg provides the same privacy and dropout-resiliency guarantees as state-of-the-art protocols, reduces aggregation overhead, and supports synchronous and asynchronous FL.

  • Takeaways & Limitations

    In asynchronous FL, the convergence impact of LightSecAgg’s model quantization requires separate analysis, addressed in the appendix for smooth non-convex settings.

Abstract

from arXiv · show

Secure model aggregation is a key component of federated learning (FL) that aims at protecting the privacy of each user's individual model while allowing for their global aggregation. It can be applied to any aggregation-based FL approach for training a global or personalized model. Model aggregation needs to also be resilient against likely user dropouts in FL systems, making its design substantially more complex. State-of-the-art secure aggregation protocols rely on secret sharing of the random-seeds used for mask generations at the users to enable the reconstruction and cancellation of those belonging to the dropped users. The complexity of such approaches, however, grows substantially with the number of dropped users. We propose a new approach, named LightSecAgg, to overcome this bottleneck by changing the design from "random-seed reconstruction of the dropped users" to "one-shot aggregate-mask reconstruction of the active users via mask encoding/decoding". We show that LightSecAgg achieves the same privacy and dropout-resiliency guarantees as the state-of-the-art protocols while significantly reducing the overhead for resiliency against dropped users. We also demonstrate that, unlike existing schemes, LightSecAgg can be applied to secure aggregation in the asynchronous FL setting. Furthermore, we provide a modular system design and optimized on-device parallelization for scalable implementation, by enabling computational overlapping between model training and on-device encoding, as well as improving the speed of concurrent receiving and sending of chunked masks. We evaluate LightSecAgg via extensive experiments for training diverse models on various datasets in a realistic FL system with large number of users and demonstrate that LightSecAgg significantly reduces the total training time.

1 INTRODUCTION

LightSecAgg addresses the server-side reconstruction bottleneck in dropout-resilient secure aggregation by reconstructing one aggregate mask from surviving users. It also supports asynchronous FL and reduces end-to-end training time through system-level optimizations.

  • Protocol design: LightSecAgg replaces dropped-user seed reconstruction with one-shot reconstruction of surviving users’ aggregate masks.Users encode and share local masks so the server can reconstruct the aggregate mask directly.
  • Protocol design: The server reconstructs only one mask during recovery, independent of the number of dropped users.This reduces aggregation complexity while preserving the stated privacy and dropout-resiliency guarantees.
  • System optimization: Parallelizing independent random-mask generation with local-model computation reduces total running time across evaluated secure aggregation protocols.The system also overlaps model training with on-device encoding and improves concurrent chunked-mask transfers.
  • Asynchronous FL: LightSecAgg enables secure aggregation for asynchronous FL, unlike SecAgg and SecAgg+, without relying on differential privacy or trusted execution environments.The protocol protects individual updates when users’ local updates are not synchronized.
  • Evaluation: 8.5×-12.7× performance gain over SecAgg and 2.9×-4.4× over SecAgg+ is achieved in realistic user-bandwidth settings.Experiments cover up to 200 users, diverse models, and datasets with low- and high-resolution images.

2 PROBLEM SETTING

The problem setting formalizes federated learning as aggregation of private local updates under an honest-but-curious threat model and possible user dropouts. The desired protocol must preserve privacy, tolerate dropouts, scale with users, and support both synchronous and asynchronous operation.

  • Federated learning: Federated learning minimizes a global objective using data held at edge devices and aggregates users’ local models through a server.The global model has dimension d, and each user contributes a weighted local objective.
  • Federated learning: At each iteration, users receive the current global model, create local updates, and send them to the server for aggregation.Users may drop during training because of unreliable communication or other operational conditions.
  • Secure aggregation: Secure aggregation aims to recover the aggregate of local models without exposing individual models to the server.The protocol operates over a finite field Fq because its cryptographic primitives require finite-field operations.
  • Security model: The honest-but-curious threat model permits up to T colluding users and a server, while requiring that nothing beyond the aggregate model is learned.The privacy requirement is expressed through a conditional mutual-information constraint.
  • Design requirements: The protocol must recover the aggregate despite up to D dropped users, with at least N−D surviving users.Dropouts can result from interrupted processing, poor wireless conditions, or low battery.
  • Design requirements: The stated goal is an efficient, scalable protocol with strong privacy and dropout resilience that scales linearly with N and supports synchronous and asynchronous FL.Asynchronous FL avoids synchronizing all users but introduces update staleness and incompatibility with existing secure aggregation protocols.

3 OVERVIEW OF BASELINE PROTOCOLS: SE CAG G AND SE CAG G+

SecAgg and SecAgg+ protect models with pairwise masks and secret-shared seeds, but server-side mask reconstruction becomes a scaling bottleneck and prevents asynchronous aggregation.

  • SecAgg workflow: SecAgg uses pairwise random seeds to mask models and secret-shares private seeds and keys for dropout recovery.The server reconstructs surviving users’ private seeds or dropped users’ pairwise seeds, then computes the aggregate model.
  • Guarantees: SecAgg protects model privacy against T colluding users and tolerates D dropouts when N − D > T.
  • SecAgg workflow: 4d is the server’s mask-reconstruction cost in the three-user example when user 1 drops.The server reconstructs n2, n3, z1,2, and z1,3 before recovering x2 + x3.
  • Scaling bottleneck: O(N^2) mask-reconstruction overhead dominates SecAgg, while SecAgg+ reduces it to O(N log N) using a sparse random graph.SecAgg+ still relies on mask reconstructions, so pairwise-mask recovery remains a scaling bottleneck.
  • Asynchronous FL: SecAgg and SecAgg+ cannot support asynchronous FL because pairwise-mask cancellation is not guaranteed when the aggregated users are unknown beforehand.FedBuff provides an alternative through a TEE-enabled buffer, but its use is limited by TEE memory and availability.
  • Comparison: d is the server cost for LightSecAgg in the corresponding three-user dropout example, compared with 4d for SecAgg.LightSecAgg directly recovers the aggregate-mask in one shot rather than reconstructing individual seeds.

4 LIGHTSECAGG PROTOCOL

LightSecAgg encodes each user’s local mask so the server can recover surviving users’ aggregate-mask in one shot. Its MDS-based design supports privacy and dropout resilience, including asynchronous and weighted aggregation settings.

  • Core design: LightSecAgg shifts secure aggregation from reconstructing dropped users’ pairwise seeds to one-shot reconstruction of surviving users’ aggregate-mask.Each user generates one mask, encodes it, and shares encoded masks with other users.
  • Security guarantees: LightSecAgg provides robustness against D dropped users and privacy against T colluding users through its encoded-mask construction.The three-user example uses encoding and sharing that tolerates one dropped user and one curious user.
  • Offline encoding and sharing: Each user partitions its mask into U − T pieces and creates encoded masks with a T-private MDS matrix.The matrix provides privacy against T colluding users while supporting recovery from surviving-user messages.
  • Protocol phases: The protocol has three phases: offline mask encoding and sharing, masked-model uploading, and server reconstruction of surviving users’ aggregate masks.Local models are quantized into a finite field before masking.
  • One-shot aggregate-model recovery: After identifying survivors, the server collects their aggregated encoded sub-masks and uses MDS decoding to recover the aggregate-mask.Any U messages suffice to recover the relevant U − T aggregated mask components, after which the server subtracts the mask aggregate from the masked models.
  • Dropout resilience: LightSecAgg requires at least U surviving users at any time, so up to N − U users may drop or be delayed.This removes the need for a stable communication link between every pair of users.
  • Weighted aggregation: LightSecAgg directly applies to weighted local models because mask sharing does not require prior knowledge of users’ weight coefficients.This supports aggregation when users hold datasets of unequal sizes.
  • Asynchronous FL: LightSecAgg supports asynchronous FL with bounded staleness by aggregating updates received into a server buffer.The server updates the global model when the buffer reaches size K.

5 THEORETICAL ANALYSIS

LightSecAgg provides formal privacy and dropout-resiliency guarantees while reducing server-side aggregation complexity. Its resource costs are characterized for storage, communication, and computation.

  • Theoretical Guarantees: LightSecAgg guarantees privacy against up to T colluding users and tolerates up to D dropped users whenever T + D < N.The theorem applies to both synchronous and asynchronous FL settings.
  • Theoretical Guarantees: LightSecAgg trades privacy for dropout resiliency: increasing one guarantee requires reducing the other.Any D < N − T dropped users can be tolerated for a given privacy guarantee T.
  • Theoretical Guarantees: Unlike several comparison protocols, LightSecAgg provides worst-case dropout resiliency rather than only probabilistic recovery under random dropouts.SecAgg+ and other cited protocols relax the worst-case constraint to random dropouts.
  • Complexity Analysis: Each user stores its random mask plus coded masks, giving total offline storage cost (1 + N/(U−T))d.The design parameter satisfies N − D ≥ U > T.
  • Complexity Analysis: During recovery, the server receives U/(U−T)d coded-mask elements and decodes the aggregate mask using an O(U log U) computation.Recovery communication is independent of the number of dropped users once the surviving set is sufficiently large.
  • Complexity Analysis: With U chosen as (1 − p)N, LightSecAgg keeps server complexity nearly constant in N, compared with O(dN^2) for SecAgg and O(dN log N) for SecAgg+.This reduction targets the server computation bottleneck that limits aggregation time as user counts grow.

6 SYSTEM DESIGN AND OPTIMIZATION

The system design separates foundational communication and training components from algorithm-specific secure aggregation components. It reduces runtime by overlapping training with mask processing and by accelerating concurrent tensor communication.

  • System Architecture: The implementation includes a federated training system for reducing secure-aggregation overhead and evaluating LightSecAgg in cross-device FL.The system is designed for realistic deployment and evaluation rather than protocol analysis alone.
  • System Architecture: The software architecture has a foundation layer and an algorithm layer to remain lightweight and maintainable.The foundation layer contains the communicator and training engine, while the algorithm layer integrates client and server secure aggregation components.
  • System Architecture: The server-side Secure Aggregator caches masked models and starts reconstruction when the cache is full before synchronizing the updated global model.The architecture marks seven sequential steps in a single FL round.
  • Runtime Optimization: Training and offline mask generation can run simultaneously because random-mask generation is independent of local-model computation.The design uses parallel processes; multi-threading performs relatively worse because of Python GIL.
  • Runtime Optimization: Tensor-aware sending and receiving queues accelerate devices that transmit and receive masks concurrently.The design parallelizes transmission and reception of chunked masks and uses PyTorch RPC for tensor-aware communication.
  • Evaluation System: The evaluation covers four image-classification tasks spanning low- and high-resolution datasets and shallow and larger edge-oriented models.The models include LR, CNN, MobileNetV3, and EfficientNet-B0.

7 EXPERIMENTAL RESULTS

Experiments evaluate LightSecAgg against SecAgg and SecAgg+ across models, datasets, user counts, dropout rates, bandwidths, and asynchronous training. LightSecAgg reduces server-side aggregation cost and substantially improves total running time while achieving comparable asynchronous convergence.

  • 7.1 Setup: Experiments compare LightSecAgg with SecAgg and SecAgg+ using four models across computer-vision datasets and up to 200 users.The evaluation measures one-round total running time, including local model training and secure aggregation.
  • 7.2 Performance Comparison: SecAgg and SecAgg+ running time increases monotonically with dropout rate because server-side mask recovery grows quadratically with the number of users.The experiments consider dropout rates p = 0.1, 0.3, and 0.5.
  • 7.3 Impact of Model Size: LightSecAgg’s speedup remains substantial across model sizes, but is smallest for EfficientNet-B0 on GLD-23K because training dominates total time.Aggregation-only comparisons retain the speedup across model sizes.
  • 7.4 Convergence Performance in Asynchronous FL: For asynchronous FL on CIFAR-10, LightSecAgg has almost the same convergence performance as FedBuff while adding quantization noise for privacy.The comparison uses Constant and Poly staleness-mitigation strategies, with Poly configured as α = 1.

8 CONCLUSION AND FUTURE WORKS

The paper concludes that LightSecAgg supports secure aggregation in synchronous and asynchronous FL while preserving privacy and dropout resilience and reducing aggregation overhead. It also identifies Byzantine-robust privacy-preserving aggregation as future work.

  • Conclusion: LightSecAgg provides the same privacy and dropout-resiliency guarantees as state-of-the-art protocols while reducing model-aggregation overhead.Its design uses one-shot aggregate-mask reconstruction of surviving users.
  • Conclusion: LightSecAgg is designed for secure aggregation in both synchronous and asynchronous federated learning.The conclusion presents this as the paper’s scope.
  • Future Work: Future work would combine LightSecAgg with Byzantine-robust aggregation protocols while maintaining privacy.The stated threat model in this work is honest but curious.

B PROOF OF THEOREM 1

The proof establishes that LightSecAgg simultaneously provides privacy against colluding users and dropout resilience by decoding aggregate masks from sufficiently many surviving-user messages.

  • Guarantees: For T + D < N, the protocol selects U such that T < U ≤ N − D to target both privacy and dropout resilience.The same parameters support privacy against up to T colluding users and resilience against up to D dropped users.
  • Dropout-resiliency guarantee: At least U surviving users remain after up to D dropouts, enabling recovery of aggregate mask sub-masks through MDS decoding.Any set of U surviving-user messages suffices for decoding.
  • Dropout-resiliency guarantee: Recovering the aggregate mask allows the server to recover the aggregate model for the surviving users.The proof proceeds from decoded sub-masks to the summed masks and then to the aggregate model.
  • Privacy guarantee: The privacy proof assumes jointly uniform random sub-masks and analyzes an arbitrary colluding set of T users.The guarantee is proved for one round and extends across rounds because randomness is independently generated each round.

C DISCUSSION

LightSecAgg improves scalability by replacing repeated mask recovery with aggregate-mask decoding, reducing server computation while retaining stated privacy and dropout-resiliency guarantees. Its offline randomness and storage also scale linearly with the number of users rather than exponentially in a compared scheme.

  • Computational Complexity: For U = O(N), LightSecAgg has server aggregation complexity O(d log N), compared with SecAgg’s O(N^2d).The lower complexity targets the server bottleneck that dominates SecAgg execution time for large user populations.
  • Computational Complexity: LightSecAgg significantly improves server computational efficiency during aggregation compared with SecAgg.The comparison attributes the improvement to aggregate-mask reconstruction rather than reconstructing each dropped user’s masks.
  • Storage and Randomness: LightSecAgg’s required offline randomness and per-user storage scale linearly with N, unlike the compared scheme’s exponential scaling.LightSecAgg generates randomness locally and does not require a trusted third party.
  • Guarantees: LightSecAgg retains worst-case privacy and dropout resilience under T + D < N, whereas SecAgg+ provides probabilistic guarantees.LightSecAgg selects T, D, and U subject to T < U ≤ N − D.

D EXPERIMENTAL DETAILS

This section reports runtime experiments comparing LightSecAgg with SecAgg and SecAgg+ across models, datasets, user counts, and dropout rates.

  • Experimental setup: The experiments compare LightSecAgg, SecAgg, and SecAgg+ while increasing the number of users under various dropout rates.The evaluated tasks include logistic regression on MNIST, MobileNetV3 on CIFAR-10, and EfficientNet-B0 on GLD23k.
  • Experimental setup: The reported runtime measure is the total running time for training and secure aggregation.

E PROOF OF LEMMA 1

This section develops LightSecAgg’s asynchronous extension and its convergence analysis, covering finite-field quantization, mask recovery, privacy, and dropout resilience.

  • Asynchronous FL: Asynchronous FL buffers local updates and applies a global update when the buffer reaches size K.The paper focuses on buffered asynchronous FL in FedBuff, while noting that the approach can generalize to varying group sizes.
  • Asynchronous FL: Conventional pairwise masks may not cancel asynchronously because users have mismatched staleness values.The identity of each user’s staleness is not known beforehand, preventing reuse of the same pairwise random seeds.
  • Asynchronous LightSecAgg: LightSecAgg encodes local masks so the server can recover the aggregate mask in one computation even when masks originate in different rounds.In asynchronous operation, encoded masks carry timestamps, and coding commutes with addition.
  • Quantization and masking: Local updates are quantized into a finite field because masking and MDS encoding operate there for information-theoretic privacy.The field size q is assumed large enough to avoid wrap-around during secure aggregation.
  • Quantization and masking: Stochastic quantization is unbiased, and increasing c reduces its variance.The method combines stochastic rounding with two’s-complement representation to support finite-field computation.
  • Convergence analysis: Quantized staleness compensates asynchronous updates while keeping the computation in the finite field.The original staleness value is real-valued, so the paper introduces a finite-field quantized staleness function.
  • Convergence analysis: Theorem 2 establishes an ergodic convergence rate for asynchronous LightSecAgg under constant learning rates satisfying ηlηgKE ≤1.The quantized gradient estimator remains unbiased with bounded variance, while quantization adds variance relative to FedBuff.

F.5 Experiments for Asynchronous LightSecAgg

This section evaluates asynchronous LightSecAgg against FedBuff on MNIST and CIFAR-10, examining weighting strategies, staleness compensation, and quantization levels.

  • Experimental setup: The asynchronous experiments use MNIST with LeNet and CIFAR-10 with a CNN, with N = 100 users and buffer size K = 10.User staleness is uniformly distributed over [0, 10], and the field size is q = 2^32 − 5.
  • Experimental setup: LightSecAgg and FedBuff are evaluated using constant and polynomial staleness-weighting functions.The polynomial function is sα(τ) = (1 + τ)^−α with α = 1.
  • Empirical results: LightSecAgg achieves almost the same performance as FedBuff on both MNIST and CIFAR-10 despite adding quantization noise for privacy.The experiments attribute the similar performance to negligible quantization noise.
  • Empirical results: Using cg = 2^6 gives quantized staleness compensation with the same performance in mitigating staleness as the original real-valued function.
  • Quantization levels: The quantization parameter cl = 2^16 performs best, while smaller or larger values perform poorly because rounding and wrap-around errors trade off.
Loading 2109.14236v3…