PaperlayerView exact revision on arXiv

Source-linked AI summary

RoFL: Robustness of Secure Federated Learning

Hidde Lycklama, Lukas Burkhalter, Alexander Viand, Nicolas Küchler, Anwar Hithnawi

arXiv:2107.03311v4cs.CRcs.LG

TL;DR

Targeted attacks threaten FL integrity, while their mechanisms and practical defenses compatible with secure aggregation remain insufficiently understood. The paper analyzes attack mechanisms and introduces RoFL, which adds privacy-preserving validation to secure aggregation; norm bounds reduce some attack risks, though memorization-based attacks remain challenging.

  • Problem

    Targeted attacks threaten FL integrity, while the mechanisms enabling them and practical defenses compatible with secure aggregation remain insufficiently understood.

    Source: §1 · pp. 1–2

  • Method

    The paper analyzes targeted-attack mechanisms and introduces RoFL, which extends secure aggregation with zero-knowledge validation of constraints on encrypted updates.

    Source: §1 · p. 2

  • Results

    Norm bounds significantly reduce secure FL's attack surface and prevent a class of single-client model-replacement attacks, but are limited against memorization-based attacks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Methods used to limit memorization, including regularization and differential privacy, are likely to improve robustness for tail targets.

    Source: §2.5 · p. 7

    The paper's analysis assumes fewer than 5% of clients are compromised per round; tradeoffs of regularization and noise across accuracy, privacy, robustness, and fairness are beyond its scope.

    Source: §2.1 · p. 3, §2.5 · p. 8

Abstract

from arXiv · show

Even though recent years have seen many attacks exposing severe vulnerabilities in Federated Learning (FL), a holistic understanding of what enables these attacks and how they can be mitigated effectively is still lacking. In this work, we demystify the inner workings of existing (targeted) attacks. We provide new insights into why these attacks are possible and why a definitive solution to FL robustness is challenging. We show that the need for ML algorithms to memorize tail data has significant implications for FL integrity. This phenomenon has largely been studied in the context of privacy; our analysis sheds light on its implications for ML integrity. We show that certain classes of severe attacks can be mitigated effectively by enforcing constraints such as norm bounds on clients' updates. We investigate how to efficiently incorporate these constraints into secure FL protocols in the single-server setting. Based on this, we propose RoFL, a new secure FL system that extends secure aggregation with privacy-preserving input validation. Specifically, RoFL can enforce constraints such as $L_2$ and $L_\infty$ bounds on high-dimensional encrypted model updates.

1. Introduction

The introduction frames secure FL as protecting client updates while leaving model integrity exposed to malicious participants, and motivates defenses compatible with secure aggregation. It presents a robustness analysis and RoFL, which adds privacy-preserving validation for encrypted updates.

1. Introduction

  • Secure aggregation protects individual gradient updates, but submitted gradients can still leak sensitive information through data reconstruction.Source: §1 · p. 1
  • Targeted attacks threaten FL integrity by inserting backdoors that make the global model misbehave on specific inputs.Source: §1 · p. 1
  • Existing anomaly-detection and Byzantine-robust defenses become prohibitively expensive in secure FL because they require general-purpose secure computation.The paper motivates practical defenses compatible with existing secure aggregation protocols.Source: §1 · pp. 1–2
  • Norm bounds significantly reduce the attack surface and prevent practical single-client model-replacement attacks, but remain limited against attacks exploiting model memorization.The paper links the need to memorize tail subpopulations with implications for ML integrity.Source: §1 · p. 2
  • Memorization of tail subpopulations has implications for ML integrity, and norm bounds have limited effectiveness against attacks exploiting that memorization.Source: §1 · p. 2
  • RoFL extends secure aggregation with zero-knowledge proofs that let a single server enforce constraints such as L2 and L∞ bounds on high-dimensional encrypted updates.Its design uses efficient homomorphic commitments and range proofs compatible with masking-based secure aggregation.Source: §1 · p. 2

2. Analysis of FL Robustness

The analysis traces targeted FL attacks to client-controlled update manipulation and the rarity of their target subpopulations, then examines how norm bounds affect these attacks. It finds that bounds can limit many attacks, but continuous attacks on tail targets exploit the model’s learning and memorization behavior.

2. Analysis of FL Robustness

  • FL’s open participation, direct update manipulation, and attackers’ ability to adapt across rounds create targeted attack surfaces that norm bounds do not resolve uniformly.The analysis distinguishes frequent prototypical targets from rare tail subpopulations; attack effectiveness varies with target type.Source: §2 · p. 3

2.1. Attack Strategies

  • The analysis compares model poisoning, which permits arbitrary malicious updates, with data poisoning, which is limited to changing compromised clients’ training data.It considers projected-gradient, Neurotoxin, and Anticipate adaptive model-poisoning strategies, and assumes the attacker controls fewer than 5% of clients in a round.Source: §2.1 · pp. 3–4

2.2. Experimental Setup

  • Experiments use FMN and CIFAR-10 tasks with non-IID client data, testing prototypical and tail backdoors under single-shot or repeated attacker participation.The tasks use 3,383/100 clients and 30/40 selected per round for FMN/C10, respectively; the study measures main-task and backdoor accuracy.Source: §2.2 · p. 4

2.3. Attacks on Prototypical Targets

  • An appropriate median-based bound with r = 1.5 prevents prototypical attacks without slowing main-task convergence, while overly tight bounds slow convergence.Single-shot scaling attacks are prevented by L2 bounds of 4.0 for FMN and 5.0 for C10; bound effectiveness depends on choosing a suitable value.Source: §2.3 · pp. 4–5, Figure 4 · p. 5
  • As more clients are compromised, prototypical backdoors become more effective despite norm bounds, and data poisoning approaches model poisoning’s effectiveness.For FMN, malicious-objective success is above 50% at α = 26.7%; for C10, it exceeds 50% at α = 50%.Source: Figure 7 · p. 6, §2.3 · p. 6

2.4. Attacks on Tail Targets

  • With norm bounds, continuous tail backdoors reach 50% accuracy only after 75 FMN rounds or 500 C10 rounds, while prototypical backdoor success stays below 10%.A single-shot tail backdoor persists at least 200 FMN rounds; adding an artificial trigger to prototypical targets improves backdoor accuracy by more than 85% for FMN and 25% for C10.Source: §2.4 · pp. 6–7, Figure 8 · p. 6, Figure 9 · p. 7

2.5. Discussion

  • Norm bounds offer practical robustness, but tail attacks remain possible and defenses that limit memorization may introduce accuracy, privacy, and fairness tradeoffs.The authors note that persistent attacks require an attacker to compromise a significant portion of clients (>1%), and that defenses against every possible attack may be unattainable.Source: §2.5 · pp. 7–8, §2.4 · p. 7
  • Enforcing bounds on private updates in secure aggregation is non-trivial: client self-regulation is unreliable against malicious clients, while generic secure computation is prohibitively expensive.Figure 11 depicts RoFL augmenting secure FL with proof and verification steps to check update constraints.Source: §2.5 · p. 8, Figure 11 · p. 8

3. RoFL Design

RoFL extends secure aggregation so a server can validate constraints on clients’ secret updates. It combines homomorphic commitments and zero-knowledge proofs with optimizations for federated-learning workloads.

3.1. Preliminaries

  • Commitments hide values, while zero-knowledge proofs let a prover establish a fact without revealing additional information.The paper uses these cryptographic building blocks to validate secret client inputs.Source: §3.1 · p. 8

3.2. Setup and Threat Model

  • RoFL targets a single-server deployment and verifies that each client update satisfies a predefined constraint while preserving secure aggregation’s input-privacy guarantees.It provides L2 and L∞ bounds and correctness against malicious clients when the server is semi-honest.Source: §3.2 · p. 8

3.3. Secure Aggregation with Commitments

  • RoFL adapts secure aggregation’s homomorphic masking with commitments, enabling efficient checks that the unmasking key is correct and detection of malformed aggregations.ElGamal commitments support the required homomorphisms and can be paired with zero-knowledge range proofs.Source: §3.3 · p. 9

3.4. ZKP of Norm Bounds

  • Clients prove L∞ bounds with per-parameter range checks; L2 proofs additionally commit to squared parameters and prove their sum is bounded.Bulletproofs aggregate range proofs, reducing proof bandwidth from linear to logarithmic in the number of parameters.Source: §3.4 · pp. 10–11

3.5. Optimizations

  • Probabilistic range checks reduce L∞ proof work, random-subspace compression reduces L2 update size, and optimistic continuation overlaps verification with next-round training.Optimistic continuation can require a one-round rollback if verification fails; probabilistic checking cannot be applied to L2 bounds because one out-of-bound value can enable overflow.Source: §3.5 · pp. 11–12

4. Evaluation

RoFL adds measurable computation and communication costs to secure aggregation, but its optimizations substantially reduce end-to-end overhead. The evaluation reports practical training performance and speedups from overlapping proof verification with training.

4.1. Input Validation with ZKP

  • 157s (L2) and 154s (L∞) per 262k-parameter proof on strong clients rose to 514s and 477s on less powerful machines.Proof creation costs were within an order of magnitude of model training cost.Source: Figure 12 · p. 13, §4.1 · p. 13
  • 75MB (72x plaintext) for L2 and 51MB (48x) for L∞ were transmitted per client for 262k parameters.Probabilistic checking reduced L∞ computation cost 6x–17x for clients and 6x for the server; the reported 262k-parameter example fell from 154s to 9s on a large client and from 19s to 3s per client server-side.Source: Figure 12 · p. 13, §4.1 · p. 13

4.2. RoFL End-to-End Performance

  • 13x from 63x, 29x from 153x, and 10x from 106x: optimized L2 training reduced per-round computation overhead versus secure aggregation on MNIST, CIFAR-10 S, and CIFAR-10 L.The L2 variant uses random-subspace learning; accuracy reductions from quantization are less than 1% overall.Source: Table 4 · p. 14, §4.2 · p. 14
  • 39% for CIFAR-10 L and 18% for Shakespeare: optimistic continuation sped training by overlapping proof verification with next-round training and proof creation.For Shakespeare, verification added no wall-clock time because the next round’s training and proof creation took longer.Source: Figure 14 · p. 14, §4.2 · p. 14

5. Related Work

RoFL addresses integrity checks that prior secure aggregation systems did not provide, while other privacy-preserving approaches impose different trust or performance trade-offs.

Secure Data Collection

  • Existing secure aggregation protocols protect inputs but do not address integrity; RoFL’s underlying protocol is tailored to FL and does not assume a trusted setup.Kursawe et al. provide correct-execution guarantees without constraining client inputs, while Canny et al. support L2 checks but have overheads described as unsuitable for FL.Source: §5 · p. 15
  • Non-colluding-server protocols can enable efficient input checks, but their trust model remains difficult to deploy in practice.The authors report that organizations have struggled to deploy this model, including with an external non-profit organization.Source: §5 · p. 15

FL with Differential Privacy

  • Local differential privacy requires substantial noise to hide client updates and can significantly reduce utility; RoFL can compose with differential privacy for output privacy.The paper describes perturbation-based differential privacy as an orthogonal feature that may be combined with RoFL.Source: §5 · p. 15

Appendix A.

The appendix compares untargeted and model-poisoning attacks under different norm bounds and attacker fractions, and reports an L∞ bound that defends against prototypical backdoors.

Untargeted attacks

  • Less than 6% untargeted attack impact under tight bounds contrasts with targeted attacks becoming successful at a lower median-based bound.Untargeted attacks start becoming effective at r-M >= 30, whereas targeted attacks already have significant success at r-M >= 15.Source: Figure 15 · p. 19, Appendix A · p. 19

Comparison of model poisoning attacks

  • At M-r = 1.5, none of MP-AT, MP-PD, and MP-NT succeeds; at M-r = 15, MP-PD leads for FMN and MP-NT for C10.The comparison uses adaptive norm bounds.Source: Figure 16 · p. 19, Appendix A · p. 19

Growing number of attackers

  • MP-AT improves malicious accuracy over data poisoning by at most 40% as the compromised-client fraction varies.The compared attacks show similar performance relative to the percentage of compromised clients per round.Source: Figure 17 · p. 19, Appendix A · p. 19

L∞-norm bound

  • For an L2-norm bound, setting the bound too tightly impacts model convergence, whereas setting it too loosely allows the attacker to successfully inject the prototypical backdoor into the model.Source: Appendix A · p. 19

Appendix B.

Appendix B analyzes the failure probability of random parameter checking for L∞ constraints, then tests whether the assumed fraction of above-bound parameters is plausible for poisoning attacks. The experiments show that probabilistic checking detects attacks as the checked fraction grows, while top-K attacks lose effectiveness when few parameters are scaled.

Appendix B.

  • The server's miss probability is the chance that every uniformly sampled parameter lies below the bound, modeled as a hypergeometric draw without replacement.The malicious update has length ℓ, with a fraction pv of parameters above B; the server checks a fraction pc.Source: Appendix B · p. 19
  • In generic FMN-P attacks, 81.8% of malicious-update parameters exceed the 0.01 bound, versus 0.87% for benign updates; continuous attacks exceed 47% per round.The plotted setting assumes pv = 0.005 and includes 44,426 weights.Source: Appendix B · p. 20, Figure 20 · p. 20
  • The security setting depends on assumed fractions pv and pc, with pc chosen by the server and pv treated as a fixed minimum.Figure 19b relates the number of checked parameters to the assumed pv and failure probability δ.Source: Appendix B · p. 20, Figure 19 · p. 20
  • Top-K attacks are always successful for K ≥10%, fall below 50% success at K = 5%, and reach 0% success for K ≤2%.The FMN-P adaptive attack still requires more than 2% of parameters above the bound; the assumed pv = 0.005 is below observed generic and adaptive attack fractions.Source: Appendix B · p. 20
  • Detection rates for top-K updates quickly converge to 1.0 as the server checks a larger fraction pc of parameters.The analysis uses pv = 0.005 as a conservative estimate and sets the failure probability to 1e-8.Source: Appendix B · p. 20, Figure 19 · p. 20

C.1. Analysis Setup

The analysis uses non-IID FMN and CIFAR-10 tasks with prototypical and tail backdoors. It compares data-poisoning and norm-adaptive model-poisoning attacks under specified training and attack configurations.

C.1. Analysis Setup

  • The analysis uses FMN digit classification across 3,383 clients, with 30 selected each round, and CIFAR-10 across 100 clients, with 40 selected each round and Dirichlet non-IID allocation.FMN data are grouped by writer; CIFAR-10 uses Dirichlet parameter 0.9 in all dimensions.Source: §C.1 · pp. 20–21
  • Backdoor tasks include FMN prototypical sevens-to-ones and European-style tail sevens, plus CIFAR-10 green-cars-to-birds and Southwest-airplane tail attacks.The target sets contain 310, 660, 30, and 784 images, respectively.Source: §C.1 · p. 21
  • Attack success is measured on held-out backdoor images, with three unseen C10-P samples expanded into 200 test variants each.For FMN-P, success is malicious classification accuracy; tail attacks use their designated test sets.Source: §C.1 · p. 21
  • Model-poisoning adversaries use PGD to project updates into an Lp norm constraint after SGD steps, with final scaling applied after each epoch.The attack projects to a slightly larger space during the epoch before using γ to satisfy the bound.Source: §C.1 · p. 21
  • The Neurotoxin configuration masks frequently updated weights at ratio k = 5, while Anticipate uses k = 9 steps on C10 and k = 5 on FMN.The Anticipate settings follow the original attack as closely as possible.Source: §C.1 · p. 22

C.2. RoFL Evaluation Setup

The end-to-end evaluation benchmarks four models with 48 clients selected in every round. Client updates use 8-bit probabilistic quantization with 7 fractional bits for encryption.

C.2. RoFL Evaluation Setup

  • End-to-end tests cover LeNet5 and ResNet-20 on CIFAR-10, a 19k-parameter MNIST CNN, and an 818k-parameter Shakespeare LSTM.The evaluation increases deployment size to 48 clients, all selected in each training round.Source: §C.2 · p. 22
  • RoFL applies 8-bit probabilistic quantization with 7 fractional bits to client updates for encryption.The reported microbenchmarks average four measurements after one warm-up mock execution.Source: §C.2 · p. 22

C.2.1. Model Architectures.

The paper evaluates CNN, ResNet-20, and LSTM architectures across its analysis and end-to-end experiments.

C.2.1. Model Architectures.

  • The MNIST CNN has two convolution layers, max pooling, a 32-unit fully connected layer, and 19k trainable parameters.The CIFAR-10 S variant changes the input dimensions and channel count.Source: §C.2.1 · p. 22
  • ResNet-20 experiments use the Keras CIFAR-10 example implementation, corresponding to ResNet20v1.It is used in FMN analysis and CIFAR-10 L evaluation.Source: §C.2.1 · p. 22
  • The LSTM follows LEAF and the original federated-learning paper, with an 8-size embedding and two stacked layers of 256 hidden units.Its dense output width is 80, matching the vocabulary size.Source: §C.2.1 · p. 22
Loading 2107.03311v4…