PaperlayerView exact revision on arXiv

Source-linked AI summary

Local and Central Differential Privacy for Robustness and Privacy in Federated Learning

Mohammad Naseri, Jamie Hayes, Emiliano De Cristofaro

arXiv:2009.03561v5cs.CRcs.AI

TL;DR

Federated Learning keeps data local but remains exposed to backdoor, membership, and property inference attacks. This paper evaluates Local and Central Differential Privacy as defenses, finding protection against backdoor and white-box membership inference attacks, but not property inference, under utility trade-offs.

  • Problem

    Prior work had not jointly evaluated defenses for robustness and privacy in FL, including white-box membership inference attacks.

  • Method

    The paper conducts an extensive measurement study of LDP and CDP for mitigating backdoor and inference attacks in FL while measuring utility.

  • Results

    Both LDP and CDP defend against backdoor and white-box membership inference attacks, whereas neither provides effective property-inference protection without substantial utility trade-offs.

  • Takeaways & Limitations

    DP offers a reusable way to quantify robustness, privacy, and utility trade-offs in FL, but its protection depends on the attack and deployment setting.

  • Takeaways & Limitations

    The robustness evaluation covers only backdoor attacks, while membership and property-inference evaluations use the datasets and tasks from prior studies.

Abstract

from arXiv · show

Federated Learning (FL) allows multiple participants to train machine learning models collaboratively by keeping their datasets local while only exchanging model updates. Alas, this is not necessarily free from privacy and robustness vulnerabilities, e.g., via membership, property, and backdoor attacks. This paper investigates whether and to what extent one can use differential Privacy (DP) to protect both privacy and robustness in FL. To this end, we present a first-of-its-kind evaluation of Local and Central Differential Privacy (LDP/CDP) techniques in FL, assessing their feasibility and effectiveness. Our experiments show that both DP variants do d fend against backdoor attacks, albeit with varying levels of protection-utility trade-offs, but anyway more effectively than other robustness defenses. DP also mitigates white-box membership inference attacks in FL, and our work is the first to show it empirically. Neither LDP nor CDP, however, defend against property inference. Overall, our work provides a comprehensive, re-usable measurement methodology to quantify the trade-offs between robustness/privacy and utility in differentially private FL.

1 Introduction

The paper examines whether Local and Central Differential Privacy can jointly mitigate robustness and privacy attacks in Federated Learning while preserving utility. Experiments find protection against backdoor and white-box membership inference attacks, but not property inference, with trade-offs depending on the DP variant and setting.

  • Motivation: FL keeps participant datasets local while exchanging model parameters, but remains vulnerable to poisoning, membership, and property inference attacks.Backdoor attacks can corrupt model behavior on specific sub-tasks.
  • Research question: The paper asks whether LDP and CDP can mitigate both backdoor and inference attacks and what utility trade-offs result.It evaluates different application scenarios through an extensive measurement study.
  • Findings: Both LDP and CDP defend against backdoor attacks, with protection and utility trade-offs that are better overall than prior defenses.With 2,400 EMNIST participants, attack accuracy fell from 88% to 10% with LDP and 6% with CDP at ϵ = 3; norm bounding and weak DP reached 37% and 16%.
  • Findings: Both LDP and CDP protect against white-box membership inference without necessarily imposing a high utility cost.On CIFAR100, active attack accuracy fell from 75% to 55% with LDP and 52% with CDP.
  • Findings: LDP does not protect against property inference, while CDP can require significant utility loss to provide theoretical protection.The paper also warns that applying LDP only to non-attackers can boost backdoor attack accuracy.
  • Contributions: The work contributes a reusable framework for measuring robustness, privacy, and utility trade-offs in differentially private FL.It reports first-of-kind evaluations of LDP and CDP for these combined goals.

2 Preliminaries

The preliminaries introduce Federated Learning, Differential Privacy, and the local and central DP mechanisms used in FL. They explain how local noise protects records while server-side perturbation provides participant-level privacy, each with a privacy-utility trade-off.

  • Federated Learning: FL trains models across N participants that keep datasets private, send local parameter updates, and receive aggregated global models over multiple rounds.At each round, K participants are selected and the server averages their updates.
  • Differential Privacy: Differential Privacy bounds the influence of one record by randomizing algorithm outputs with noise.The privacy budget ϵ measures privacy loss, while δ allows a small probability that the bound does not hold.
  • Differential Privacy: DP-SGD learns private model parameters by bounding gradients and adding noise to stochastic gradient descent.The paper uses DP-SGD for local training and tracks spent privacy with a moments accountant.
  • Local Differential Privacy: With LDP, each participant perturbs its update locally before sending it to the server, protecting individual records according to ϵ.The local mechanism is applied through participant-side DP-SGD.
  • Central Differential Privacy: With CDP, the server clips participant updates, aggregates them, and adds Gaussian noise, providing participant-level DP.CDP requires participants to trust the server with updates and correct perturbation.

3 Defending Against Backdoor Attacks in FL

The paper evaluates LDP, CDP, norm bounding, and weak DP against targeted backdoor attacks in federated learning, measuring both backdoor and main-task accuracy. LDP and CDP mitigate attacks more effectively than norm bounding and weak DP, but their protection–utility trade-offs differ and CDP performs best overall.

  • Motivation: Backdoor attacks implant targeted behaviors in federated models through malicious clients, while data-dependent defenses can violate federated learning’s privacy premise.The experiments use single-pixel and semantic backdoors across EMNIST, CIFAR10, Reddit-comments, and Sentiment140.
  • Setting 1: Reproducing Sun et al. [103]: Around 88% backdoor accuracy remained after 300 rounds on EMNIST, while main-task accuracy fell only from 94% to 92% under one unconstrained attacker per round.The corresponding attack reached around 90% on CIFAR10 and around 95% on Sentiment140.
  • Setting 1: Reproducing Sun et al. [103]: Norm bounding reduced EMNIST backdoor accuracy from 37% to 7% with bound 3 versus 5, without affecting main-task accuracy, which remained around 90%.On CIFAR10, bound 10 reduced backdoor accuracy from 90% to 26% without affecting utility.
  • Setting 2: Increasing Number of Attackers: With 50 EMNIST attackers, weak DP reduced backdoor accuracy to 42% versus 47% for norm bounding, but reduced utility further to 84%.Increasing attackers generally improved backdoor accuracy and reduced utility, although utility declines alone were insufficient to identify attacks.
  • Setting 2: Increasing Number of Attackers: Applying LDP to all participants reduced EMNIST backdoor accuracy from 97% to 35% with 30 attackers, while main-task accuracy fell from around 96% to around 70%.Across both settings, LDP and CDP were more effective than norm bounding and weak DP, and CDP generally provided the best robustness–utility trade-off but required trust in the central server.

4 Defending against Inference Attacks in FL

The paper evaluates LDP and CDP against white-box membership and property inference attacks in FL, alongside existing robustness defenses. Both DP variants mitigate membership inference, whereas neither provides an effective property-inference defense at acceptable utility.

  • Attack scope: The evaluation covers white-box membership and property inference attacks, including active and passive attackers operating from server or participant sides.The experiments use CIFAR100, Purchase100, and Texas100 for membership inference, while property inference is evaluated on the datasets and tasks from prior work.
  • Baselines: Prior backdoor defenses based on norm bounding and weak DP do not protect privacy against inference attacks.The paper evaluates these defenses against inference attacks and reports that they do not defend privacy.
  • Property inference: Neither LDP nor CDP successfully defends against property inference at privacy budgets where the model converges.For LDP at ϵ = 10.7 and CDP at ϵ = 8.1, AUC does not significantly change compared with no defense; stronger CDP protection would incur unacceptable utility loss.

5 Related Work

Prior work studies separate robustness and privacy threats in FL, while this paper evaluates differential privacy as a joint defense against backdoor and inference attacks.

  • Prior FL defenses primarily use Byzantine-robust aggregation for robustness, while homomorphic encryption limits server access but does not eliminate aggregate leakage.
  • Backdoor attacks: Backdoor research focuses on targeted model-update poisoning, and prior defenses include norm bounding and Gaussian noise.
  • Privacy attacks: Privacy attacks in machine learning include membership inference, which detects whether target records were in training data, alongside model, input, and property inference.
  • Privacy attacks: In FL, white-box membership attacks can be active or passive and launched from server or participant sides during training.
  • Privacy attacks: Property inference can reveal sensitive attributes of other parties’ datasets or properties of subsets of training inputs.
  • Differential privacy: This paper applies CDP at client level and LDP at record level, evaluating both against backdoor and white-box membership attacks in FL.

6 Discussion & Conclusion

The paper evaluates LDP and CDP as joint defenses for robustness and privacy in FL, finding protection against backdoor and membership attacks but not property inference.

  • The study addresses a gap in prior work by evaluating defenses for both backdoor robustness and membership or property privacy attacks in FL.
  • LDP: LDP reduces backdoor and membership attack success, but its protection can impose a moderate utility cost.On EMNIST, LDP (ϵ = 3) reduces backdoor accuracy from 88% to 10% while utility falls from 92% to 62%.
  • LDP: LDP can reduce backdoor accuracy on Reddit-comments, but this defense assumes attackers cannot modify the algorithm running on their devices.With LDP (ϵ=1.7), task accuracies decrease from 83%, 78%, and 81% to around 45%, 43%, and 55%, while utility falls from 19% to 15%.
  • CDP: CDP provides a viable defense against backdoor and membership attacks and can outperform LDP on backdoor mitigation while preserving greater utility.On EMNIST, CDP (ϵ = 3) reduces backdoor accuracy from 88% to 6%, with utility decreasing from 90% to 78%.
  • Limitations: Neither LDP nor CDP strongly mitigates property inference when the number of participants is small.
  • Limitations: The privacy bounds of LDP and CDP cannot be directly compared because they capture different concepts.
  • Limitations: The evaluation covers only backdoor attacks for robustness and uses the datasets and tasks from prior membership and property-inference attacks.

A Hyperparameters

The appendix reports the hyperparameters used for LDP and CDP in the robustness and privacy experiments.

  • Tables 2 and 3 report hyperparameters for LDP and CDP in the robustness and privacy experiments, respectively.

B Property Inference Attack

The appendix describes the property-inference evaluation through main-task accuracy and attack AUC under no defense, LDP, and CDP.

  • Table 4 reports federated gender-classification main-task accuracy under no DP, LDP, and CDP.
  • Table 5 reports the property-inference attack AUC under no DP, LDP, and CDP.
  • The appendix lists LDP and CDP hyperparameter settings for the robustness and privacy experiments.
  • The reported comparison includes no defense alongside LDP and CDP conditions.
Loading 2009.03561v5…