PaperlayerView exact revision on arXiv

Source-linked AI summary

MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients

Xiaoyu Cao, Neil Zhenqiang Gong

arXiv:2203.08669v2cs.CRcs.LG

TL;DR

Existing untargeted model poisoning attacks assume access to many compromised genuine clients, an assumption the paper considers unrealistic in production FL. MPAF injects fake clients that use only global models to steer training toward a low-accuracy base model, and it remains effective against classical defenses and norm clipping. The paper concludes that more advanced defenses are needed for fake-client poisoning attacks.

  • Problem

    Existing model poisoning attacks assume many compromised genuine clients, but that assumption is unrealistic for production federated learning systems involving millions of clients.

  • Method

    MPAF injects fake clients that craft and scale local model updates toward an attacker-chosen low-accuracy base model using only received global models.

  • Results

    MPAF significantly decreases global-model test accuracy across multiple datasets and FL methods, including a 32% decrease with Trimmed-mean when 10% fake clients are injected on Purchase.

  • Takeaways & Limitations

    MPAF remains effective despite classical defenses and norm clipping, highlighting the need for more advanced defenses against fake-client model poisoning.

  • Takeaways & Limitations

    Extending MPAF to targeted poisoning and improving it with extra knowledge, such as training data or models from similar tasks, remain future work.

Abstract

from arXiv · show

Existing model poisoning attacks to federated learning assume that an attacker has access to a large fraction of compromised genuine clients. However, such assumption is not realistic in production federated learning systems that involve millions of clients. In this work, we propose the first Model Poisoning Attack based on Fake clients called MPAF. Specifically, we assume the attacker injects fake clients to a federated learning system and sends carefully crafted fake local model updates to the cloud server during training, such that the learnt global model has low accuracy for many indiscriminate test inputs. Towards this goal, our attack drags the global model towards an attacker-chosen base model that has low accuracy. Specifically, in each round of federated learning, the fake clients craft fake local model updates that point to the base model and scale them up to amplify their impact before sending them to the cloud server. Our experiments show that MPAF can significantly decrease the test accuracy of the global model, even if classical defenses and norm clipping are adopted, highlighting the need for more advanced defenses.

1. Introduction

Federated learning is vulnerable to untargeted model poisoning, but existing attacks assume many compromised genuine clients. MPAF instead uses cheaply injected fake clients with no extra system knowledge to steer training toward a low-accuracy base model.

  • Motivation: Existing untargeted attacks require a large fraction of compromised genuine clients and become less effective when that fraction is small.This requirement is considered unrealistic for production FL systems involving millions of clients.
  • Contribution: MPAF is the first model poisoning attack to FL based on fake clients.The attack addresses the cost and practicality limits of compromising genuine clients.
  • Contribution: Fake clients can be emulated cheaply using open-source projects or Android emulators, reducing the cost of injecting many clients.The attacker can run multiple emulator instances on their own machines.
  • Method: MPAF crafts fake local model updates using only received global models, without genuine local training data or other extra system knowledge.The attack chooses a low-accuracy base model and points scaled fake updates from the current global model toward it.
  • Evaluation: MPAF remains effective across multiple datasets and FL methods even when classical defenses and norm clipping are used.The experiments evaluate the attack against these countermeasures.

2. Related Work

Related work establishes how FL aggregates updates, categorizes poisoning attacks, and motivates MPAF by showing that existing untargeted attacks depend on compromised genuine clients. Classical robust aggregation and existing security guarantees do not fully address fake-client attacks.

  • Federated Learning: In each FL round, the server sends a global model to clients, which fine-tune local models and return updates for aggregation.The global model is then updated using the aggregated global model update.
  • Aggregation Rules: FedAvg averages local updates, while Median takes coordinate-wise medians and Trimmed-mean removes extreme coordinate values before averaging.These aggregation rules differ in their treatment of adversarial outliers.
  • Defenses: Trimmed-mean trades adversarial robustness against non-adversarial test accuracy, and experiments assume its trimming parameter equals the number of fake clients.This assumption represents a strong defender that knows the fake-client count.
  • Model Poisoning Attacks: Untargeted poisoning decreases global-model test accuracy, whereas targeted poisoning seeks attacker-chosen labels for selected inputs.Existing attacks described here rely on compromised genuine clients.
  • Model Poisoning Attacks: Existing untargeted attacks require many compromised genuine clients and may be unaffordable in production FL.Their malicious updates are computed from genuine local training data and then perturbed.
  • Defenses: Existing robustness claims for FedAvg and classical defenses against compromised-genuine-client attacks do not extend to fake-client attacks.The paper argues that attackers can inject many fake clients and use MPAF to degrade the global model.

3. Threat Model

The threat model gives the attacker control over many fake clients that can send arbitrary updates, while assuming no knowledge beyond received global models. Injecting fake clients is presented as cheaper than compromising genuine devices.

  • Attacker Goal: The attacker’s objective is to decrease the learnt global model’s test accuracy.A larger accuracy difference between attacked and unattacked models indicates a stronger attack.
  • Attacker Capability: The attacker can inject many fake clients and control them to send arbitrary fake local model updates.This capability distinguishes the threat model from attacks requiring compromised genuine clients.
  • Attacker Capability: Compromising genuine clients can require bypassing anti-malware protections or buying costly zombie devices.The cost increases with the number of genuine clients compromised.
  • Attacker Capability: Fake clients can be emulated cheaply with open-source projects or Android emulators supporting multiple instances.A single machine can emulate many client instances.
  • Knowledge Assumption: The attacker is assumed to know no FL-system information beyond the global models received during training.Fake clients generally have no genuine local training data.

4. Our Attack

MPAF addresses limited-knowledge fake-client poisoning by steering the global model toward a low-accuracy attacker-chosen base model. Baseline attacks use random noise or reversed historical updates, whereas MPAF maintains a consistent direction across rounds.

  • Baseline attacks: The random attack sends scaled Gaussian noise as fake local model updates.Each fake client uses g_i^t = −λε, where ε is sampled from N(0, I).
  • Baseline attacks: The history attack reverses the estimated benign global update from the previous round and scales it by λ.It sends g_i^t = −λ(w_t − w_{t−1}) without requiring the exact global learning rate.
  • Baseline attacks: The baseline attacks may cancel across rounds because their deviation directions are inconsistent.This limits their impact when classical defenses such as Trimmed-mean are applied.
  • MPAF: MPAF chooses a same-architecture base model with low test accuracy and uses fake updates to drag the global model toward it.A randomly initialized model can serve as the base model.
  • MPAF: MPAF formulates the attack over the entire training process, keeping every round directed toward the same attacker-chosen base model.The optimization minimizes the distance between the final global model and the base model using the ℓ2 norm.
  • MPAF: In each round, fake clients compute d = w′ − w_t and scale it by λ before aggregation.A sufficiently large λ is intended to preserve effectiveness after combining fake and genuine updates.

5. Evaluation

The evaluation measures untargeted poisoning using test accuracy across MNIST, Fashion-MNIST, and Purchase under varying attack and training settings. MPAF remains effective against classical defenses, client sampling, and unknown learning-rate settings.

  • Datasets and settings: The evaluation uses MNIST, Fashion-MNIST, and Purchase with image and tabular global-model architectures.MNIST and Fashion-MNIST use CNNs, while Purchase uses a fully connected network.
  • FL and attack settings: The default setup has 1,000 genuine clients, 100 fake clients, full client sampling, and 200 FL rounds.The fake-client fraction is defined as m/n, with a default value of 10%.
  • Evaluation metric: Test accuracy is the metric for untargeted attacks, with lower accuracy indicating a stronger attack.The experiments repeat each attack 20 times with different random seeds and report averages.
  • Impact of fake-client fraction: With FedAvg, random attack, history attack, and MPAF reduce global-model accuracy to random guessing using only 1% fake clients.This observation is reported across the three datasets in Figure 2.
  • Impact of fake-client fraction: 32%: MPAF reduces Purchase accuracy under Trimmed-mean with 10% fake clients, versus at most 4% for baseline attacks.The reduction increases from 32% to 49% when the fake-client fraction rises from 10% to 25%.
  • Impact of sample rate: MPAF significantly decreases Trimmed-mean accuracy when the sample rate β ranges from 0.01 to 1.00.The paper reports that β has little impact on MPAF.
  • Impact of scaling factor: As λ increases, test accuracy first decreases and then remains almost unchanged; λ ≥ 1 is sufficient for significant reduction in these experiments.This effectiveness does not require knowing FL hyperparameters such as the global learning rate η.

6. Norm Clipping as A Countermeasure

Norm clipping limits the impact of MPAF updates but does not eliminate the attack. Lower clipping bounds also reduce benign-update quality, creating a trade-off between attack mitigation and clean-model accuracy.

  • Norm clipping mechanism: Norm clipping bounds local model updates by scaling any update with ℓ2-norm above M down to norm M.Updates at or below M remain unchanged, limiting the largest clipped-update norm to M.
  • Attack effectiveness: MPAF remains effective against Trimmed-mean on Fashion-MNIST when norm clipping is deployed.Figure 5 evaluates the global model’s test accuracy under different clipping bounds.
  • Attack effectiveness: 0.17 accuracy loss separates the no-attack maximum of 0.85 at M →∞ from MPAF’s maximum of 0.68 around M = 100.The comparison uses test accuracy of the global model learnt by Trimmed-mean on Fashion-MNIST.
  • Accuracy trade-off: As M decreases, the accuracy gap between MPAF and no attack narrows because more fake updates are clipped.The mitigation comes with reduced clean accuracy because benign updates are clipped too, especially for M < 100.

7. Conclusion and Discussion

The paper concludes that MPAF enables model poisoning through fake clients under a minimum-knowledge setting and remains effective with classical defenses and norm clipping. It also identifies extensions involving extra knowledge, global optimization, and targeted behavior.

  • Conclusion: MPAF is presented as the first model poisoning attack to federated learning based on fake clients.The attacker can inject fake clients at lower cost than compromising genuine clients, according to the paper’s discussion.
  • Conclusion: The attack remains effective when classical defenses and norm clipping are applied, motivating more advanced defenses against fake-client poisoning.This conclusion is stated for the paper’s minimum-knowledge attacker setting.
  • Future directions: Future work could improve MPAF using extra knowledge such as training data or a model from a similar learning task.The discussion frames this as an extension because compromising a large fraction of genuine clients is unrealistic.
  • Optimization formulation: MPAF formulates a global optimization problem toward a fixed base model, avoiding possible cross-round cancellation from independent round-wise attack problems.The paper contrasts this with existing attacks based on compromised genuine clients.
  • Future directions: Targeted model poisoning is proposed as future work by choosing a base model with attacker-desired behavior, such as a backdoored base model.The paper states that forcing the global model toward such a base model may reproduce its backdoor behavior.
Loading 2203.08669v2…