PaperlayerView exact revision on arXiv

Source-linked AI summary

Backdoor Attacks on Federated Meta-Learning

Chien-Lun Chen, Leana Golubchik, Marco Paolieri

arXiv:2006.07026v2cs.LGcs.CRcs.DCstat.ML

TL;DR

Federated meta-learning can adapt to new classification tasks from few examples, but whether those adaptations resist backdoor attacks was unclear. The paper studies poisoning attacks and introduces a matching-network-inspired local decision defense. One-shot attacks can succeed and persist after benign training, while the defense reduces attack success in the reported settings.

  • Problem

    Whether federated meta-learning’s rapid adaptation prevents backdoor attacks from succeeding or persisting after benign updates and fine-tuning had not been investigated.

    Source: §1 · pp. 1–2

  • Method

    The paper investigates poisoning attacks in federated meta-learning and proposes a matching-network-inspired defense that predicts classes from feature similarity to labeled support examples.

    Source: §1 · p. 2

  • Results

    From as high as 90% to less than 20% on Omniglot training/validation and mini-ImageNet training, the defense reduces attack success; one-shot attacks persist after benign updates.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The local decision defense is compatible with secure aggregation of encrypted updates and does not require third-party examination of user updates.

    Source: §1 · p. 2

    The matching-network defense comes with some main-task accuracy reduction.

    Source: §5 · p. 6

Abstract

from arXiv · show

Federated learning allows multiple users to collaboratively train a shared classification model while preserving data privacy. This approach, where model updates are aggregated by a central server, was shown to be vulnerable to poisoning backdoor attacks: a malicious user can alter the shared model to arbitrarily classify specific inputs from a given class. In this paper, we analyze the effects of backdoor attacks on federated meta-learning, where users train a model that can be adapted to different sets of output classes using only a few examples. While the ability to adapt could, in principle, make federated learning frameworks more robust to backdoor attacks (when new training examples are benign), we find that even 1-shot~attacks can be very successful and persist after additional training. To address these vulnerabilities, we propose a defense mechanism inspired by matching networks, where the class of an input is predicted from the similarity of its features with a support set of labeled examples. By removing the decision logic from the model shared with the federation, success and persistence of backdoor attacks are greatly reduced.

1 Introduction

Federated meta-learning can adapt to new classification tasks from few examples and accommodate users with different data distributions, but its backdoor vulnerability had not been investigated. The paper reports successful, persistent attacks and proposes a matching-network-inspired defense.

1 Introduction

  • Federated learning aggregates users’ local model updates without sharing their private data, but malicious updates can implant backdoors that evade detection.Backdoors can succeed after a single malicious update while minimally reducing accuracy, and malicious updates can be masked among benign ones.Source: §1 · p. 1
  • Meta-learning aims to adapt model parameters to new classification tasks with different output classes using only a few examples, while allowing users with different data distributions to train jointly.This approach offers an alternative to federated learning’s shared-output-class requirement.Source: §1 · p. 1
  • The influence of backdoor attacks on federated meta-learning had not been investigated, leaving their success and persistence after benign updates or task-specific fine-tuning unclear.The paper examines whether adaptation can undo attacks when benign users share updates or fine-tune the meta-model.Source: §1 · p. 2
  • After one malicious update, backdoor examples are misclassified at high rates, and the attacks persist through benign meta-training or user fine-tuning.On Omniglot, 80% of backdoor examples are misclassified after one poisoned update with a 1% meta-testing accuracy reduction; benign fine-tuning reduces attack success by less than 10% on both datasets.Source: §1 · p. 2
  • A matching-network-inspired local decision mechanism reduces attack success from as high as 90% to below 20% in several settings.For mini-ImageNet validation, success falls from 50% to 20%; the defense uses similarities between input features and a support set of examples.Source: §1 · p. 2

2 Backdoors in Federated Meta-Learning

The section describes federated meta-training and a poisoning attack in which a participant trains on mislabeled backdoor examples and sends a boosted update. The attack requires the target class to appear in the task and backdoor-class inputs to be used.

2 Backdoors in Federated Meta-Learning

  • In each round, the server selects users, sends them the shared model, and applies received user updates to produce the next shared model.Each selected user initializes a local model from the shared model, performs training steps, and returns an update.Source: §2 · p. 2
  • Users meta-train with Reptile on batches of episodes, each containing N classes and K examples per class, to adapt the model to new classification tasks.Within each episode, the NK examples drive SGD steps; episode-trained models are averaged for the local update.Source: §2 · p. 2
  • Evaluation uses new episodes with unseen classes and K + 1 examples per class, fine-tuning on K examples before testing on the held-out example.The shared model is fine-tuned with a few SGD steps for each episode.Source: §2 · p. 2
  • An attacker poisons examples from backdoor classes by labeling them as the target class, then sends a boosted model update intended to persist after benign fine-tuning.The attacker uses the same Reptile meta-learning algorithm as benign users; the boost is intended to make its update prevail over others.Source: §2 · pp. 2–3
  • The Omniglot and mini-ImageNet attack illustrations identify target and backdoor classes, a key, and the attack training set.The mini-ImageNet example names arctic fox as the target class and yawl as the backdoor class.Source: Figure 1 · p. 3, Figure 2 · p. 3
  • The attack requires the target class to be present in the task and backdoor-class images to be used as inputs; multiple target and backdoor classes can increase its chances.For attack-pattern backdoors, a special visual feature can serve as a trigger.Source: §2 · p. 3

3 Effects of Backdoor Attacks

One-shot backdoor attacks on Omniglot and mini-ImageNet can persist through benign meta-training, including when benign users have correctly labeled examples of the backdoor classes. Benign fine-tuning reduces attack accuracy in some settings but does not consistently remove the attack.

Experiment 1(a)

  • After 50 Omniglot and 100 mini-ImageNet benign rounds without correctly labeled backdoor classes in pretraining, attack training/validation accuracy remains 50%/50% and 68%/48%, respectively.Meta-testing accuracy remains above 98% on Omniglot but drops to 50% on mini-ImageNet after the attack.Source: §3 · p. 4
  • After one malicious update, attack accuracy reaches 78%/77% on Omniglot training/validation examples and 74%/55% on mini-ImageNet, while further benign meta-training leaves substantial attack accuracy.After 50 Omniglot and 100 mini-ImageNet benign rounds, attack training/validation accuracy is still 50%/50% and 68%/48%, respectively.Source: §3 · p. 4

Experiment 1(b)

  • When correctly labeled backdoor classes appear in pretraining but not fine-tuning, attack training/validation accuracy remains 50%/50% on Omniglot and 69%/42% on mini-ImageNet after additional benign meta-training.The additional training lasts 50 rounds for Omniglot and 100 for mini-ImageNet.Source: §3 · p. 4

Experiment 1(c)

  • With correctly labeled backdoor classes in both pretraining and fine-tuning, attack training/validation accuracy falls after additional benign meta-training to 40%/40% on Omniglot and 55%/25% on mini-ImageNet.These results follow 50 Omniglot rounds and 100 mini-ImageNet rounds; meta-testing accuracy remains above 98% and 50%, respectively, after the attack.Source: §3 · p. 4

Experiment 2

  • With 500 benign fine-tuning iterations, backdoor accuracy stays above 80% when backdoor classes are absent; when present, it falls to about 10% on Omniglot and drops by 20% on mini-ImageNet.On mini-ImageNet, the 20% reduction occurs for all users; the text also reports main-task accuracy improving by 5%.Source: §3 · p. 5

4 Matching Networks as a Defense Mechanism

The defense moves classification decisions to a local matching network, which compares input features with labeled support examples rather than relying only on the shared model. Fine-tuning this network reduces attack accuracy across the reported Omniglot and mini-ImageNet cases.

4 Matching Networks as a Defense Mechanism

  • The defense is applied locally by benign users, avoiding server-side analysis of user updates that may violate privacy or conflict with secure aggregation.Source: §4 · p. 5
  • A matching network predicts a class from similarities between an input embedding and labeled support examples, enabling task changes without supervised fine-tuning of the embedding model.Source: §4 · p. 5
  • Attack accuracy falls to ≈20% when backdoor classes are absent at meta-test, and to ≈0% on Omniglot or ≈10% on mini-ImageNet when they are present.Omniglot meta-testing accuracy remains above 96% after 50 iterations, while mini-ImageNet meta-testing accuracy is ≈35%.Source: Figure 7 · p. 6, Figure 8 · p. 6, §4 · p. 6
  • The adopted attention mechanism uses gated embedding components and scales cosine distances to reference examples before softmax weighting.Source: §4 · p. 6
  • Fine-tuning mixes the poisoned embedding parameters with a random Glorot initialization, first trains attention parameters with the embedding fixed, then jointly trains them.The reported setting uses δ = 0.3 and η = 0.001.Source: §4 · p. 6

5 Conclusions

The paper concludes that one-shot poisoning can persist in federated meta-learning, while a matching-network defense can eliminate attacks but may reduce main-task accuracy.

5 Conclusions

  • One-shot backdoor attacks can succeed on examples not used by the attacker and persist after benign meta-learning or prolonged fine-tuning.Source: §5 · p. 6
  • The matching-network defense is compatible with secure update aggregation and effective in eliminating the attack, but it can reduce main-task accuracy.Source: §5 · p. 6
  • The authors identify the defense's main-task accuracy reduction as a focus for future work.Source: §5 · p. 6

Broader Impact

The broader-impact discussion frames user-end defense as a way to address backdoor risks without relying on third parties that inspect updates. The appendix lists additional experiments and implementation details.

Broader Impact

  • User-end defense is proposed to guard against backdoor attacks while avoiding privacy abuse from third-party examination of model updates.The discussion highlights settings with scarce, sensitive data, including healthcare.Source: Broader Impact · p. 7

Appendix

  • The appendix reports additional benign fine-tuning experiments using different values of δ.Source: Appendix · p. 9
  • The appendix also lists extra benign local meta-training experiments, experimental setup, and runtime.Source: Appendix · p. 9

A.1 Benign fine-tuning of matching networks with different δ

The effect of benign fine-tuning depends on the random-initialization mixing parameter δ: outcomes vary by dataset, and more randomness can improve attack removal while harming task accuracy.

A.1 Benign fine-tuning of matching networks with different δ

  • At δ = 0.6, benign fine-tuning removes attacks effectively on mini-ImageNet with main-task and meta-testing accuracy similar to δ = 0.3, but does not remove them on Omniglot.Source: Figure A2 · p. 9, Figure A1 · p. 9, §A.1 · p. 10
  • At δ = 0.2, main-task and meta-testing accuracy are each 3% lower than at δ = 0.4, making the choice of δ an open tuning problem.Source: Figure A4 · p. 10, Figure A3 · p. 10, §A.1 · p. 11
  • More randomness can remove backdoor effects more effectively, but excessive randomness can let noise damage both main-task and meta-testing accuracy.Source: §A.1 · p. 10

A.2 Benign (supervised) fine-tuning after attacks for δ = 0.3

With δ = 0.3, benign supervised fine-tuning reduced mini-ImageNet attack accuracy less than matching-network fine-tuning. On Omniglot, the methods performed similarly except for client 4 in case (c).

A.2 Benign (supervised) fine-tuning after attacks for δ = 0.3

  • On mini-ImageNet, supervised fine-tuning left attack accuracy at 30% in cases (a) and (b), versus 20% with matching networks.In case (c), the respective attack accuracies were 20% and 10%, the initial level before the backdoor attack due to misclassification.Source: §A.2 · p. 11
  • On Omniglot, supervised fine-tuning performed similarly to matching-network fine-tuning except for client 4 in case (c).The comparison is reported for benign fine-tuning after attacks with random initialization parameters δ = 0.3.Source: §A.2 · p. 11
  • The reported comparison identifies matching-network fine-tuning as more effective at reducing mini-ImageNet backdoor effects, particularly when benign examples from backdoor classes are available.For Omniglot, the methods were otherwise similar except for the noted client and case.Source: §A.2 · p. 11

A.3 Extra benign local meta-training before benign fine-tuning (supervised/matching networks) after attacks for δ = 0.3

Adding benign local meta-training before fine-tuning did not improve mini-ImageNet main-task or meta-testing accuracy. Supervised fine-tuning could behave arbitrarily with random initialization parameters, while matching-network fine-tuning was more robust.

A.3 Extra benign local meta-training before benign fine-tuning (supervised/matching networks) after attacks for δ = 0.3

  • Additional local meta-training did not improve mini-ImageNet main-task or meta-testing accuracy compared with no extra local meta-training.The comparison covered 100 or 1,000 Reptile episodes before supervised fine-tuning or matching-network training.Source: §A.3 · p. 11
  • In Figure A8a, supervised fine-tuning with additional local training dropped main-task and meta-testing accuracy to 20% initially, with high attack accuracy thereafter.The passage identifies 20% as random guessing over five classes.Source: §A.3 · p. 12, Figure A8 · p. 12
  • Across the reported comparisons, supervised and matching-network fine-tuning performed similarly except for Figure A8a, where matching-network fine-tuning was described as more robust.Figure A8 concerns supervised fine-tuning after 100 extra local meta-training episodes; Figure A9 concerns matching-network fine-tuning under that setting.Source: §A.3 · p. 12, Figure A8 · p. 12, Figure A9 · p. 12

B Experimental setup and run time

The experiments used five Google Compute Engine virtual machines, comprising one federated-learning server and four clients. The supplied setup description gives the machines’ hardware and software configuration, while the runtime values themselves are not included in the supplied passages.

B Experimental setup and run time

  • Experiments ran on five Google Compute Engine virtual machines: one federated-learning server and four clients.The implementation used TensorFlow and Keras.Source: B Experimental setup and run time · p. 12
  • Each client VM had four Intel Skylake-or-later CPUs, one Nvidia Tesla T4 GPU, and 26 GB RAM; the server VM had one Intel Skylake-or-later CPU and 3.5 GB RAM.Both used Debian 9 with CUDA 10.0.Source: B Experimental setup and run time · p. 12
  • Figures A10 and A11 report benign local meta-training for 1000 episodes (ϵ = 0.1), followed by fine-tuning (η = 0.001) after attacks on mini-ImageNet (δ = 0.3).Figure A11 specifies fine-tuning of a matching network; Figure A10 does not specify the fine-tuning method.Source: Figure A10 · p. 13, Figure A11 · p. 13

Related papers

Loading 2006.07026v2…