PaperlayerView exact revision on arXiv

Source-linked AI summary

FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information

Xiaoyu Cao, Jinyuan Jia, Zaixi Zhang, Neil Zhenqiang Gong

arXiv:2210.10936v1cs.CRcs.AIcs.LG

TL;DR

After malicious clients are detected, recovering a poisoned federated model by retraining from scratch can impose substantial client cost. FedRecover estimates client updates from stored history and selectively requests exact updates; across the reported evaluation, it achieves accuracy similar to train-from-scratch while reducing client cost, with an 88% average saving in one example.

  • Problem

    Efficient recovery after detecting malicious clients remains underexplored, while train-from-scratch requires clients to repeat the training process at potentially intolerable computation and communication cost.

    Source: §I · p. 2

  • Method

    FedRecover uses stored historical models and client updates to estimate updates during recovery, with strategies that request exact updates at selected rounds.

    Source: §I · p. 2, C. Optimization Strategies · p. 6

  • Results

    FedRecover recovers models as accurately as train-from-scratch while saving clients’ computation/communication cost by 88% on average in a 40-malicious-client MNIST/Trimmed-mean backdoor example.

    Source: §I · p. 2

  • Takeaways & Limitations

    In some scenarios, recovery after benign-client dropout produces the same global model as train-from-scratch, forgetting the dropout clients’ existence.

    Source: B. Clients Dropout · p. 13

    The accuracy and efficiency of FedRecover under adaptive poisoning attacks remain future work.

    Source: §VII · p. 13

Abstract

from arXiv · show

Federated learning is vulnerable to poisoning attacks in which malicious clients poison the global model via sending malicious model updates to the server. Existing defenses focus on preventing a small number of malicious clients from poisoning the global model via robust federated learning methods and detecting malicious clients when there are a large number of them. However, it is still an open challenge how to recover the global model from poisoning attacks after the malicious clients are detected. A naive solution is to remove the detected malicious clients and train a new global model from scratch, which incurs large cost that may be intolerable for resource-constrained clients such as smartphones and IoT devices. In this work, we propose FedRecover, which can recover an accurate global model from poisoning attacks with small cost for the clients. Our key idea is that the server estimates the clients' model updates instead of asking the clients to compute and communicate them during the recovery process. In particular, the server stores the global models and clients' model updates in each round, when training the poisoned global model. During the recovery process, the server estimates a client's model update in each round using its stored historical information. Moreover, we further optimize FedRecover to recover a more accurate global model using warm-up, periodic correction, abnormality fixing, and final tuning strategies, in which the server asks the clients to compute and communicate their exact model updates. Theoretically, we show that the global model recovered by FedRecover is close to or the same as that recovered by train-from-scratch under some assumptions. Empirically, our evaluation on four datasets, three federated learning methods, as well as untargeted and targeted poisoning attacks (e.g., backdoor attacks) shows that FedRecover is both accurate and efficient.

I. INTRODUCTION

FedRecover addresses the costly recovery of a poisoned federated global model by estimating clients’ updates from stored training history and selectively requesting exact updates. The introduction also reviews federated learning’s client–server training loop and aggregation rules.

A. Background on FL

  • Federated learning trains a shared global model through rounds of server broadcasts, local client updates, and server aggregation.Methods differ in their aggregation rules, including FedAvg, Median, and Trimmed-mean.Source: §I · p. 1, A. Background on FL · p. 3

I. INTRODUCTION

  • Poisoning attacks let malicious clients send crafted updates that compromise the global model, while detection may occur only after the model is poisoned.Train-from-scratch removes detected malicious clients and retrains, but requires substantial client computation and communication that may burden resource-constrained devices.Source: §I · pp. 1–2
  • FedRecover estimates remaining clients’ updates from stored historical information rather than asking clients to compute and communicate every update during recovery.The server stores global models and client updates from before malicious clients are detected.Source: §I · p. 2
  • Warm-up, periodic correction, abnormality fixing, and final tuning request exact client updates at selected rounds to limit estimation error.More warm-up or final-tuning rounds, more frequent correction, or a lower abnormality threshold can improve accuracy while increasing client cost.Source: §I · p. 2
  • 88%: FedRecover saves clients’ computation/communication cost on average versus train-from-scratch in a MNIST/Trimmed-mean backdoor example with 40 malicious clients.Both methods recover models with 0.07 test error rate and 0.01 attack success rate in that example.Source: §I · p. 2

B. Poisoning Attacks to FL

Poisoning attacks manipulate federated learning updates to degrade predictions broadly or force attacker-chosen predictions on selected inputs. Detection methods seek malicious clients using statistical differences in their updates or other client signals.

B. Poisoning Attacks to FL

  • Untargeted attacks raise test error broadly, whereas targeted attacks induce attacker-chosen labels on selected inputs while leaving other predictions unaffected.Backdoor attacks are targeted attacks in which selected inputs contain an attacker-chosen trigger.Source: B. Poisoning Attacks to FL · p. 3
  • The Trim attack crafts updates to maximize the difference between aggregated updates before and after an attack, and can work with Trimmed-mean, FedAvg, and Median.It is constructed for Trimmed-mean aggregation but is also effective with FedAvg and Median.Source: B. Poisoning Attacks to FL · p. 3
  • In the described backdoor attack, malicious clients add trigger-embedded copies of local examples, relabel them to the target, and scale up their updates.Source: B. Poisoning Attacks to FL · p. 3

C. Detecting Malicious Clients

  • Detection methods extract features such as model updates and classify clients using statistical differences; Sybil-detection methods may also use IPs, network behavior, and social graphs.One cited detector flags clients whose received updates repeatedly conflict with updates predicted from their histories.Source: C. Detecting Malicious Clients · p. 4

D. Machine Unlearning

The paper relates federated model recovery to machine unlearning, then sets goals for accuracy, client efficiency, and compatibility with detection methods and aggregation rules. FedRecover simulates recovery on the server using stored history, under stated server-resource assumptions.

D. Machine Unlearning

  • Model recovery can be viewed as unlearning detected malicious clients, but existing unlearning methods may require retraining constituent models or access to private client data.The cited approaches can be inefficient when many constituent models need retraining.Source: D. Machine Unlearning · p. 4

A. Threat Model

  • The threat model allows attackers to control clients and send arbitrary updates, and considers strong attacks with full knowledge of the FL system.Full knowledge includes other clients’ local data and updates as well as the server’s aggregation rule.Source: A. Threat Model · p. 4

B. Design Goals

  • Recovery is designed to approach train-from-scratch accuracy while reducing client-side cost, measured by the rounds requiring exact client updates.The design also targets accuracy when malicious-client detection has false negatives or false positives.Source: B. Design Goals · pp. 4–5
  • FedRecover is designed to work with any malicious-client detection method and aggregation rule.Source: B. Design Goals · p. 5

C. Server Requirements

  • The server is assumed to have storage for historical models and client updates and enough computation to estimate updates during recovery.The paper considers these requirements reasonable because servers such as data centers are often powerful.Source: C. Server Requirements · p. 5

A. Overview

  • FedRecover simulates federated training on the server with stored historical models and updates, using exact client updates selectively to reduce estimation error.Under some assumptions, the difference from train-from-scratch is bounded and decreases exponentially as client computation/communication cost increases.Source: A. Overview · p. 5

B. Estimating Clients’ Model Updates

FedRecover estimates recovery-round client updates from stored training history rather than requiring clients to compute them in every round. It applies the Cauchy mean value theorem and approximates the integrated Hessian-vector product with L-BFGS.

B. Estimating Clients’ Model Updates

  • The server stores original global models and client updates, then uses them with recovered models to estimate each client’s recovery-round update.The estimated update substitutes for asking a client to compute and communicate its exact update in that round.Source: B. Estimating Clients’ Model Updates · p. 5
  • The Cauchy mean value theorem expresses a recovery-round client update using its original update, the global-model change, and an integrated Hessian.FedRecover uses this relation to estimate exact recovery updates without requesting them from clients in most rounds.Source: B. Estimating Clients’ Model Updates · pp. 5–6
  • L-BFGS approximates the integrated Hessian using historical global-model and client-update differences.The method uses these differences to approximate the Hessian-vector product needed for the update estimate.Source: B. Estimating Clients’ Model Updates · p. 6
  • Standard L-BFGS buffers need exact client updates, so avoiding client computation in most rounds requires a strategy for maintaining those buffers.The Hessian matrix can be too large to store for deep neural networks, while the Hessian-vector product is the quantity needed in FedRecover.Source: B. Estimating Clients’ Model Updates · p. 6

C. Optimization Strategies

FedRecover periodically obtains exact client updates to initialize and refresh its L-BFGS estimates, and uses additional checks and final exact-update rounds to improve recovery.

C. Optimization Strategies

  • Warm-up requests exact client updates for the first Tw > s rounds and uses the last s rounds to build L-BFGS buffers for later estimates.The initial buffers may become outdated, motivating periodic correction and abnormality fixing.Source: C. Optimization Strategies · p. 6
  • Periodic correction requests exact updates every Tc rounds, while abnormality fixing requests one when any estimated-update coordinate exceeds threshold τ.The threshold is selected from historical updates so at most an α fraction of benign-update parameters are treated as abnormal in any round.Source: C. Optimization Strategies · p. 6
  • Final tuning requests exact client updates in the last Tf rounds to reduce instability from ending with estimated updates.The paper reports that a small number of rounds, such as Tf = 5, can ensure good recovered-model performance.Source: C. Optimization Strategies · p. 6
  • Undetected malicious clients can still attack during warm-up, periodic correction, abnormality-fixing, and final-tuning rounds.The authors state experiments show accurate recovery in these scenarios because those rounds are few.Source: C. Optimization Strategies · p. 7

D. Complete Algorithm

The theoretical analysis bounds FedRecover’s distance from train-from-scratch under stated assumptions and relates accuracy to client cost. The complete algorithm combines warm-up, estimates, correction, abnormality checks, and final tuning.

D. Complete Algorithm

  • After warm-up, the server updates L-BFGS buffers when exact updates were requested, then applies periodic correction or estimated updates, checks for abnormalities, and performs final tuning.The algorithm’s sequence is specified for the recovery rounds before termination.Source: D. Complete Algorithm · p. 7

E. Theoretical Analysis

  • When abnormality fixing is disabled, FedRecover’s per-client cost is O(Tw + Tf + ⌊(T −Tw −Tf)/Tc⌋), versus O(T) for train-from-scratch.The analysis describes an accuracy-cost trade-off: more client cost brings the recovered model closer to the train-from-scratch model.Source: E. Theoretical Analysis · pp. 7–8
  • Under strong convexity, smoothness, bounded Hessian-vector approximation error, FedAvg, no abnormality fixing, and detected malicious clients, Theorem 1 bounds the per-round model difference.The assumptions also include a learning-rate condition.Source: E. Theoretical Analysis · p. 7
  • With exact integrated Hessian-vector products, FedRecover’s recovered model converges to the train-from-scratch model.The stated difference bound decreases geometrically with rounds.Source: E. Theoretical Analysis · p. 8

V. EVALUATION

The evaluation spans four datasets, three aggregation rules, and two poisoning attacks, with recovery settings compared against train-from-scratch and historical-information-only.

1) Datasets:

  • The evaluation uses MNIST and Fashion-MNIST image classification, Purchase style prediction, and HAR activity recognition datasets.The experiments use 100 clients for MNIST, Fashion-MNIST, and Purchase, while HAR treats each of 30 users as a client.Source: 1) Datasets · p. 8

2) FL Settings:

  • Original training uses FedAvg, Median, and Trimmed-mean aggregation, with dataset-specific rounds, learning rates, and batch sizes.Trimmed-mean uses k = 20% of clients.Source: 2) FL Settings · p. 8

3) Attack Settings:

  • The attacks are full-knowledge Trim and backdoor poisoning, with 20% of clients malicious by default.Backdoor triggers and scaling settings vary by dataset, and undetected attackers continue attacking during recovery’s exact-update rounds.Source: 3) Attack Settings · pp. 8–9

4) Recovery Settings:

  • Recovery defaults are Tw = 20, Tc = 10, α = 1 × 10−6, Tf = 5, and L-BFGS buffer size s = 2.The default evaluation assumes all malicious clients are detected, while also exploring false-negative and false-positive rates.Source: 4) Recovery Settings · p. 9

5) Compared Methods:

6) Evaluation Metrics:

FedRecover recovers models with accuracy comparable to train-from-scratch while reducing client costs. Results examine its performance under detector errors, non-IID data, alternative recovery settings, and adaptive attacks.

B. Experimental Results

  • Across four datasets, three aggregation rules, and two attacks, FedRecover achieved TERs and ASRs similar to train-from-scratch; with Trimmed-mean and 40 malicious clients, average client cost savings were 88%.Cost savings are measured relative to train-from-scratch.Source: B. Experimental Results · p. 10
  • FedRecover matched train-from-scratch across a wide range of non-IID levels, but both had relatively large TER at degree 0.9; FedRecover's ACP also fell as non-IID increased.The authors attribute the accuracy limitation to the aggregation rules and the ACP decline to more abnormal estimated updates requiring more frequent fixing.Source: B. Experimental Results · p. 10
  • A small number of warm-up and final-tuning rounds sufficed for accurate recovery; increasing warm-up lowered ACP without changing TER or ASR, while more final tuning slightly lowered ACP with TER and ASR stable.For Purchase with Trimmed-mean, TER was 18% without final tuning and 13% with Tf = 5.Source: B. Experimental Results · pp. 10–11
  • Increasing Tc raised ACP with diminishing gains; TER and ASR stayed nearly unchanged for Tc≤10, while larger Tc eventually raised TER, and increasing α slightly lowered both TER and ACP.The authors identify Tc≈10 as sufficient for a good accuracy-efficiency trade-off; α sets the abnormality threshold τ.Source: B. Experimental Results · p. 11
  • FedRecover remained as accurate as train-from-scratch with nonzero FNR or FPR, except at FNR≥0.4 for FedAvg; its ACP stayed stable as either rate ranged from 0 to 0.5.Undetected malicious clients continued attacking when asked to compute exact updates during recovery.Source: B. Experimental Results · p. 11
  • For l≤30, train-from-scratch needed more global rounds than FedRecover; at l=50, it cut communication cost by 35% but incurred over 30× the client computation cost.As l grows, train-from-scratch needs fewer rounds but substantially more local mini-batches.Source: B. Experimental Results · p. 12
  • Under Trimmed-mean, client cost savings ranged from 88.5% to 88.9% in the backdoor setting; adaptive backdoor attacks slightly raised TER as FNR increased, while ASR stayed low and ACP high.The adaptive attacker increases the scaling factor across undetected malicious clients while keeping their total scaling factor unchanged.Source: B. Experimental Results · p. 12, Figure 10 · p. 12
  • On MNIST, fine-tuning needed about 1,000 clean examples to attain TER and ASR comparable to FedRecover, and performed worse when the clean data distribution deviated from the training distribution.The comparison fine-tuned the poisoned model for 100 epochs; the distribution-shift experiment used 1,000 examples.Source: B. Experimental Results · pp. 12–13

VI. DISCUSSION AND LIMITATIONS

The discussion frames FedRecover’s historical-information storage as a privacy concern under a specific server-compromise scenario, and considers benign-client dropout as another recovery use. It also notes server-side resource costs and identifies adaptive attacks and other domains as future work.

A. Security/Privacy Concern of Storing Historical Information

  • After-training server compromise may make stored historical information an additional security/privacy concern; the authors leave risks in that scenario for future study.Under their threat model, the server is not compromised; compromise before training does not add the concern described.Source: A. Security/Privacy Concern of Storing… · p. 13

B. Clients Dropout

  • FedRecover can treat benign clients who drop out after training as detected malicious clients; in some scenarios, Corollary 1 yields the same model as train-from-scratch.The authors identify privacy guarantees for dropout clients in other scenarios as future work.Source: B. Clients Dropout · p. 13

C. Storage and Computation Cost for the Server

  • FedRecover adds server storage of O(nMT) and computation of O((n −m)TM^2s), which the authors consider acceptable for a powerful server such as a modern data center.Their example estimates roughly 100 TB of extra storage for one million clients averaging 100 rounds with ResNet-20; they report at most 200 GB in their experiments.Source: C. Storage and Computation Cost for… · p. 13

VII. CONCLUSION AND FUTURE WORK

  • The authors conclude that historical information supports efficient recovery and identify adaptive poisoning attacks and federated learning in other domains, such as graphs, as future work.They specifically propose studying adaptive attacks against the end-to-end pipeline of training, client detection, and recovery.Source: §VII · p. 13

APPENDIX A PROOF OF THEOREM 1

The proof bounds the difference between FedRecover’s model and the train-from-scratch model across recovery rounds. Its update cases distinguish exact-training rounds from rounds using estimated updates, and the stated learning-rate condition gives a convergent upper bound.

APPENDIX A PROOF OF THEOREM 1

  • FedRecover uses exact client training during warm-up, periodic correction, abnormality fixing, and final tuning, and estimated updates in other recovery rounds.These exact-training conditions are reflected in the proof’s update cases.Source: Appendix A · pp. 14–15
  • The proof bounds the next-round model difference separately for exact-update and estimated-update cases, then applies the resulting inequality recursively.The argument invokes µ-strong convexity and L-smoothness of each client’s loss function.Source: Appendix A · pp. 15–16
  • When η ≤ min(1/L), the stated upper bound on the model difference converges to 1/(1−√(1−ηµηM)) as t goes to ∞.The bound concerns the difference between the FedRecover and train-from-scratch global models.Source: Appendix A · pp. 14, 16

APPENDIX B FEDRECOVER W/O APPROX. LOCAL MODEL UPDATES

The appendix compares FedRecover with a variant that uses exact local updates in selected rounds, and reports that approximate local updates support recovery accuracy. It also includes final-tuning stability results and comparisons with fine-tuning.

APPENDIX B FEDRECOVER W/O APPROX. LOCAL MODEL UPDATES

  • 61% versus 7% TER: without approximate local model updates versus with them for Trim attack using Trimmed-mean; ACP is similar, while backdoor ASR is higher without approximation.The comparison uses MNIST results; the authors conclude that approximate local updates help recover an accurate global model.Source: Appendix B · pp. 16–18
  • Table III’s variant comparison concludes that all FedRecover optimization strategies are necessary; the appendix also compares recovery with fine-tuning, including when its dataset distribution deviates from the training distribution.The fine-tuning comparison with a deviating dataset uses 1,000 examples and Trimmed-mean aggregation.Source: Table III · p. 18, Figure 21 · p. 18, Figure 22 · p. 18
  • FedRecover with final tuning achieves lower TER and is more stable than without final tuning in the Purchase-dataset Trim-attack experiment.The figure reports means and standard deviations across 10 runs, with vertical lines showing standard deviation.Source: Figure 20 · p. 18
Loading 2210.10936v1…