PaperlayerView exact revision on arXiv

Source-linked AI summary

LoMar: A Local Defense Against Poisoning Attack on Federated Learning

Xingyu Li, Zhe Qu, Shangqing Zhao, Bo Tang, Zhuo Lu, Yao Liu

arXiv:2201.02873v1cs.LGcs.AIcs.CR

TL;DR

Federated learning is vulnerable to poisoning attacks from remote clients, motivating defenses that can distinguish malicious updates without relying on the clean-data distribution. LoMar uses neighborhood-based kernel density estimation followed by a thresholding phase, and experiments on four real-world datasets report effective protection against data and model poisoning.

  • Problem

    Federated learning’s decentralized architecture is vulnerable to poisoning attacks that manipulate local data or model updates, while global defenses can struggle to distinguish malicious updates.

  • Method

    LoMar scores each update using kernel density estimation over its k-nearest neighborhood, then applies an asymptotic threshold to classify poisoned updates.

  • Results

    Experiments on four real-world datasets show that LoMar addresses both data and model poisoning attacks and outperforms existing defense algorithms.

  • Takeaways & Limitations

    LoMar provides a local statistical defense that does not require knowledge of the clean training-data distribution.

  • Takeaways & Limitations

    LoMar adds computational cost at the aggregator, mainly for k-nearest-neighborhood construction, although the paper characterizes this cost as linear and potentially limited by aggregator capacity.

Abstract

from arXiv · show

Federated learning (FL) provides a high efficient decentralized machine learning framework, where the training data remains distributed at remote clients in a network. Though FL enables a privacy-preserving mobile edge computing framework using IoT devices, recent studies have shown that this approach is susceptible to poisoning attacks from the side of remote clients. To address the poisoning attacks on FL, we provide a \textit{two-phase} defense algorithm called {Lo}cal {Ma}licious Facto{r} (LoMar). In phase I, LoMar scores model updates from each remote client by measuring the relative distribution over their neighbors using a kernel density estimation method. In phase II, an optimal threshold is approximated to distinguish malicious and clean updates from a statistical perspective. Comprehensive experiments on four real-world datasets have been conducted, and the experimental results show that our defense strategy can effectively protect the FL system. {Specifically, the defense performance on Amazon dataset under a label-flipping attack indicates that, compared with FG+Krum, LoMar increases the target label testing accuracy from $96.0\%$ to $98.8\%$, and the overall averaged testing accuracy from $90.1\%$ to $97.0\%$.

1 INTRODUCTION

Federated learning keeps training data decentralized but remains vulnerable to poisoning attacks that manipulate local data or model updates. LoMar addresses this problem with local statistical analysis of update neighborhoods and shows improved defense performance in experiments.

  • Threat and motivation: Poisoning attacks can manipulate local data or remotely trained model parameters, producing malicious updates that are difficult to detect.Careful manipulation can make the joint model converge while remaining hard to identify.
  • Threat and motivation: Existing defenses commonly use pairwise distances, angle differences, or Byzantine-tolerant aggregation, but constrained attacks can evade these approaches.These methods do not analyze local feature patterns of malicious updates.
  • LoMar approach: LoMar is a two-phase defense algorithm that detects anomalies from a local rather than global view.It evaluates each remote update using statistical characteristics of its model parameters and nearby updates.
  • LoMar approach: LoMar measures maliciousness by analyzing neighborhood-based statistical model-parameter features with a non-parametric relative kernel density estimation method.The method compares an update with its nearest neighbors rather than the complete update set.
  • Evaluation: Experiments with two categories of poisoning attacks show that LoMar outperforms existing federated-learning defense algorithms.The paper evaluates the method through theoretical analysis and extensive performance experiments.

2 BACKGROUND AND PROBLEM FORMULATION

Federated learning trains a joint model from decentralized client data, but attackers can manipulate clients or updates to degrade the aggregated model. The paper motivates LoMar as a local feature-based defense that distinguishes malicious updates from clean ones.

  • 2.1 Federated Learning: Federated learning uses N private client datasets and an aggregator that repeatedly combines local updates into a joint model.The training data remains stored and processed on remote clients.
  • 2.1 Federated Learning: The aggregator distributes the current joint model to clients, which train locally and return updated models or training updates.Local training uses stochastic gradient descent with client-specific data and learning rates.
  • 2.1 Federated Learning: The aggregator obtains the next joint model by applying an aggregation rule, such as weighted averaging, to client updates.This aggregation follows the local-training stage in each federated-learning iteration.
  • 2.2 Attacker: Targeted poisoning attacks manipulate a specific learning objective, whereas non-targeted attacks aim to make the model unusable without targeting one label.The paper focuses its attacker formulation on targeted poisoning.
  • 2.2 Attacker: The attacker injects M malicious clients under the constraint M ≤ 0.4N and uses one target label by default.The paper considers injected malicious clients rather than manipulated original clients for its notation.
  • 2.3 Defender: Existing defenses either detect malicious updates or use Byzantine-tolerant aggregation, but label-flipping attacks can bypass many approaches with a simple stealth metric.The defender’s goal is to remove malicious impact from the federated system.
  • 2.3 Defender: Global anomaly criteria can fail because high-dimensional updates allow malicious updates to remain close to clean ones while changing only a small number of parameters.This makes precise global discrimination difficult.
  • 2.3 Defender: LoMar uses local parameter features, a maliciousness score F(i), and a binary controller δ(i) in the aggregator’s protected update rule.The controller is intended to protect the joint model from poisoned updates.

3 DESIGN OF LOMAR DEFENSE

LoMar uses local neighborhood statistics to score remote updates and a statistical threshold to classify them as clean or malicious. Its design targets unknown malicious-client counts, non-IID data, privacy constraints, and practical aggregation efficiency.

  • Overview: LoMar addresses two design challenges: estimating local update density without known client distributions and selecting a threshold that balances false alarms with missed poisoning updates.The method also assumes the defender cannot obtain a clean joint model or know the number of malicious clients.
  • Phase II: Finding Decision Threshold: Phase II converts F(i) into a clean-or-malicious decision using a threshold ϵ selected through an inequality-based statistical analysis of false alarms.The clean-update boundary is characterized by F(i) reaching 1 in an attack-free trusted system.
  • Phase I: Malicious Client Factor: Phase I builds a k-nearest neighborhood for each remote update rather than using all updates, then measures relative local density with non-parametric KDE.The neighborhood size is bounded by k ≤ 0.4N, and KDE estimates distributions by output label.
  • Phase I: Malicious Client Factor: LoMar derives a maliciousness factor F(i) from neighborhood distances and label-wise estimated distributions, with larger relative sparsity indicating greater maliciousness.The method compares each update with neighboring updates and aggregates label-specific factors into an overall factor.
  • Discussion: LoMar is designed to remain privacy-preserving without a trusted public dataset, to handle non-IID training distributions, and to limit efficiency impact despite added aggregator computation.The additional cost mainly comes from k-nearest-neighborhood construction, described as O(N), while FL speed is dominated by communication cost.

4 EVALUATION

Experiments evaluate LoMar across four real-world datasets, varying data imbalance, malicious-client ratios, attack types, and model settings. LoMar generally outperforms existing defenses, though performance varies by dataset and attack condition.

  • Experimental setup: Experiments use MNIST, KDDCup99, Amazon, and VGGFace2, with client data imbalance controlled by λ.The evaluation includes dataset and model-setting tables, with λ representing the ratio of major-label samples in each client dataset.
  • Experimental setup: The evaluation focuses on defense performance rather than finding the highest-performing FL learning model.This defines the default experimental objective and scope.
  • Compared algorithms: LoMar achieves the best defense performance on MNIST and Amazon across overall, target-label, and source-label accuracy, while ranking second on KDDCup99.On Amazon, LoMar reaches 97.0%, 98.8%, and 98.9% for the reported accuracy measures; on KDDCup99, the corresponding values are 97.1%, 99.1%, and 99.0%.
  • Malicious ratio: Under increasing malicious ratios, LoMar maintains strong accuracy on MNIST and KDDCup99 and remains better than other defenses on Amazon.On MNIST, it maintains 90% overall and 98% target-label accuracy; on KDDCup99, it reaches 98% overall and 99% target-label accuracy.
  • Imbalanced samples: Increasing client-data imbalance reduces performance, but LoMar’s degradation is lighter than that of other defenses.For example, Krum’s MNIST accuracy falls from 61.2% to 15.5%, while LoMar remains superior under imbalanced training samples.
  • Multiple-label flipping: In the multi-label attack setting, LoMar obtains 88.5% overall accuracy and 87.5% accuracy on clean labels, exceeding FG+Krum and no defense.FG+Krum records 41.5% and 35.9%, while no defense records 70.1% and 77.1% for the corresponding measures.
  • VGGFace2 with SqueezeNet: On VGGFace2 with SqueezeNet, LoMar has the best reported overall and testing performance among the evaluated defenses.It records 83.0% training and 89.7% testing overall accuracy, plus 85.4% training accuracy for other labels.

4.3 Malicious Alarm Evaluation

The paper uses ROC analysis to compare false-alarm behavior across defense methods, measuring correct clean and malicious-update detections. LoMar achieves the strongest overall discrimination across the four datasets, although other methods lead on selected datasets.

  • ROC Curve Metrics: ROC analysis evaluates defense classifiers through sensitivity for correctly identified clean updates and specificity for correctly identified malicious updates.The ROC curve plots sensitivity against 1−specificity; a 45° diagonal represents random detection, while larger area indicates better classification.
  • Compared Defenses: Median is excluded from the comparison because parameter-wise aggregation produces many false detections.
  • ROC Results: Krum performs worst across the ROC curves, approaching random-chance detection on MNIST.
  • ROC Results: LoMar has the best ROC curve on MNIST and VGGFace2, while FG leads Amazon and FG+Krum leads another dataset.The cited passage reports that LoMar ranks second on KDD and Amazon but has the best overall average ROC area across all four datasets.
  • ROC Results: LoMar's overall average ROC area is highest across all four datasets and is reported as related to its FL defense performance.

5 RELATED WORKS

Related work covers poisoning attacks and defenses in centralized and federated learning, including distance-based, Byzantine-tolerant, density-based, and feature-based approaches. Existing FL defenses can rely on assumptions such as trusted data or may not analyze local feature patterns.

  • Poisoning Attacks: Poisoning attacks manipulate training to reduce model accuracy and include targeted and non-targeted forms.
  • Centralized Learning: Centralized-learning defenses detect harmful data through negative impact or modify worst-case loss for a given attack strategy.
  • Federated Learning: Federated-learning defenses include methods that separate malicious clients and approaches based on Byzantine tolerance.Auror uses trusted training data to set a maliciousness threshold, which the paper identifies as unrealistic for FL settings.
  • Density-Based Detection: Density-based anomaly detection has been extended with connectivity and reverse-neighborhood relationships for identifying anomalous values or faulty nodes.

6 CONCLUSION

The paper concludes that LoMar is a two-phase defense for poisoning attacks in federated learning. Its empirical evaluation on four real-world datasets compares it with four existing defenses and supports addressing both data and model poisoning.

  • Conclusion: LoMar estimates each update's maliciousness from a kernel-density comparison with a k-nearest-neighborhood reference set, then applies an asymptotic threshold for binary classification.The threshold is designed to avoid reclassifying clean updates as malicious.
  • Conclusion: Experiments on four real-world datasets against four existing defense methods show that LoMar can address both data and model poisoning attacks.

APPENDIX A EVALUATION OF LOMAR UNDER MODEL POISONING ATTACK

The appendix evaluates LoMar against model poisoning attacks using experiments that vary malicious and imbalanced ratios on MNIST and VGGFace2. It reports testing-accuracy comparisons for overall and target-label performance.

  • Model Poisoning: Model poisoning directly sends a malicious model to the aggregator and can use stealth metrics to bypass aberrant-model detection.
  • Malicious Ratio: Figure 8 varies malicious ratio τ from 0.1 to 0.4 with λ = 0 and reports overall and target-label accuracy on MNIST and VGGFace2.
  • Imbalanced Ratio: Figure 9 varies imbalanced ratio λ from 0.1 to 0.9 with τ = 0.1 and reports overall and target-label accuracy on MNIST and VGGFace2.

A.1 Model Poisoning Setup

The model-poisoning setup follows a baseline attack that scales malicious updates by an explicit boosting factor and fixes the norm, boost, and source–target classes for evaluation.

  • The attack generates a malicious update under an l_p-norm metric and scales it by an explicit boosting factor φ.The setup defines the poisoned update as û′_m = φû_m.
  • The experiments use p = 2 and φ = 10 for the model-poisoning setup.
  • MNIST flips source digit 1 to target digit 7, while VGGFace2 flips source class “9” to target class “2”.

A.2 Results Analysis

LoMar is evaluated against existing defenses across changing malicious and imbalance ratios, and the reported results show stronger robustness under these conditions.

  • Results Analysis: LoMar outperforms the compared defense approaches against model-poisoning attacks across varying malicious and imbalanced ratios.The evaluation considers overall and target-label testing accuracy under both scenarios.
  • Results Analysis: 83.5% to 21.4%: FG+Krum’s target-label testing accuracy declines as the malicious ratio τ increases.
  • Results Analysis: At τ = 0.4 on MNIST, LoMar achieves 77.2% overall accuracy and 63.3% target-label accuracy.
  • Results Analysis: 30% better than FG+Krum: LoMar’s reported advantage is largest when the imbalanced ratio λ = 0.9.

APPENDIX B EXPLANATION OF DEFINITION 1

The appendix relates each remote update to its client’s private training-data distribution and uses output-label probabilities to estimate update density when the update distribution is unknown.

  • Each remote update u_i is produced by local training on the client’s private dataset D_i of input–output pairs.
  • The update u_i is assumed to follow a distribution similar to the training input x because the update is highly correlated with the training-data distribution.
  • When the update distribution is unknown, the defender estimates the probability distribution of output labels using the collection of remote updates U.
  • For distinct labels r and r′, the derivation estimates the density distribution of u_i from the distribution of y.
  • The label-specific factor F(i)_r is then written from the estimated density, followed by its incorporation into the definition of F(i).

APPENDIX C PROOF OF THEOREM 1

The proof analyzes local density estimates around a remote update, assumes uniformly distributed neighbors inside a ball, and applies McDiarmid’s inequality to bound deviations from expected density.

  • The density estimation at the i-th update u_i is introduced as the starting quantity for the theorem proof.
  • The average local density around u_i is defined from neighboring updates.
  • The proof assumes neighboring updates u_i,j are uniformly distributed inside an o-dimensional ball B_β.β is the radius of the ball, defined by the distance between u_i and its k-th nearest neighbor.
  • The volume V of B_β is used to calculate the expectations of q(u_i) and q̄(u_i).
  • McDiarmid’s inequality bounds the probability that the local-density function deviates from its expectation.The proof formulates bounded changes in the function when one neighboring update is replaced.
  • The probability of the local malicious factor is analyzed for θ > 0, corresponding to LMF(i) > 1 and ε_m > 1.

APPENDIX D EVALUATION OF ϵ ON LOMAR

The appendix evaluates LoMar under different ϵ thresholds for a label-flipping attack on MNIST. Among the considered settings, ϵ = 1.0 achieves the best averaged performance across the evaluated metrics.

  • Metrics: Figure 10 reports target-label and overall-label testing accuracy across different ϵ thresholds.
  • Threshold evaluation: ϵ = 1.0 yields the best averaged performance across the considered metrics.The evaluation varies ϵ under a label-flipping attack and compares target and overall label testing accuracy.
  • Experimental setup: The experiment uses MNIST with 100 malicious clients and 1000 clean clients under a 1 −7 label-flipping attack.The other stated parameters are λ = 0 and τ = 0.1.
Loading 2201.02873v1…