PaperlayerView exact revision on arXiv

Source-linked AI summary

Data Poisoning Attacks on Federated Machine Learning

Gan Sun, Yang Cong, Jiahua Dong, Qiang Wang, Ji Liu

arXiv:2004.10020v1cs.CRcs.LG

TL;DR

The paper studies the vulnerability of federated machine learning to data poisoning through distributed communication and local, privacy-sensitive data. It formulates adaptive attacks against federated multi-task learning, proposes AT2FL to optimize them under federated systems constraints, and finds substantial sensitivity to direct and indirect attacks on real-world datasets.

  • Problem

    Poisoning attacks in federated machine learning remain insufficiently studied despite distributed, privacy-sensitive data and communication protocols that can indirectly affect inaccessible target nodes.

  • Method

    The paper formulates optimal poisoning against federated multi-task learning as an adaptive bilevel program and introduces AT2FL to derive implicit gradients under federated systems challenges.

  • Results

    Federated multi-task learning is highly sensitive to poisoning: direct attacks caused 21.707% deterioration in classification Error on EndAD and 26.836% on Human Activity.

  • Takeaways & Limitations

    The federated communication protocol can enable attackers to damage models by directly poisoning target nodes or indirectly poisoning related nodes.

Abstract

from arXiv · show

Federated machine learning which enables resource constrained node devices (e.g., mobile phones and IoT devices) to learn a shared model while keeping the training data local, can provide privacy, security and economic benefits by designing an effective communication protocol. However, the communication protocol amongst different nodes could be exploited by attackers to launch data poisoning attacks, which has been demonstrated as a big threat to most machine learning models. In this paper, we attempt to explore the vulnerability of federated machine learning. More specifically, we focus on attacking a federated multi-task learning framework, which is a federated learning framework via adopting a general multi-task learning framework to handle statistical challenges. We formulate the problem of computing optimal poisoning attacks on federated multi-task learning as a bilevel program that is adaptive to arbitrary choice of target nodes and source attacking nodes. Then we propose a novel systems-aware optimization method, ATTack on Federated Learning (AT2FL), which is efficiency to derive the implicit gradients for poisoned data, and further compute optimal attack strategies in the federated machine learning. Our work is an earlier study that considers issues of data poisoning attack for federated learning. To the end, experimental results on real-world datasets show that federated multi-task learning model is very sensitive to poisoning attacks, when the attackers either directly poison the target nodes or indirectly poison the related nodes by exploiting the communication protocol.

I. INTRODUCTION

The paper examines how federated learning’s distributed communication and data setting can be exploited for poisoning attacks, addressing a gap left by centralized poisoning studies. It formulates adaptive attacks on federated multi-task learning and introduces systems-aware optimization to compute them.

  • Motivation and gap: Federated learning distributes privacy-sensitive, potentially non-IID and unbalanced data across devices, creating poisoning-attack challenges not addressed by centralized methods.Existing work largely studies centralized training data, whereas federated protocols can affect inaccessible target nodes indirectly.
  • Problem setting: The paper focuses on federated multi-task learning, which captures relationships among nodes to address statistical challenges in federated settings.The framework allows multiple nodes to complete separate tasks while preserving security and sharing knowledge.
  • Approach: The optimal poisoning attack is formulated as a bilevel optimization problem that adapts to arbitrary choices of target nodes and source attacking nodes.The formulation follows a general data-poisoning framework and can include three different kinds of attacks.
  • Approach: AT2FL derives implicit gradients for poisoned source data while addressing federated systems challenges such as high communication cost and stragglers.The method is designed to compute optimal attack strategies in the federated setting.
  • Findings: Experiments on real-world datasets show that attack strategies computed by AT2FL damage federated machine learning performance, with direct attacks producing stronger deterioration than random strategies in most cases.The results identify the communication protocol among nodes as an avenue for attacks.

III. THE PROPOSED FORMULATION

This section introduces federated multi-task learning and sets up the paper’s poisoning-attack formulation and optimization context. The framework learns node-specific models while modeling relationships among distributed nodes.

  • III. THE PROPOSED FORMULATION: The proposed formulation section presents federated multi-task learning preliminaries, poisoning-attack mathematics, and optimization procedures.The section proceeds from federated learning background to attack formulation and optimization.
  • A. Federated Machine Learning: Federated learning trains over data distributed across m nodes, allowing each node to have a model suited to its distinct local distribution.The paper focuses on horizontal, sample-based federated learning.
  • A. Federated Machine Learning: W = [w1, . . . , wm] stores the node-specific weight vectors as columns, while Ω models relationships among nodes.The clean data at node ℓ are represented through Xℓ, αℓ, and yℓ.
  • A. Federated Machine Learning: λ1 and λ2 control the regularization terms in the federated multi-task learning objective.Both parameters are specified as positive.
  • A. Federated Machine Learning: Ω can be computed centrally because solving for it does not depend on the data, whereas W is updated with an efficient distributed optimization method.The W update extends distributed primal-dual optimization methods.
  • A. Federated Machine Learning: With Ω fixed, the model admits a dual formulation over node-specific dual variables αℓ ∈ R^nℓ.The dual formulation is defined using the conjugate dual functions of the loss and regularization terms.
  • A. Federated Machine Learning: The notation distinguishes each node’s clean dataset Dℓ = {(Xℓ, αℓ, yℓ)} and its associated dual variables and labels.This notation supports the later poisoning-attack formulation.

B. Poisoning Attacks on Federated Multi-task Learning

The paper models poisoning as injecting malicious data at source nodes to degrade selected target nodes. It distinguishes direct, indirect, and hybrid attacks and expresses the optimal strategy as a bilevel problem.

  • B. Poisoning Attacks on Federated Multi-task Learning: The attack formulation targets nodes Ntar by injecting poisoned data into source nodes Nsou.Nodes outside Nsou receive no injected data.
  • B. Poisoning Attacks on Federated Multi-task Learning: Direct attacks set Ntar = Nsou, allowing poisoned data to be injected directly into every target node.The paper illustrates this with counterfeit mobile-phone sensor data.
  • B. Poisoning Attacks on Federated Multi-task Learning: Indirect attacks require Ntar ∩ Nsou = ∅, using communication among nodes to affect targets without directly injecting into them.The attacker poisons other mobile phones instead of the target nodes.
  • B. Poisoning Attacks on Federated Multi-task Learning: Hybrid attacks combine direct and indirect poisoning by injecting samples into target and source attacking nodes simultaneously.The paper formulates this strategy to maximally degrade target-node performance.
  • B. Poisoning Attacks on Federated Multi-task Learning: The upper-level problem chooses injected data, while the lower-level problem trains federated multi-task learning on clean and injected samples.The lower-level optimization acts as a constraint on the upper-level attack problem.

IV. ATTACK ON FEDERATED LEARNING (AT2FL)

AT2FL optimizes poisoning data through projected stochastic gradient ascent. It addresses the bilevel problem’s nonlinear lower level by updating injected features while fixing their labels and enforcing feasibility.

  • A. Attacking Alternating Minimization: AT2FL uses projected stochastic gradient ascent to increase target-node empirical loss and damage classification or regression performance.The method is designed for optimal attack strategies in federated learning.
  • A. Attacking Alternating Minimization: The bilevel attack is difficult because its lower-level problem is highly nonlinear and non-convex.The upper-level problem is comparatively simple, but depends on the lower-level solution.
  • A. Attacking Alternating Minimization: AT2FL reduces attack complexity by optimizing injected features while fixing the labels of injected data.The feature updates follow the direction that maximizes the target-node objective.
  • A. Attacking Alternating Minimization: Injected features are updated iteratively with step size η1 and projected onto a feasible region X.The feasible region is specified by the upper-level constraints.
  • A. Attacking Alternating Minimization: The corresponding dual variables are updated gradually as the injected features change.The feature and dual-variable updates are coupled within the attack optimization.

B. Gradients Computation

The gradient-computation procedure derives attack gradients through the lower-level federated optimization. It updates dual variables locally, reconstructs model variables, and supports least-square and hinge losses.

  • B. Gradients Computation: The gradient of a target node’s loss with respect to injected data is computed using the chain rule.This connects direct loss sensitivity to the injected data’s effect on the target model.
  • B. Gradients Computation: The lower-level dual subproblem is obtained after fixing Ω to eliminate lower-level constraints.The resulting formulation supports distributed computation of attack gradients.
  • B. Gradients Computation: Because lower-level optimality constrains the upper-level attack, Ω is treated as constant while gradients are computed.W and α remain connected through a differentiable formulation involving R*(Xα).
  • B. Gradients Computation: The key computational task is updating α and then computing the corresponding injected-data gradients.The method separates the gradient contribution from the target loss from the lower-level optimality contribution.
  • B. Gradients Computation: AT2FL initializes with target and source nodes plus an attacker budget, then updates local dual variables across nodes in parallel.Source nodes additionally update malicious-data dual variables.
  • B. Gradients Computation: After local dual updates, the procedure updates W and Ω and then updates injected features for source nodes.The full optimization procedure is summarized in Algorithm 1.
  • B. Gradients Computation: Local dual subproblems use quadratic approximations and randomly selected clean and injected samples for distributed updates.The method supports least-square and hinge-loss cases through corresponding dual updates.
  • B. Gradients Computation: The resulting gradients are computed for least-square regression and hinge-loss classification settings.The gradient derivation uses the lower-level optimality conditions and associated loss-specific updates.

V. EXPERIMENTS

This section evaluates the proposed poisoning attacks and their convergence using several datasets and analyses.

  • The evaluation covers adopted datasets, experimental attack results, and convergence analysis of the proposed model.

A. Real Datasets

The experiments use three classification datasets and one regression dataset, with performance measured by classification Error or RMSE.

  • The benchmark includes three classification datasets and one regression dataset.
  • Datasets: EndAD contains 9,769 lesion images and 9,768 healthy images from 1,063 volunteers, represented using 128-dimensional VGG features.
  • Datasets: Human Activity Recognition models 30 individuals as separate tasks using 561-dimensional sensor feature vectors.
  • Datasets: Landmine contains 14,820 samples divided across 29 geographical regions and uses binary labels for landmines and clutter.
  • Datasets: Parkinson predicts symptom scores from 16 biomedical features across 42 patient-specific regression tasks, producing Parkinson-Total and Parkinson-Motor datasets.
  • Evaluation: Classification Error and RMSE evaluate classification and regression performance, respectively; smaller values indicate better model performance and weaker attacks.

B. Evaluating Attack Strategies

The study compares direct, indirect, hybrid, and randomized attacks across federated datasets, showing broad vulnerability and dependence of indirect attacks on node correlations.

  • Attack setup: Each dataset uses half the nodes as targets and the remaining nodes as free nodes; EndAD has three target nodes and Human Activity has 15.
  • Attack setup: Direct attacks use target nodes as sources, indirect attacks use the remaining nodes, and hybrid attacks randomly select sources.
  • Attack setup: Randomized variants retain these source configurations but randomly choose injected samples; each source injects 20% of clean data, with results averaged over ten repetitions.
  • Results: All attack strategies affect every dataset, while direct attacks significantly damage classification or regression performance across datasets.
  • Results: 21.707% and 26.836% deterioration occur on EndAD and Human Activity classification Error, respectively, under the reported direct attacks.
  • Results: Optimized attacks generally produce greater deterioration than random attacks, indicating learned attack strategies outperform random injection in most cases.
  • Results: Indirect attacks perform better on Landmine than Parkinson because target and source nodes are more highly correlated in Landmine.

C. Sensitivity Study and Convergence Analysis

This section studies how injected-data ratios and step sizes affect attack strategies, and analyzes convergence of AT2FL.

  • The sensitivity studies vary the injected-data ratio and step size η, followed by convergence analysis of AT2FL.

1) Effect of Ratio of Injected Data:

Experiments examine how injected-data ratio, step size, and convergence affect AT2FL attack strategies across federated-learning datasets. The results indicate that attack effectiveness varies by strategy and dataset, while AT2FL converges to local optima.

  • Effect of Ratio of Injected Data: Direct attacks outperform indirect attacks as the injected-data ratio varies because they directly involve target-node data.Indirect attacks have almost no effect on EndAD, where node correlations are relatively low and nodes can learn classifiers without relying heavily on others.
  • Effect of Ratio of Injected Data: Classification Error on the test data slightly decreases after a 10% injection ratio, while reaching its upper bound at 10%.The paper attributes this behavior to the attack objective maximizing training loss rather than test-data classification Error.
  • Effect of Step Size: Increasing the step size η improves attack-strategy performance, which tends toward a fixed point as η increases.The experiments vary η over {0.01, 0.1, 1, 10, 100, 1000} on Human Activity Recognition and Landmine.
  • Convergence of Proposed AT2FL: AT2FL converges to a local optimum after a few iterations for three attack types on EndAD and Human Activity Recognition.Convergence is assessed using target-node classification Error and lower-level primal sub-optimality for Eq. (3).
  • Conclusion: The study reports that communication protocols can enable effective indirect poisoning attacks, particularly between strongly correlated nodes.The paper identifies vertical and federated transfer learning poisoning attacks as future work.

APPENDIX A DEFINITION OF LEAST-SQUARE AND HINGE LOSSES

The appendix introduces least-square loss and presents associated dual formulations and dual variables for node-level regression or classification problems.

  • The regression problem uses least-square loss.
  • The appendix gives a dual formulation and a dual problem for the regression setup.
  • The symbol ℓ identifies the corresponding dual variable for the ℓ-th node in regression or classification problems.
Loading 2004.10020v1…