Source-linked AI summary
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C. Lupu, Fabio Roli
TL;DR
Machine-learning services are vulnerable to training-data poisoning, while prior attacks were limited mainly to binary learners. The paper extends poisoning to multiclass settings and uses back-gradient optimization to make attacks more computationally efficient, demonstrating substantial degradation across spam, malware, and handwritten-digit classification tasks.
Problem
Training-data poisoning threatens machine-learning services, but prior poisoning attacks primarily targeted binary learning algorithms.
Method
The paper extends the threat model to multiclass poisoning and uses reverse-mode automatic differentiation while reversing learning updates to optimize poisoning points efficiently.
Results
The attacks significantly degrade classifiers, increasing Spambase error up to 30% with 15% poisoned training data and nearly doubling handwritten-digit error with less than 6% poisoning points.
Takeaways & Limitations
Poisoning attacks can substantially compromise multiclass classifiers and affect linear and neural models across spam, malware, and handwritten-digit recognition settings.
Takeaways & Limitations
The study does not provide an extensive evaluation of poisoning attacks against deep networks, leaving their security insufficiently assessed.
Abstract
from arXiv · showhide
A number of online services nowadays rely upon machine learning to extract valuable information from data collected in the wild. This exposes learning algorithms to the threat of data poisoning, i.e., a coordinate attack in which a fraction of the training data is controlled by the attacker and manipulated to subvert the learning process. To date, these attacks have been devised only against a limited class of binary learning algorithms, due to the inherent complexity of the gradient-based procedure used to optimize the poisoning points (a.k.a. adversarial training examples). In this work, we rst extend the de nition of poisoning attacks to multiclass problems. We then propose a novel poisoning algorithm based on the idea of back-gradient optimization, i.e., to compute the gradient of interest through automatic di erentiation, while also reversing the learning procedure to drastically reduce the attack complexity. Compared to current poisoning strategies, our approach is able to target a wider class of learning algorithms, trained with gradient- based procedures, including neural networks and deep learning architectures. We empirically evaluate its e ectiveness on several application examples, including spam ltering, malware detection, and handwritten digit recognition. We nally show that, similarly to adversarial test examples, adversarial training examples can also be transferred across di erent learning algorithms.
1 INTRODUCTION
Machine-learning services face data-poisoning threats because attackers can manipulate training data or test inputs. This work extends poisoning to multiclass settings and introduces back-gradient optimization for broader gradient-trained models.
- Poisoning attacks let attackers control part of the training data to subvert learning or facilitate later evasion.
- Prior poisoning methods targeted algorithms such as SVMs, LASSO, logistic regression, and ridge regression, but excluded neural networks and deep learning.The technical challenge is solving a bilevel optimization problem for poisoning samples; prior methods used implicit differentiation tied to limited algorithm classes.
- The paper extends poisoning attacks to multiclass classification and distinguishes specific from generic misclassification goals.
- Back-gradient optimization reverses the learning procedure and uses reverse-mode automatic differentiation to compute poisoning gradients without storing all parameter updates.The approach requires smooth parameter updates during training, such as gradient descent.
- The authors evaluate poisoning in spam filtering, malware detection, and handwritten digit recognition, including attacks using a small fraction of poisoning points.They also investigate whether attacks transfer across different learning algorithms.
2 THREAT MODEL
The threat model characterizes attacks by their goals, attacker knowledge, and data-manipulation capabilities, then formalizes poisoning as optimization over manipulated training samples. For multiclass settings, it adds error specificity and defines surrogate-data objectives for evaluating poisoning impact.
- Threat-model framework: The framework distinguishes training-time poisoning from test-time evasion and represents attack construction as an optimization problem over adversarial examples.It accounts for attacker goals, system knowledge, and input-data manipulation capabilities.
- Attack goals: Multiclass attacks require error specificity because misclassification can target multiple incorrect classes rather than one unique alternative.The extension applies to multiclass classification and is detailed for poisoning scenarios.
- Attacker’s Knowledge: Attacker knowledge ranges from training data, features, and learning procedure to trained parameters, with perfect-knowledge and surrogate-learner settings covering different information assumptions.Surrogate learners are used when the attacker knows the data but not the targeted learning algorithm.
- Attacker’s Capability: Attack capability includes causative versus exploratory influence and practical manipulation constraints, including label-related constraints in poisoning.Causative attacks affect training and test data, whereas exploratory attacks affect only test data.
- Poisoning Attack Scenarios: For poisoning, surrogate data is split into training and validation sets: poisoning points train the surrogate model, while untainted validation data measures attack impact.The attack objective depends on poisoning through the parameters of the poisoned surrogate model.
- Poisoning Attack Scenarios: Desired-label poisoning evaluates targeted misclassification by assigning attacker-chosen labels to validation samples and minimizing loss on that desired-label set.Some desired labels may remain equal to the true labels for integrity or targeted attacks.
3 POISONING ATTACKS WITH BACK-GRADIENT OPTIMIZATION
Back-gradient optimization replaces exact inner optimization with reversible gradient-based training iterations, making poisoning-gradient computation more efficient and applicable to neural networks and deep learning architectures.
- Poisoning with Back-gradient Optimization: Back-gradient optimization enables poisoning attacks against neural networks and deep learning architectures without using a surrogate model.The authors describe it as more computationally efficient and stabler than the preceding gradient-based procedure.
- Gradient-based Poisoning Attacks: The attack optimizes one poisoning point at a time, keeps its attacker-chosen label fixed, and projects updates onto a feasible manipulation set.The feasible set can impose bounds on manipulated feature values and label-dependent constraints.
- Gradient-based Poisoning Attacks: Previous gradient-based poisoning attacks were limited to learning algorithms with convex objectives because they relied on stationarity conditions for the inner problem.The approach also requires costly Hessian inversion and linear-system solves, which can make practical assessment prohibitive.
- Poisoning with Back-gradient Optimization: Back-gradient optimization replaces the inner optimization with a finite sequence of learning iterations and computes the outer gradient by reversing those updates with automatic differentiation.This permits gradients based on incompletely optimized inner problems after T training iterations.
- Poisoning with Back-gradient Optimization: The method avoids storing all parameter updates required by naïve back-propagation, while Hessian-vector products efficiently compute the reverse-pass terms.Naïve back-propagation can be extremely memory-demanding because it stores the full sequence of parameter updates and forward derivatives.
- Poisoning with Back-gradient Optimization: For multiclass problems, poisoning can be error-generic or error-specific, with the latter targeting selected misclassifications while preserving other labels.Figure 1 illustrates these attacks against a multiclass logistic classifier on a three-class synthetic dataset using one blue-labeled poisoning point.
4 EXPERIMENTAL ANALYSIS
The experiments evaluate back-gradient poisoning across spam, malware, handwritten-digit, and deep-network settings, showing substantial degradation with few poisoning points and asymmetric transferability across models.
- Experimental scope: The evaluation covers spam filtering, malware detection, multiclass handwritten-digit recognition, transferability, and an end-to-end deep-network poisoning proof of concept.Experiments use Spambase, Ransomware, and MNIST, with MLPs, logistic regression, Adaline, and a CNN.
- Spam and malware detection: On Spambase, PK attacks raise ADA and LR classification error to 30% and MLP error to 25% while controlling only 15% of training data.Ransomware shows similar results, with MLP vulnerability comparable to ADA and LR.
- Transferability: Poisoning samples transfer effectively between linear algorithms and noticeably affect neural networks, whereas transfers from nonlinear to linear models are less effective.Linear-surrogate attacks have lower impact on MLPs, while MLP-surrogate attacks produce smaller effects on linear models.
- Handwritten digit recognition: Less than 6% poisoning nearly doubles multiclass LR error under error-generic attacks and affects classes comparably, though digit 5 is easier to poison.The comparison uses the confusion matrix under 6% poisoning versus no attack.
- Handwritten digit recognition: Under error-specific poisoning, fewer than 4% poisoning points increase digit-8 error from approximately 20% to 50%, with most 8s misclassified as 3s.Digit-3 error also rises slightly, but not significantly.
- Deep neural networks: The CNN transfer experiment is asymmetric: LR-crafted poisoning is totally ineffective against the CNN, while CNN-crafted samples affect LR similarly to random label flips.The authors identify this as a proof-of-concept experiment and leave systematic deep-network security evaluation for future work.
5 RELATED WORK
Earlier poisoning attacks targeted binary or selected learning algorithms but were computationally demanding and poorly suited to gradient-descent-trained deep networks. This work addresses those constraints with end-to-end poisoning across all deep-network parameters without a surrogate model.
- Prior poisoning attacks: Prior practical poisoning attacks targeted two-class classification algorithms and methods including SVMs, LASSO, logistic regression, and ridge regression.These attacks were applied in settings such as spam filtering and anomaly detection.
- Computational limitations: KKT-based poisoning methods require full retraining and complex gradient computation, making them computationally demanding and limiting applicability to deep neural networks.Gradient-descent optimization is often truncated before the stationarity precision needed for effective poisoning gradients.
- Deep-network poisoning: Compared with prior deep-network work using a logistic surrogate on fixed penultimate-layer features, this work poisons the network end-to-end across all parameters and layers without a surrogate model.The authors characterize this as the first such demonstration to their knowledge.
6 CONCLUSIONS, LIMITATIONS AND FUTURE WORK
The paper extends poisoning attacks from binary to multiclass classification and proposes a back-gradient method that applies to a wider class of learning algorithms. It demonstrates that small poisoning fractions can compromise real-world systems, while acknowledging that deep-network security remains insufficiently evaluated.
- Conclusions: The work introduces multiclass poisoning attacks and distinguishes specific misclassification errors from generic errors.Error specificity captures whether poisoned models cause samples to be assigned to a particular wrong class or any incorrect class.
- Conclusions: Small fractions of poisoning points can drastically compromise learning performance in applications including cybersecurity and other data-driven systems.The paper frames these systems as increasingly important targets as machine-learning use and cyberattack sophistication grow.
- Conclusions: Back-gradient optimization removes the need for attacked algorithms to satisfy stationarity conditions, broadening the applicable learner class.The approach requires smooth parameter updates and can estimate gradients from truncated, incomplete optimization.
- Conclusions: The attack can efficiently target large neural networks, deep learning architectures, and other gradient-trained algorithms.This follows from estimating gradients through parameters obtained after a reduced number of training iterations.
- Limitations and future work: The authors did not extensively evaluate poisoning attacks against deep networks, so their security to poisoning remains incompletely assessed.Preliminary experiments suggest greater resilience than other learning algorithms, but systematic analysis is still needed.