Empirical Evaluation of Data Poisoning Attacks in Supervised Learning

Toshif Khan; Muhammad Abusaqer

Source-linked AI summary

Empirical Evaluation of Data Poisoning Attacks in Supervised Learning

Toshif Khan, Muhammad Abusaqer

arXiv:2609.10952v1Sep 10, 2026cs.CRcs.LG

TL;DR

Data poisoning can either degrade overall predictive quality or implant targeted malicious behavior while preserving ordinary performance. This study compares label flipping and backdoor poisoning across MNIST, Fashion-MNIST, and three classical classifiers, finding that backdoors were highly effective and comparatively covert.

Problem
Compact, reproducible evidence is needed to compare representative poisoning attacks across datasets and baseline classifiers because training-data trust is a basic assumption of supervised learning.
Method
The study evaluates label flipping and trigger-based backdoor poisoning across MNIST and Fashion-MNIST using Logistic Regression, Linear SVM, and Random Forest under a consistent protocol.
Results
Backdoor attack success remained extremely high across classifiers and datasets while clean accuracy and macro-F1 often stayed close to baseline; label flipping more clearly degraded clean performance, especially for linear models.
Takeaways & Limitations
Conventional clean-test metrics alone are insufficient for assessing poisoning risk, so security-oriented measures such as attack success rate are needed.
Takeaways & Limitations
The benchmark uses classical rather than deep models, a simple visible trigger, and moderate poisoning rates, so it is not a comprehensive adversarial evaluation.

Abstract

from arXiv · show

Data poisoning corrupts training data to degrade a model or to plant attacker-controlled behavior. This study evaluates two representative training-time attacks, label flipping and backdoor poisoning, on MNIST and Fashion-MNIST with three baseline classifiers: Logistic Regression, Linear SVM, and Random Forest. Clean training is compared with poisoning rates of 5%, 10%, and 20% using clean-test accuracy, macro-precision, macro-recall, macro-F1, and, for backdoors, attack success rate. Label flipping caused clear degradation, largest for Logistic Regression and Linear SVM, while Random Forest stayed comparatively stable. Backdoor poisoning reached attack success rates from 0.9667 to 1.0000 on both datasets and all three models while often keeping clean-test performance near baseline. The results separate indiscriminate poisoning, which shows up in standard metrics, from targeted backdoor poisoning, which stays comparatively stealthy while embedding highly effective malicious behavior, and they support security-oriented evaluation beyond conventional clean-test metrics.

1. Introduction

Data poisoning threatens supervised-learning systems by corrupting training data, degrading predictive quality, or implanting attacker-directed behavior. This paper addresses the need for controlled empirical comparisons of representative poisoning attacks across standard datasets and models.

1. Introduction: The study compares label flipping, which targets overall model quality, with backdoor poisoning, which embeds trigger-based attacker behavior.These attacks represent indiscriminate degradation and targeted compromise, respectively.
1. Introduction: The study provides a unified, reproducible comparison intended to clarify how poisoning affects models under controlled conditions.Its contribution is empirical rather than algorithmic.
1. Introduction: The benchmark evaluates MNIST and Fashion-MNIST with Logistic Regression, Linear SVM, and Random Forest.It measures clean-test classification metrics and attack success rate for backdoors.

2. Related Work

Prior work distinguishes indiscriminate poisoning from targeted backdoors and shows that clean accuracy alone can miss compromise. This paper extends that literature with a compact comparison of representative attacks, datasets, and interpretable baseline models.

2. Related Work: The literature includes foundational attacks on classical classifiers, standardized backdoor benchmarks, online poisoning, and surveys of emerging threat surfaces.These strands establish a broad research context for the present benchmark.
2. Related Work: Backdoor studies show that models can retain normal clean-input behavior while reliably misclassifying triggered inputs.This motivates evaluation beyond clean accuracy.
2. Related Work: Poisoning research distinguishes label flipping that reduces overall accuracy from backdoors that implant trigger-based behavior.The distinction spans classical, deep-learning, federated, and newer model settings.
2. Related Work: The paper addresses the value of concise common-protocol studies by comparing label flipping and backdoors on MNIST and Fashion-MNIST with three classical classifiers.The resulting design is a streamlined empirical baseline rather than a new attack or defense.

3. Methodology and Experimental Setup

The paper uses a controlled benchmark protocol to compare label flipping and trigger-based backdoors across two image datasets, three classical classifiers, and multiple poisoning rates. Clean and triggered test conditions separate ordinary predictive quality from targeted attack behavior.

3. Methodology and Experimental Setup: The study compares label flipping and backdoor poisoning across datasets, poisoning rates, and baseline classifiers under one unified protocol.It is explicitly an empirical study, not a proposal for a new attack or defense.
3. Methodology and Experimental Setup: MNIST and Fashion-MNIST provide 28 × 28 grayscale, 10-class image benchmarks for the experiments.Fashion-MNIST is used as a more challenging drop-in replacement for MNIST.
3. Methodology and Experimental Setup: Logistic Regression, Linear SVM, and Random Forest serve as established, efficient, and interpretable baseline classifiers.Their selection supports transparent comparison across model families.
3. Methodology and Experimental Setup: Label flipping randomly replaces selected true labels with incorrect labels at poisoning rates of 5%, 10%, and 20%, leaving images unchanged.The attack corrupts supervision while preserving image content.
3. Methodology and Experimental Setup: Backdoor poisoning stamps a visible 3 × 3 white square onto selected images and relabels them to target class 0.The trigger is applied to non-target training samples at the same poisoning rates.
3. Methodology and Experimental Setup: Clean test data measures accuracy, macro-precision, macro-recall, and macro-F1, while triggered non-target test samples measure backdoor attack success rate.Poisoning is applied only to training data, with a separate triggered test condition for targeted behavior.

4. Results

Clean baselines were strongest for Random Forest and on MNIST, while label flipping most visibly degraded Logistic Regression and Linear SVM as poisoning increased. These results establish the clean-performance reference and the main label-flipping pattern.

4. Results: Random Forest produced the strongest clean baselines, while Linear SVM produced the weakest across both datasets.MNIST was consistently easier than Fashion-MNIST for all three models.
4. Results: On MNIST, accuracy and macro-F1 ranged from 0.8400/0.8358 for Linear SVM to 0.9475/0.9470 for Random Forest.These values define the clean reference range before poisoning.
4. Results: On Fashion-MNIST, accuracy and macro-F1 ranged from 0.7665/0.7656 to 0.8455/0.8435 across the models.The lower range than MNIST reflects the dataset’s more difficult clean-baseline performance.
4. Results: As poisoning increased from 5% to 20%, label flipping most degraded Logistic Regression and Linear SVM, while Random Forest remained comparatively stable.Macro-F1 followed the same overall direction as accuracy, indicating broad predictive degradation.

F1 F1 Drop

Label flipping visibly degraded clean-test performance, especially for Logistic Regression and Linear SVM, while Random Forest remained comparatively stable; backdoor poisoning achieved highly effective targeted behavior while often preserving clean performance.

Label flipping: At 20% poisoning on MNIST, Logistic Regression lost 0.0705 accuracy and 0.0716 macro-F1, while Random Forest’s accuracy drop stayed at or below 0.0075.These results show a substantial sensitivity gap between the linear model and Random Forest in this setting.
Label flipping: On Fashion-MNIST, Linear SVM’s worst condition was 10% poisoning, with accuracy and macro-F1 drops of 0.0660 and 0.0680, while Random Forest’s accuracy drop stayed at or below 0.0070.Logistic Regression also degraded, but Random Forest changed only modestly.
Backdoor poisoning: Backdoor attack success rates ranged from 0.9667 to 1.0000 while often preserving clean-test performance near baseline across MNIST, Fashion-MNIST, and all three classifiers.This contrast makes backdoor poisoning comparatively stealthy under standard clean-test evaluation.
Backdoor poisoning: Random Forest reached a backdoor attack success rate of 1.0000 at every poisoning rate on MNIST while maintaining clean accuracy close to baseline.On Fashion-MNIST, it reached 1.0000 at 10% and 20% poisoning while preserving strong clean performance.
Security interpretation: Standard clean-test metrics alone were insufficient to detect backdoor compromise, unlike label flipping’s visible degradation of ordinary performance.Backdoor poisoning embedded attacker-controlled behavior while remaining comparatively stealthy.

5. Discussion

The experiments distinguish visible indiscriminate degradation from comparatively stealthy targeted compromise. Backdoors retained much clean-test performance while achieving highly reliable trigger-based misclassification, whereas label flipping affected models differently.

Label flipping reduced accuracy and macro-F1 most clearly for Logistic Regression and Linear SVM, while Random Forest showed much smaller declines.The comparison indicates sensitivity varied by model family under the study’s settings.
Backdoor poisoning preserved clean accuracy and macro-F1 near baseline while achieving attack success rates often approaching or reaching 1.0000 across datasets and classifiers.This makes clean-test evaluation alone insufficient to reveal compromise.
Random Forest’s stronger clean performance did not imply stronger security, because it also showed perfect or near-perfect backdoor attack success under several settings.Clean-task quality and trustworthiness were therefore distinct concerns in this benchmark.
MNIST was consistently easier than Fashion-MNIST, yet both datasets showed visible label-flipping degradation and high targeted backdoor effectiveness.The benchmark influenced absolute outcomes without changing the qualitative attack trends.
The study’s compact scope limits interpretation to classical models, a simple visible trigger, and moderate poisoning rates rather than a comprehensive adversarial evaluation.Within those boundaries, the attacks still produced meaningful degradation or highly successful hidden compromise.

6. Future Work

Future work should test whether the observed poisoning trends persist across more attack formulations, model families, defenses, realistic settings, and broader trustworthy-AI applications. These extensions would move the compact baseline toward more comprehensive evaluation.

Additional poisoning strategies, including clean-label, optimization-based, and adaptive backdoor attacks, could test whether the observed trends persist under subtler or more powerful formulations.The proposed extensions broaden attack coverage beyond the two representative attacks studied here.
Comparisons with convolutional neural networks, vision transformers, and other deep architectures could assess whether representational capacity changes poisoning susceptibility or stealth.The current baselines were selected for transparency and computational manageability.
Systematic defense benchmarking should evaluate sanitization, robust training, influence-based filtering, and mitigation–clean-performance trade-offs under the same protocol.The present study intentionally prioritized attack comparison rather than defense evaluation.
More realistic evaluations could vary datasets, class imbalance, attacker knowledge, trigger placement, attacker control, and pre-training data validation.These changes would expand the threat models and data conditions represented by the benchmark.
Connecting benchmark studies to trustworthy-AI education could help students and practitioners understand how security weaknesses persist despite strong ordinary performance metrics.The paper positions this instructional framework as relevant to poisoning research across deep learning, federated learning, and language models.

7. Conclusion

The paper benchmarks label flipping and backdoor poisoning across two image datasets and three classical classifiers. It finds that label flipping visibly reduces clean performance, whereas backdoors achieve highly effective targeted misclassification while preserving much ordinary behavior, establishing a comparative baseline for security-oriented evaluation.

Backdoor poisoning achieved very high attack success rates across both datasets and all three classifiers, while label flipping reduced clean-test performance especially for the two linear models.The conclusion summarizes the paper’s cross-dataset and cross-model result pattern.
Label flipping was visible through declines in ordinary accuracy and macro-F1, whereas backdoor poisoning covertly embedded attacker-controlled misclassification while preserving much clean-task behavior.The attacks therefore differed according to their objectives and observable effects.
The benchmark demonstrates that conventional evaluation metrics alone are insufficient for assessing model security against targeted poisoning.Security-oriented measures are needed alongside clean-task performance metrics.
The study offers a clear comparative baseline for classroom use, further experimentation, and subsequent research on poisoning resilience without proposing a new attack or defense.Its value is empirical and benchmark-oriented rather than algorithmic.