PaperlayerView exact revision on arXiv

Source-linked AI summary

De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks

Jian Chen, Xuxin Zhang, Rui Zhang, Chen Wang, Ling Liu

arXiv:2105.03592v1cs.CRcs.LGcs.PF

TL;DR

Data-poisoning defenses are often attack-specific, despite attacks that can disrupt learning by injecting malicious training samples. De-Pois uses GAN-based augmentation and a mimic model to compare predictions and identify poisoned data. It reports accuracy and F1-score above 0.9 on average across four attack types, while performance declines as the trusted-data ratio decreases.

  • Problem

    Existing poisoning defenses are largely attack-specific, and few general strategies address the different principles used by poisoning attacks.

    Source: §I · p. 2, D. Regression Attack (R-attack) · p. 3

  • Method

    De-Pois uses GAN-based data augmentation and conditional WGAN-GP to train a mimic model, then compares predictions to recognize poisoned samples.

    Source: §I · pp. 1–2

  • Results

    Accuracy and F1-score are each over 0.9 on average across four poisoning-attack types, according to the reported evaluation.

    Source: §I · p. 2

  • Takeaways & Limitations

    The authors report that De-Pois works across the evaluated attack types and performs better in most cases than existing defense techniques.

    Source: §VII · p. 13

    On average, accuracy and F1-score decrease from 90% to 70% as the trusted-data ratio falls from 5% to 0.5%.

    Source: H. Evaluation on Limitations of De-Pois · p. 11

Abstract

from arXiv · show

Machine learning techniques have been widely applied to various applications. However, they are potentially vulnerable to data poisoning attacks, where sophisticated attackers can disrupt the learning procedure by injecting a fraction of malicious samples into the training dataset. Existing defense techniques against poisoning attacks are largely attack-specific: they are designed for one specific type of attacks but do not work for other types, mainly due to the distinct principles they follow. Yet few general defense strategies have been developed. In this paper, we propose De-Pois, an attack-agnostic defense against poisoning attacks. The key idea of De-Pois is to train a mimic model the purpose of which is to imitate the behavior of the target model trained by clean samples. We take advantage of Generative Adversarial Networks (GANs) to facilitate informative training data augmentation as well as the mimic model construction. By comparing the prediction differences between the mimic model and the target model, De-Pois is thus able to distinguish the poisoned samples from clean ones, without explicit knowledge of any ML algorithms or types of poisoning attacks. We implement four types of poisoning attacks and evaluate De-Pois with five typical defense methods on different realistic datasets. The results demonstrate that De-Pois is effective and efficient for detecting poisoned data against all the four types of poisoning attacks, with both the accuracy and F1-score over 0.9 on average.

I. INTRODUCTION

Data poisoning can manipulate a model’s decision boundary, while existing defenses often target particular attack types and few general strategies exist. De-Pois trains a mimic model to identify poisoned samples by comparing its predictions with the target model’s, using GAN-based data augmentation and mimic-model construction.

I. INTRODUCTION

  • Existing poisoning defenses are largely attack-specific, and few general strategies address attacks that follow different principles.The introduction notes that a defense for targeted clean-label attacks may fail against gradient-based poisoning, while a regression defense may not apply to classification.Source: §I · p. 1
  • De-Pois compares mimic-model and target-model predictions to distinguish poisoned from clean samples, without requiring explicit knowledge of the ML algorithm or poisoning type.It uses cGANs for training-data augmentation and conditional WGAN-GP to construct the mimic model.Source: §I · pp. 1–2
  • De-Pois is presented as a generic defense for classification and regression that needs no explicit knowledge of the ML algorithm or attack type.The paper evaluates it against four poisoning-attack types on four realistic datasets, comparing it with five typical defenses.Source: §I · p. 2
  • Accuracy and F1-score are each over 0.9 on average across the four poisoning-attack types, according to the reported evaluation.The authors describe De-Pois as effective and efficient for detecting poisoned data.Source: §I · p. 2

A. Targeted Clean-Label Poisoning Attack (TCL-attack)

The section reviews four poisoning-attack types and defenses tailored to them, contrasting these with De-Pois’s attack-agnostic approach. Existing defenses use differing detection principles and may not transfer across attack types or learning tasks.

A. Targeted Clean-Label Poisoning Attack (TCL-attack)

  • Targeted clean-label poisoning uses cleanly labeled, minimally perturbed samples to make a specific test sample misclassify; Deep-kNN detects samples through neighboring labels.The reviewed stronger attack relaxes constraints on poisons to avoid obvious target patterns, while Deep-kNN flags a sample when most of its k neighbors do not share its label.Source: A. Targeted Clean-Label Poisoning… · p. 2

B. Poisoning Attack with GAN (pGAN-attack)

  • pGAN-attack uses a GAN to craft adversarial training examples, while certified defense filters outliers and uses margin-based loss and testing-loss bounds to identify some poisons.The attack’s α hyperparameter trades off poison detectability and effectiveness; higher α indicates greater chances of evading detection.Source: B. Poisoning Attack with GAN… · p. 3

C. Label-Flipping Attack (LF-attack)

  • Label-flipping attacks alter training labels, and reviewed defenses include label sanitization and DUTI’s bi-level optimization with trusted items and expert inspection.Label sanitization relies on poisoned samples being farther from the SVM decision boundary and thus more likely to be relabeled.Source: C. Label-Flipping Attack (LF-attack) · p. 3

D. Regression Attack (R-attack)

  • Regression poisoning defenses include TRIM, which iteratively estimates residuals and treats the smallest-residual subset as pristine, and DUTI.The review says existing defenses rely on different analyses, including task-specific losses or nearest neighbors, making cross-task detection difficult.Source: D. Regression Attack (R-attack) · p. 3
  • De-Pois is described as a generic, attack-agnostic defense, unlike methods that address particular attacks or tasks.The review notes that attack-specific approaches may be ineffective when the defender does not know which attack was carried out.Source: D. Regression Attack (R-attack) · p. 3

III. THREAT MODEL AND DEFENSE CAPABILITY

The threat model describes attackers’ goals, knowledge, and control over training data, while De-Pois assumes access to some trusted clean data. Its three-step design augments that data, constructs a mimic model, and uses a detection boundary to recognize poisoned samples.

III. THREAT MODEL AND DEFENSE CAPABILITY

  • The attacker may seek indiscriminate prediction degradation or specific errors, and may have white-box or black-box knowledge of the target model.Poisoning capability can include changing features and labels, features only, or labels only; the poison fraction is often constrained below 30%.Source: §III · pp. 3–4

IV. DE-POIS DESIGN

  • De-Pois classifies samples in a potentially poisoned training set using a small amount of trusted clean data and prediction differences from a mimic model.The mimic model is intended to reproduce the target model’s prediction behavior when trained on clean data.Source: §IV · p. 4
  • De-Pois generates synthetic training data with cGAN and an authenticator, then uses conditional WGAN-GP to construct a mimic model.The mimic model is the WGAN-GP discriminator; a detection boundary then marks a sample as poisoned when the mimic output falls below it.Source: §IV · p. 4, A. Synthetic Data Generation · p. 4

1) cGAN-based Generator:

  • The conditional generator receives additional information such as labels, and regression values are conditioned with additive unimodal noise to cover outputs.The paper notes that an unconditioned GAN offers no control over the modes of generated data.Source: 1) cGAN-based Generator · p. 4

2) Authenticator:

The authenticator supervises cGAN augmentation by evaluating synthesized samples against their labels or regression values. The paper also notes that cGAN-generated data may lack diversity, especially in low-volume data settings.

2) Authenticator:

  • cGAN-generated data may have less diverse expressions because of a single input distribution, and its fidelity and variety are uncertain with low-volume data.The paper identifies this as a limitation of the generation approach.Source: 2) Authenticator · p. 4
  • The authenticator evaluates synthesized samples against their true labels or regression values and back-propagates its loss to supervise cGAN augmentation.The discriminator minimizes LcGAN + LA, while the generator minimizes LcGAN − LA; the paper extends the authenticator from classification to regression.Source: 2) Authenticator · pp. 4–5
  • For classification, the authenticator uses a CNN and cross-entropy loss; for regression, it uses a regression module such as LASSO and mean squared error.The classification loss uses predicted class probabilities and class labels; the regression loss compares the predicted and true regression values.Source: 2) Authenticator · pp. 4–5

3) Synthetic Data Generation:

Synthetic data augment trusted samples because the posterior cannot be calculated directly; an iterative E-step/M-step procedure estimates the augmented posterior and optimizes the model parameters.

3) Synthetic Data Generation:

  • Because p(θ|s) cannot be calculated directly, the method adds synthesized data zs = (xs, ys) to estimate an augmented posterior in the E-step.Here xs is synthesized data and ys its associated class label or regression value.Source: 3) Synthetic Data Generation · p. 5
  • The M-step maximizes the Q function, and training stops when successive parameter values differ by sufficiently little.The last iteration supplies the optimal θ*.Source: 3) Synthetic Data Generation · p. 5
  • Monte Carlo sampling approximates the E-step integration, while SGD updates parameters using subsets of trusted and augmented data.The resulting synthetic training data are intended to have a distribution similar to clean data; the generated amount is adjusted to match the augmentation setting.Source: 3) Synthetic Data Generation · p. 5

B. Mimic Model Construction

De-Pois constructs a mimic model with conditional WGAN-GP, then uses its prediction values and a boundary derived from augmented data to identify poisoned samples.

B. Mimic Model Construction

  • Conditional WGAN-GP conditions both generator and discriminator on y to construct a supervised mimic model, addressing GAN instability and low-data overfitting concerns.The method regards the discriminator as the expected mimic model after alternating generator and discriminator optimization converges.Source: B. Mimic Model Construction · p. 6

C. Poisoned Data Recognition

  • De-Pois treats mimic-model outputs below a detection boundary as poisoned and higher outputs as unpoisoned.The output is the Wasserstein distance between generated and real samples; clean samples generally have higher prediction values than poisoned ones.Source: C. Poisoned Data Recognition · p. 6
  • The boundary is derived from the mean and standard deviation of augmented-sample predictions, with standardized test predictions compared against a significance-level threshold.The paper gives z_s = −1.96 as an example threshold for significance level 0.05.Source: C. Poisoned Data Recognition · p. 6
  • After testing samples outside the trusted set, De-Pois excludes identified poisoned data before model training.Source: C. Poisoned Data Recognition · p. 6

V. DISCUSSION

The discussion distinguishes De-Pois’s data-removal use from robust-training defenses and describes its application to identifying poisoning sources and defending against GAN-generated attacks.

A. Application Scope of De-Pois

  • When poisoned data must be identified or sources traced, De-Pois fits the data-removal strategy and may help developers fix systematic training-set errors.The discussion contrasts this with defenses that reduce poison impact through robust optimization.Source: A. Application Scope of De-Pois · pp. 6–7

B. Defending Scope of De-Pois

  • De-Pois is reported to defend against GP-attack, a GAN-based poisoning method, and to outperform centroid-based CD in experiments.GP-attack uses an autoencoder to generate poisoned samples and a discriminator to calculate GAN loss.Source: B. Defending Scope of De-Pois · p. 7

1) Datasets:

  • Evaluation uses four datasets spanning digit recognition, image classification, nonlinear binary classification, and house-price prediction.The named datasets are MNIST, CIFAR-10, Fourclass, and House Pricing.Source: 1) Datasets · p. 7
  • For both Fourclass and House Pricing, data were randomly split 70%/30% into training/testing; trusted clean samples were randomly selected by the corresponding proportion and augmented to the original training-set size for mimic construction.Source: 1) Datasets · p. 7

2) Baselines:

The evaluation compares De-Pois with typical defenses only on attacks those defenses are reported to address, and specifies metrics for defense and synthetic-data performance.

2) Baselines:

  • Existing defenses are reported only against their corresponding attacks; ineffective attack-defense pairings are omitted from the comparison.Source: 2) Baselines · p. 7

3) Evaluation Metrics:

  • Defense performance is evaluated with Accuracy, Recall, and F1-score, with higher F1 indicating better defense performance.Experiments are repeated five times to average out randomness.Source: 3) Evaluation Metrics · pp. 7–8
  • Synthetic-data evaluation uses Inception Score, Fréchet Inception Distance, Wasserstein Distance, and Average Euclidean Distance.The supplied text describes IS as evaluating generated-image quality and FID as measuring similarity between real and synthetic image embedding distributions.Source: 3) Evaluation Metrics · p. 7

4) Model Settings:

De-Pois sets dataset-specific detection boundaries and uses GAN-based components for synthetic data generation and mimic-model construction. On CIFAR-10, the reported boundary distinguishes clean from TCL-poisoned samples.

4) Model Settings:

  • For synthetic-data generation, cGAN takes 100-dimensional noise and trusted labels or regression values as input, using a mini-batch of 128 with |St| = 20%|So|.The generator uses three Leaky ReLU fully connected layers and a sigmoid output layer; the discriminator uses three fully connected layers and a sigmoid unit, with dropout 0.4.Source: 4) Model Settings · p. 8
  • The authenticator uses a CNN with cross-entropy loss for image classification and LASSO with MSE loss for regression.Source: 4) Model Settings · p. 8
  • Mimic-model construction conditions WGAN-GP on labels or regression values, adding convolutional layers for images and training the discriminator five times per generator update.The reported RMSprop learning rate is 0.00005.Source: 4) Model Settings · p. 8

B. Determining Detection Boundary

  • For CIFAR-10, the detection boundary was very close to the ideal boundary and distinguished poisoned samples generated by TCL-attack from clean samples; 12,000 of each were tested.The reported boundaries were −0.073 for Fourclass and 0.683 for House Pricing.Source: B. Determining Detection Boundary · p. 8

C. Effectiveness of Synthetic Data Generation

The synthetic-data evaluation reports results close to a same-size clean-data baseline and favorable comparisons with cGAN. Under TCL-attack, De-Pois trails Deep-kNN on average but retains high accuracy and F1-score in most cases.

1) Effectiveness under TCL-attack on CIFAR-10 Dataset:

C. Effectiveness of Synthetic Data Generation

  • The augmented model's accuracy, recall, and F1-score were quite close to those of a same-size clean-data baseline across CIFAR-10, Fourclass, and House Pricing tests.The tests used 15,000 clean and poisoned TCL samples for CIFAR-10, 500 each for LF on Fourclass, and 1,000 each for R on House Pricing.Source: C. Effectiveness of Synthetic Data… · pp. 8–9
  • Synthetic generation reached the reported performance of real data and outperformed the cGAN model on the listed image and regression-dataset measures.Table II reports IS and FID for CIFAR-10, and WD and AED for Fourclass and House Pricing.Source: C. Effectiveness of Synthetic Data… · p. 9, Table II · p. 9

D. Impact of Poisoning Rate

  • Across the four poisoning-attack types, results were reported as confirming De-Pois is attack-agnostic, unlike other defenses described as attack-specific.Source: D. Impact of Poisoning Rate · p. 9

2) Effectiveness under pGAN-attack on CIFAR-10 Dataset:

Across four attack settings, De-Pois performs strongly against several baselines, while its results depend on the amount and quality of trusted data and dataset complexity. The component evaluation also reports gains from both major design components.

2) Effectiveness under pGAN-attack on CIFAR-10 Dataset:

  • Against pGAN-attack, De-Pois’s accuracy and recall exceed CD’s by 15% and 7% on average, while CD’s F1-score is 3% higher on average.The comparison uses CIFAR-10 with trusted data at 20% of the original dataset and attack stealth parameter α = 0.5.Source: 2) Effectiveness under pGAN-attack on… · p. 10

3) Effectiveness under LF-attack on Fourclass Dataset:

  • Under LF-attack, De-Pois outperforms DUTI, Sever, and CD as the poisoned rate increases; DUTI’s accuracy and F1-score exceed 0.85, while Sever’s exceed 0.8 and 0.85, respectively.Source: 3) Effectiveness under LF-attack on… · p. 10

4) Effectiveness under R-attack on House Pricing Dataset:

  • Under R-attack, De-Pois outperforms TRIM at all tested poisoned rates and beats Sever and DUTI below 20%, but not at higher rates.The passage attributes the latter comparison to Sever and DUTI’s retraining after outlier detection.Source: 4) Effectiveness under R-attack on… · p. 10

E. Impact of Trusted Clean Data Size

  • Under TCL-, LF-, and R-attacks, De-Pois’s accuracy, recall, and F1-score average above 0.9 and fluctuate less than 10% as trusted-data size increases.With trusted data at 5% of the original dataset, De-Pois also has effectiveness comparable to Deep-kNN under TCL-attack.Source: E. Impact of Trusted Clean Data Size · p. 10

F. Impact of Each Component in De-Pois

G. Effectiveness under GP-attack on MNIST Dataset

H. Evaluation on Limitations of De-Pois

I. Evaluation on Run-time Overhead

De-Pois adds runtime for detection and training, with detection time varying by dataset. The conclusion presents its GAN-based data augmentation and mimic model as supporting attack-agnostic poisoned-data detection.

I. Evaluation on Run-time Overhead

  • De-Pois detects 1,000 CIFAR-10 images in 0.108 seconds, while CD takes less time; De-Pois takes the longest on Fourclass and House Pricing.The authors attribute overhead mainly to loading the mimic model and report longer detection time on larger datasets.Source: §I · p. 12, Table IV · p. 13
  • For every 500 samples, synthetic-data generation takes 544 seconds on CIFAR-10, 27.1 seconds on Fourclass, and 32.3 seconds on House Pricing; mimic-model construction takes 654, 30.75, and 179.5 seconds, respectively.Source: §I · p. 12, Table V · p. 13

VII. CONCLUSION

  • De-Pois uses modified cGAN to generate training data and conditional WGAN-GP to construct a mimic model; experiments across four datasets and several attack types validate its effectiveness, with better performance than existing defenses in most cases.Source: §VII · p. 13
Loading 2105.03592v1…