PaperlayerView exact revision on arXiv

Source-linked AI summary

SAGE: Similarity-Based Cleaning of Poisoned Training Data from Verified Examples

Chaeeun Han, Soodeh Atefi, Yevgeniy Vorobeychik, Aron Laszka

arXiv:2610.01788v1cs.LGcs.CR

TL;DR

Poison detection is costly when defenses need large verified-clean sets, while clean-label poisons are difficult to recognize. SAGE uses a learned feature extractor and similarity-weighted predictions from a small verified set containing clean and poisoned examples. The paper reports effective suppression across evaluated clean-label attacks, with a remaining assumption that verified-clean examples are correct.

  • Problem

    Existing defenses either lack ground-truth information about poisons or require large verified-clean sets, which are costly to obtain for visually indistinguishable clean-label poisons.

    Source: §I · p. 1

  • Method

    SAGE trains a feature extractor on a separate dataset and uses non-parametric, similarity-weighted prediction over a small set of verified clean and poisoned examples.

    Source: §I · p. 2, C. Applying SAGE as a Poison Detector · p. 4

  • Results

    Across evaluated clean-label attacks, SAGE suppresses attacks while limiting data removal; on CIFAR-10 it finds at least 480 of 500 poisons in every triggerless setting.

    Source: §I · p. 2, C. Conclusion · p. 8

  • Takeaways & Limitations

    Verified poisoned examples provide useful attack-region information, and the distribution of verified clean examples across classes matters more than their number.

    Source: §I · p. 1, C. Conclusion · p. 8

    SAGE assumes examples identified as clean are truly clean, and the experiments do not evaluate contamination of that verified-clean set.

    Source: B. Limitations · p. 8

Abstract

from arXiv · show

As machine learning increasingly relies on public, untrusted data sources, data poisoning attacks, which inject malicious examples into training data to induce misclassification of a chosen target, pose a growing threat. Existing defenses either assume zero ground-truth information about which examples are poisoned, or they assume access to a large set of examples verified to be clean. Satisfying the latter assumption incurs significant cost since reliable verification can be very resource- or labor-intensive. This cost is particularly high for clean-label attacks, where poisoned examples are visually indistinguishable from clean data. Since requiring a large set of verified examples is impractical, we propose relying on a small set of verified examples including both clean and poisoned ones, i.e., each example verified either to be clean or poisoned through inspection by a forensic expert. The challenge is then to detect poisons based on a set of verified examples that is so small that most classification models would overfit. To address this challenge, we propose Similarity-based Approach for Ground-truth-driven Exclusion (SAGE), which trains a generic feature extractor on a separate dataset and then flags poisoned training examples using a non-parametric, similarity-weighted prediction based on the verified set. On standard benchmarks against seven clean-label attack methods, we demonstrate that having access to even a handful of verified poisoned examples provides a substantial advantage. We also find that the distribution of verified clean examples across classes matters more than the number of verified examples.

I. INTRODUCTION

SAGE uses a small verified set containing both clean and poisoned examples to detect poisons without fitting a detector to that set. On CIFAR-10, it suppresses attacks while removing little training data.

I. INTRODUCTION

  • Clean-label poisons are difficult and costly to verify, making the hundreds or thousands of verified clean examples assumed by existing defenses often impractical.Professional inspectors struggle to identify over 86% of feature-perturbing poisons.Source: §I · p. 1
  • Verified poisons mark attack-targeted feature-space regions, providing information that a verified-clean-only set does not contain.The paper studies using both kinds of verified examples to detect poisons from much smaller sets.Source: §I · p. 1
  • SAGE trains a feature extractor on a separate dataset and detects poisons through non-parametric, similarity-weighted prediction over verified clean and poisoned examples.The extractor is trained with an objective that mirrors detection, using similarity-weighted predictions from reference examples.Source: §I · p. 2
  • SAGE finds at least 480 of 500 poisons in every CIFAR-10 triggerless setting, limits attack success to at most 5%, and discards less than 2% of training data.Against backdoor attacks, it suppresses attacks without discarding so much training data that application-model accuracy falls below 0.50.Source: §I · p. 2

A. Data Poisoning Attacks

The paper frames clean-label poisoning as a threat that can evade label checks and reviews defenses by what ground truth they assume. SAGE uses a small verified set of both clean and poisoned examples, within a setting that balances attack suppression, retained accuracy, and limited data removal.

A. Data Poisoning Attacks

  • Clean-label attacks perturb features while preserving labels, and the paper considers targeted attacks including feature collision, bullseye polytope, and gradient matching.Backdoor attacks instead target inputs carrying a trigger patch.Source: A. Data Poisoning Attacks · p. 2, §III · p. 3

B. Defense Strategies against Data Poisoning

  • Label-dependent defenses trust training labels, ground-truth-agnostic methods use outlier or clustering criteria, and base-set defenses compare against verified clean examples.The paper describes base-set defenses as vulnerable to poisoned contamination and notes that inspectors miss many clean-label poisons.Source: B. Defense Strategies against Data… · pp. 2–3
  • SAGE instead uses a small verified set labeled as clean or poisoned, trading reliance on many presumed-correct labels for fewer integrity judgments.Unlike base-set defenses, it does not assume the verified set contains only clean examples.Source: B. Defense Strategies against Data… · p. 3

III. PROBLEM SETTING AND THREAT MODEL

  • The defender is assumed to have a small, class-balanced verified set, and must suppress target attacks while preserving cleaned-data accuracy and discarding few examples.The attacker leaves labels unchanged and seeks misclassification of chosen test inputs while largely preserving overall accuracy.Source: §III · p. 3
  • Attack success alone is insufficient for comparing defenses because discarding most data may stop attacks while failing the retention and accuracy requirements.The paper therefore reports both attack success and the amount of data discarded.Source: §III · p. 3

IV. PROPOSED APPROACH: SAGE

SAGE detects poisoned training examples by comparing them in a learned feature space with a small verified set of clean and poisoned examples.

IV. PROPOSED APPROACH: SAGE

  • SAGE identifies poisoned training examples through comparison with a small verified set containing both clean and poisoned examples.The comparison takes place in a learned feature space.Source: §IV · p. 3
  • The feature extractor is first trained on CIFAR-100 to place images sharing a label close together.The extractor is later reused frozen on the target dataset.Source: §IV · p. 3
  • At defense time, SAGE uses similarity to interpolate the verified examples’ poison or clean designations.The verified set serves as the reference set for this prediction.Source: §IV · p. 3

A. Feature Extractor

SAGE learns a feature space for similarity-based prediction rather than fitting a detector to the small verified set. Its training uses reference-query prediction, with stratified splits and a reference-set size chosen to suit detection.

A. Feature Extractor

  • The extractor is trained from scratch on a separate dataset because the verified set is too small for training and pretrained features are optimized for linear classification rather than cosine similarity.Training the extractor directly also permits tuning its output dimension for similarity-based detection.Source: A. Feature Extractor · p. 3

B. Training the Feature Extractor

  • Training predicts query labels as similarity-weighted combinations of reference labels, and cross-entropy updates the extractor and scale parameter.The loss compares each prediction with the query sample’s own label, while gradients flow through both reference and query features.Source: B. Training the Feature Extractor · p. 4
  • The extractor is trained on CIFAR-100, disjoint from the attacked CIFAR-10 and Tiny ImageNet datasets, to avoid biasing the defense with target data.The paper expects transfer because these are image benchmarks with broadly similar visual statistics.Source: B. Training the Feature Extractor · p. 4
  • Class-stratified reference-query splits ensure each query class has reference examples, since unseen classes cannot receive prediction weight.Rotating the subsets lets samples serve in both roles across the inner iterations.Source: B. Training the Feature Extractor · p. 4
  • The reference-set size is tuned because the prediction scale is calibrated during training but reused with the much smaller verified set at detection.A large mismatch between training and detection reference-set sizes leaves the scale tuned for a different setting.Source: B. Training the Feature Extractor · p. 4

C. Applying SAGE as a Poison Detector

At detection time, SAGE reuses its frozen feature extractor and similarity structure to estimate poison status from the verified set. It removes candidates whose estimated poison probability is at least 0.5, while retaining verified clean examples.

C. Applying SAGE as a Poison Detector

  • The extractor and scale remain frozen, and the verified set supplies scalar poison indicators instead of class-label vectors.Verified poisons receive indicator 1 and verified clean examples receive 0; each reference image retains its class label as well as its poison status.Source: C. Applying SAGE as a Poison Detector · p. 4
  • Each remaining training example is treated as a query against the verified set.The similarity-weighted fraction of verified poisons estimates the candidate’s poison probability.Source: C. Applying SAGE as a Poison Detector · pp. 4–5
  • SAGE removes a candidate when its estimated poison probability is at least 0.5, but retains verified clean examples regardless.No loss is computed and no parameters are updated during detection.Source: C. Applying SAGE as a Poison Detector · p. 5

V. EXPERIMENTAL SETUP

The experiments use CIFAR-10 and Tiny ImageNet benchmarks, a CIFAR-100-trained feature extractor, and a small verified set whose default composition is specified for CIFAR-10.

A. Datasets

  • CIFAR-10 has 50,000 training images across 10 classes, while Tiny ImageNet has 100,000 across 200 classes; neither trains the feature extractor.The extractor is trained on CIFAR-100. Once trained, it is reused across attacks, datasets, and verified-set configurations; training takes roughly 30 minutes, and detecting on 50,000 images takes 4 minutes.Source: A. Datasets · p. 5

B. Feature Extractor Training

  • The feature extractor uses a ResNet-18 backbone trained from scratch on CIFAR-100, with a two-layer head of hidden width 512 and output dimension 256.Training runs for 200 epochs with SGD, random crops, and horizontal flips; the training-time reference set has 200 images.Source: B. Feature Extractor Training · p. 5

C. Verified Set

  • The default CIFAR-10 verified set contains two clean examples per class and two confirmed poisons from the base class, for 22 examples total.Verified examples are drawn from the poisoned training set itself, and later experiments vary the set’s size and per-class composition.Source: C. Verified Set · p. 5

D. Victim Model Training

The setup specifies victim-model training by attack type, defines the evaluated poisoning settings, and compares SAGE with defenses that use different detection strategies and information.

D. Victim Model Training

  • Victim models use ResNet-18 on CIFAR-10 and VGG-16 on Tiny ImageNet, with transfer learning or from-scratch training depending on the attack.The trigger-based attacks share a fixed CIFAR-10 ResNet-18 training recipe, while each attack retains its published trigger design, poison rate, and target-class restriction.Source: D. Victim Model Training · p. 5

E. Poisoning Attacks

  • The evaluation covers three triggerless clean-label attacks and four clean-label backdoor attacks, using benchmark protocols or released implementations and default settings.Triggerless attacks use 20 distinct base–target trials, budgets 8, 16, and 22, and 500 CIFAR-10 or 250 Tiny ImageNet poisons per setting; backdoor ASR is measured on triggered test sets.Source: E. Poisoning Attacks · p. 5

F. Baseline Defenses

  • Baselines include label-disagreement or clustering filters, clean-subset selection, intentionally corrupted training, and representation-based poison ranking.Deep k-NN uses training labels; EPIC and Activation Clustering use clustering, Meta-Sift selects a clean subset, Confusion Training uses randomly mislabeled clean samples, and SPECTRE and FLARE analyze representations or activations.Source: F. Baseline Defenses · pp. 5–6
  • FLARE aggregates abnormal activations across hidden layers, selects an adaptive subspace, and flags the more stable of two clusters as poisoned.Its approach differs from Activation Clustering, which clusters final-layer activations within each class.Source: F. Baseline Defenses · p. 6

G. Metrics

The section defines attack and utility metrics and reports SAGE’s triggerless results across CIFAR-10 and Tiny ImageNet, including accuracy trade-offs and a setting where accuracy slightly improves.

G. Metrics

  • The evaluation reports ASR, clean test accuracy, training-set removal fraction, and the false-positive rate among removed examples.For targeted attacks, ASR is the fraction of trials in which the single target image receives the attacker’s chosen base label.Source: G. Metrics · p. 6

A. Triggerless Attacks

  • SAGE reaches 0% ASR for Feature Collision and Bullseye Polytope at every CIFAR-10 perturbation budget, and 0–5% for Gradient Matching.SAGE achieves the lowest ASR in every setting, including ties; EPIC also reaches 0% for Feature Collision at all three budgets.Source: A. Triggerless Attacks · p. 6
  • At ε = 16 for Bullseye Polytope, SAGE and Deep k-NN tie at 0% ASR, versus 15% for EPIC, 35% for Meta-Sift, and 70% for Confusion Training.Source: A. Triggerless Attacks · p. 6
  • Under CIFAR-10 transfer learning, SAGE accuracy ranges from 0.9258–0.9428, below EPIC’s 0.9455–0.9466 and Deep k-NN’s 0.9389–0.9481.For from-scratch Gradient Matching, SAGE retains 0.8980–0.9033 versus EPIC’s 0.8787–0.8806, while Confusion Training is best at 0.9040–0.9122; SAGE’s accuracy cost versus undefended models is 0.3–5.1 percentage points.Source: A. Triggerless Attacks · p. 6
  • For Tiny ImageNet Bullseye Polytope, SAGE reduces ASR from 30% to 10% and improves accuracy from 0.6153 to 0.6169.It is the only defense in this setting that avoids an accuracy loss; Deep k-NN reduces ASR to 20% but accuracy falls to 0.5584, while EPIC leaves ASR at 30% and reduces accuracy to 0.5569.Source: A. Triggerless Attacks · p. 6

B. Backdoor Attacks

On CIFAR-10 backdoor attacks, SAGE reduced ASR while retaining substantially higher clean accuracy than FLARE. Removal results show strong poison recovery with limited data removal, though outcomes vary by setting.

B. Backdoor Attacks

  • Across four CIFAR-10 backdoors, SAGE reduced ASR to 0.1310 on Narcissus and 0.2720 on Wicked-BadNets, with 0.9281–0.9494 clean accuracy versus FLARE’s 0.4586–0.5507.Each attack poisoned 10% of its selected target class, or 1% of the full training set.Source: B. Backdoor Attacks · p. 6, Table II · p. 7
  • On the three Wicked variants, SAGE had lower ASR than Activation Clustering, SPECTRE, and Confusion Training while keeping clean accuracy above 0.94.Source: B. Backdoor Attacks · p. 6

C. Precision of Removal

  • On CIFAR-10, SAGE recovered more than 480 of 500 poisons while removing 769.2–995.1 images (1.5–2.0% of training data), with 35.4–50.8% false positives.The removal results compare total samples removed with the number of true poisons among them.Source: C. Precision of Removal · p. 6
  • For Gradient Matching at ϵ = 16, SAGE and EPIC both reached 5% ASR but removed 896.9 versus 20,174.2 images, respectively.Source: C. Precision of Removal · p. 7
  • On Tiny ImageNet, SAGE removed 187 images, including 126 of 250 poisons (50.4% recall); 32.6% of removed examples were benign, and ASR fell from 30% to 10%.Source: C. Precision of Removal · p. 7

D. Composition of the Verified Set

Class-balanced clean-example coverage improves verified-set composition results, while adaptive attacks expose assumptions and threat-model boundaries that remain to be evaluated.

D. Composition of the Verified Set

  • Two verified poisons suffice for comparable recall with class-balanced clean examples, versus approximately five with unstratified examples.Class-balanced sampling removes approximately 180–220 clean images, compared with approximately 1,500–2,200 for unstratified sampling, even when unstratified sampling achieves lower recall.Source: D. Composition of the Verified Set · p. 7, Figure 1 · p. 8

A. Adaptive Attacks

  • An attacker without access to the extractor weights cannot directly compute or differentiate through SAGE’s similarity function, and the verified examples are selected independently after poisoning.The threat model makes no robustness claim against an attacker who knows the verified set when constructing poisons.Source: A. Adaptive Attacks · p. 8
  • Diverse poisons could challenge the assumption that verified and unverified poisons occupy related feature-space regions, at the cost of reducing individual poison effectiveness.This strategy targets SAGE’s central premise rather than specific reference examples.Source: A. Adaptive Attacks · p. 8
  • Future work could test evasion terms that penalize proximity to a verified set and diversity-regularized attacks to measure the effectiveness–detectability trade-off.These adaptive strategies have not yet been evaluated.Source: A. Adaptive Attacks · p. 8

B. Limitations

SAGE reports strong results across the evaluated clean-label attacks, but depends on verified-clean examples being correct and reuses a CIFAR-100-trained extractor on target datasets without fine-tuning.

B. Limitations

  • SAGE assumes examples marked clean are genuinely clean, and the experiments do not measure performance under contamination of that set.The authors identify contamination testing as future robustness work.Source: B. Limitations · p. 8
  • The extractor is trained on CIFAR-100 and reused without fine-tuning on CIFAR-10 and Tiny ImageNet; performance may decline outside natural-image domains.The method does not guarantee separation elsewhere, and non-visual modalities require separate evaluation.Source: B. Limitations · p. 8

C. Conclusion

  • On CIFAR-10, SAGE holds attack success rate to at most 5% while recovering at least 480 of 500 poisons on average; on Tiny ImageNet, it reduces Bullseye Polytope ASR from 30% to 10%.On Tiny ImageNet, SAGE recovers 126 of 250 poisons.Source: C. Conclusion · p. 8
  • Against Gradient Matching on CIFAR-10, SAGE removes fewer than 900 images on average at every tested perturbation budget, whereas EPIC removes more than 19,000.The conclusion also reports suppression of four clean-label backdoor attacks without the accuracy loss of the one baseline reaching comparable success rates.Source: C. Conclusion · p. 8

APPENDIX

The appendix selects the feature extractor’s output dimension and training reference-set size using clean CIFAR-100 performance under the detection-time prediction rule.

APPENDIX

  • The feature extractor is trained once on CIFAR-100, with output dimension and reference-set size chosen using clean-test cross-entropy and accuracy under Equation (3).Both design choices are fixed before any poisoned data are seen.Source: APPENDIX · p. 10

A. Feature Extractor Output Dimension

B. Reference Set Size

  • On the clean CIFAR-100 test set, accuracy under the similarity-weighted prediction rule is highest when the training reference set has |R| = 200, decreasing for both smaller and larger sets.Source: B. Reference Set Size · p. 10, Table V · p. 10
  • At |R| = 22, references cover only about one fifth of the 100 classes, and the learned τ approaches zero as most queries lack a same-class reference.The authors identify class coverage, rather than reference-set size alone, as important.Source: B. Reference Set Size · p. 10

Related papers

Loading 2610.01788v1…