PaperlayerView exact revision on arXiv

Source-linked AI summary

Poisoned classifiers are not only backdoored, they are fundamentally broken

Mingjie Sun, Siddhant Agarwal, J. Zico Kolter

arXiv:2010.09080v2cs.LGcs.CR

TL;DR

The paper asks whether a poisoned classifier can be controlled without access to its secret trigger or training data. It uses Denoised Smoothing, adversarial examples, and human inspection to construct alternative triggers, finding comparable or better performance and broader practical detectability. The authors conclude that poisoned classifiers are not exclusively controllable by the original attacker.

  • Problem

    The paper addresses the assumption that only the attacker possessing a backdoor trigger can control a poisoned classifier.

  • Method

    The method robustifies a poisoned classifier with Denoised Smoothing, analyzes smoothed adversarial examples, and uses human inspection to extract alternative color or cropped-patch triggers.

  • Results

    Alternative triggers achieve comparable or better attack performance than the initial backdoor, including 89.20% and 85.80% attack success rates versus 72.60% for the original trigger in one ImageNet result.

  • Takeaways & Limitations

    Access to a poisoned classifier can enable third parties to control it and can support human-in-the-loop identification of such backdoors.

  • Takeaways & Limitations

    The method is mainly limited to patch-based backdoor attacks and cannot be directly applied to more sophisticated trigger forms.

Abstract

from arXiv · show

Under a commonly-studied backdoor poisoning attack against classification models, an attacker adds a small trigger to a subset of the training data, such that the presence of this trigger at test time causes the classifier to always predict some target class. It is often implicitly assumed that the poisoned classifier is vulnerable exclusively to the adversary who possesses the trigger. In this paper, we show empirically that this view of backdoored classifiers is incorrect. We describe a new threat model for poisoned classifier, where one without knowledge of the original trigger, would want to control the poisoned classifier. Under this threat model, we propose a test-time, human-in-the-loop attack method to generate multiple effective alternative triggers without access to the initial backdoor and the training data. We construct these alternative triggers by first generating adversarial examples for a smoothed version of the classifier, created with a procedure called Denoised Smoothing, and then extracting colors or cropped portions of smoothed adversarial images with human interaction. We demonstrate the effectiveness of our attack through extensive experiments on high-resolution datasets: ImageNet and TrojAI. We also compare our approach to previous work on modeling trigger distributions and find that our method are more scalable and efficient in generating effective triggers. Last, we include a user study which demonstrates that our method allows users to easily determine the existence of such backdoors in existing poisoned classifiers. Thus, we argue that there is no such thing as a secret backdoor in poisoned classifiers: poisoning a classifier invites attacks not just by the party that possesses the trigger, but from anyone with access to the classifier.

1 INTRODUCTION

The paper challenges the assumption that only the original attacker can control a poisoned classifier, showing that third parties can generate effective alternative triggers from model access alone. Its human-in-the-loop method robustifies the classifier, inspects smoothed adversarial examples, and extracts colors or cropped patches as triggers.

  • 1 INTRODUCTION: A third party without the original trigger or training data can reliably generate alternative triggers that control a poisoned classifier as effectively as, or better than, the original.This reframes backdoor poisoning as granting control to anyone with access to the trained model.
  • 1 INTRODUCTION: The attack converts the poisoned classifier into a robust smoothed classifier using Denoised Smoothing, then analyzes its adversarial examples for backdoor patterns.Robust models have perceptually aligned gradients, making their adversarial examples more interpretable.
  • 1 INTRODUCTION: Human inspection extracts colors or cropped patches from smoothed adversarial examples to construct novel triggers.The procedure is designed to make trigger construction interpretable rather than fully automated.
  • 1 INTRODUCTION: Experiments on ImageNet and TrojAI show that the method is effective across commonly used backdoor poisoning methods and improves over prior trigger-distribution approaches in efficiency and effectiveness.The paper also evaluates the approach through a user study involving poisoned-classifier identification.

2 RELATED WORK

Related work spans backdoor attacks and defenses, adversarial robustness, and randomized smoothing. The paper connects these areas by using robustified classifiers and human inspection to reconstruct alternative triggers rather than automatically recovering only the original trigger.

  • Backdoor Attacks: Backdoor attacks poison a subset of training data so a trigger causes clean test images to be classified as a chosen target class.Representative attacks include BadNet, clean-label backdoors, and hidden-trigger backdoors.
  • Backdoor Attacks: Most trigger-reconstruction defenses automatically seek the original backdoor, whereas this work uses adversarial examples and human inspection to construct interpretable alternatives.The paper contrasts its approach with Qiao et al. (2019), which models a distribution of possible triggers.
  • Adversarial Robustness: Adversarially robust classifiers have perceptually aligned gradients, so their adversarial examples can reveal salient characteristics of other classes and support image synthesis.These examples generally require larger perturbations to expose such characteristics.
  • Randomized Smoothing: Randomized smoothing creates classifiers certifiably robust in l2 norm, but typically requires specialized training of the base model.Denoised Smoothing addresses this requirement by adding a pretrained denoiser before smoothing a standard classifier.

3 BACKGROUND

The background defines the perceptual rationale and smoothing machinery behind the attack. Randomized smoothing averages predictions under Gaussian noise, while Denoised Smoothing makes this strategy applicable to standard pretrained classifiers; attack success rate measures trigger effectiveness.

  • Perceptual property of adversarially robust classifiers: Adversarially robust models can produce adversarial examples with salient characteristics of the class they are misclassified into.This perceptual alignment motivates inspecting robustified adversarial examples for backdoor patterns.
  • Randomized Smoothing and Denoised Smoothing: Randomized smoothing converts a base classifier into a classifier that is certifiably robust in l2 norm under Gaussian noise.The noise level σ controls the tradeoff between robustness and accuracy.
  • Randomized Smoothing and Denoised Smoothing: Denoised Smoothing prepends a custom-trained denoiser to a standard pretrained classifier before applying randomized smoothing, producing a robust smoothed classifier without additional classifier training.The denoiser maps noisy inputs toward clean images that the base classifier is assumed to classify well.
  • Backdoor poisoning model: The backdoor setting uses image-patching attacks, where attack success rate is the percentage of triggered test inputs classified into the target class.The evaluated poisoned classifiers contain backdoors that redirect some source classes to a target class.

4 METHODOLOGY

The paper introduces a human-in-the-loop threat model in which third parties can control poisoned classifiers without the original trigger or training data, using smoothed adversarial examples to extract alternative triggers.

  • 4 METHODOLOGY: A third party can manipulate poisoned classifiers without access to the original trigger or training data, using only model analysis with test data.The threat model applies to users of pretrained classifiers or cloud machine-learning services.
  • 4.1 MOTIVATION FOR GENERATING SMOOTHED ADVERSARIAL EXAMPLES: Backdoor images can be treated as adversarial examples, motivating robustification because standard classifier gradients are noisy and difficult to interpret.The paper formalizes f(x_a) = a and f(B(x_a)) = b, then frames B(x_a) as a bounded adversarial perturbation.
  • 4.1 MOTIVATION FOR GENERATING SMOOTHED ADVERSARIAL EXAMPLES: Denoised Smoothing converts the poisoned classifier into a robust model whose adversarial examples are perceptually meaningful and can expose backdoor patterns.The method generates smoothed adversarial examples with Monte Carlo gradient estimates and an l2 norm bound.
  • 4.2 BREAKING POISONED CLASSIFIERS: Most adversarial examples for the investigated binary poisoned classifiers contain backdoor patterns, with visually apparent color regions that are less prevalent for clean classifiers.The regions become more saturated at ϵ = 60, and their colors usually correspond to the original trigger except for random triggers.
  • 4.2 BREAKING POISONED CLASSIFIERS: The attack robustifies the classifier, generates large-ϵ adversarial examples, visually identifies suspicious regions, and constructs color or cropped alternative triggers.Color patches use representative pixels, while cropped patches directly reuse regions containing backdoor patterns.
  • 4.2 BREAKING POISONED CLASSIFIERS: Triggers extracted from only a handful of adversarial examples generalize well to test images, while clean classifiers are not broken by the method.The authors use this finding to argue that the original secret trigger is unnecessary for manipulating poisoned classifiers.
  • 4.3 DISCUSSION: Human inspection is required to select suspicious regions, which the authors view as practical for forensic analysis and as protection against adversarial attacks on automated detectors.The method’s main limitation is that it is designed primarily for patch-based triggers and lacks an exact algorithmic standard for patch selection.

5 EXPERIMENTS

Experiments on ImageNet and TrojAI show that model-only, human-in-the-loop attacks can generate effective alternative triggers for poisoned classifiers, while clean classifiers remain unaffected.

  • 5.1 IMAGENET: Denoised Smoothing produced smoother adversarial images with more distinctive backdoor patterns than Basic Adv or Smoothing baselines.The Smoothing baseline produced no obvious pattern, highlighting the role of the denoiser.
  • 5.1 IMAGENET: 89.20% and 85.80% attack success rates exceeded the original Trigger A’s 72.60% on an ImageNet BadNet classifier.The alternative triggers differed greatly from Trigger A, and larger perturbations produced better results.
  • 5.1 IMAGENET: Our method achieved 98.80% ASR for a binary BadNet classifier, compared with 65.29% for MESA.Human interaction helped find effective triggers beyond randomly cropped patches.
  • 5.1 IMAGENET: On clean ImageNet classifiers, patched test data had low error rates, indicating that the attack did not apply to clean models.The experiment used larger perturbations because no obvious pattern appeared at ϵ = 20.
  • 5.1 IMAGENET: 88.60% and 83.00% ASR exceeded the original camouflaged trigger’s 75.80%, despite the absence of obvious backdoor patterns.Cropped patterns from large-ϵ adversarial examples remained effective even when the trigger resembled clean-image colors.
  • 5.2 TROJAI: Across 20 sampled classifiers per TrojAI round, constructed triggers had high ASR, with cropped triggers outperforming color triggers for both illustrated classifiers.The evaluation excluded filter-based triggers in round 2 and round 3 datasets.
  • 5.2 TROJAI: In the user study, Denoised Smoothing helped participants identify poisoned classifiers and construct more effective triggers than Basic Adv or random crops.Random cropping failed to find effective backdoor triggers, supporting the necessity of human interaction.

6 CONCLUSION

The paper introduces a human-in-the-loop attack that generates alternative triggers from smoothed adversarial examples, without the original trigger. Experiments and a user study show poisoned classifiers are vulnerable to users with model access alone.

  • The attack constructs alternative triggers from adversarial examples of a robustified poisoned classifier, achieving comparable or better performance than the initial backdoor.The approach is demonstrated on high-resolution datasets and compared with prior trigger-distribution modeling.

A.1 TRAINING DETAILS

The experiments combine poisoned-classifier training, smoothed-classifier attacks, a user study, and an interactive tool for constructing and testing alternative triggers.

  • A.1 TRAINING DETAILS: The ImageNet poisoned-classifier setup fine-tunes pretrained AlexNet on poisoned datasets, using 400 poisons except 1000 for the BadNet multiclass classifier.HTBA divides each class into 200 poisoning, 800 training, and 100 testing images; CLBD uses robustly initialized ResNets with specified l2 perturbations.
  • A.2 COMPUTING ADVERSARIAL EXAMPLE: The attack computes adversarial examples of a smoothed classifier using SMOOTHADV with projected gradient descent and a denoiser.The implementation uses 16 Monte-Carlo noise vectors, 100 PGD steps, and an l2 perturbation bound.
  • B.1 DETAILS ON USER STUDY: The user study gives participants classifier labels and saliency maps, then tests whether they can identify poisoned models using alternative-trigger attacks or saliency maps.The study involved 15 participants divided evenly among three groups and sampled 50 TrojAI classifiers.
  • B.2 TROJAI INTERACTIVE TOOL: The interactive tool visualizes adversarial examples and lets users create patch triggers while observing predictions on patched images.Its two parts support adversarial-example inspection and alternative-trigger testing.

C.1 IMAGENET BINARY POISONED CLASSIFIER

Alternative triggers are effective across poisoned binary and multiclass ImageNet classifiers and TrojAI models, while clean classifiers remain substantially less vulnerable.

  • C.1 IMAGENET BINARY POISONED CLASSIFIER: All three binary poisoned ImageNet classifiers yielded effective alternative triggers.The CLBD classifier’s original Trigger A had a 90.00% attack success rate.
  • C.2 IMAGENET MULTI-CLASS POISONED CLASSIFIER: The method constructed effective triggers for both HTBA and CLBD poisoned multiclass ImageNet classifiers.The HTBA classifier’s original Trigger A had a 74.55% attack success rate.
  • C.3 IMAGENET CLEAN CLASSIFIERS: Clean ImageNet classifiers were not vulnerable to the triggers constructed by the attack.This result covers both binary and multiclass clean-classifier experiments.
  • C.4 TROJAI: For all eight poisoned TrojAI classifiers, at least one of four alternative triggers achieved a 100% attack success rate.On two clean TrojAI classifiers, more than half of test images remained correctly classified when patched with constructed triggers.

D.1 ADVERSARIAL EXAMPLES ON TROJAI DATASET

Smoothed adversarial examples can expose backdoor patterns in poisoned TrojAI classifiers, but not every visually salient region forms a usable trigger.

  • D.1 ADVERSARIAL EXAMPLES ON TROJAI DATASET: Adversarial examples with clear backdoor patterns were all classified into the backdoor target class, whereas dense black regions did not yield successful triggers.The black regions appeared in class-4 examples and were interpreted as mimicking features of the predicted class-0 images.
  • D.2 COMPARISON OF DIFFERENT ADVERSARIAL EXAMPLES: The paper compares different adversarial examples at an l2 perturbation size of 20.The supplied passage identifies the comparison but does not report an outcome.

E IMAGENET CLASSIFIERS WITH MORE CLASSES

The method is evaluated against a poisoned ImageNet classifier trained on 10 randomly selected classes, with alternative triggers achieving attack success rates comparable to or exceeding the original trigger.

  • Alternative triggers achieve attack success rates similar to or higher than the original trigger.The classifier is trained with BadNet and Trigger A before testing the alternative triggers.
  • The experiment randomly selects 10 classes from ImageNet's 1,000 classes to evaluate the method on a larger class setting.
  • 59.71% is the original backdoor's success rate in the 10-class ImageNet attack.
Loading 2010.09080v2…