PaperlayerView exact revision on arXiv

Source-linked AI summary

Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information

Yi Zeng, Minzhou Pan, Hoang Anh Just, Lingjuan Lyu, Meikang Qiu, Ruoxi Jia

arXiv:2204.05255v2cs.CRcs.LG

TL;DR

Existing clean-label backdoor attacks typically require access to the entire training set, which is often difficult when data come from multiple independent sources. The paper introduces NARCISSUS, a target-class-only attack, and reports high attack success with minimal poisoning, physical-world generalization, and limited defense robustness.

  • Problem

    Existing clean-label backdoor attacks require knowledge of all-class training data, although such data may be difficult or impossible to access when collected from independent sources.

  • Method

    NARCISSUS optimizes a trigger toward the inside of the target class using only representative target-class training data.

  • Results

    85.81% of test examples from any class were classified as the desired target class with a trigger after manipulating 0.05% of the total data.

  • Takeaways & Limitations

    Clean-label backdoor attacks remain feasible under minimal attacker knowledge, can generalize to the physical world, and may evade existing defenses.

Abstract

from arXiv · show

Backdoor attacks insert malicious data into a training set so that, during inference time, it misclassifies inputs that have been patched with a backdoor trigger as the malware specified label. For backdoor attacks to bypass human inspection, it is essential that the injected data appear to be correctly labeled. The attacks with such property are often referred to as "clean-label attacks." Existing clean-label backdoor attacks require knowledge of the entire training set to be effective. Obtaining such knowledge is difficult or impossible because training data are often gathered from multiple sources (e.g., face images from different users). It remains a question whether backdoor attacks still present a real threat. This paper provides an affirmative answer to this question by designing an algorithm to mount clean-label backdoor attacks based only on the knowledge of representative examples from the target class. With poisoning equal to or less than 0.5% of the target-class data and 0.05% of the training set, we can train a model to classify test examples from arbitrary classes into the target class when the examples are patched with a backdoor trigger. Our attack works well across datasets and models, even when the trigger presents in the physical world. We explore the space of defenses and find that, surprisingly, our attack can evade the latest state-of-the-art defenses in their vanilla form, or after a simple twist, we can adapt to the downstream defenses. We study the cause of the intriguing effectiveness and find that because the trigger synthesized by our attack contains features as persistent as the original semantic features of the target class, any attempt to remove such triggers would inevitably hurt the model accuracy first.

I. INTRODUCTION

The paper asks whether clean-label backdoor attacks remain feasible when attackers know only target-class data, and introduces NARCISSUS to answer yes. It reports strong attack efficacy at low poisoning rates, physical-world transfer, and resilience against defenses.

  • Motivation: Existing clean-label attacks require training data from all classes, which may be inaccessible when datasets are assembled from independent sources.Face-recognition data, for example, may combine images supplied by different users.
  • Method: NARCISSUS synthesizes a trigger oriented toward the inside of the target class using only representative target-class training data.This class-oriented design avoids relying on accurate knowledge of non-target classes.
  • Results: Under the same poison ratio, existing clean-label attacks misclassified 1.72% of test examples despite access to the full training data.The comparison supports higher efficacy under a weaker attacker-knowledge assumption.
  • Defenses: Popular and state-of-the-art defenses did not robustly mitigate NARCISSUS, while low-frequency trigger constraints helped it evade frequency-based detection.The authors attribute the difficulty of trigger removal to persistent trigger features whose removal would hurt model accuracy.
  • Physical-world setting: Tailoring triggers to real-world variations enabled a clean-label attack that generalized to physical objects.The paper presents this as the first clean-label attack claimed to robustly generalize to the physical world.

II. BACKGROUND AND RELATED WORK

This section frames supervised learning as empirical-risk minimization for a classifier and explains why deep neural networks typically use stochastic gradient descent to find local minima.

  • Supervised Machine Learning: Supervised learning trains a classifier f_θ: X → [k] to predict labels for inputs across k classes.The training data consist of labeled examples, and the learned model is evaluated on test examples.
  • Supervised Machine Learning: The learning algorithm seeks parameters θ that minimize empirical risk, measuring classifier performance on the training data.For deep neural networks, this objective is generally non-convex.
  • Supervised Machine Learning: Because finding a global minimum is generally impossible for deep neural networks, standard practice uses stochastic gradient descent to seek a local minimum.The trained model then serves predictions during testing.

B. Backdoor Attacks

Backdoor attacks poison training data so triggered inputs are mapped to a target class while clean inputs remain correctly classified. Clean-label methods avoid obvious label inconsistencies, but prior approaches require broader training-data knowledge than NARCISSUS.

  • Backdoor attacks train models to associate a trigger with a target label while preserving accuracy on clean inputs.
  • Dirty-label attacks relabel triggered non-target examples as the target class, making the poisoning detectable through human inspection.
  • HTBA and SAA indirectly mimic non-target poisoning through feature-space or gradient alignment, requiring information from non-target classes or the full training set.
  • NARCISSUS requires only target-class training data, supports all-to-one attacks, and extends clean-label performance to large datasets and physical-world demonstrations.

III. METHODOLOGY

NARCISSUS models an attacker who controls supplied training data and knows representative target-class examples plus general task information. It aligns the trigger with target-class features using a surrogate model built without access to the victim’s full training set.

  • The threat model assumes a victim trains on data aggregated from multiple sources, allowing an adversary to control part of the supplied dataset.
  • The attacker seeks targeted predictions on triggered inputs while preserving clean accuracy and making perturbations bounded and clean-label.
  • The attacker knows representative target-class examples and general information about the learning task, but not the identities or classes in the victim’s training data.
  • NARCISSUS addresses arbitrary-trigger misalignment by optimizing a trigger toward better alignment with the target class.
  • The trigger optimization minimizes target-class prediction loss for target examples under allowable trigger designs, treating the resulting trigger as a representative target-class feature.
  • A surrogate model trained from POOD and target-class examples replaces the unavailable oracle model during trigger synthesis.
  • The synthesized trigger is reported to remain robust when victim and surrogate models differ in architecture or training data.

C. Attack Workflow

NARCISSUS uses a four-stage workflow: warm up a surrogate model, synthesize an inward-pointing trigger, insert it into target-class data, and magnify it during test queries.

  • Poi-warm-up builds a surrogate model from a POOD-data-pre-trained model using only target-class samples.
  • Trigger generation uses the warmed-up surrogate as a feature extractor to synthesize inward-pointing noise from target-class data.
  • Trigger insertion poisons a small amount of target-class data, followed by test-query manipulation that magnifies the trigger.
  • POOD pretraining supplies generally useful low-level features, while target-class fine-tuning enables discrimination of the target class.
  • The trigger-generation algorithm iteratively updates the trigger under an allowable perturbation set to produce the NARCISSUS trigger.
  • The two-pronged surrogate-training approach is more efficient than retraining from combined POOD and target-class data whenever target classes change dynamically.
  • An l∞ constraint bounds each trigger dimension by ϵ, while alternative constraint sets can support adaptive attacks such as low-frequency perturbations.

IV. EVALUATION

The evaluation measures NARCISSUS against existing attacks across datasets, studies design choices, and tests defenses. It covers effectiveness, robustness to configuration choices, and defense resistance.

  • The evaluation compares NARCISSUS with existing backdoor attacks across different datasets.
  • It studies the effects of surrogate-model architecture, Poi-warm-up fine-tuning iterations, and perturbation constraints.
  • It evaluates popular and state-of-the-art defenses against the proposed attack.

A. Experimental Setup

The evaluation compares NARCISSUS with clean-label and adapted dirty-label attacks across CIFAR-10, PubFig, and Tiny-ImageNet under very low poison ratios. Results assess clean accuracy, target-class accuracy, attack success, and trigger budgets.

  • Experimental datasets and models: The experiments cover CIFAR-10, PubFig, and Tiny-ImageNet using target-class-only surrogate training with out-of-distribution data.NARCISSUS uses target-class data plus POOD examples, unlike existing surrogate models that use in-distribution data from all classes.
  • Baselines: The comparison includes HTBA, SAA, LC, adapted clean-label BadNets and Blend, and their original dirty-label variants.The adapted attacks poison only target-class data while retaining the original labels.
  • Evaluation protocol: 0.05%, 0.024%, and 0.05% are the poison ratios used for CIFAR-10, PubFig, and Tiny-ImageNet, respectively.These ratios are substantially below the 5%–20% range commonly studied in prior work.
  • Main comparison: NARCISSUS achieves the highest ASR across three datasets while changing clean accuracy by at most 0.39%, 0.36%, and 0.17%.The clean-accuracy changes correspond to CIFAR-10, PubFig, and Tiny-ImageNet, respectively.
  • Dataset-specific findings: Three PubFig target-class images suffice at a 0.024% poison ratio, reflecting the attack’s effectiveness on this less diverse face dataset.The passage attributes the result partly to finding a pattern that conflicts with robust features across classes while indicating the target class.
  • Main comparison: 85.81% ASR is achieved on Tiny-ImageNet at a 0.05% poison ratio, versus 1.71% for the best existing all-to-one attack.The paper reports this as 50× greater effectiveness, while randomized training augmentations are applied.

2) Impact of target-class poison ratio:

Target-class poison ratio creates a trade-off between attack success and target-class accuracy. NARCISSUS reaches near-perfect ASR at only 0.5% while largely preserving target-class accuracy.

  • Trade-off: A higher target-class poison ratio increases ASR but impairs Tar-ACC.The figure compares these two metrics for different triggers on CIFAR-10.
  • Baseline behavior: Below 1% target-class poisoning, BadNets-c, Blend-c, and LC cannot obtain satisfactory ASR.Their ASR rises as the target-class poison ratio increases.
  • Baseline behavior: 90%+ ASR requires a 70% target-class poison ratio for all three baselines, while Tar-ACC falls from 94% to around 85%.The resulting classwise accuracy difference could reveal the attack during inspection.
  • NARCISSUS: 0.5% target-class poisoning gives NARCISSUS an almost perfect ASR while largely maintaining target-class accuracy.This contrasts with the much larger poison ratio needed by the baselines.

C. Ablation Study

The ablation study examines surrogate architecture, perturbation radius, and optimization iterations. NARCISSUS remains effective across surrogate-target pairs and with small perturbation budgets.

  • Surrogate architecture: All evaluated surrogate-target model pairs achieve satisfying ASR, with ACC above 95% and Tar-ACC above 93%.Using the same architecture for surrogate and target does not necessarily produce the best ASR.
  • Perturbation budget: 30.6% ± 8.12% ASR remains at a 2/255 radius and 0.05% poison ratio.The results are averaged over three runs.
  • Surrogate architecture: NARCISSUS does not require matching surrogate and target architectures to maximize attack performance.Performance depends more on learning-task performance than architectural similarity; the paper recommends the most advanced architecture for the task.
  • Perturbation budget: ASR increases with the l∞-norm radius, while ACC and Tar-ACC remain largely unaffected.The radius controls the perturbation budget used in trigger generation.
  • Optimization iterations: More than 70% ASR is possible without fine-tuning the POOD-pretrained surrogate on target-class data.The ablation varies fine-tuning rounds and trigger-synthesis gradient-descent iterations while maintaining ACC above 95% and Tar-ACC above 93%.

D. Defenses

The paper evaluates model-based defenses against NARCISSUS and finds that existing unlearning methods struggle to remove the trigger without damaging clean accuracy. I-BAU remains erratic after extended unlearning because it begins removing robust semantic features before fully mitigating the trigger.

  • D. Defenses: Five defenses are considered across unlearning, poisoned-sample detection, and robust-training approaches.The section includes Neural Cleanse, Fine-Pruning, I-BAU, and ABL-related evaluations.
  • D. Defenses: 95.34% ACC, 93.44% Target Class ACC, and 97.10% ASR define the NARCISSUS-poisoned CIFAR-10 baseline before defense.The poisoned model uses a 0.05% poison ratio.
  • D. Defenses: NARCISSUS triggers point inside the target class, making trigger removal likely to hurt clean-model performance.The trigger can activate neurons similar to those used by target-class samples.
  • D. Defenses: I-BAU leaves NARCISSUS partially effective after 100 rounds, with performance that does not converge smoothly.The evaluation uses SGD and Adam with learning rates 0.001 and 0.0001, respectively.
  • D. Defenses: I-BAU synthesizes and unlearns semantically meaningful robust features, and Adam produces clearer features alongside a larger ACC drop than SGD.This correlates robust-feature unlearning with clean-accuracy degradation.

2) Model-agnostic Backdoor Detection:

The paper studies frequency-based, model-agnostic detection of NARCISSUS and adapts the attack to evade it. A low-frequency constraint makes the adaptive trigger visually and spectrally stealthy while retaining strong attack efficacy.

  • 2) Model-agnostic Backdoor Detection:: Frequency-based detection assumes benign samples are dominated by low-frequency features while poisoned samples contain high-frequency artifacts.This detector operates without relying on a particular model.
  • 2) Model-agnostic Backdoor Detection:: The vanilla l∞-constrained NARCISSUS trigger is easy for frequency-based detection to identify because it contains high-frequency artifacts.The detector exploits the frequency characteristics of the synthesized noise.
  • 2) Model-agnostic Backdoor Detection:: A low-pass filter applied after each gradient update adapts NARCISSUS to optimize triggers over low-frequency patterns.This construction follows the setup used for the Smooth attack.
  • 2) Model-agnostic Backdoor Detection:: The adaptive NARCISSUS trigger achieves frequency-domain and visual stealth comparable to the original Smooth trigger.Both attacks use the same low-pass filter constraint.
  • 2) Model-agnostic Backdoor Detection:: 77.59% higher ASR than the original Smooth trigger is achieved by adapted NARCISSUS under clean-label poisoning.Its ASR is also slightly higher than the dirty-label Smooth attack.

3) Robust Training over Poisoned Dataset:

The paper evaluates robust training on NARCISSUS-poisoned data and extends the attack to physical triggers. ABL fails to reduce attack success, while physical deployment must address sensor information loss and geometric variation.

  • 3) Robust Training over Poisoned Dataset:: ASRs remain close to 100% after ABL on poisoned WideResNet-16-1 models, showing ineffective mitigation of NARCISSUS.The evaluation targets CIFAR-10 bird samples with a NARCISSUS trigger generated using ResNet-18.
  • 3) Robust Training over Poisoned Dataset:: At a 0.05% poison ratio, ABL isolates only three target-class samples, none of which are poisoned.At 0.5%, it isolates ten target-class samples, including three poisoned samples.
  • 3) Robust Training over Poisoned Dataset:: Physical clean-label attacks face information loss from camera capture and variation in viewing angles, rotations, and backgrounds.Camera capture can change hue and lose pixels, while physical placement introduces affine transformations.
  • 3) Robust Training over Poisoned Dataset:: Randomized augmentation, an 8 × 8 square trigger, random placement, padding, hue changes, rotation, and EOT are used to optimize a physical-world trigger.These choices address location, transformation, and sensor-related variation during physical deployment.

B. Evaluation

The evaluation shows that NARCISSUS succeeds as a clean-label backdoor with only representative target-class knowledge and very little poisoned data. It also generalizes across datasets, models, defenses, and physical-world triggers.

  • B. Evaluation: NARCISSUS is the only evaluated trigger that enables a successful clean-label backdoor in the physical-world Tiny-ImageNet experiment.The comparison uses bullfrog as the target class, 0.05% poisoning, and white-square and random-noise baselines.
  • B. Evaluation: NARCISSUS requires only representative target-class examples and manipulates 0.05% or less of the training data.The attack is model-agnostic under the stated evaluation scope.
  • B. Evaluation: NARCISSUS ASR is reported as 30.33× to 64.45× higher than existing clean-label attacks that require full training-data knowledge.The attack also generalizes to large datasets and the physical world.
Loading 2204.05255v2…