PaperlayerView exact revision on arXiv

Source-linked AI summary

Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer

Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, Maosong Sun

arXiv:2110.07139v1cs.CLcs.AIcs.CR

TL;DR

Existing NLP models may be vulnerable to adversarial and backdoor attacks that exploit task-irrelevant text style. The paper introduces style transfer-based methods for both attack settings and reports attack success rates exceeding 90% in many cases, with advantages over baselines in several evaluations.

  • Problem

    The paper addresses the lack of investigation into adversarial and backdoor attacks based on text style, a task-irrelevant feature for most NLP tasks.

  • Method

    The paper transforms inputs into multiple styles for adversarial attacks and poisons training samples with a selected trigger style for backdoor attacks.

  • Results

    Style transfer-based adversarial attacks achieve over 90% attack success rates on SST-2 against all models, while backdoor attacks exceed 90% in almost all cases.

  • Takeaways & Limitations

    The results reflect that popular NLP models have limited ability to handle text style under security threats, and style transfer-based attacks can outperform baselines in several aspects.

  • Takeaways & Limitations

    The authors note that style-transfer-based attacks may be maliciously used and that defenses such as test-time style transfer may have side effects on normal samples.

Abstract

from arXiv · show

Adversarial attacks and backdoor attacks are two common security threats that hang over deep learning. Both of them harness task-irrelevant features of data in their implementation. Text style is a feature that is naturally irrelevant to most NLP tasks, and thus suitable for adversarial and backdoor attacks. In this paper, we make the first attempt to conduct adversarial and backdoor attacks based on text style transfer, which is aimed at altering the style of a sentence while preserving its meaning. We design an adversarial attack method and a backdoor attack method, and conduct extensive experiments to evaluate them. Experimental results show that popular NLP models are vulnerable to both adversarial and backdoor attacks based on text style transfer -- the attack success rates can exceed 90% without much effort. It reflects the limited ability of NLP models to handle the feature of text style that has not been widely realized. In addition, the style transfer-based adversarial and backdoor attack methods show superiority to baselines in many aspects. All the code and data of this paper can be obtained at https://github.com/thunlp/StyleAttack.

1 Introduction

NLP models face adversarial and backdoor threats that exploit task-irrelevant features, and text style is a suitable target because it is generally independent of semantics. This paper introduces style transfer-based attacks and finds high success rates and advantages over baselines.

  • Motivation: Adversarial and backdoor attacks exploit task-irrelevant features while preserving or inducing task-relevant behavior.Adversarial attacks alter irrelevant features at inference time, whereas backdoor attacks alter training data to implant trigger-dependent behavior.
  • Motivation: Text style comprises lexical and syntactic patterns independent from semantics, making style transfer suitable for both attack types.Style transfer changes sentence style while preserving meaning, and prior work had not investigated these attacks based on style transfer.
  • Approach: The paper introduces iterative style transformation for adversarial examples and trigger-style poisoning for backdoor attacks.The adversarial method transforms test inputs into multiple styles, while the backdoor method transforms training samples into a selected trigger style.
  • Results: Over 90% attack success rates are reported for style transfer-based adversarial attacks on SST-2 against all evaluated models.Experiments cover 3 popular NLP models on 3 tasks.
  • Results: Style transfer-based adversarial attacks consistently outperform baselines across attack success rates, adversarial example quality, and attack validity.Style transfer-based backdoor attacks also exceed 90% success rates in almost all cases and substantially outperform baselines when defenses are deployed.
  • Conclusion: The experiments indicate that existing NLP models handle text style poorly under security threats, motivating greater attention to this vulnerability.The authors frame this as a limited ability that has not been widely recognized.

2 Background

Textual adversarial attacks perturb correctly classified inputs so models misclassify them, while backdoor attacks poison training data to create trigger-dependent behavior. Both attack settings distinguish normal behavior from attacker-controlled outcomes.

  • Adversarial Attacks: An adversarial attacker perturbs a correctly classified test input while retaining its ground-truth label and causing victim-model misclassification.The formalization uses a victim classifier Fθ and a test sample (x_t, y_t).
  • Adversarial Attacks: Adversarial attacks vary by perturbation level—character, word, or sentence—and by model access, including white-box and black-box settings.Black-box attacks require no full knowledge of the victim model and are described as more practical.
  • Backdoor Attacks: Backdoor training creates poisoned samples by modifying normal training data, then mixes poisoned and normal samples into a backdoor training set.The resulting set D_b = D* ∪ D trains the backdoored model Fθ*.
  • Backdoor Attacks: During backdoor inference, the model classifies normal inputs correctly but assigns the attacker-specified target label to trigger-embedded inputs.This behavior defines the intended separation between benign operation and trigger activation.

3 Methodology

The paper uses text style transfer to create adversarial examples and inject style-based backdoors while preserving sentence semantics. Its methodology combines STRAP-based transformations with black-box victim-model querying or trigger-style training.

  • 3.1 Text Style Transfer Model: STRAP is an unsupervised style-transfer model based on controlled paraphrase generation that aims to preserve task-relevant attributes such as sentiment.It creates pseudo-parallel data, trains style-specific inverse paraphrase models, and applies the model for a target style.
  • 3.2 Style Transfer-based Adversarial Attacks: StyleAdv generates multiple style-transferred paraphrases of each test input and queries the victim model until one produces an incorrect output.If several adversarial examples succeed, the method selects the paraphrase most similar to the original input and iterates until success or the query limit.
  • 3.2 Style Transfer-based Adversarial Attacks: StyleAdv is a sentence-level black-box attack requiring only the victim model’s output during attack generation.The method can use multiple styles and random seeds to generate different paraphrases.
  • 3.3 Style Transfer-based Backdoor Attacks: StyleBkd selects a trigger style, transforms training samples into that style, changes their labels to a target label, and mixes them with normal training data.During inference, transforming a test sample into the trigger style is intended to produce the target label.
  • 3.3 Style Transfer-based Backdoor Attacks: StyleBkd trains with L = L_t + L_a, adding an auxiliary classification loss so the victim model learns the abstract style trigger.The paper states that an ablation study demonstrates the effectiveness of this auxiliary loss.

4 Experiments of Adversarial Attacks

The adversarial-attack experiments evaluate StyleAdv across three tasks, datasets, victim models, baselines, and automatic and human-centered metrics. StyleAdv generally achieves strong attack performance, while attack validity remains limited and HateSpeech is a difficult setting.

  • 4.1 Experimental Settings: StyleAdv is evaluated on sentiment analysis, hate speech detection, and news topic classification using SST-2, HateSpeech, and AG’s News.The experiments use BERT, ALBERT, and DistilBERT as victim models.
  • 4.1 Experimental Settings: The automatic evaluation measures attack success rate, adversarial-example quality, and attack validity across the compared attacks.Quality includes fluency measured by GPT-2 perplexity and grammaticality.
  • 4.2 Attack Results of Automatic Evaluation: StyleAdv consistently achieves the highest ASR and best overall adversarial-example quality, outperforming the sentence-level baselines in the reported comparisons.The baselines are GAN and SCPN; GAN could only be evaluated on SST-2 because usable training on the other datasets was unsuccessful.
  • 4.2 Attack Results of Automatic Evaluation: Over 90% ASR is achieved on SST-2 against all victim models, indicating strong vulnerability to style-transfer adversarial attacks.Both SCPN and StyleAdv perform poorly on HateSpeech, possibly because special abusive words dominate classification and are difficult to substitute by paraphrasing.
  • 4.3 Validity Results of Human Evaluation: StyleAdv’s human-evaluated validity is 49.5%, compared with 43% for SCPN and 3% for GAN.The evaluation samples 200 SST-2 adversarial examples per method, with three annotators per example.
  • 4.3 Validity Results of Human Evaluation: A style-transferred Tweets paraphrase preserves the original example’s semantics and fluency while changing the victim model’s prediction from Positive to Negative.The example illustrates a successful adversarial example generated from one of five transferred styles.

5 Experiments of Backdoor Attacks

The experiments evaluate StyleBkd against insertion-based backdoor baselines, with and without ONION defense, and assess attack effectiveness, stealth, trigger-style selection, and auxiliary training loss. StyleBkd maintains ASR above 90%, resists defense, and produces highly inconspicuous poisoned samples.

  • Experimental Settings: The study compares StyleBkd, RIPPLES, and InsertSent using attack success rate and clean accuracy, with evaluations conducted under non-defense and ONION-defense settings.The experiments use common datasets and victim models, tune poisoning rates for high ASR while keeping clean-accuracy decrements below 3%.
  • Backdoor Attack Results: Without defense, all three attacks achieve ASRs over 90 while maintaining clean accuracy across victim models and datasets.This indicates substantial susceptibility of NLP models to backdoor attacks in the evaluated settings.
  • Backdoor Attack Results: StyleBkd’s ASRs are lower than those of the insertion-based baselines but exceed 90 without exception.The authors attribute this difference to text style being more abstract and harder for victim models to remember than inserted content.
  • Backdoor Attack Results: Under defense, average ∆ASRs are -66.92 for RIPPLES and -45.49 for InsertSent, compared with -0.83 for StyleBkd.The authors report that StyleBkd is comparatively resistant because abstract style features are harder to damage.
  • Invisibility Evaluation: StyleBkd achieves the lowest poisoned-sample F1, down to 15.09, and the best PPL and GE quality results.Lower poisoned F1 means greater invisibility; the authors also report that StyleBkd samples are natural, fluent, and semantically preserved.
  • Trigger Style Selection: Bible has the highest probing classification accuracy and the highest overall StyleBkd performance, whereas Tweets has the lowest values.The result supports selecting the trigger style using probing classification accuracy.
  • Auxiliary Classification Loss: Adding the auxiliary classification loss improves StyleBkd from ASR 92.16 →94.70 and 91.94 →94.59 across two attack settings.It also improves defended InsertSent from ASR 30.92 → 47.69, while having little effect on RIPPLES in the corresponding situation.

6 Related Work

Related work covers text style transfer, textual adversarial attacks, and textual backdoor attacks. The paper positions style transfer as a previously unused basis for adversarial and backdoor triggers, emphasizing its potential invisibility.

  • Text Style Transfer: Most text style transfer research is unsupervised because parallel corpora are scarce, using disentangled representations or alternative methods such as reinforcement learning.Applications include formality alteration, dialogue diversification, and privacy-oriented personal-attribute obfuscation.
  • Backdoor Attacks on Text: Style transfer had not previously been used for adversarial or backdoor attacks, while style-based triggers are presented as more invisible than grammaticality-breaking alternatives.The related-work discussion also notes contemporaneous triggers based on syntactic structures and context-aware word substitution.
  • Adversarial Attacks on Text: Textual adversarial attacks are categorized by perturbation level into character-, word-, and sentence-level methods, with most existing work at character or word level.The supplied related-work passage identifies sentence-level attacks as another category.
  • Backdoor Attacks on Text: Existing textual backdoor attacks commonly insert fixed words or sentences as triggers into normal samples.These insertion-based triggers can impair grammaticality or fluency, making poisoned samples easier to detect and remove.

7 Conclusion and Future Work

The paper presents style-transfer-based adversarial and backdoor attacks and reports that popular NLP models are susceptible to both. Future work targets stronger handling of text style and defenses, while warning that proposed defenses may affect normal samples.

  • Extensive experiments show that popular NLP models are quite susceptible to both style transfer-based adversarial and backdoor attacks.
  • Future work will improve models’ robustness and learning ability on text style and develop defenses against style-transfer-based attacks.Suggested defenses include style-transfer augmentation during training and preprocessing test samples, but side effects on normal samples require careful consideration.

Ethical Considerations

The paper frames its attacks as exposing an underinvestigated vulnerability in NLP models while acknowledging possible malicious use. Its experiments use open datasets, fair annotation compensation, privacy-preserving evaluation, limited energy, and no demographic or identity characteristics.

  • The attacks aim to reveal NLP models’ inability to handle text style, a security issue not widely studied in previous work.
  • The authors acknowledge that the presented attacks could be maliciously used but prioritize raising awareness of existing NLP model vulnerabilities.
  • The study uses open, free datasets and reports fair annotator compensation, preserved privacy, limited experimental energy consumption, and no demographic or identity characteristics.
Loading 2110.07139v1…