Source-linked AI summary
Generating Fluent Adversarial Examples for Natural Languages
Huangzhao Zhang, Hao Zhou, Ning Miao, Lei Li
TL;DR
NLP adversarial attackers must handle discrete sentence space and preserve fluency. MHA uses Metropolis-Hastings sampling with language-model and gradient guidance, and experiments report faster, more successful attacks plus improved adversarial-training outcomes.
Problem
NLP attack construction is difficult because sentence space is discrete and generated adversarial examples may be unfluent.
Method
MHA uses Metropolis-Hastings sampling, combining language-model and adversarial terms, with white-box proposals incorporating adversarial gradients.
Results
MHA generates attacks faster than the genetic baseline, with higher success rates and fewer victim-model invocations, while adversarial training improves accuracy across tested settings.
Takeaways & Limitations
MHA produces more fluent adversarial examples and may improve adversarial robustness and classification accuracy after adversarial training.
Takeaways & Limitations
MHA may return incomplete sentences or alter meanings through unconstrained entity and verb replacements.
Abstract
from arXiv · showhide
Efficiently building an adversarial attacker for natural language processing (NLP) tasks is a real challenge. Firstly, as the sentence space is discrete, it is difficult to make small perturbations along the direction of gradients. Secondly, the fluency of the generated examples cannot be guaranteed. In this paper, we propose MHA, which addresses both problems by performing Metropolis-Hastings sampling, whose proposal is designed with the guidance of gradients. Experiments on IMDB and SNLI show that our proposed MHA outperforms the baseline model on attacking capability. Adversarial training with MAH also leads to better robustness and performance.
1 Introduction
NLP adversarial attack construction must reconcile discrete sentence perturbations with fluent, effective examples. MHA addresses these challenges with Metropolis-Hastings sampling and reports faster, more successful attacks plus improved adversarial-training outcomes.
- Challenges: NLP attackers struggle to follow gradient directions because sentence space is discrete, while unfluent examples are easier for victim models to recognize.These two problems limit attack effectiveness and adversarial training performance.
- Prior work: HotFlip uses gradient-guided character flipping, but it can produce meaningless words such as “mood” becoming “mooP”.
- Prior work: Genetic attack filters word replacements with a language model, yet its examples remain insufficiently fluent and it discards gradient information.
- MHA: MHA applies Metropolis-Hastings sampling to generate fluent adversarial examples, with black-box and white-box variants incorporating language-model and gradient guidance.The black-box and white-box variants are described as b-MHA and w-MHA, respectively.
- Results: On IMDB and SNLI, MHA generates attacks faster, achieves higher success rates with fewer victim-model invocations, and improves robustness and accuracy after adversarial training.
2 Preliminary
Adversarial attacks perturb inputs to induce incorrect model outputs, while adversarial training incorporates those examples into training to improve victim models. The paper distinguishes attacks by the model information available to the attacker.
- Adversarial learning: Adversarial attacks perturb samples to mislead neural models, producing erroneous outputs such as wrong labels in text classifiers.
- Adversarial learning: Adversarial training mixes perturbed examples into the training set to improve model performance and robustness.
- Definitions: Victim models are word-level classifiers that take tokenized sentences as input and output labels.
- Attack settings: Black-box attacks access only model outputs, whereas white-box attacks also access gradients, outputs, and model parameters.
- Definitions: For adversarial training, the victim model is trained from scratch on an updated dataset containing adversarial examples.
3 Proposed Method: MHA
MHA applies Metropolis-Hastings sampling to generate adversarial sentences, combining language-model fluency with attack objectives and, in its white-box variant, gradient guidance. Its proposal mechanism uses word-level transitions and pre-selection to make sampling more efficient, while white-box gradients restrict the available operations.
- MHA overview: MHA applies Metropolis-Hastings sampling to generate adversarial examples for natural language.At each iteration, the method proposes a transition and accepts or rejects it according to the M-H acceptance procedure.
- Black-box attack: b-MHA designs its stationary distribution to balance fluent sentences, classifier deception, and few classifier invocations.Its language-model term supports fluency, while the adversarial term targets an erroneous victim-model label.
- Black-box attack: The black-box proposal uses replacement, insertion, and deletion transitions, combined as a weighted sum and supported by pre-selected candidate words.Pre-selection avoids repeatedly evaluating every vocabulary word when computing the replacement proposal.
- Black-box attack: Candidate pre-selection ranks words with a bidirectional language-model score using the sentence prefix and suffix, reducing the candidate set used by replacement.Without pre-selection, the candidate set would contain the full vocabulary and require repeated classifier invocations.
- White-box attack: w-MHA adds gradient similarity to the pre-selection score, guiding substitutions toward the gradient direction and increasing the target error probability and acceptance rate.The gradient identifies the steepest direction, while the embedding difference represents the actual substitution direction.
- White-box attack: w-MHA excludes insertion and deletion because their gradients are difficult to compute after discrete sentence changes.The gradient through an intermediate inserted sentence is not the actual gradient for the original sentence, whose discrete change is non-differentiable.
4 Experiments
Experiments on IMDB and SNLI evaluate MHA’s attack efficiency, fluency, robustness, and accuracy effects after adversarial training. MHA generally improves attack efficiency and the downstream benefits of adversarial training, while w-MHA can reduce human-rated fluency and MHA can alter sentence meaning.
- Adversarial Attack: MHA’s invocation-success curves exceed the genetic baseline, and w-MHA rises faster than b-MHA when gradients guide the proposal distribution.The evaluation samples 1,000 IMDB and 500 SNLI correctly classified test examples.
- Adversarial Attack: MHA generates examples with lower perplexity than the compared models, indicating greater likelihood under the evaluation language model.Success rates are measured with at most 6,000 victim-model invocations, and the success-rate gaps are not large because all models achieve high rates.
- Adversarial Attack: b-MHA receives the best human fluency ranking, while w-MHA is ranked below the genetic approach.The mean rankings are 1.80 for b-MHA, 1.93 for the genetic approach, and 2.03 for w-MHA, where lower is more fluent.
- Adversarial Attack: On SNLI, MHA uses a language model to avoid some grammatical mistakes, but unrestricted replacements can change the original sentence’s meaning.The examples may replace entities or verbs with irrelevant words because MHA does not require replacements to have similar meanings.
- Adversarial Training: Adversarial training with b-MHA or w-MHA decreases the success rate of genetic attacks, whereas genetic adversarial training cannot defend against MHA.This comparison is reported for robustness testing on IMDB.
- Adversarial Training: w-MHA significantly improves SNLI classification accuracy across all three training-data settings and outperforms the genetic baseline with 10K and 30K data.The reported p-values are below 0.02; with 100K training data, w-MHA achieves comparable improvements, and gains are larger with less training data.
5 Future Works
The current MHA has limitations involving sentence completeness and unconstrained entity or verb replacements, motivating additional generation constraints.
- 5 Future Works: Returning an example immediately after the label changes can produce incomplete, human-unfluent sentences.Forcing the model to generate ⟨EOS⟩ before returning may address this issue.
- 5 Future Works: Unconstrained entity and verb replacements negatively affect adversarial example generation for NLI tasks.Limiting similarity during word operations or embedding distance may help address this problem.
6 Conclusion
The paper proposes MHA, which uses Metropolis-Hastings sampling to generate natural-language adversarial examples. Experiments report faster generation than the genetic baseline, with examples that are more fluent and potentially more effective for adversarial training.
- 6 Conclusion: MHA generates natural-language adversarial examples by adopting the Metropolis-Hastings sampling approach.
- 6 Conclusion: MHA generates adversarial examples faster than the genetic baseline.
- 6 Conclusion: MHA's generated adversarial examples are more fluent and may be more effective for adversarial training.