On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

1 Introduction

• Text attacks are difficult to assess because even a single word or character change is perceptible, and textual similarity lacks a canonical distance metric.Robustness attacks can reveal model vulnerabilities, but input changes must not legitimately alter the expected output.Source: §1 · p. 1
• The proposed criterion requires adversarial examples to preserve source meaning while disrupting target meaning, so drastic output changes on equivalent inputs indicate a model fault.This focus on source-side meaning preservation contrasts with the cited prior seq2seq attack work (Cheng et al., 2018).Source: §1 · pp. 1–2
• The paper formalizes meaning-preserving perturbations for evaluation with human judgments or automatic proxies, and constrains gradient-based word substitutions to increase the chance of preserving meaning.The constrained attacks are then used in adversarial training experiments.Source: §1 · p. 2
• chrF correlates with human judgments better than BLEU and METEOR, distinguishes meaning preservation across languages and models, and constrained adversarial training improves robustness without reducing original-distribution test performance.The reported results also find that constrained attacks can be more meaning-preserving while still largely changing output sequences.Source: §1 · p. 2

2 A Framework for Evaluating Adversarial Attacks

• The framework denotes source and target sentences as x and y, the model translation as yM, and their perturbed-input counterpart as x̂ and its translation as ŷM.The attack-generation procedure is not restricted by this notation.Source: §2 · p. 2

2.1 The Adversarial Trade-off

• The trade-off uses source similarity and relative target-score decrease; report both, or use S = ssrc + dtgt, where S > 1 means target meaning is destroyed more than source meaning.Relative target decrease makes scores comparable across models and languages; the framework can also be adapted to targeted keyword attacks and other tasks.Source: §2.1 · p. 2

2.2 Similarity Metrics

• Semantic similarity scores are left separate from any particular metric because automatic sentence-similarity evaluation remains an open research area.The framework permits manual or automatic measures of source and target similarity.Source: §2.2 · p. 3

2.2.1 Human Judgment

• Human judgments are the gold standard; the proposed symmetric six-level scheme rates meaning similarity and considers fluency at the lowest and highest levels.The scale ranges from completely different or meaningless sentences to essentially equal meanings expressed in well-formed language.Source: §2.2.1 · p. 3

2.2.2 Automatic Metrics

• BLEU uses exact word n-gram matches, METEOR aligns stems and synonyms, and chrF uses character n-grams (Popović, 2015).chrF can reflect morphological variation without language-specific resources; the metrics are compared with human judgments using Pearson correlation.Source: §2.2.2 · p. 3

3.1 Attack Paradigm

• The attacks replace one word to maximize an adversarial loss, with added substitution constraints intended to make perturbations more likely to preserve meaning.Table 1 gives examples of the different adversarial inputs and highlights the substituted word.Source: §3.1 · p. 3, §3.2 · p. 4, Table 1 · p. 4

3.1.1 General Approach

• The general approach selects a word position and replacement by maximizing a differentiable adversarial objective, using a first-order approximation around the original word vectors.The paper describes greedy search as an adequate approximation and says the one-substitution search is sufficiently fast for adversarial training.Source: §3.1.1 · p. 4

3.1.2 The Adversarial Loss Ladv

• The adversarial loss maximizes the probability of an error at the next decoding step, assuming the model has produced the correct output prefix so far.The loss is expressed as a sum over output steps of the log probability of not producing the next target token.Source: §3.1.2 · p. 4

3.2 Enforcing Semantically Similar Adversarial Inputs

• The kNN constraint restricts replacements to the 10 nearest neighbors in the source embedding space, while CharSwap uses character swaps and requires the resulting word to be out of vocabulary.If swapping cannot produce an out-of-vocabulary word, the procedure repeats the word’s last character; for subword models, it applies perturbations to the original segmentation and then resegments the result. Unconstrained denotes the base attack without constraints.Source: §3.2 · pp. 4–5

4.1 Experimental setting

• The experiments use IWSLT2016 French, German, and Czech to English data with LSTM and Transformer models in word-based and BPE-subword forms.Validation data comprises earlier test sets; the 2015 and 2016 test sets are held out for testing.Source: §4.1 · p. 5

4.3 Attack Results

• Constrained kNN and CharSwap attacks preserve source meaning better than unconstrained attacks; CharSwap alone consistently exceeds 1 average attack success.In the French manual evaluation, 90% of sentence pairs rated 4 or 5 for semantic similarity had chrF > 78.Source: Table 2 · p. 6, §4.3 · pp. 7–8
• Word-based models show high target RDchrF under CharSwap, while Transformers appear less robust than LSTMs to kNN embedding perturbations.The authors relate the CharSwap result to word-based models’ known difficulty with out-of-vocabulary words, even with <unk> replacement.Source: Table 2 · p. 6, §4.3 · p. 7

4.2 Correlation of Automatic Metrics with Human Judgment

• chrF correlates with human judgments better than METEOR and BLEU on both source and target sides.Its advantage over METEOR is significant in French, but not English; the authors point to English synonym resources and French character-level errors as possible explanations.Source: Table 3 · p. 6, §4.2 · p. 6

5.1 Adversarial Training

• Training replaces the original-sample NLL objective with an interpolation of losses on original and adversarial samples.The experiments generate three perturbations on the fly per step and use top-3 instead of iteratively solving Equation (2), keeping adversarial training under 2× slower than normal training.Source: §5.1 · p. 8
• The section asks whether adversarial training's possible test-performance cost persists when attacks are largely meaning-preserving.The investigation compares unconstrained attacks with CharSwap-constrained attacks using a word-based LSTM model.Source: §5.1 · p. 8

5.2 Results

• With α = 1.0, Unconstrained-adv loses test performance, while CharSwap-adv remains on par with the original model.At α = 0.5, the effect disappears because the model also sees original samples; Table 5 reports chrF (BLEU) on the original test set.Source: Table 5 · p. 8, §5.2 · p. 8
• CharSwap-adv is more robust to CharSwap attacks for both α values, with α = 1.0 giving the best results.Unconstrained-adv is similarly or more vulnerable than the baseline; Table 6 reports validation-set RDchrF, where lower is better.Source: Table 6 · p. 8, §5.2 · p. 8

• Adversarial attacks and defenses developed from gradient-based attacks and adversarial training introduced for computer vision.The cited examples include later attack and defense mechanisms.Source: §6 · pp. 8–9
• Earlier NLP adversarial attacks target tasks including sentiment, malware, gender, and toxicity classification.Machine-translation attacks address word-based and character-based models, but largely focus on target-side evaluation rather than source meaning preservation.Source: §6 · p. 9
• Meaning-preserving NLP attacks have used paraphrase generation or rule-based approaches, but are highly engineered and focused on English.Source: §6 · p. 9

• The paper proposes a general framework for evaluating adversarial perturbations and compares automatic metrics as proxies for human judgment.Source: §7 · p. 9
• In machine translation, naive attacks do not preserve meaning in general, and the paper proposes alternatives to remedy this issue.Source: §7 · p. 9
• The paper reports the utility of adversarial training and hopes its framework helps future work evaluate meaning conservation more consistently.Source: §7 · p. 9

• If scrambling fails or the word is too short, the procedure repeats its last character until the result is out of vocabulary.Source: §A.1 · p. 12
• For words longer than three letters, the procedure repeatedly swaps adjacent internal characters and returns the result once it falls outside the vocabulary.It tries up to the specified maximum number of scrambling steps.Source: §A.1 · p. 12

• Table 7 reports statistics on the IWSLT2016 corpus used in the experiments.Source: Table 7 · p. 12, §A.2 · p. 12

A.3 Breakdown of Correlation with Human Judgement

• BLEU < METEOR < chrF in source-side correlation with human judgments across edit-count and constraint breakdowns.The subgroup coefficients come from Tables 8 and 9; their samples are smaller than Table 3’s, and not all differences are statistically significant at p < 0.01.Source: §A.3 · p. 12, Table 8 · p. 12, Table 9 · p. 12
• The subgroup correlations follow the same metric ranking, but their smaller sample sizes mean not all differences are significant at p < 0.01.Source: §A.3 · p. 12
• chrF’s good correlation with human judgments is not explained solely by distinguishing perturbations with different numbers of edits.Source: §A.3 · p. 12, Table 8 · p. 12