PaperlayerView exact revision on arXiv

Source-linked AI summary

Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models

Yimu Wang, Peng Shi, Hongyang Zhang

arXiv:2307.12507v2cs.CLcs.AIcs.CRcs.CY

TL;DR

The paper addresses obstinate adversarial examples, where word substitutions change input meaning without changing model predictions, while prior methods rely mainly on manual antonym strategies. It introduces GRADOBSTINATE, a gradient-based method that searches broadly and filters semantic changes. Across diverse benchmarks, it achieves higher attack success rates than prior methods, and its substitutions transfer successfully to other models, including GPT-3 and ChatGPT.

Abstract

from arXiv · show

In this paper, we study the problem of generating obstinate (over-stability) adversarial examples by word substitution in NLP, where input text is meaningfully changed but the model's prediction does not, even though it should. Previous word substitution approaches have predominantly focused on manually designed antonym-based strategies for generating obstinate adversarial examples, which hinders its application as these strategies can only find a subset of obstinate adversarial examples and require human efforts. To address this issue, in this paper, we introduce a novel word substitution method named GradObstinate, a gradient-based approach that automatically generates obstinate adversarial examples without any constraints on the search space or the need for manual design principles. To empirically evaluate the efficacy of GradObstinate, we conduct comprehensive experiments on five representative models (Electra, ALBERT, Roberta, DistillBERT, and CLIP) finetuned on four NLP benchmarks (SST-2, MRPC, SNLI, and SQuAD) and a language-grounding benchmark (MSCOCO). Extensive experiments show that our proposed GradObstinate generates more powerful obstinate adversarial examples, exhibiting a higher attack success rate compared to antonym-based methods. Furthermore, to show the transferability of obstinate word substitutions found by GradObstinate, we replace the words in four representative NLP benchmarks with their obstinate substitutions. Notably, obstinate substitutions exhibit a high success rate when transferred to other models in black-box settings, including even GPT-3 and ChatGPT. Examples of obstinate adversarial examples found by GradObstinate are available at https://huggingface.co/spaces/anonauthors/SecretLanguage.

Introduction

Obstinate adversarial examples change input meaning while preserving model predictions, a complementary vulnerability to fickle attacks. GRADOBSTINATE addresses the limits of manual antonym-based generation with gradient-based substitutions and shows strong attack and transfer performance.

  • Obstinate examples alter input semantics while models retain their predictions, unlike fickle examples that change outputs after small perturbations.Source: Introduction · pp. 1–2, Figure 1 · p. 1
  • Existing word-substitution methods rely on manual or antonym-based strategies, limiting the search space and hindering defenses.Source: Introduction · p. 2
  • GRADOBSTINATE is the first gradient-based method that autonomously generates obstinate examples without human expertise or predefined search constraints.Source: Introduction · p. 2
  • Across five benchmarks and five representative language models, GRADOBSTINATE outperforms previous methods on obstinate-example generation.Source: Introduction · p. 2
  • Obstinate substitutions transfer with consistently high success rates to other models, including GPT-3 and ChatGPT in black-box settings.Source: Introduction · p. 2

Related works

Prior work has extensively studied fickle robustness, whereas obstinate robustness has received less attention. GRADOBSTINATE differs from earlier manual strategies by using gradients to search automatically across the vocabulary.

Fickle adversarial robustness in NLP

  • Fickle robustness methods seek stable predictions for semantically similar inputs and include synonym-based, character-level, and word-level attacks.Source: Related works · p. 2
  • GRADOBSTINATE also uses gradients to change words, but targets obstinate rather than fickle robustness and directly perturbs sentences instead of embeddings.Source: Related works · pp. 2–3

Obstinate adversarial robustness in NLP

  • Obstinate robustness concerns semantic-changing perturbations that preserve model outputs, a vulnerability previously explored mainly through antonym-based strategies and adversarial training.Source: Related works · p. 3
  • GRADOBSTINATE automatically searches an effectively unlimited vocabulary space without human effort and reports superior attacks across five language-based models.Source: Related works · p. 3

Our approach

GRADOBSTINATE searches for word substitutions that change semantics while preserving model outputs. It combines gradient-guided one-hot perturbations with vocabulary decoding and semantic filtering to produce valid obstinate examples.

Problem overview

  • An obstinate adversarial example is semantically different from its original sentence but receives the same model output.Source: Problem overview · p. 3

GRADOBSTINATE for finding obstinate adversarial examples

Our approach

GRADOBSTINATE for finding obstinate adversarial examples

Experiments

Experiments evaluate GRADOBSTINATE across diverse NLP and cross-modal settings, showing strong attack performance, semantic changes with preserved predictions, and transferability across models.

Experimental settings

  • Experiments use five models across sentiment, paraphrase, NLI, QA, and image-text retrieval, targeting task-specific text inputs and evaluating attack success over ten restarts.The settings use selected benchmarks and attack targets, with important words identified by Integrated Gradients.Source: Experimental settings · pp. 4–5

Quantitative results

  • GRADOBSTINATE consistently outperforms antonym-based attacks across SST-2, MRPC, SNLI, and SQuAD, reaching 99.51% and 97.54% on SST-2 and 96.60% to 86.60% across multi-sentence tasks.On MRPC, the antonym-based method achieves only 0.85% and 0.58%, while GRADOBSTINATE reaches 96.60% and 96.77%.Source: Quantitative results · pp. 5–6, Table 2 · p. 5
  • GRADOBSTINATE achieves 100% attack success on both MSCOCO train and validation sets, versus 3.9% and 3.6% for the antonym-based method.Source: Table 2 · p. 5, Quantitative results · p. 6

Qualitative results

  • Qualitative examples show that substitutions can change the semantic meaning of MRPC, SNLI, and SQuAD inputs while preserving model outputs.Replacing “sculpted” with “volunteered” changes a question’s meaning while Roberta produces the same answer.Source: Qualitative results · p. 6, Table 3 · p. 6

Black-box settings via transferability

  • Obstinate substitutions transfer across contexts and models, including 10/11 success for “yoga”→“burg” and 58/63 for “scientists”→“sunderland”.Transfers from GPT-2 and Roberta to GPT-3 and ChatGPT also succeed, although ChatGPT can distinguish some examples better than GPT-3.Source: Table 4 · p. 6, Black-box settings via transferability · p. 7

Conclusion

GRADOBSTINATE generates obstinate adversarial examples through gradient-based word substitution without human effort or predefined rules, outperforming previous methods across diverse benchmarks. Its substitutions also transfer across models, including black-box GPT-3 and ChatGPT, while the method remains a white-box approach and has not been evaluated for adversarial training.

  • GRADOBSTINATE uses gradient-based word substitution to generate obstinate adversarial examples without human effort or predefined rules.Source: Conclusion · p. 7
  • Higher attack success rates across SST-2, MRPC, SNLI, SQuAD, and MSCOCO demonstrate superiority over previous methods.Source: Conclusion · p. 7
  • Obstinate substitutions transfer across different models, including black-box GPT-3 and ChatGPT.Source: Conclusion · p. 7, Table 5 · p. 7
  • The method is gradient-based and white-box, and its effectiveness for adversarial training remains unexamined.Source: Conclusion · p. 7

Evaluation

The evaluation spans five benchmarks, integrated-gradient constructions, multi-word attacks, qualitative analyses, and black-box transfer tests. Results show that GradObstinate can find semantically dissimilar substitutions while preserving model outputs, with transferability varying across target models.

Details of experimental settings

  • GradObstinate is evaluated on SNLI, MRPC, SQuAD, SST-2, and MSCOCO using sampled training and testing subsets.The evaluation uses the first 10,000 training and testing samples for each dataset, with smaller subsets for some SQuAD and MSCOCO settings.Source: Details of experimental settings · p. 11
  • For multi-sentence tasks, changing one sentence while keeping the other fixed tests whether the model preserves an unchanged relationship despite altered input meaning.This setting is motivated as a direct test of obstinate behavior in sentence-pair and question-answering tasks.Source: Details of experimental settings · p. 11, Details of integrated gradient · p. 11

Details of integrated gradient

  • Integrated gradients rank input words by attribution, using word-stacked inputs and reference-token baselines before aggregating scores across dimensions.The construction differs by task: paired tasks stack two segments, while sentiment and image-text retrieval use one text segment.Source: Details of integrated gradient · p. 11

Quantitative results

  • Accuracy for semantically dissimilar replacements generally decreases as more words are changed on multi-sentence benchmarks, with the highest MRPC and SNLI values occurring at two replacements.The authors attribute the pattern partly to replacement randomness and sentence-length constraints.Source: Quantitative results · p. 12, Table 6 · p. 12, Figure 4 · p. 14
  • 59.47% and 59.30% accuracy remain possible on SNLI train and test splits after replacing 10 words, even when changed hypotheses share no words or semantic similarity with originals.The authors report that completely changed hypotheses can retain the same classification result, including “neutral” outputs for unrelated premises.Source: Quantitative results · pp. 12–13
  • On SST-2, accuracy is high and behaves differently across splits: training accuracy rises with more replacements, whereas test accuracy falls.The authors suggest this may reflect memorization in the sentence-classification task.Source: Table 7 · p. 12, Quantitative results · pp. 13–14, Figure 5 · p. 14

Qualitative results

  • Qualitative analyses show models often select punctuation, pronouns, adverbs, and determiners, while substitutions frequently become nouns, proper nouns, or verbs.ALBERT, DistillBERT, and RoBERTa show partially consistent word selection across splits, but train and test substitutions differ because of randomness.Source: Qualitative results · pp. 14–15

Black-box settings via transferability

Loading 2307.12507v2…