PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating the Robustness of Neural Language Models to Input Perturbations

Milad Moradi, Matthias Samwald

arXiv:2108.12237v1cs.CLcs.AI

TL;DR

Benchmark results may not capture how NLP systems behave on noisy real-world text. The paper evaluates language models using character- and word-level perturbations, finding that small changes can reduce performance. It concludes that perturbed-input evaluations should complement benchmark testing.

  • Problem

    Benchmark performance may not reflect NLP-system reliability and robustness on noisy real-world inputs, and comprehensive evaluation under such conditions is missing.

    Source: §1 · pp. 1–2

  • Method

    The paper evaluates four language models on varied NLP tasks using systematic character- and word-level perturbations that simulate real-world noise.

    Source: §1 · p. 2

  • Results

    Across tasks, the evaluated models were sensitive to different perturbations, and slight input noise could decrease their performance.

    Source: §7 · p. 9

  • Takeaways & Limitations

    Perturbed-input testing can complement methods such as CheckList to assess NLP-system robustness and reliability in real-world settings.

    Source: §7 · p. 9

    Future work may help users assess whether perturbed text preserves meaning and grammatical correctness semi-automatically.

    Source: §7 · p. 9

Abstract

from arXiv · show

High-performance neural language models have obtained state-of-the-art results on a wide range of Natural Language Processing (NLP) tasks. However, results for common benchmark datasets often do not reflect model reliability and robustness when applied to noisy, real-world data. In this study, we design and implement various types of character-level and word-level perturbation methods to simulate realistic scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained. Conducting comprehensive experiments on different NLP tasks, we investigate the ability of high-performance language models such as BERT, XLNet, RoBERTa, and ELMo in handling different types of input perturbations. The results suggest that language models are sensitive to input perturbations and their performance can decrease even when small changes are introduced. We highlight that models need to be further improved and that current benchmarks are not reflecting model robustness well. We argue that evaluations on perturbed inputs should routinely complement widely-used benchmarks in order to yield a more realistic understanding of NLP systems robustness.

1 Introduction

High benchmark performance does not ensure reliable behavior on noisy real-world inputs. The paper evaluates language models with systematic perturbations and finds that small, understandable changes can expose instability.

1 Introduction

  • A comprehensive methodology for evaluating NLP models under real-world conditions is still missing, despite the gap between benchmark performance and real-world behavior.Small input changes can also cause high-performing models to make errors.Source: §1 · pp. 1–2
  • Typos and minor grammatical errors may preserve meaning and should not change system outcomes, while meaning-changing edits should be reflected in the output.Source: §1 · p. 2
  • BERT, RoBERTa, XLNet, and ELMo can be unstable under small, human-manageable changes such as misspellings, missing or repeated words, and synonyms.The authors caution against assessing NLP performance only with benchmark accuracy.Source: §1 · p. 2
  • The framework treats NLP systems as black boxes and complements CheckList by testing robustness to noisy inputs rather than linguistic capabilities.Source: §1 · p. 2
  • Most introduced perturbations can generate meaningful, understandable samples automatically, unlike CheckList test types that require time-consuming, creative human authoring.Source: §1 · p. 2
  • Unlike adversarial perturbations, which are described as infrequent worst-case noise requiring model internals and human imperceptibility, this study targets non-adversarial input changes.Source: §1 · p. 2
  • The paper presents a comprehensive set of non-adversarial perturbations on non-synthetic text, evaluates several models across NLP tasks, and includes a user study of sample meaningfulness.Source: §1 · p. 2

2 NLP tasks

The section introduces datasets used across five NLP tasks and gives details for several of them, spanning question classification, sentiment analysis, named-entity recognition, and semantic similarity.

2 NLP tasks

  • The experiments use five datasets covering five different NLP tasks.Source: §2 · p. 2
  • TREC is a text-classification dataset with more than 6,000 questions and 50 question-type labels.Source: §2 · p. 3
  • SST contains more than 11,000 Rotten Tomatoes movie reviews, each labeled as one of five sentiment classes.Source: §2 · p. 3
  • CoNLL-2003 contains Reuters news stories with more than 200,000 tokens annotated for person, organization, location, miscellaneous, or other entities.Source: §2 · p. 3
  • The STS benchmark contains more than 8,000 sentence pairs from image captions, news headlines, and user forums, each scored from 0 to 5.Source: §2 · p. 3

3 Language models

The section describes the four language models used in the experiments and how they were fine-tuned and evaluated on downstream tasks.

3 Language models

  • The reported BERTLARGE setup used the cased model, with 24 transformer layers, 1024 hidden units per layer, 16 attention heads, and 340 million parameters.Its pretraining objectives were masked language modelling and next sentence prediction.Source: §3 · p. 3
  • RoBERTaLARGE uses an architecture similar to BERT but optimizes pretraining with more data, bigger batches, longer sequences, dynamic masking, and no next sentence prediction objective.The section describes RoBERTa as further optimizing the BERTLARGE model.Source: §3 · p. 3
  • XLNetLARGE uses decoder transformers and permutation language modelling with generalized autoregressive pretraining, and has the same architecture hyperparameters and model size as BERTLARGE.Source: §3 · p. 3
  • ELMo contextualized embeddings were passed through task-specific output layers, including a softmax for TC and QA, a linear layer for SA and SS, and a CRF for NER.The original ELMo model uses character convolutions and forward and backward LSTMs.Source: §3 · p. 3
  • The pretrained models were fine-tuned separately on each downstream task using training and development sets, then tested on the test sets.Source: §3 · p. 3

4 Perturbation methods

The section presents character- and word-level perturbations for simulating textual noise, with examples from TREC and methods aimed at different input changes. It also distinguishes perturbations previously used in adversarial testing from those newly used for non-adversarial robustness testing.

4 Perturbation methods

  • The perturbations simulate real-world noise, can be applied regardless of the language model or NLP system, and are illustrated with TREC question-classification examples.Several word-level methods had not previously been used for robustness testing, while some character-level methods had been used in adversarial settings.Source: §4 · pp. 3–4, Table 2 · p. 4

4.1 Character-level perturbation

  • Character-level methods randomly select a word and insert, delete, replace, swap, or repeat a character within it.Insertion and deletion require words of at least three characters; the first and last characters are not deleted.Source: §4.1 · p. 4
  • Other character-level methods replace common misspellings and toggle the case of either the first character or all characters in a selected word.Common misspellings are sourced from a Wikipedia corpus.Source: §4.1 · p. 4

4.2 Word-level perturbation

  • Word-level deletion removes a randomly selected word, repetition inserts a copy after a selected word, and synonym replacement uses WordNet synonyms.Source: §4.2 · p. 4
  • Singular/plural verb changes are intended to preserve the output when subject-verb agreement is irrelevant, while word-order changes test sensitivity to the ordering of consecutive words.Source: §4.2 · p. 5
  • Verb-tense changes test sensitivity where tense should not affect the output, while negation changes positive verbs to negative or vice versa to test whether outputs reflect that change.The tense method identifies verbs and subjects with POS tags and uses mlconjug3; the negation method uses POS-based rules to construct samples.Source: §4.2 · pp. 4–5

5 Experimental results

The experiments show that language models are sensitive to small input perturbations, while the user study distinguishes mostly understandable meaning-preserving edits from word-level edits that need monitoring or filtering.

5.1 Performance on perturbed inputs

  • Meaning-changing Deletion, Negation, and synonym Replacement samples were manually corrected or filtered, with three annotators curating 200 samples per dataset and PPS; other word-level perturbations were generated automatically.PPS sets the maximum number of perturbations in a sample.Source: §5.1 · p. 6
  • Models were generally more sensitive to character-level than word-level perturbations; sentiment analysis was affected more often than other tasks, while question answering suffered less.Table 5 reports absolute performance decreases by task, perturbation level, and PPS.Source: Table 5 · p. 7, §5.1 · p. 7
  • Models’ performance decreased on slightly perturbed inputs; RoBERTa still performed better than the other models, while ELMo obtained the lowest scores generally.The models were not pretrained or fine-tuned on perturbed samples; perturbations were applied only to test sets.Source: §5.1 · p. 7
  • ELMo sometimes matched XLNet and RoBERTa on character-level noise, XLNet handled word-order changes more efficiently, and RoBERTa and XLNet were more robust to synonym replacement.Models were less stable on negation when it affected the task, such as sentiment analysis; ELMo was more sensitive to word order than transformer models.Source: §5.1 · p. 7
  • RoBERTa made errors on perturbed inputs despite getting the corresponding originals right, including cases with understandable character noise, repeated words, and a deleted word.Other examples include replacing “diameter” with “diam” and “golf” with “golf_game.”Source: Figure 1 · p. 8, §5.1 · p. 8

5.2 User study

  • In a user study with 20 participants, on average, 94% of perturbed samples in the evaluated set were judged understandable and to convey the same meaning as the original text.Source: §5.2 · p. 8
  • For word-level methods that may change meaning, 39% of samples remained label-consistent, 12% were meaningful but needed relabeling, and 49% were no longer meaningful.These results support monitoring, correcting, or filtering such samples.Source: §5.2 · pp. 8–9

6 Related work

Prior work examined targeted linguistic capabilities, task-specific robustness, biases, and adversarial vulnerabilities, but the paper situates its work as testing a broader range of real-world noise across tasks.

6 Related work

  • Adversarial perturbations represent a specific kind of noise, and much prior NLP work on them focused on misspelling attacks.Other methods mentioned include behavioral testing and model explanations.Source: §6 · p. 9
  • The paper’s perturbations cover a wider range of noise that NLP systems may encounter in real-world situations.Source: §6 · p. 9
  • Prior approaches studied specific linguistic features or tasks, including syntax-sensitive dependencies, machine translation, bias detection, and susceptible entities in documents.The paper contrasts those narrower investigations with its evaluation across a wide range of tasks and character- and word-level noise.Source: §6 · p. 9

7 Conclusion

The paper concludes that non-adversarial perturbations reveal sensitivity to noisy inputs that benchmark accuracy alone may not capture. It presents the methods as complementary testing tools, while identifying semi-automatic assessment of meaning and grammaticality as future work.

7 Conclusion

  • Evaluations across NLP tasks indicate that high-performance language models are sensitive to character- and word-level perturbations, and performance can decrease with slight noise.The authors caution that benchmark accuracy alone may be too simplistic for assessing robustness.Source: §7 · p. 9
  • Most perturbations can generate noisy test samples automatically, although a few require monitoring; the methods can complement CheckList and serve as a baseline for adversarial attacks.The experiments found the methods effective for evaluating systems against noisy data.Source: §7 · p. 9
  • Future work may help users assess meaning preservation and grammatical correctness semi-automatically.Source: §7 · p. 9

Appendix A. Fine-tuning hyperparameters

Appendix A reports fine-tuning hyperparameters by task for BERT, RoBERTa, XLNet, and ELMo; unlisted hyperparameters use the original models’ defaults.

Appendix A. Fine-tuning hyperparameters

  • Hyperparameters omitted from the tables use the defaults specified by the original models.Source: Appendix A · p. 12
  • BERT and RoBERTa each have fine-tuning hyperparameter tables covering multiple NLP tasks.The tables identify TC, SA, QA, NER, and SS as task categories.Source: Table 6 · p. 12, Table 7 · p. 12
  • XLNet has a fine-tuning hyperparameter table covering multiple NLP tasks.The table identifies TC, SA, QA, NER, and SS as task categories.Source: Table 8 · p. 12
  • ELMo has a fine-tuning hyperparameter table covering multiple NLP tasks.The table identifies TC, SA, QA, NER, and SS as task categories.Source: Table 9 · p. 12

Appendix B. Manual curation of perturbed samples

Appendix B reports how many perturbed samples were checked during manual curation to reach 200 test samples for each dataset across PPS values.

Appendix B. Manual curation of perturbed samples

Loading 2108.12237v1…