PaperlayerView exact revision on arXiv

Source-linked AI summary

TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack

Yu Cao, Dianqi Li, Meng Fang, Tianyi Zhou, Jun Gao, Yibing Zhan, Dacheng Tao

arXiv:2210.15221v1cs.CLcs.AI

TL;DR

QA-specific robustness and adversarial attacks have received limited study, despite concerns that models may rely on keyword matching rather than contextual relations. TASA attacks those biases with perturbed and distracting answer sentences while preserving gold answers for humans. Across five datasets and three victim models, it outperforms baselines in attack capability while maintaining textual quality and human-identifiable answers.

  • Problem

    Few studies investigate QA-model robustness, including whether answers reflect contextual comprehension or shortcuts such as keyword matching.

    Source: §1 · p. 1

  • Method

    TASA automatically edits contexts with a perturbed answer sentence that lowers focus on the gold answer and a distracting answer sentence that steers models toward a wrong answer.

    Source: §1 · pp. 1–2, §3.3 · p. 4

  • Results

    TASA achieves better attack capability than baselines across five extractive QA datasets and three victim models while preserving textual quality and human-identifiable gold answers.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The experiments support TASA as a way to study QA-model vulnerabilities while retaining gold answers that humans can identify.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

We present Twin Answer Sentences Attack (TASA), an adversarial attack method for question answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers. Despite phenomenal progress on general adversarial attacks, few works have investigated the vulnerability and attack specifically for QA models. In this work, we first explore the biases in the existing models and discover that they mainly rely on keyword matching between the question and context, and ignore the relevant contextual relations for answer prediction. Based on two biases above, TASA attacks the target model in two folds: (1) lowering the model's confidence on the gold answer with a perturbed answer sentence; (2) misguiding the model towards a wrong answer with a distracting answer sentence. Equipped with designed beam search and filtering methods, TASA can generate more effective attacks than existing textual attack methods while sustaining the quality of contexts, in extensive experiments on five QA datasets and human evaluations.

1 Introduction

The introduction frames QA robustness as underexplored and identifies keyword matching and ignored contextual relations as model biases. TASA uses paired answer-sentence edits to attack these biases while preserving the gold answer for humans, and outperforms existing baselines in attack capability while maintaining text quality.

1 Introduction

  • QA robustness is underexplored, and adversarial attacks tailored to QA can investigate model vulnerability and biases beyond benchmark performance.The introduction warns that model vulnerabilities and biases can lead to catastrophic failures outside benchmark datasets.Source: §1 · p. 1
  • TASA targets keyword matching and ignored shared entities by changing the answer sentence into paired perturbed and distracting sentences.The perturbed sentence substitutes shared keywords with synonyms while preserving meaning and the gold answer; the distracting sentence changes associated entities while retaining keywords.Source: §1 · pp. 1–2
  • Beam search and pretrained-model filtering address the efficiency and fluency challenges of generating adversarial contexts.TASA is described as a black-box attack that perturbs context without hurting fluency or changing the gold answer.Source: §1 · pp. 1–2
  • Across five extractive QA datasets and three victim models, TASA achieves better attack capability than baselines while preserving textual quality and human-identifiable gold answers.The evaluated models are BERT, SpanBERT, and BiDAF; human evaluations also support preserved text quality and gold answers.Source: §1 · p. 2

2 Predicting Bias in Question Answering

Tests of BERT and BiDAF on modified SQuAD 1.1 validation samples show that the models rely strongly on shared lexical words in the answer sentence. The findings motivate attacks that manipulate these cues to redirect predictions.

2 Predicting Bias in Question Answering

  • The study tests BERT and BiDAF trained on SQuAD 1.1 against modified validation samples, including contexts containing only the answer sentence.It compares removing or retaining shared entities, lexical words, and function words while keeping gold answer words and the rest of the context intact.Source: §2 · pp. 2–3
  • The answer sentence is the context sentence containing the gold answer, treated as key for prediction; sensitive words are shared entities, lexical words, and function words.Lexical words include content-bearing parts of speech, while function words lack lexical meaning; the experiments exclude the gold answer from removal or retention operations.Source: §2 · pp. 2–3
  • Keeping only shared lexical words in the answer sentence supports surprisingly satisfactory performance, while removing them causes a larger drop and shuffling them may even help.Keeping other word types instead is associated with a 30%–60% drop; the comparison covers EM and F1 for BERT and BiDAF.Source: §2 · p. 3, Table 1 · p. 3
  • The observed reliance on shared lexical cues motivates manipulating existing keywords and adding misleading ones to shift predictions toward crafted wrong answers.The section asks whether lowering focus on gold answers and adding misleading cues can redirect model predictions.Source: §2 · p. 3

3 Methodology

TASA is a black-box attack that changes the context while leaving the question and human-identifiable gold answer intact. It combines a perturbed answer sentence with a distracting answer sentence, using beam search and filtering to select edits.

3.1 Remove Coreferences

  • TASA removes context coreferences by replacing pronouns with the entity names they refer to, making target words easier to edit.Source: §3.1 · p. 4

3.2 Perturb the Answer Sentence

  • TASA selects answer-sentence keywords outside the answer span that share a lemma with question tokens, ranks them by masking-based importance, and replaces them with inflected synonyms.The answer sentence is located by matching the gold-answer text; these substitutions form perturbed answer sentences while preserving the gold answer.Source: §3.2 · p. 4

3.3 Add a Distracting Answer Sentence

  • TASA adds a distracting answer sentence by changing subjects or objects while preserving keywords, then substitutes a pseudo answer for the original answer span.Candidate subjects or objects exclude keywords and answer tokens and are named entities or nouns; replacement candidates match entity or noun type.Source: §3.3 · pp. 4–5

3.4 Beam Search and Filtering

  • Beam search ranks candidate edits by effect score, retains the top M at each step, and stops when edits are exhausted or the minimum score exceeds a threshold.PAS candidates replace the original answer sentence, while DAS candidates are appended; DAS generation follows PAS generation.Source: §3.4 · p. 5
  • Filtering retains PAS contexts judged answerable and meeting semantic-similarity and fluency thresholds before DAS generation.The quality score combines sentence similarity and perplexity; DASs are not filtered because they do not affect the gold answer.Source: §3.4 · p. 5

4 Experiments

Experiments evaluate TASA against attack baselines on five extractive QA datasets and three victim models, measuring attack effectiveness and text quality. Ablations and further analyses examine its components, beam search, pseudo-answer targeting, and adversarial training.

4.1 Setup

  • Evaluation uses dev sets from five extractive QA datasets, three span-prediction models, and EM, F1, GErr, and PPL metrics.The attack experiments use beam size M = 5 and a PAS synonym set combining WordNet and PPDB.Source: §4.1 · p. 6

4.2 Main Results

  • TASA has the lowest overall PPL and comparably low GErr; its token- and sentence-level edits avoid significant textual quality loss.TextFooler usually has the lowest GErr, while T3 has worse GErr and PPL; TASA also fools models better than AddSent, whose distractors do not perturb influential context.Source: §4.2 · pp. 6–7
  • TASA achieves the overall best performance among methods on BERT and SpanBERT, leading on three datasets and matching the best results on NewsQA and TriviaQA.The authors attribute its performance to combining PAS and DAS rather than only editing tokens or adding distracting text.Source: §4.2 · p. 6, Table 2 · p. 7
  • In human evaluation, TASA trails T3 on answer preservation but matches TextFooler and has significantly better textual quality than T3.The evaluation used 150 sets of samples from SQuAD 1.1, with 63 non-expert annotators rating answer preservation and fluency and grammaticality.Source: §4.2 · p. 7, Table 3 · p. 7

4.3 Ablation Studies

  • Ablations find that both PAS and DAS materially contribute to successful attacks and feasible sample counts; PAS quality filtering trades attack success against text quality.Removing coreference slightly benefits suitable-sample quantity; changing the answer in DAS is crucial, and using both named entities and nouns benefits attack effectiveness and sample quantity.Source: §4.3 · pp. 7–8, Table 4 · p. 8

4.4 More Analysis

  • Larger beam sizes improve EM, F1, and adversarial-sample diversity but slow generation; the authors use M = 5 because gains above five are limited.Figure 3 reports these measures across three victim models.Source: Figure 3 · p. 8, §4.4 · p. 8
  • On adversarial samples containing DAS, incorrect predictions substantially overlap with pseudo answers, with F1 values close to attack-induced performance drops.The authors interpret this as evidence that DAS can draw model attention and prompt incorrect predictions.Source: Table 5 · p. 8, §4.4 · p. 8
  • With a suitable mixture ratio, TASA adversarial training improves robustness to attacks without significant loss on original data, and its defense capability transfers to AddSent.Figure 4 compares performance on original dev data, TASA adversarial samples, and AddSent samples across mixture ratios.Source: §4.4 · pp. 8–9, Figure 4 · p. 9

5 Related Work

Related work spans extractive QA models and textual adversarial attacks, while highlighting limited attention to QA-specific vulnerabilities and methods.

Question answering

  • Extractive QA predicts a context span, and current models include end-to-end architectures and pretrained backbones, whose contextual understanding remains a concern.The discussion asks whether models capture contextual information or rely on token-level knowledge.Source: §5 · p. 9

Textual adversarial attack

  • Textual attacks use character-, token-, phrase-, and sentence-level edits, but the cited general-task methods had not demonstrated effectiveness on QA.The cited examples include token attacks (Ren et al., 2019; Alzantot et al., 2018; Zang et al., 2020; Li et al., 2021) and phrase or sentence edits (Iyyer et al., 2018; Lei et al., 2022).Source: §5 · p. 9
  • QA attacks include human-created distractors and automatic methods such as T3 and Universal Trigger; TASA is distinguished by connecting questions and contexts for broader conditions.The passage characterizes human-based methods as difficult to scale and limited in extensibility.Source: §5 · p. 9

6 Conclusion

The paper presents TASA as an automatic attack for extractive QA that uses twin answer sentences to exploit model reliance on sensitive keywords. Experiments report strong attacks with satisfactory sample quality, and additional analysis points to the potential for improving model robustness.

6 Conclusion

  • TASA constructs adversarial contexts with a perturbed answer sentence and a distracting answer sentence to target QA models’ keyword-matching pitfalls.The attack is evaluated on five datasets and three victim models.Source: §6 · p. 9
  • Experiments report remarkable attack performance with satisfactory sample quality, while additional analysis suggests TASA may help improve QA robustness.The robustness statement is framed as a future possibility.Source: §6 · p. 9

A Implementation Details

The appendix documents victim-model training, TASA implementation choices, and baseline configurations.

A.1 Training Victim Models

  • BERT and SpanBERT are fine-tuned for three epochs, while BiDAF is trained for 20 epochs with its best dev-set model retained.The appendix specifies separate maximum sequence lengths, optimizers, and batch sizes for the models.Source: §A.1 · p. 12

A.2 TASA

  • TASA selects the answer sentence using answer-span labels, replaces selected keywords with inflected synonyms, and removes context coreferences before editing.PAS synonyms are drawn from WordNet and PPDB; the appendix also describes sampling replacements for DAS and limiting perturbations to five.Source: §A.2 · p. 12
  • Filtering uses USE similarity and GPT-2 perplexity, while beam search stops at a specified effect threshold or when replacements are exhausted.For SQuAD 1.1, a RoBERTa model trained on SQuAD 2.0 judges answerability; other datasets use separately trained RoBERTa models.Source: §A.2 · pp. 12–13
  • The authors state that TASA applies only to extractive QA and leaves question and answer perturbation for future work.Hyperparameters were empirically tuned to balance attack effectiveness and textual quality.Source: §A.2 · p. 13, Table 7 · p. 13

A.3 Baselines

  • TextFooler is adapted to perturb context tokens while excluding answer tokens; T3 uses its black-box targeted configuration to misguide predictions toward pseudo answers.The authors report running baseline code and attempting to reproduce published results despite configuration differences.Source: §A.3 · p. 13

A.4 Datasets

The appendix specifies the datasets’ source and formatting, then reports BiDAF results as an additional victim-model evaluation. TASA performs best on three of five datasets and remains comparable on the other two.

A.4 Datasets

  • The experiments use the official SQuAD 1.1 release and MRQA versions of the other four datasets, converted to the SQuAD 1.1 format.Dataset statistics are provided in Table 8.Source: §A.4 · p. 13

B.1 Using BiDAF as the Victim Model

  • BiDAF is included as a representative end-to-end RNN-based victim model, although its results are omitted from the main paper because of space and its fair performance relative to SOTA models.The appendix reports its results across five datasets.Source: §B.1 · p. 13
  • TASA achieves the best attack effectiveness on 3 of 5 datasets and remains comparable on the other 2 when BiDAF is the victim model.It also has the lowest overall PPL and grammar-error performance close to TextFooler.Source: Table 9 · p. 14, §B.1 · p. 14

B.2 Shift to Pseudo Answers

The analysis compares model predictions with pseudo answers in distracting answer sentences against predictions on original samples. Models’ predictions overlap more with DAS pseudo answers on adversarial samples, supporting DAS’s misguiding effect.

B.2 Shift to Pseudo Answers

  • Across all three models and five datasets, predictions on TASA samples with DAS overlap more with DAS pseudo answers than predictions on original samples do.The comparison uses F1 scores for predictions on adversarial and original samples against the corresponding pseudo answers.Source: Table 10 · p. 14, §B.2 · p. 14
  • The prediction-overlap difference is smaller on NewsQA and TriviaQA, where TASA’s attack capability is consistently weaker.The authors state that DAS’s ability to draw model attention affects attack performance when combined with PAS.Source: §B.2 · p. 14

B.3 Analysis of Computational Complexity

On SQuAD 1.1, TASA is reported as the fastest attack and uses fewer victim-model queries than the compared baselines. T3’s query count is constant, but its complexity depends on target-model embedding scale.

B.3 Analysis of Computational Complexity

  • On SQuAD 1.1 across all victim-model types, TASA is the fastest attack and makes fewer queries than the compared baselines before finding an adversarial sample.Measurements were obtained on a single NVIDIA V100 GPU.Source: §B.3 · p. 14
  • T3 has a constant query count, but its complexity depends on the scale of the target model’s embedding.T3 is excluded from the query-count comparison because its query number is constant.Source: §B.3 · pp. 14–15

B.4 The Composition of Samples Generated by TASA

TASA’s PAS+DAS samples make up most adversarial examples on nearly all datasets and have the strongest attack effectiveness. Samples with only one component are less effective, with the difference from PAS+DAS narrowing on longer-context datasets.

B.4 The Composition of Samples Generated by TASA

  • PAS-only and DAS-only samples arise when the corresponding edits are inapplicable; on longer-context datasets, the PAS+DAS and PAS attack-performance difference narrows.The authors link the smaller PAS influence on longer inputs to its smaller share of the full input and lower attention weights per token.Source: §B.4 · p. 15
  • PAS+DAS samples form the majority on nearly all datasets and have the best attack effectiveness among the sample compositions.The table reports composition ratios and QA-model performance across five datasets and three victim models.Source: §B.4 · p. 15
  • Only PAS or only DAS lowers attack effectiveness relative to PAS+DAS, which uses both perturbed keywords and distracting sentences with pseudo answers.The authors attribute the difference to a single attack source being insufficient to fool models.Source: §B.4 · p. 15

C Qualitative Samples

This section presents sample adversarial contexts from TextFooler, T3, and TASA alongside model predictions, plus screenshots of the human-evaluation instructions. The context examples compare attacks with the original SQuAD 1.1 context using BERT as the victim model.

C Qualitative Samples

Loading 2210.15221v1…