Source-linked AI summary
Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA
Yichen Jiang, Mohit Bansal
TL;DR
The paper examines whether HotpotQA truly tests multi-hop reasoning, given examples that permit shortcut answers through word matching. It constructs adversarial documents and introduces a dynamically controlled 2-hop model; adversarial evaluation exposes shortcut reliance, while adversarially trained 2-hop reasoning improves robustness and surpasses the trained 1-hop baseline.
Problem
HotpotQA examples often contain reasoning shortcuts that let models locate answers without connecting the intended multiple evidence pieces.
Method
The paper constructs adversarial documents that preserve the correct answer while disrupting shortcut paths, and uses a control unit to dynamically guide question attention across two reasoning hops.
Results
The adversarially trained 2-hop model improves over its regular-data counterpart and outperforms the adversarially trained 1-hop baseline on adversarial evaluation.
Takeaways & Limitations
Robust multi-hop QA benefits from combining explicit compositional reasoning with adversarial training.
Takeaways & Limitations
Adversarially trained models may learn new shortcuts by ignoring adversarial documents whose randomly sampled titles are unique within the context.
Abstract
from arXiv · showhide
Multi-hop question answering requires a model to connect multiple pieces of evidence scattered in a long context to answer the question. In this paper, we show that in the multi-hop HotpotQA (Yang et al., 2018) dataset, the examples often contain reasoning shortcuts through which models can directly locate the answer by word-matching the question with a sentence in the context. We demonstrate this issue by constructing adversarial documents that create contradicting answers to the shortcut but do not affect the validity of the original answer. The performance of strong baseline models drops significantly on our adversarial evaluation, indicating that they are indeed exploiting the shortcuts rather than performing multi-hop reasoning. After adversarial training, the baseline's performance improves but is still limited on the adversarial evaluation. Hence, we use a control unit that dynamically attends to the question at different reasoning hops to guide the model's multi-hop reasoning. We show that this 2-hop model trained on the regular data is more robust to the adversaries than the baseline model. After adversarial training, this 2-hop model not only achieves improvements over its counterpart trained on regular data, but also outperforms the adversarially-trained 1-hop baseline. We hope that these insights and initial improvements will motivate the development of new models that combine explicit compositional reasoning with adversarial training.
1 Introduction
HotpotQA examples can permit reasoning shortcuts that let models answer by matching question words rather than connecting multiple evidence pieces. The paper constructs adversarial documents to eliminate these shortcuts and finds that adversarial training plus explicit 2-hop control improves robustness.
- 1 Introduction: HotpotQA examples often permit models to locate answers by matching question keywords with one context sentence instead of following the intended reasoning chain.The shortcut is especially problematic because distractor documents may not contain information that breaks the direct match.
- 1 Introduction: Adversarial documents create conflicting shortcut answers while preserving the original answer, forcing models to connect evidence across multiple hops.Phrase-level perturbations alter answer spans and supporting-document titles, making single-hop matching yield two possible answers.
- 1 Introduction: The strong single-hop baseline performs poorly under adversarial evaluation, revealing reliance on shortcuts rather than genuine multi-hop reasoning.Supporting-fact supervision significantly improves adversarial performance, but the result remains below regular evaluation because the architecture is not designed for multi-hop reasoning.
- 1 Introduction: A 2-hop model dynamically attends to different question words at each reasoning step and is more robust to adversaries than the single-hop baseline.The control unit guides multi-hop bi-attention through a sequence of question-focused reasoning hops.
- 1 Introduction: Adversarial training further improves the 2-hop model, which outperforms the adversarially-trained baseline with or without supporting-fact supervision.These results motivate combining explicit compositional reasoning with adversarial training.
2 Adversarial Evaluation
HotpotQA’s distractor setting can leave bridge-type questions answerable through keyword-matching shortcuts rather than the intended two-document reasoning. The paper constructs adversarial documents that preserve the original answer while creating a misleading shortcut answer.
- 2.1 The HotpotQA Task: HotpotQA combines two supporting Wikipedia documents with eight TF-IDF-selected distractors, but distractors are not guaranteed to make both supports necessary.The dataset contains 113k questions, and crowd workers generate questions without seeing the distractors.
- 2.1 The HotpotQA Task: 26 of 50 sampled bridge-type development questions contain a reasoning shortcut that bypasses the intended first hop.The shortcut matches question keywords with a context fact instead of discovering the intended reasoning path.
- 2.2 Adversary Construction: ADDDOC perturbs an answer-containing support document to insert a fake answer that satisfies the shortcut without changing the question’s true answer.For multiword answers, it substitutes a non-stopword using close GloVe candidates or sampled answers, then replaces mentions of the original answer.
- 2.2 Adversary Construction: The adversary can also replace the bridge entity so the fake answer no longer forms a valid reasoning chain for the question.ADDDOC mixes the original context with constructed adversarial documents, while preserving the total number of context documents.
- 2.2 Adversary Construction: ADDDOC adversaries are model-independent, but randomly sampled titles can create a new shortcut because they may appear nowhere else in the context.The authors add another document containing each adversarial title to reduce this title-occurrence bias.
3 Models
The paper combines bi-attention with a recurrent control unit that dynamically focuses on different question words across reasoning hops. This architecture aims to guide compositional reasoning and supervise attention toward bridge entities connecting supporting documents.
- 3.2 Single-Hop Baseline: The architecture builds on a bi-attention and self-attention baseline that predicts answer spans, yes/no answers, and sentence-level supporting facts.The baseline uses contextual encodings, bi-attention, self-attention, span logits, a three-way answer classifier, and supporting-fact supervision.
- 3.3 Compositional Attention over Question: The 2-hop model uses a recurrent control unit to compute a question-word distribution at each reasoning hop.The distribution identifies which part of the question is relevant to the current hop and updates the recurrent control state.
- 3.3 Compositional Attention over Question: The control unit is intended to model sequential reasoning by first locating a bridge-related phrase and then attending to the next question component.The motivating example shifts from “father of Kasper Schmeichel” to “voted by IFFHS in 1992” across two hops.
- 3.3 Compositional Attention over Question: The control unit biases bi-attention so query-to-context attention focuses on context relevant to the current sub-question.The model replaces the standard context representation with a control-state-conditioned representation and computes context attention from the hop-specific question distribution.
- 3.3 Compositional Attention over Question: Because the control distribution lacks direct supervision, the model identifies and supervises a bridge entity connecting the two supporting documents.This supervision is introduced to help the control unit learn which question component matters at each reasoning step.
4 Experimental Setup
The experiments evaluate HotpotQA models under regular and adversarial conditions, using adversarial documents with varied insertion settings and EM as the primary metric. Adversarial training augments the regular training data with randomly inserted adversarial examples.
- Adversarial Evaluation and Training: The study constructs four adversarial dev sets by varying adversarial-document count and whether documents are randomly inserted or prepended.The settings are Add4Docs-Rand, Add4Docs-Prep, Add8Docs-Rand, and Add8Docs-Prep.
- Dataset and Metrics: Table 1 reports exact-match scores for regular-data and adversarially-trained models on regular and Add4Docs-Rand adversarial development sets.The table distinguishes 1-hop Base and 2-hop models without sentence-level supporting-fact supervision.
- Adversarial Evaluation and Training: Adversarial training uses Add4Docs-Rand examples, adding 40% of them to the regular training set.Fake titles and answers are sampled from the original training set, and random insertion is selected for robustness across evaluation settings.
- Dataset and Metrics: The evaluation uses HotpotQA’s distractor setting and reports EM rather than F1 because generated fake answers usually overlap lexically with original answers.The authors state that overall trends and takeaways are the same for F1.
5 Results
The experiments show that adversarial training improves robustness, while the control-unit 2-hop model and bridge-entity supervision further strengthen multi-hop performance across regular and adversarial evaluations.
- Regularly-Trained Models: The regular-data single-hop baseline performs poorly against adversarial documents, while supporting-fact supervision significantly improves adversarial evaluation but remains more than 9 points below regular performance.The result indicates that the baseline exploits reasoning shortcuts and that stronger supervision improves, but does not eliminate, the robustness gap.
- Adversarially-Trained Models: Adversarial training significantly improves both baseline and control-unit 2-hop models on adversarial evaluations.The comparison is supported by the reported horizontal comparisons across training conditions.
- Adversarially-Trained Models: The 2-hop model outperforms the single-hop baseline on both regular and adversarial evaluations, and sentence-level supporting-fact supervision provides further improvements.These comparisons are reported after adversarial training, with statistically significant differences over the baseline.
- Adversary Ablation: With eight adversarial documents per supporting document, all models lose more than 1 point, but the 2-hop models continue to outperform their single-hop counterparts.Prepending adversaries does not affect either model family in the reported ablation.
- Control Unit Ablation: Removing the control unit lowers performance in all four training and evaluation settings, validating its contribution to multi-hop robustness.The ablation compares models trained on regular or adversarial data and evaluated on both regular and adversarial dev sets.
- Bridge-Entity Supervision Ablation: Removing bridge-entity supervision causes large EM drops in both 1-hop and 2-hop models, showing that intermediate supervision supports compositional reasoning.The ablation uses a 2-hop model with unshared bi-attention layers and compares it with the supervised configuration.
6 Analysis
The analysis verifies that the adversaries preserve the original answers, identifies adversarial-document spans as a dominant failure pattern, and shows that some questions remain inherently single-hop.
- Manual Verification of Adversaries: In 50 randomly sampled ADD4DOCS-RAND examples, no adversarial fake answer contradicted the original answer, supporting the validity of the adversary construction.The adversaries therefore break the logical connection between supporting documents without invalidating the original answer.
- Model Error Analysis: For regular-data 1-hop baseline failures, 96.3% of predictions span an adversarial document, compared with 95.4% after adversarial training.The analysis attributes this vulnerability partly to separately predicted start and end indexes that can be affected by different adversarial documents.
- Adversary Failure Analysis: Some model successes remain because certain questions can be answered from a single supporting document, so adversaries cannot change their single-hop nature.The paper illustrates this with the question about the producer of Jennifer Kent’s directorial debut.
- Toward Better Multi-Hop QA Datasets: For future multi-hop datasets, final sub-questions should avoid being overly specific because semantic overlap with answer-context wording creates shortcuts.Less-specific final sub-questions make it easier for distractors to disrupt shortcut matching.
7 Related Works
Prior QA work largely focused on single-document, knowledge-base, table, synthetic, or limited cross-sentence settings before datasets such as QAngaroo and HotpotQA expanded multi-hop evaluation.
- Multi-hop Reading Comprehension: Earlier QA datasets mainly used single-document contexts, limiting the need to connect evidence across documents.The paper contrasts these settings with multi-hop reading comprehension.
- Multi-hop Reading Comprehension: Earlier multi-hop efforts reasoned over knowledge bases, tables, or synthetic text contexts, while TriviaQA included only a small portion of cross-sentence questions.These approaches provided related forms of multi-step reasoning before newer multi-document benchmarks.
- Multi-hop Reading Comprehension: QAngaroo and HotpotQA construct settings where evidence can be distributed across multiple documents, directly targeting composite reasoning.QAngaroo uses Wikipedia contexts and subject-relation queries, while HotpotQA is described as a newer multi-hop dataset.
- Adversarial Evaluation and Training: Adversarial QA evaluation previously showed that syntactically similar distractor sentences can significantly reduce state-of-the-art performance on SQuAD.The paper positions its adversarial documents as preserving semantic overlap while targeting reasoning shortcuts.
8 Conclusion
The paper identifies reasoning shortcuts in HotpotQA, evaluates them with answer-preserving adversaries, and finds that explicit 2-hop reasoning combined with adversarial training improves robustness over a single-hop baseline.
- 8 Conclusion: Adversarial documents expose HotpotQA reasoning shortcuts by creating conflicting shortcut answers without invalidating the original answer.Strong baseline performance drops under this adversarial evaluation.
- 8 Conclusion: A control unit dynamically attends to question words across reasoning hops to guide multi-hop bi-attention.The resulting 2-hop model is more robust than the baseline when trained on regular data.
- 8 Conclusion: Adversarial training further improves the 2-hop model, which outperforms the adversarially trained single-hop baseline.The paper presents these as initial improvements toward combining explicit compositional reasoning with adversarial training.
A Examples
Figure 4 presents a single-hop HotpotQA example that the proposed adversary cannot fix without introducing a contradiction.
- The question can be answered directly from the second document in the Golden Reasoning Chain.
- Because the example is single-hop, creating an adversarial document to defeat it would require introducing a contradiction.
- The figure illustrates a boundary of the adversarial-document approach rather than a successful adversarial case.