Finding Generalizable Evidence by Learning to Convince Q&A Models

• The approach is motivated by the difficulty of extracting answers from large volumes of text or complex arguments.Source: §1 · p. 1
• Agents select passage sentences that maximize a QA judge model’s probability of a designated answer, using exhaustive search or learned prediction.The system trains one agent per answer to identify sentences the judge regards as strong evidence.Source: §1 · p. 1
• Agent-selected evidence makes both humans and other QA models favor the supported answer over alternatives.This tests whether evidence generalizes beyond the model that selected it.Source: §1 · p. 1
• Humans can answer QA benchmark questions accurately using evidence for each answer that contains only 20% of the full passage’s sentences.Source: §1 · p. 2
• Using selected evidence helps QA models generalize from short training passages to longer passages and from middle-school to high-school questions.The reported improvement occurs when models answer from the most convincing evidence rather than the full passage.Source: §1 · p. 2

Evidence Selection

• Answer-specific agents select passage sentences that maximize a judge model’s probability for their assigned answer instead of using the full passage.Each agent seeks evidence E(i) that maximizes pφ(i|E(i), Q, A), with the judge trained to predict the correct answer from the passage, question, and options.Source: §2 · p. 2
• Sequential agents build evidence greedily by repeatedly adding the sentence that most improves support for one answer.The selected evidence is an ordered subsequence of passage sentences, and adding an already selected sentence is a no-op.Source: §2 · p. 2
• Competing agents jointly compose evidence pools, either through free-for-all interaction or round-robin pairwise contests between answers.Free-for-all agents condition choices on other agents’ prior selections, while round-robin aggregation averages an answer’s probability across its pairwise rounds.Source: §2 · p. 2

Judge Models

• Judge models score answer options using TFIDF, fastText, or BERT, providing diverse model classes for training and evaluating evidence generality.TFIDF and fastText use cosine similarity, while BERT computes answer scores from the concatenated passage, question, and answer.Source: §2.1 · p. 3

Evidence Agents

• Evidence agents are implemented either by exhaustive search over candidate sentences or by learned models predicting how sentences influence the judge.Learned agents predict one scalar per passage sentence, can reuse predictions across later timesteps, and can model judge behavior without answer labels for each question.Source: §2.2 · pp. 3–4

Evaluation Desiderata

• Agents are evaluated on questions with more than one passage-supported answer so performance reflects evidence selection rather than only the answer being defended.This desideratum is why multiple-choice QA is preferred over extractive span-based QA for the experiments.Source: §3.1 · p. 4

Multiple-choice QA: RACE and DREAM

• RACE and DREAM provide multiple-choice passage-based QA questions with plausible distractors, enabling evidence evaluation across defensible answer options.RACE has four options per question and DREAM has three; each has exactly one correct option.Source: §3.1 · p. 4

Other datasets we considered

• On SQuAD, agents could learn to convince the judge only when supporting the correct answer because each question has one answer.This limits SQuAD’s usefulness for evaluating answer-independent evidence selection.Source: §3.1 · p. 4

Judge models

• The study evaluates judge models with full-passage accuracy spanning word-based baselines and BERT models, with BERTLARGE achieving the best overall accuracy.No model reaches the estimated human ceiling on either dataset.Source: §3.2 · p. 4, Table 2 · p. 4

Experimental setup

• The setup is not directly comparable to standard QA accuracy because it targets evidence quality, while judge-model accuracy is reported for analysis.The code and human-evaluation implementation use AllenNLP, PyTorch, and Mechanical Turk.Source: §3.2 · p. 4

Human evaluation setup

• Humans evaluate agent-selected evidence from a single sentence alongside the question and answer options, without seeing the full passage.The human study uses a 100-question subset of the test set.Source: §4.1 · p. 5, Evaluating an agent’s evidence across… · p. 6

Human Evaluation of Evidence

• 42% of RACE human choices favor evidence from the learned p(i) agent, matching 41.6% for human-selected evidence and exceeding the 25% chance baseline.Across agents, human selection rates range from 33.5–42.0% on RACE and 41.7–50.5% on DREAM.Source: §4.1 · p. 5
• 74.6% versus 31.1%: on RACE, the learned p(i) agent finds strong evidence more than twice as often for correct than incorrect answers.BERT-based agents generally find stronger evidence than word-based agents, and learned BERTBASE agents outperform directly querying BERTBASE on RACE.Source: §4.1 · p. 5, Table 3 · p. 5
• Learned agents may find more general evidence because they predict sentences with large effects on judge confidence, which humans are more likely to regard as valid evidence.This mechanism is offered to explain why learned agents can outperform search agents in human evaluation.Source: §4.1 · pp. 5–6

Evaluating an agent’s evidence across models

• Other judge models select learned p(i) or ∆p(i) agents’ answers over 46% of the time, indicating evidence generalizes beyond its target model.All agents exceed the 25% no-evidence baseline, although agents are not uniformly convincing across judges.Source: Evaluating an agent’s evidence across… · p. 6, Figure 2 · p. 6
• Agents are most convincing to their target judge, while similar models tend to find similar evidence convincing across judges.Search agents are especially effective for their target because they directly query that model; similar trends also appear on DREAM.Source: Evaluating an agent’s evidence across… · p. 6

5 Evidence Agents Aid Generalization

• The experiments compare full-passage QA with evidence-selected QA using judges trained on easier examples and tested on longer or harder examples.Agents are trained without labels on the full RACE dataset, and random-sentence and answer-only baselines provide reference points.Source: §5 · pp. 6–7

5.1 Generalizing to Longer Passages

• 50.2% vs. 40.2%: agent-chosen sentences improve judge accuracy on longer passages over the answer-only baseline, exceeding full-passage accuracy of 44.1%.Both search and learned agents help, while random sentence selection reaches only 44.7%, showing the gain is not merely from shorter input.Source: §5.1 · p. 7, Table 4 · p. 7

5.2 Generalizing Across Domains

• BERT-based evidence agents improve RACE-to-DREAM transfer when the judge answers from selected evidence rather than the full passage.The transfer setting changes content, writing style, sentence length, and discourse format, and the learned agent is not fine-tuned on DREAM.Source: §5.2 · p. 7, Table 4 · p. 7

5.3 Generalizing to Harder Questions

• 53.0% and 51.9% with BERT agents outperform 50.7% with the full passage and 50.4%-51.0% with search-based TFIDF and fastText agents on harder questions.The judge is trained on RACE Middle School questions and tested on High School questions.Source: §5.3 · p. 7, Table 5 · p. 7
• The harder-question gains come mainly from categories where full-passage performance is worse, according to Figure 3.Providing key sentences may remove extraneous or distracting material, especially in passages with uncommon words or complex sentence structure.Source: Figure 3 · p. 8, §5.3 · p. 8

Human evaluation setup

• The human evaluation presents competing agents’ selected sentences and collects five answers for each of 100 test questions.Humans act as the judge in the free-for-all setup.Source: §6 · p. 8

Humans can answer using evidence sentences alone

• Humans retain 90% of full-passage QA accuracy while reading 3.3 versus 18.2 sentences on RACE and 2.4 versus 12.2 on DREAM.Accuracy is 73.2% versus 82.3% on RACE and 83.8% versus 93.0% on DREAM.Source: §6 · p. 8, Table 6 · p. 8
• BERTBASE evidence yields 83.8% human accuracy, exceeding BERTLARGE evidence at 75.0% because it better supports right answers and less supports wrong ones.BERTBASE supports right answers at 82.5% versus 79.4% for BERTLARGE, and supports wrong answers at 34.6% versus 38.7%.Source: §6 · p. 8

Related work

• The paper places these connections alongside broader discussion of similarities and differences between patterns learned by humans and neural networks.This discussion is identified as extending material from §4.1.Source: §7 · p. 8

Evidence Extraction

• Prior work extracts evidence for correct-answer prediction or to explain what a particular neural model has learned.The cited approaches include Wang et al. (2018a), Lei et al. (2016), Li et al. (2016), and Yu et al. (2019).Source: §7 · p. 8

Generic Summarization

• Other approaches optimize generic, question-agnostic summaries using downstream QA accuracy as a reward rather than extracting answer-specific evidence.The cited work includes Arumae and Liu (2018, 2019) and Eyal et al. (2019).Source: §7 · p. 9

Debate

• Evidence extraction can be framed as debate, and prior evidence-based debate improves crowdsourced label accuracy similarly to this paper’s QA findings.The related debate work is Irving et al. (2018), Irving and Askell (2019), and Chen et al. (2018).Source: §7 · p. 9

8 Conclusion

• Training evidence agents to convince a judge model distills general insights for passage-based QA.The agents select evidence for candidate answers, and the selected evidence is evaluated with humans and other QA models.Source: §8 · p. 9
• Humans correctly answer questions using only 20% of the full passage, while agent-selected evidence also helps QA models generalize to longer passages and harder, out-of-distribution questions.The conclusion characterizes the selected evidence as generalizable across evaluators.Source: §8 · p. 9

• Evidence-agent examples show that a BERTLARGE search agent can select a sentence making its designated answer receive over 99% confidence from the judge.Source: Table 7 · p. 11
• The implementation uses BERT tokenization and sentence boundaries, truncates passages to fit 512 tokens while retaining the question and answer, and predicts evidence at sentence-ending positions.Learned agents use BERTBASE, and their training applies early stopping based on validation loss because they do not predict the correct answer directly.Source: §B.1 · p. 11, §B.2.2 · p. 11, §B.3 · pp. 11–12
• The evaluation also exposes adversarial evidence: sentences can bias an answerer toward incorrect options, while presenting evidence for multiple answers can help it weigh competing support.Source: Table 9 · p. 12
• Human evaluations present workers with agent-selected evidence sentences and assess whether they choose the supported answer without the full passage.Workers were filtered using representative-task performance and completion-time criteria, and were recruited from predominantly English-speaking countries.Source: C Human Evaluation Details · pp. 12–13
• Models that perform better generally achieve gains across question types, with neural methods showing larger improvements on multi-sentence reasoning questions on RACE.Source: D Human Evaluation of Agent Evidence… · p. 13

• Search-predicting agents more accurately identify the search-selected sentence when that sentence has a larger effect on the judge’s target-answer probability.Source: E Analysis · p. 13, Figure 6 · p. 13
• Table 11 evaluates search and learned agents by having humans answer questions using only each selected evidence sentence.Source: Table 11 · p. 14
• Judge models’ sentence confidence aligns with human evidence judgments: humans tend to select the target answer when the judge assigns that sentence high target-answer probability.Source: Figure 7 · p. 14
• Because strong model evidence tends to be strong human evidence, learned agents are more accurate at predicting sentences humans find convincing.Source: E Analysis · pp. 13–14, Figure 7 · p. 14

• All evidence agents exceed the 33% no-evidence baseline in convincing judge models to select the agent-supported answer.Source: Figure 8 · p. 14
• Learned agents predicting p(i) or ∆p(i) find the most broadly convincing evidence across judge models.The DREAM findings are reported as similar to those obtained on RACE.Source: Figure 8 · p. 14, F Model Evaluation of Evidence on DREAM · p. 14