Source-linked AI summary
Hierarchical Question Answering for Long Documents
Eunsol Choi, Daniel Hewlett, Alexandre Lacoste, Illia Polosukhin, Jakob Uszkoreit, Jonathan Berant
TL;DR
Long-document QA is difficult because sequential RNNs are slow and commonly require document truncation. The paper selects relevant sentences with a fast model, then generates answers with an RNN, jointly learning selection from answer supervision through reinforcement learning. The framework improves performance on a challenging WIKIREADING subset and speeds document encoding by 3.5x-6.7x while accessing longer documents.
Problem
Sequential RNN-based QA over long documents is difficult to parallelize, making it slow and often forcing models to truncate documents.
Method
A fast sentence selector creates a document summary for a slower answer-generation RNN, with sentence selection learned jointly from answers using reinforcement learning.
Results
The framework improves performance on a challenging WIKIREADING subset and speeds document encoding by 3.5x-6.7x while allowing access to much longer documents.
Takeaways & Limitations
Coarse-to-fine processing quickly focuses QA on relevant portions of long documents while retaining access to substantially more text.
Takeaways & Limitations
WIKIREADING is not ideal for testing sentence selection because most documents are short and answers often occur in the first sentence.
Abstract
from arXiv · showhide
We present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of-the-art models. While most successful approaches for reading comprehension rely on recurrent neural networks (RNNs), running them over long documents is prohibitively slow because it is difficult to parallelize over sequences. Inspired by how people first skim the document, identify relevant parts, and carefully read these parts to produce an answer, we combine a coarse, fast model for selecting relevant sentences and a more expensive RNN for producing the answer from those sentences. We treat sentence selection as a latent variable trained jointly from the answer only using reinforcement learning. Experiments demonstrate the state of the art performance on a challenging subset of the Wikireading and on a new dataset, while speeding up the model by 3.5x-6.7x.
1 Introduction
The paper addresses the difficulty of applying sequential RNN-based question answering to long documents by selecting relevant sentences before answer generation. Its hierarchical framework improves performance on a challenging WIKIREADING subset and speeds document encoding by 3.5x-6.7x.
- RNN-based document QA is slow because sequential processing over thousands of tokens is difficult to parallelize and often requires truncation.
- The model first selects a few relevant sentences with a fast model, then uses a slow RNN to generate the answer from those sentences.The RNN processes a fixed number of tokens regardless of document length.
- Sentence selection is treated as a latent variable trained jointly with answer generation from the answer alone using reinforcement learning.This addresses cases where answers are absent verbatim or occur repeatedly in spurious contexts.
- The experiments report improved performance on a WIKIREADING subset and comparable performance on other datasets.The evaluation includes a new WIKISUGGEST dataset with more natural questions gathered from a search engine.
2 Problem Setting
The task learns to answer questions from documents by assuming that a small latent subset of document sentences is sufficient. The paper illustrates this setting with document, question, and answer examples and reports answer-string statistics as context for sentence selection.
- The model learns from question-document-answer triples to produce an answer for each question-document pair.
- Table 1 summarizes whether the answer string appears in the document and how often the matching sentence is the first sentence.Its statistics motivate treating sentence selection as a distinct modeling problem.
- A document is represented as a list of sentences, and the answer is assumed to be producible from a small latent subset.The example identifies sentence s5 as belonging to that subset.
3 Data
The paper evaluates existing WIKIREADING data, constructs WIKIREADING LONG to make sentence selection more challenging, and introduces WIKISUGGEST from natural search queries. WIKIREADING LONG contains 1.97M long-document examples, while the original data often places answer matches near the beginning.
- WIKIREADING is automatically generated from Wikipedia and Wikidata, with the goal of inferring a target property value from a document.
- WIKIREADING does not annotate answer spans, making sentence selection more challenging.
- In original WIKIREADING, the answer appears verbatim in 47.1% of examples and the match is in the first sentence in 75% of those cases.Because most documents have a median of 9 sentences, the answer can often be inferred from the beginning.
- WIKIREADING LONG filters short documents and properties with accuracy below 60% for the prior best model to distribute answers more evenly.
- WIKIREADING LONG contains 1.97M examples with 1.2k tokens on average; the answer appears in 50.4% of examples and in the first sentence 31% of the time.Its documents are longer on average than those in SQuAD and CNN.
- WIKISUGGEST uses natural-language questions from Google Suggest and Wikipedia answers returned through Google Search.Pairs without an exact answer string in the document were pruned.
4 Model
The model first selects relevant sentences with a fast sentence-selection component, then uses an answer-generation model to produce the final answer from a focused document summary. It supports hard or soft attention and uses efficient BoW, chunked BoW, or convolutional sentence representations.
- Model architecture: The model defines a sentence distribution p(s | x, d) and uses a costly answer generator conditioned on a summary focused on relevant document parts.The two components separate fast document selection from answer generation.
- Sentence selection: Sentence selection uses efficient feed-forward representations based on bag-of-words, chunks, or convolutional composition.The convolutional model computes filters in parallel but does not fully capture sequential text structure.
- Sentence selection: Chunked BoW scores seven-token chunks separately and marginalizes chunk probabilities to obtain sentence probabilities.This provides finer granularity when questions are answered by subsentential units.
- Document summary: The document summary can use hard attention, which samples sentences, or soft attention, which averages sentence tokens using selection probabilities.Hard attention can focus on multiple sentences, while soft attention is easier to optimize and retains information from multiple sentences.
- Answer generation: The answer generator is a word-level sequence-to-sequence model using GRUs to encode the question and document summary and decode the answer.The architecture can generate answers that do not appear verbatim in the selected sentence.
5 Learning
The paper compares supervised, reinforcement-learning, and differentiable training strategies for jointly learning sentence selection and answer generation. Reinforcement learning treats sentence selection as an action whose reward is the answer model’s log probability, while curriculum learning reduces sampling variance.
- Learning objectives: The training approaches are distant supervision, REINFORCE for hard attention, and end-to-end backpropagation for soft attention.Distant supervision trains sentence selection independently using heuristic labels.
- Distant supervision: Distant supervision labels the first sentence containing a full answer-string match, or the first document sentence when no full match exists.The selected sentence serves as the document summary during supervised training.
- Distant supervision: At test time, the model replaces the unavailable target sentence with the highest-probability sentence under the sentence-selection model.This prediction is used as the document summary for answer generation.
- Reinforcement learning: Reinforcement learning treats sentence selection as the action and rewards each sentence by the log probability of the correct answer given that sentence.The objective maximizes expected reward under the sentence-selection distribution.
- Reinforcement learning: Curriculum learning begins with distant supervision and gradually transitions to reinforcement learning to reduce the variance caused by sampling.The probability of using the supervised objective decays across training epochs.
6 Experiments
The experiments evaluate hierarchical QA models on answer accuracy, sentence selection, runtime, and error patterns across WIKIREADING and WIKISUGGEST variants. The models reduce RNN steps while generally matching or improving BASE, with especially strong gains on long documents.
- Runtime: 3.5x-6.7x speedup is obtained with REINFORCE for K=2 or K=1, and it is at least three times faster than BASE in all settings.The runtime comparison measures document encoding at test time.
- Answer Accuracy Results: REINFORCE outperforms PIPELINE, with a larger improvement on WIKIREADING LONG where approximate sentence-selection supervision is missing for 51% of examples versus 22% in WIKISUGGEST.PIPELINE relies on a noisy supervision signal, while REINFORCE jointly learns answer generation and sentence selection.
- Answer Accuracy Results: On WIKIREADING LONG, REINFORCE outperforms all other models excluding ORACLE, while BASE is slightly better on other datasets at the cost of speed.BASE is advantageous for categorical questions that require gathering evidence throughout the document.
- Answer Accuracy Results: Sampling an additional sentence improves performance across datasets, and REINFORCE performs better than SOFTATTEND.Additional sampling helps recover from sentence-selection mistakes in WIKIREADING LONG.
- Sentence Selection Results: 74.4% of WIKIREADING LONG examples with the answer in the document yield the approximate gold sentence under REINFORCE selection.On WIKISUGGEST, sentence-selection performance is 67.5%, partly due to noise in the data.
7 Related Work
Prior work applied hierarchical latent sentence selection across several tasks, but this paper extends that setup to question answering and answer-string generation.
- Unlike extractive approaches, the model treats answer sentence selection as latent and generates answer strings instead of selecting text spans.
- Hierarchical latent sentence selection had been applied to text categorization, extractive summarization, machine translation, and sentiment analysis.
- This work is presented as the first use of document hierarchy for question answering.
- Related reinforcement-learning work includes hard and soft attention, curriculum learning, and applications to coreference resolution, information extraction, semantic parsing, and textual games.
8 Conclusion
The paper presents a coarse-to-fine framework that focuses quickly on relevant portions of long documents for question answering. It identifies deeper structural modeling and multi-document question answering as future directions.
- The framework quickly focuses on relevant portions of long documents for question answering.
- Future work will use richer structural clues and answer questions over multiple documents, including paragraph, title, and section structure.