Source-linked AI summary
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Andrew McCallum
TL;DR
Open-domain QA struggles when retrieval fails to supply the contexts that readers need. This paper introduces an iterative retriever-reader framework with query reformulation and fast search, and reports uniform gains across four datasets while scaling to millions of paragraphs.
Problem
Open-domain QA systems can degrade substantially when a reader is paired with a retriever that fails to provide the needed context.
Method
The framework uses a gated multi-step reasoner to update the query from the reader state, re-rank paragraphs with fast retrieval, and support different reader architectures.
Results
The framework uniformly improves machine reading over the base reader across four open-domain QA datasets and scales retrieval to millions of paragraphs, improving DrQA and BiDAF.
Takeaways & Limitations
Iterative interaction helps retrieve informative paragraphs and combine evidence across multiple paragraphs while retaining scalable retrieval.
Takeaways & Limitations
The framework requires access to token-level hidden representations from a span-extractive reader, limiting its reader-architecture agnosticism to compatible models.
Abstract
from arXiv · showhide
This paper introduces a new framework for open-domain question answering in which the retriever and the reader iteratively interact with each other. The framework is agnostic to the architecture of the machine reading model, only requiring access to the token-level hidden representations of the reader. The retriever uses fast nearest neighbor search to scale to corpora containing millions of paragraphs. A gated recurrent unit updates the query at each step conditioned on the state of the reader and the reformulated query is used to re-rank the paragraphs by the retriever. We conduct analysis and show that iterative interaction helps in retrieving informative paragraphs from the corpus. Finally, we show that our multi-step-reasoning framework brings consistent improvement when applied to two widely used reader architectures DrQA and BiDAF on various large open-domain datasets --- TriviaQA-unfiltered, QuasarT, SearchQA, and SQuAD-Open.
1 INTRODUCTION
The paper proposes an open-domain QA framework in which retriever and reader interact iteratively, addressing retrieval failures while retaining fast, scalable paragraph search. A gated multi-step reasoner reformulates the query from the reader state, enabling later retrieval of additional evidence.
- Open-domain QA performance can degrade sharply when sophisticated reading models are combined with retrieval; DrQA falls from 69.5% to 28.4% exact match on SQuAD.
- The framework requires a fast retriever and interaction that lets the reader obtain more relevant context when the initial retrieval is insufficient.
- The retriever caches query-independent paragraph representations, performs fast inner-product search, and passes highly ranked contexts to an architecture-agnostic reader.
- A gated recurrent unit uses the reader state and current query vector to generate a new query that re-ranks context and supports combining evidence across paragraphs.
- The framework iteratively couples retriever and reader through a multi-step-reasoning component that retrieves and combines information from multiple paragraphs.
- Large-scale experiments demonstrate scalability to millions of paragraphs and improvements across two neural reading architectures, DrQA and BiDAF.
2 BASIC COMPONENTS OF OUR MODEL
The model combines a paragraph retriever, a neural reader, and a multi-step reasoner. Query-independent representations enable efficient retrieval, while cross-paragraph score aggregation lets the reader combine evidence from retrieved contexts.
- The architecture has three components: a paragraph retriever, a reader that predicts answer spans, and a gated recurrent unit enabling iterative retriever-reader interaction.
- Paragraph representations are computed independently of the query, cached offline, and ranked against query vectors using inner products.
- Fast nearest-neighbor search replaces exhaustive comparison over million-paragraph corpora after preprocessing, reducing per-query search to logarithmic time in the number of paragraphs.
- The reader receives top-ranked paragraphs and outputs an answer span, while the framework remains applicable to both DrQA and BiDAF.
- Reader training normalizes start and end scores across retrieved paragraphs and aggregates answer occurrences to gather evidence across contexts.
- During inference, spans are scored by start and end scores, with matching surface forms aggregated across paragraphs retrieved at different steps.
3 MULTI-STEP-REASONER
The multi-step-reasoner lets the reader and retriever communicate iteratively: reader state updates the query, which reranks paragraphs for subsequent reading. It is trained with reinforcement learning and integrated into a repeated retrieval–reading loop.
- Multi-step interaction: The multi-step-reasoner uses a GRU to combine the current query with the reader state and produce a reformulated query for paragraph reranking.The reader state summarizes information encoded from previously retrieved paragraphs.
- Reader state: The framework requires access to token-level hidden representations from span-extractive readers to compute the reader state.Soft attention between paragraph-token representations and the reader’s final query representation produces the state.
- Training: Because query reformulation lacks direct supervision, the multi-step-reasoner is trained with reinforcement learning using reader F1 as its reward.The formulation treats interaction as a deterministic finite-horizon POMDP and optimizes expected reward.
- Training: The policy assigns probabilities to retrieved paragraphs by applying a softmax to their retriever scores.The top-ranked paragraph at each step is treated as the selected action.
- Training: Pretraining ranks answer-containing paragraph vectors above randomly sampled paragraph vectors while keeping paragraph representations fixed.Gradient flows through the reformulation query and multi-step-reasoner parameters.
- Putting it all together: Algorithmically, each step retrieves top-ranked paragraphs, reads them, updates the query from the reader state, and finally returns the highest-scoring answer span.The loop repeats for a specified number of multi-steps T.
4 RELATED WORK
Prior open-domain QA systems either use non-trainable retrievers or query-dependent representations that do not scale well and lack iterative recovery. The paper relates its approach to query reformulation, iterative reasoning, and nearest-neighbor search, while reporting benchmark and retrieval comparisons.
- Open-domain QA: Earlier open-domain QA retrievers either cannot overcome retrieval mistakes or use query-dependent paragraph representations that do not scale to millions of paragraphs.These systems also lack iterative reasoning for recovering from ranking errors or aggregating evidence across paragraphs.
- Query reformulation: Query reformulation has been used in information retrieval, but this work applies it to iterative interaction between an open-domain retriever and reader.Related approaches use relevance feedback or reinforcement learning to improve retrieval objectives.
- Iterative reasoning: The framework extends iterative reasoning beyond repeatedly reading one paragraph by updating retrieval across multiple paragraphs.The related-work discussion connects the model to memory-network-style controller updates.
- Nearest-neighbor search: The retriever uses exact k-nearest-neighbor search with SGTree, while the proposed multi-step reasoner is not tied to one particular k-NN technique.The paper contrasts SGTree with other exact and approximate nearest-neighbor strategies.
- Evaluation context: Table 2 reports test-set performance across various open-domain QA datasets, while Table 3 reports QUASAR-T retrieval performance using P@k.The supplied comparison note also records that DS-QA results on Quasar-T could not be reproduced and some TriviaQA-unfiltered results came from the authors.
5 EXPERIMENTS
Experiments evaluate retriever scalability, benchmark performance, and the effects of iterative interaction. Across datasets and analyses, reformulated queries improve retrieval and QA, while gains typically peak after several interaction steps.
- Experimental setup: The experiments evaluate the retriever on multiple open-domain QA datasets, including large-scale settings and paragraph-retrieval analyses.The study reports EM and F1 for QA performance and examines retrieval quality as interaction steps increase.
- Paragraph retriever: Query reformulation improves paragraph re-ranking after seven interaction steps, although the retriever remains below R3 in P@k.The comparison separates the scalability-oriented inner-product retriever from stronger query-dependent retrievers.
- QA performance: The multi-step reasoner consistently improves the base Dr.QA reader across four open-domain QA datasets and performs competitively with most baselines.The reported comparison uses exact match and F1 as evaluation metrics.
- Interaction steps: Performance rises uniformly over the no-interaction baseline as steps increase, generally peaking around 5 to 7 steps before further steps add little benefit.The analysis covers QUASAR-T, SEARCHQA, and TRIVIAQA-unfiltered.
- Large-scale QA: On TRIVIAQA-open with 1.6M paragraphs per query, three interaction steps improve DrQA from EM = 37.45 and F1 = 42.16 to EM = 39.76 and F1 = 44.30.The experiment shows that the framework scales to millions of paragraphs, although the large-context setting remains difficult.
- Analysis of interaction: Iterative retrieval finds relevant paragraphs missed initially and can combine evidence or aggregate answer-span scores across retrieved paragraphs.The SEARCHQA analysis links later retrieval to better paragraph quality, while the examples illustrate query reformulation and span-score aggregation.
6 CONCLUSION
The paper concludes that iterative retriever–reader interaction improves open-domain QA across four datasets and supports scalable retrieval over millions of paragraphs. The framework is reader-architecture agnostic when token-level hidden representations are available, and improves both Dr.QA and BiDAF.
- Conclusion: Iterative retriever–reader interaction improves machine reading over the base reader across four open-domain QA datasets.The conclusion reports consistent improvement from the resulting framework.
- Conclusion: Fast retrieval scales to millions of paragraphs, beyond the capability of existing open-domain systems with a trained retriever module.The conclusion characterizes scalability as a central outcome of the retrieval method.
- Conclusion: The method is agnostic to the machine-reading architecture when token-level hidden representations from the reader are available.The conclusion specifically reports gains for Dr.QA and BiDAF.