1-PAGER: One Pass Answer Generation and Evidence Retrieval

1 Introduction

• Retrieve-and-read systems provide paragraph-level attribution but rely on heterogeneous retrieval and generation models that are hard to train together.Source: §1 · p. 1
• A single Transformer decoder generates a search path of keywords and an answer string contained in at least one evidence document.Source: §1 · p. 1
• 1-PAGER is competitive as a passage retriever with a widely used dense retriever and outperforms an equivalent closed-book question-answering model on answer accuracy.Source: §1 · p. 2

2 Related Work

• Unlike prior iterative retrieval that uses reformulated queries, 1-PAGER uses search paths to partition the corpus and generates answers without an external reader.Source: §2 · pp. 2–3

3 Iterative Corpus Partitioning and Answer Prediction

• The target mapping takes a question and document corpus to an answer and a relevant supporting document that contains evidence for the answer.Source: §3 · p. 3
• A Search Path is a sequence of n-gram keywords, with each prefix selecting the documents that contain its keywords.Source: §3 · p. 3
• Each additional keyword narrows the current document subset, and 1-PAGER treats the final keyword as the answer.Source: §3 · p. 3

4 Constrained Decoding and FM-Index

• Only keyword continuations present in the current document subset are allowed, so decoded search paths continue to match at least one document.The decoder modifies beam search to permit only continuations contained in the current partition.Source: §4 · p. 3
• Each generated keyword narrows the corpus to documents matching the path so far, and decoding continues with tokens valid in that reduced subset.After a keyword separator, the next keyword must occur in documents containing the previously generated keyword.Source: §4 · p. 4, Figure 3 · p. 4
• The FM-index supports listing valid token continuations and matching documents, with extra cost when operations use a subset of its original corpus.The index also supports searches for arbitrary n-grams occurring within documents.Source: §4 · p. 4

5 Training data generation

• Training paths are generated by selecting n-grams from an answer passage, simulating corpus partitions until few documents remain, and appending the answer span.The resulting examples serialize questions as inputs and search paths plus answers as targets.Source: §5 · p. 4

5.1 Keyword Selection

• Keywords come from n-grams in the gold document, are filtered and heuristically scored, then top candidates are rescored with a language model to capture semantic relevance.Titles and keywords containing the answer string receive special scoring treatment.Source: §5.1 · pp. 4–5

5.2 Search Paths

• Paths begin with the title or highest-scored keyword, add non-overlapping keywords until fewer than ten passages match, and then append the answer keyword.Training paths have a median of three keywords and one matching document.Source: §5.2 · p. 5

6.1 Datasets

• Open-NQ supplies training and in-domain evaluation data, while WebQuestions and CuratedTREC test out-of-domain performance; the index uses DPR Wikipedia passages split into 100-word chunks.Positive DPR passages provide training-path samples.Source: §6.1 · p. 5

6.2 Training

• Training uses 310k paths for 55k Open-NQ queries, with T5-XXL fine-tuned in T5X using batch size 256 and dropout 0.1.No additional hyperparameter tuning is performed.Source: §6.2 · p. 5

6.3 Inference

• Inference uses beam decoding, but only the top-beam path retrieves documents; when multiple documents match, one is sampled arbitrarily for evaluation.The best model uses a beam of 5.Source: §6.3 · p. 5

6.4 Baselines

• Baselines include closed-book QA and retrieve-and-read systems that retrieve one passage before answer generation, with comparisons using shared T5-XXL model foundations and training examples where specified.The passage-retrieval comparisons include DPR, BM25, and SEAL.Source: §6.4 · p. 5, §6.4.1 · p. 5, §6.4.2 · pp. 5–6

6.5 Evaluation; 6.6 1P configurations

• Evaluation uses the Open-NQ test split and out-of-domain WebQuestions (WQ) and CuratedTREC (TREC), measuring passage retrieval with Hits@1; experiments compare 1P (training plus constrained decoding), 1P-Unconstrained (standard unconstrained decoding), and 1P + Reader.For 1P + Reader, the top passage retrieved by 1P is input to the Reader model to extract the answer.Source: §6.5 · p. 6, §6.6 · p. 6

7 Results

• 1P outperforms CBQA in question answering, beats BM25 and SEAL, is competitive with DPR for in-domain retrieval, and generalizes significantly better out-of-domain.On in-domain QA, the DPR retrieve-and-read pipeline remains ahead; the authors attribute this mainly to its reader rather than retriever.Source: §7 · pp. 6–7, Table 1 · p. 6, Table 2 · p. 6

Utility of Search Paths

• Improvement of 1P-Unconstrained over CBQA is attributed to path-conditioned answer generation, analogous to chain-of-thought reasoning.The authors describe 1P-Unconstrained as CBQA extended with a generated search path before the answer.Source: §7 · p. 7

Effect of Constrained Decoding

• Constrained decoding lets 1P achieve higher Hits@1 than 1P-Unconstrained, yet with beam size one it causes a small drop in answer accuracy, partly because a single 100-word passage may lack needed context.Source: §7 · p. 7
• A beam size of 5 improves 1P performance by 3% through planning, even though only the top-beam result is retrieved; unconstrained decoding gains little from planning.SEAL uses all outputs from the larger beam, unlike 1P.Source: §7 · p. 7, Table 3 · p. 7

Generating Answers

• 1P trails 1P+Reader, whose access to full passage context contrasts with 1P’s search-path and constrained-index view; feeding retrieved text to 1P is future work.Source: §8 · p. 7

Path vs Keyword set

• 1P paths use sequentially generated, distinct keywords to target one or a few documents, while SEAL’s independent keyword sets suit broad retrieval; 1P leads at Hits@1 and SEAL at Hits@5.The authors note that model-size variation and differences in beam size, decode count, and reader use complicate direct comparison.Source: §8 · pp. 7–8

Qualitative Analysis

• Paths commonly start with query entities and narrowing predicates, but may rewrite entity-poor queries or, less often, state the answer before supporting keywords.The authors characterize the generated paths as fairly meaningful and interpretable.Source: §8 · p. 8

Sampling Search Paths for Training

• LM-rescored keywords improved accuracy over heuristic sampling; title-first paths helped further, while mixing in non-title starts gave the best results.The mixed paths encourage predicates for questions where no entity can be determined.Source: §8 · p. 8

Sensitivity to tokenization

• 26.4% to 28.7% answer accuracy followed a training-data change that added surrounding punctuation to match FM-index tokenization.The authors say more work is needed to make 1P robust to tokenization variations.Source: §8 · p. 8

• 1-PAGER performs question answering and passage retrieval in one pass with a single language model, using constrained decoding to partition the corpus and generate an answer.Source: Conclusion · p. 8
• The authors report competitive or improved performance over comparable baselines and analyze training strategies, decoding styles, and qualitative predictions.Source: Conclusion · p. 8
• Poor search spaces and tokenization sensitivity remain challenges for constrained decoding, and the paper presents mitigation strategies.Source: Conclusion · p. 8
• The authors hope 1P will help advance generative retrieval by demonstrating a single Transformer model for both retrieval and answering.Source: Conclusion · p. 8

• 1P is geared toward a small document set and its answer generation is not competitive with a reader because it lacks passage context.It also lags behind state-of-the-art dense retrieval even for a single passage.Source: Limitations · p. 8
• Generalization remains unstudied for corpora without rich structure or queries without central entities, given the strategy’s reliance on titles and entities.Source: Limitations · p. 8
• Constrained decoding can produce low-diversity beams, require expensive corpus-wide constraint computation, and slow inference through FM-index communication.Source: Limitations · p. 9

• Despite corpus grounding, biases in pretrained language models may persist in 1-PAGER, alongside risks of toxic generation.Source: Ethical Considerations · p. 9
• The compute-intensive method is evaluated only on English knowledge-seeking queries from Natural Questions with T5, limiting evidence about other languages, datasets, and model families.The authors note that the resource demands may also make the approach inaccessible or invalid for some low-resource languages.Source: Ethical Considerations · p. 9

A.1 Constrain Computation

• A.1 Constrained decoding uses F(D, k) for documents containing a keyword and C(k, D) for its next tokens; F is cached, while arbitrary-subset C can require traversing all documents.For the full corpus, the FM-index can fetch next tokens in O(|V| log(|V|)), independent of |D|.Source: §A.1 · p. 13
• A.1 Lazy computation, caching, and rare unconstrained generation reduce constraint costs, but inference remains expensive and may need a special next-token lookup structure.Paths absent from the corpus after unconstrained generation can be filtered later.Source: §A.1 · p. 13

A.2 Training data size

• A.2 Increasing sampled paths per query improves performance, perhaps through greater training diversity, but is limited by the relevant paths extractable for each query.Source: §A.2 · p. 13, Table 7 · p. 13
• A.2 Mixing 9M PAQ question-answer pairs with the original training data in a 1:1 ratio further boosts performance, but with diminishing returns as silver data increases.Source: §A.2 · p. 13

A.3 SEAL keywords

• A.3 SEAL’s keywords can be repetitive, requiring many keywords to narrow to meaningful documents; this suits larger retrieval sets for reranking, unlike 1P’s top-document focus.Source: §A.3 · p. 13

A.5 Normalizing sequence likelihood over constrained space

• A.5 Re-normalizing likelihood over constrained continuations inflates scores and impedes backtracking, so 1P uses the language model’s scores directly.In the example, direct scores lead from an irrelevant path to the intended “Slaughterhouse cases, Butcher” path.Source: §A.5 · pp. 13–14

A.4 Hits@5

• SEAL achieves a significantly higher Hits@5 score than 1P, with the comparison using SEAL’s all-beam outputs and 1P’s top-beam result.Source: §A.4 · p. 13, Table 9 · p. 14

A.6 Number of matching documents

• A.6 Generated paths generally match only a few corpus documents; about half of test examples match a single path.A small fraction match no documents because of inference-time pruning optimizations.Source: §A.6 · pp. 14–15, Figure 4 · p. 14