A Memory Efficient Baseline for Open Domain Question Answering

• Dense-retrieval indexes can reach tens of gigabytes because all knowledge-source vectors must be kept in memory.This motivates the question of how to store information for question answering more efficiently.Source: §1 · p. 1
• The paper studies dimension reduction, vector quantization, and document filtering on TriviaQA and NaturalQuestions.Source: §1 · p. 1
• Below 6Gb, the resulting QA systems retain competitive performance with minimal loss.Source: §1 · p. 1

• Retriever-reader systems retrieve relevant documents before a reader produces an answer, while related work also explores closed-book and retrieval-augmented generation.Source: §2 · p. 1
• Dense retrieval research has examined supervised, latent-modeling, and distillation training, while this paper tests quantization and filtering on downstream QA.It contrasts with Luan et al. (2020), who studied retrieval design choices including document length and vector dimension.Source: §2 · p. 2
• Product quantization divides a vector into subvectors and quantizes them independently using k-means.Source: §2 · p. 2

• Fusion-in-Decoder encodes each question-passage pair independently, concatenates the representations, and decodes an answer using a T5-initialized model.Source: §3 · p. 2
• The retriever embeds questions and passages in the same space, ranks passages by dot-product similarity, and uses Faiss for maximum inner-product search.Source: §3 · p. 2
• Knowledge distillation transfers aggregated reader cross-attention scores to a new retriever, whose retrieved passages then train a new reader.Source: §3 · p. 2

4.1 Initial system size

• The baseline uses BERT-base retrieval and T5-base/large readers, while its 768-dimensional float32 Wikipedia index occupies 75Gb.Wikipedia is split into non-overlapping 100-word chunks before vector indexing.Source: §4.1 · pp. 2–3

4.2 Reducing the index size

• Dimension reduction and quantization are presented as complementary index-compression strategies.Source: §4.2 · p. 3
• Dimension reduction adds a linear layer and LayerNorm to map passage embeddings to a lower dimension; the authors report stronger performance with LayerNorm.Source: §4.2 · p. 3
• Product quantization stores subvectors using a fixed number of bits, reducing storage to 0.5 bytes per dimension and compressing the index eightfold in the described setting.Source: §4.2 · p. 3
• Passage filtering trains a linear classifier on article titles and categories, using retrieved training articles as positives and iteratively selected negatives.Source: §4.2 · p. 3

5 Experiments

• Experiments use the standard open-domain QA setting on NaturalQuestions and TriviaQA, evaluating end-to-end Exact Match and top-k retrieval accuracy P@k.P@k is the percentage of questions with a gold-answer passage among the top-k retrieved passages.Source: §5 · pp. 3–4

5.1 Technical details

• The retriever uses uncased BERT base with averaged final-layer representations, while the reader is initialized with T5-base unless otherwise specified.Reader training uses 100 retrieved passages per question and selects the best model by development-set Exact Match.Source: §5.1 · p. 4

5.4 Final models

• At 5.1Gb and 2.1Gb, the compressed systems obtain competitive test performance against state-of-the-art toplines.The 5.1Gb system uses a large reader, a 256-dimensional index with 2 bits per dimension, and 18M passages; the 2.1Gb system uses a small reader, a 128-dimensional index with 2 bits per dimension, and 10M passages.Source: Table 1 · p. 4, §5.4 · p. 4

5.2 Dimension reduction vs. quantization

• A 1.6Gb index combines dimension reduction and product quantization, with losses of -0.2EM on NQ and -1.1EM on TriviaQA.Pareto-optimal systems often use both techniques.Source: §5.2 · p. 4

5.3 Passage filtering

• Filtering can discard a significant fraction of Wikipedia articles, reducing both the index and the knowledge source.Figure 2 evaluates passage counts starting from Wikipedia’s 26M passages using pipelines with 1.67Gb and 0.42Gb full-Wikipedia indices.Source: §5.3 · p. 4

• Dimension reduction, vector quantization, and passage filtering are complementary strategies; combined systems below 6Gb are competitive with the state of the art.Source: Discussion · p. 4