Source-linked AI summary
Generative Retrieval as Dense Retrieval
Thong Nguyen, Andrew Yates
TL;DR
Generative retrieval combines indexing and retrieval in one transformer, but updating its index and scaling to large collections are difficult. The paper analyzes its relationship to dense retrieval and proposes Tied-Atomic; experiments report competitive effectiveness and results on a collection of approximately 8 million documents.
Problem
Generative retrieval has difficulty updating its index and scaling to large collections, while prior research has focused mainly on collections of several hundred thousand documents.
Source: §1 · p. 1Method
The paper decomposes generative retrieval scoring into query–document vector dot products and proposes Tied-Atomic, which ties document text to DocID embeddings and uses contrastive training.
Source: §1 · p. 1Results
On MSMARCO, Tied-Atomic with T5-small performs comparably to ANCE and outperforms DSI Atomic (base) by 38% in MRR@10; on NQ320k, it is slightly worse than NCI (base) at R@1 but improves other metrics by around 2% to around 6%.
Source: §5.3 · p. 4Takeaways & Limitations
Tied-Atomic performs competitively with previous generative models while allowing index updates and scaling to large document collections.
Source: §6 · p. 4The impact of embedding refinement in hierarchical search on retrieval effectiveness remains uncertain and requires further investigation.
Source: §3.2 · p. 3
Abstract
from arXiv · showhide
Generative retrieval is a promising new neural retrieval paradigm that aims to optimize the retrieval pipeline by performing both indexing and retrieval with a single transformer model. However, this new paradigm faces challenges with updating the index and scaling to large collections. In this paper, we analyze two prominent variants of generative retrieval and show that they can be conceptually viewed as bi-encoders for dense retrieval. Specifically, we analytically demonstrate that the generative retrieval process can be decomposed into dot products between query and document vectors, similar to dense retrieval. This analysis leads us to propose a new variant of generative retrieval, called Tied-Atomic, which addresses the updating and scaling issues by incorporating techniques from dense retrieval. In experiments on two datasets, NQ320k and the full MSMARCO, we confirm that this approach does not reduce retrieval effectiveness while enabling the model to scale to large collections.
1 INTRODUCTION
Generative retrieval jointly indexes and retrieves with one transformer, but updating its index and scaling to large collections remain difficult. The paper shows how its scoring can be viewed as dense retrieval and introduces Tied-Atomic to address those challenges.
1 INTRODUCTION
- Transformer-based retrieval also includes learned sparse retrieval and cross-encoders; cross-encoders are computationally expensive and mainly used for re-ranking.Sparse and dense methods encode queries and documents into vectors for inverted or vector indexing.Source: §1 · p. 1
- Generative retrieval faces difficulty updating its index and scaling to large collections, with prior research focused mainly on collections of several hundred thousand documents.Unlike dense retrieval’s learned text-to-vector association, it treats document embeddings as randomly initialized parameters; adding documents requires re-optimization, and scaling may require a larger model.Source: §1 · p. 1
- Each generation step can be decomposed into a dot product between the transformer’s last hidden state and vocabulary embeddings for document identifiers, analogous to dense retrieval scoring.Figure 1 compares dense retrieval using FlatIndex with generative retrieval using Atomic DocIDs.Source: §1 · p. 1, Figure 1 · p. 1
- The paper analyzes Atomic DocIDs and Hierarchical Semantic DocIDs, showing that both model relevance as a dot product between query and document vectors.This analysis asks how far generative retrieval can be viewed as dense retrieval.Source: §1 · p. 1
- Tied-Atomic ties document texts to vocabulary (DocID) embeddings and uses contrastive loss; it performs competitively with DSI and NCI on NQ320K, handles unseen documents, and scales to the full MSMARCO dataset.Source: §1 · p. 1
2 RELATED WORK
Related work distinguishes retrieval paradigms by how they represent and search documents. Generative approaches produce DocIDs directly, while dense retrieval uses vector representations and indexes.
2 RELATED WORK
- Dense retrieval uses bi-encoders to represent queries and documents as dense vectors; vector indexes narrow the search space for efficiency, usually with some loss in effectiveness.Document vectors can be computed and indexed offline, separately from query encoding.Source: §2 · pp. 1–2
- Generative retrieval, also called Differentiable Search Index, trains a transformer to associate documents with DocIDs and generates relevant identifiers from queries [11, 21].Prior work reports benefits from joint indexing and fine-tuning [21] and from generated-query augmentation [20] [22] [26].Source: §2 · p. 2
- Generative retrieval decodes likely DocIDs with beam search and ranks them by generation probability; identifier structure affects decoding efficiency.Atomic DocIDs assign each document an arbitrary unique token, whereas Hierarchical Semantic DocIDs represent documents as paths through a semantic tree [22].Source: §2 · p. 2
3 GENERATIVE VS. DENSE RETRIEVAL
The section compares Atomic and Hierarchical Semantic DocIDs with dense retrieval, showing how their query–document scoring relates to vector similarity while identifying differences in document representations and search.
3.1 Atomic DocIDs
- Atomic DocIDs score a query using the decoder’s last hidden state and the identifier’s vocabulary embedding, a dot product equivalent to dense retrieval’s query–document scoring.The Softmax can be removed without changing document ranking; the two vectors act as query and document vectors.Source: §3.1 · p. 2
- Unlike dense retrieval, atomic generative retrieval uses separate identifier embeddings rather than encoding document text, so it cannot create meaningful embeddings for unseen documents.Dense retrieval’s document encoder ties vectors to document text; the two approaches also use different training objectives.Source: §3.1 · p. 2
3.2 Hierarchical Semantic DocIDs
- Hierarchical Semantic DocIDs recursively cluster documents into root-to-leaf label paths, narrowing search like a tree index; refinement’s impact is uncertain, and current approaches do not outperform Atomic DocIDs.The paths encode documents, and the tree-index analogy relates hidden states and token embeddings to query and centroid vectors.Source: §3.2 · pp. 2–3
4 METHODOLOGY: TIED-ATOMIC
Tied-Atomic adapts dense-retrieval techniques to Atomic DocIDs by tying identifier embeddings to document texts and using contrastive training, while retaining the same retrieval computation.
4 METHODOLOGY: TIED-ATOMIC
- Tied-Atomic ties each DocID vocabulary embedding to its document text, making document vectors outputs of T5’s encoder-decoder stack rather than standalone model parameters.The resulting document vectors are the stack’s output vectors for the corresponding document texts.Source: §4 · p. 3
- Tied-Atomic replaces full-vocabulary cross-entropy training with contrastive loss using BM25-ranked negatives, including in-batch negatives.The original full-vocabulary logit computation is particularly expensive for Atomic DocIDs, whose vocabulary spans the document collection.Source: §4 · p. 3
- The modifications affect only training, leaving retrieval computation unchanged; the authors say the model can theoretically insert documents or scale like dense retrieval.Despite these changes, the model retains the expressiveness of generative retrieval with Atomic DocIDs.Source: §4 · pp. 3–4
5 EXPERIMENTS AND RESULTS
Experiments on NQ320K and full MSMARCO evaluate Tied-Atomic against generative, dense, and lexical retrieval baselines. The results show competitive retrieval effectiveness across these settings.
5.1 Datasets
- NQ320K has approximately 307k training and 8k development query-document pairs; full MSMARCO has more than 8 million short passages, approximately 809k training queries, and approximately 7k development queries.The NQ320K collection contains more than 100k relevant documents gathered from training and development pairs.Source: §5.1 · p. 4
5.2 Experimental setup
- For NQ320K, Tied-Atomic was trained with a ranking loss and in-batch BM25 negatives; for MSMARCO, it used mined hard negatives and CE-score distillation with MarginMSELoss.The NQ320K T5-base model was trained for 120k steps; DocIDs were frozen during MSMARCO training.Source: §5.2 · p. 4
5.3 Results and discussion
- On NQ320K, Tied-Atomic (base) trails NCI (base) at R@1 but improves other metrics around 2% to around 6%; on MSMARCO, it is comparable to ANCE and beats DSI Atomic (base) by 38% in MRR@10.The reported comparisons are in Table 1 for NQ320K and Table 2 for full MSMARCO.Source: Table 1 · p. 4, Table 2 · p. 4, §5.3 · p. 4
6 CONCLUSION
The paper analyzes generative retrieval as a form of dense retrieval and proposes Tied-Atomic as a competitive model that supports index updating and scaling to large collections.
6 CONCLUSION
- The analysis shows that generative retrieval can be viewed as a form of dense retrieval.The experiments mainly focus on atomic identifiers.Source: §6 · p. 4
- Tied-Atomic performs competitively with previous generative models while allowing index updating and scaling to large collections.Source: §6 · p. 4
- Experiments mainly focus on atomic identifiers; recent work suggests other existing identifiers do not outperform atomic identifiers [15].Source: §6 · p. 4