Source-linked AI summary
Adversarial Retriever-Ranker for dense text retrieval
Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, Weizhu Chen
TL;DR
Dense retrieval’s dual encoders omit finer-grained query-document interactions, while training relies on negative sampling. AR2 jointly trains a dual-encoder retriever and cross-encoder ranker in an adversarial game, achieving state-of-the-art performance on three benchmarks.
Problem
Dual encoders support independent query and document encoding but omit finer-grained interactions that could make retrieval accuracy suboptimal.
Source: §1 · p. 2Method
Results
Takeaways & Limitations
The retriever and ranker can iteratively improve through ranker feedback and harder negatives produced by the retriever.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Current dense text retrieval models face two typical challenges. First, they adopt a siamese dual-encoder architecture to encode queries and documents independently for fast indexing and searching, while neglecting the finer-grained term-wise interactions. This results in a sub-optimal recall performance. Second, their model training highly relies on a negative sampling technique to build up the negative documents in their contrastive losses. To address these challenges, we present Adversarial Retriever-Ranker (AR2), which consists of a dual-encoder retriever plus a cross-encoder ranker. The two models are jointly optimized according to a minimax adversarial objective: the retriever learns to retrieve negative documents to cheat the ranker, while the ranker learns to rank a collection of candidates including both the ground-truth and the retrieved ones, as well as providing progressive direct feedback to the dual-encoder retriever. Through this adversarial game, the retriever gradually produces harder negative documents to train a better ranker, whereas the cross-encoder ranker provides progressive feedback to improve retriever. We evaluate AR2 on three benchmarks. Experimental results show that AR2 consistently and significantly outperforms existing dense retriever methods and achieves new state-of-the-art results on all of them. This includes the improvements on Natural Questions R@5 to 77.9%(+2.1%), TriviaQA R@5 to 78.2%(+1.4), and MS-MARCO MRR@10 to 39.5%(+1.3%). Code and models are available at https://github.com/microsoft/AR2.
1 INTRODUCTION
Dense retrieval’s dual encoders support independent query and document encoding, but omit finer-grained interactions; training also depends on negative sampling. AR2 jointly trains a dual-encoder retriever and cross-encoder ranker through an adversarial objective and achieves state-of-the-art performance on three benchmarks.
1 INTRODUCTION
- Dual encoders enable independent query and document encoding for fast retrieval, but omit finer-grained interactions that could reduce retrieval accuracy.Source: §1 · p. 2
- AR2 combines a dual-encoder retriever with a cross-encoder ranker and trains them interactively through a unified minimax game.The cross-encoder jointly processes concatenated query and document text to model their interactions.Source: §1 · p. 2
- The retriever tries to fool the ranker with relevant documents, while the ranker distinguishes ground-truth documents from retrieved ones and provides smooth training signals to the retriever.The authors state that these signals help alleviate harmful effects of false negatives.Source: §1 · p. 2
- AR2 adds a distillation loss between retriever and ranker to smooth training because sharp retrieval probabilities may impede exploration of different documents.Source: §1 · pp. 2–3
- AR2 achieves state-of-the-art performance on Natural Questions, Trivia QA, and MS-MARCO.The introduction also describes a comprehensive ablation study of AR2 components.Source: §1 · p. 3
2 PRELIMINARIES
Dense text retrieval typically represents queries and documents as separate vectors and trains with contrastive losses over sampled negatives. Prior work explores hard-negative sampling and adversarial retrieval, while noting false-negative risks and limits for dense retrieval.
2 PRELIMINARIES
- Dense retrieval uses separate query and document embeddings, with relevance measured by their similarity and fast ANN search used over document embeddings.Source: §2 · p. 3
- Contrastive training compares a positive document with sampled negative documents, while in-batch negatives are randomly drawn from documents in the same mini-batch.The cited sampling strategy is described by Shen et al. (2014), Chen et al. (2017), and Liu et al. (2020).Source: §2 · p. 3
- ANCE and Condenser use iteratively retrieved hard negatives, while RocketQA notes that retrieved negatives can be false negatives and may limit performance.The passages cite ANCE (Xiong et al., 2021), Condenser (Gao & Callan, 2021b), and RocketQA (Qu et al., 2021).Source: §2 · p. 3
- IRGAN uses a minimax game between generative and discriminative retrieval models, but lacks the dual encoder needed for dense retrieval indexing and fast search.The passage cites IRGAN (Wang et al., 2017).Source: §2 · p. 3
3 METHOD
AR2 pairs a dual-encoder retriever with a cross-encoder ranker and coordinates their training through a minimax objective. Its method alternates ranker and retriever learning, adds distillation regularization, and refreshes the retrieval index.
3 METHOD
- AR2 uses a dual-encoder retriever and cross-encoder ranker, with language-model encoders and a linear projector scoring concatenated question-document input.The modules are jointly optimized through a contrastive minimax objective; the retriever samples relevant documents to fool the ranker, while the ranker distinguishes them from ground truth.Source: §3 · p. 4
3.1 TRAINING THE RANKER Dφ
- The ranker maximizes the likelihood of selecting the ground-truth document from a candidate set containing it and negatives sampled by the retriever.In experiments, the negatives are sampled from the top 100 retrieved documents.Source: §3.1 · p. 4
3.2 TRAINING RETRIEVER Gθ
- Retriever optimization uses an alternative upper bound because the original objective is intractable to optimize directly.The derivation is described in the appendix, and the retriever objective is optimized with respect to its parameters.Source: §3.2 · p. 4
- A distillation regularization term encourages the retriever to learn from the ranker, with both models’ conditional document probabilities considered over the corpus.In practice, the sampling space is limited to the ground-truth document and retrieved negatives.Source: §3.2 · p. 5
3.3 INDEX REFRESH
- AR2 refreshes its document index during training by encoding corpus documents with the retriever’s document encoder and building an inner-product ANN index.Source: §3.3 · p. 5
4 EXPERIMENTS
Across Natural Questions, Trivia QA, and MS-MARCO, AR2-G achieves new state-of-the-art retrieval performance. The experiments also examine its training setup, ranker, regularization, negative-sample size, and training behavior.
4.1 DATASETS
- Experiments cover Natural Questions, Trivia QA, and MS-MARCO, using R@k for the first two and MRR@10, R@50, and R@1k on the MS-MARCO dev set.Natural Questions retrieval seeks answer-containing passages; Trivia QA retrieval seeks passages containing the answer.Source: §4.1 · pp. 5–6
4.2 IMPLEMENTATION DETAILS
- Training uses dataset-specific continuous pretraining, warm-up retriever and ranker models, then coordinated adversarial training for 10 iterations.The document index is refreshed after each iteration.Source: §4.2 · p. 6
4.3 RESULTS
- AR2-G significantly outperforms its warm-up model and achieves new state-of-the-art retrieval performance on all three datasets.Table 1 compares first-stage retrieval on Natural Questions test, Trivia QA test, and MS-MARCO dev sets.Source: Table 1 · p. 6, §4.3 · p. 7
4.4 ANALYSIS
- On NQ, the adversarially trained AR2-D ranker improves over both the retriever alone and the warm-up ranker.The evaluation reranks the retriever’s top 100 documents for each test query.Source: §4.4 · p. 7
- Increasing negative samples from 1 to 7 improves R@1 and R@5, while further increases to 15 yield marginal gains and training latency rises nearly linearly.These results are reported without the regularization term.Source: §4.4 · p. 7
- Regularization improves R@1 and R@5 and smooths retriever probabilities; average NQ entropy is 2.10 with regularization versus 1.70 without it.The entropy is calculated over the retrieved top-15 documents.Source: §4.4 · p. 8
- As adversarial iterations increase, both models’ NQ R@5 rises; the retriever improves by about 8% and the ranker by about 3% by the end.The retriever’s largest gain, about 4.5%, occurs after the first iteration, and gains slow near convergence.Source: §4.4 · p. 8, Figure 3 · p. 8
- With the same model initialization, AR2 steadily outperforms ANCE in R@5 and R@10 during training; AR2 also significantly outperforms IRGAN, and its retrieve-then-rank pipeline significantly outperforms existing public pipelines.The retrieve-then-rank pipeline was evaluated on the NQ dataset.Source: §4.4 · pp. 8–9
5 RELATED WORK
Related work frames retrieval as a retrieve-then-rank pipeline, with sparse and dense approaches for retrieval and cross-encoders for ranking. AR2 differs by dynamically adjusting ranker negatives with a progressing retriever.
5 RELATED WORK
- Retrieval-then-rank is a widely used pipeline, with sparse methods such as BM25 and neural methods expanding or reweighting terms for retrieval.Examples include BERT-based term weighting and generated query or document expansions.Source: §5 · p. 9
- Dense retrieval methods use approaches including hard-negative sampling, passage-centric training, reader-to-retriever distillation, and joint training with downstream tasks.Examples include DPR, ANCE, RocketQA, ADORE, PAIR, FID-KD, and RDR.Source: §5 · p. 9
- Cross-encoder rankers capture token-level query-document interactions, but prior methods typically use statically sampled negatives.AR2 instead dynamically adjusts ranker negatives with the progressive retriever.Source: §5 · p. 9
- IRGAN proposes a minimax retrieval framework unifying generative and discriminative retrieval models, while few works apply GANs to text retrieval.The passage identifies IRGAN as related work on adversarial retrieval.Source: §5 · p. 9
6 CONCLUSION
AR2 jointly trains a retriever and ranker in an adversarial retrieve-then-rank framework. It reports new state-of-the-art performance on three competitive benchmarks.
6 CONCLUSION
- AR2 achieves new state-of-the-art performance on all three competitive benchmarks.Its framework jointly trains an end-to-end retrieve-then-rank pipeline.Source: §6 · p. 9
- The retriever retrieves hard negatives to challenge the ranker, which learns to rank positives alongside those negatives and provide progressive rewards to the retriever.The conclusion attributes iterative improvement to the ranker guiding the retriever and the retriever supplying harder negatives for ranker training.Source: §6 · p. 9
- The conclusion attributes iterative improvement of both models to their reciprocal training signals.The ranker guides the retriever, while harder negatives from the retriever support ranker learning.Source: §6 · p. 9
A.1 PROOF
The proof appendix describes sampling negative documents from the retriever’s top-K set to approximate the implementation equation.
A.1 PROOF
- The negative-document set for query q consists of documents sampled by Gθ(q, ·).The appendix identifies this set as D−_q.Source: §A.1 · p. 13
- In implementation, D−_q is approximated by sampling n documents from the top-K retrieved negative set.The passage says this leads to an approximate equation in implementation.Source: §A.1 · p. 13
- The appendix labels the following material as a proof of Equation 10.The supplied passage gives the proof heading but no further proof steps.Source: §A.1 · p. 13
A.2 EFFICIENCY REPORT
The efficiency report compares training and inference time, with evaluation on eight NVIDIA A100 GPUs. Its training-time estimate excludes continuous pretraining and warm-up.
A.2 EFFICIENCY REPORT
- The report lists training and inference time costs in Table 7.The supplied passages do not state the table’s values.Source: §A.2 · p. 13, Table 7 · p. 13
- The efficiency evaluation uses 8 NVIDIA A100 GPUs.Source: §A.2 · p. 13
- The overall training-time estimate excludes continuous pretraining and warm-up time.Source: §A.2 · p. 13
A.3 HYPERPARAMETERS
Table 8 is identified as the reference for AR2 training hyperparameters.
A.3 HYPERPARAMETERS
- Table 8 lists hyperparameters for AR2 training.Source: Table 8 · p. 14
A.4 MODEL CONFIGURATION AND EXPERIMENT SETTINGS
The appendix reports that the retrieve-then-rank AR2 pipeline achieves state-of-the-art performance across all three benchmarks. It also compares initial models, prior approaches, and large-model settings.
A.4 MODEL CONFIGURATION AND EXPERIMENT SETTINGS
- Table 9 documents AR2 model configurations and experiment settings.Source: §A.4 · p. 15, Table 9 · p. 15
A.5 ABLATION STUDY ON DIFFERENT INITIAL MODELS
- ERNIE-Base starts slightly ahead of BERT-Base, but BERT-Base AR2-G still beats methods with the same initialization; ICT pre-training further improves AR2-G.Source: §A.5 · p. 15, Table 10 · p. 15
A.6 COMPARISON WITH SEVERAL EXISTING APPROACHES
- AR2's key distinction from prior approaches is jointly optimizing the retriever and ranker with an adversarial objective.Table 11 compares AR2 with existing retrieval approaches and indicates whether answer labels are used.Source: §A.6 · p. 16, Table 11 · p. 16
A.7 PERFORMANCE OF THE PIPELINE
- The AR2 retrieve-then-rank pipeline improves performance on all three benchmarks and achieves state-of-the-art results on each.Source: §A.7 · p. 16, Table 12 · p. 16
A.8 PERFORMANCE OF THE LARGE-SIZE MODEL
- AR2-G significantly improves over baselines initialized with large models, including methods using a 65m-pair corpus or a larger T5-Large reader.The AR2-G model uses ERNIE-2.0-Large without continuous pre-training.Source: §A.8 · p. 16, Table 13 · p. 16