PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimizing Dense Retrieval Model Training with Hard Negatives

Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, Shaoping Ma

arXiv:2104.08051v1cs.IR

TL;DR

Dense Retrieval training lacks a clear theoretical account of how negative-sampling strategies affect ranking and remains inefficient in some effective methods. The paper analyzes these objectives, identifies risks in static hard negatives, and proposes STAR and ADORE. Experiments on two retrieval benchmarks report significant improvements for either strategy, with their combination achieving the best performance.

  • Problem

    DR training depends on sampling strategies whose empirical effects are inconsistent, whose efficient use is limited, and whose optimization behavior lacks sufficient theoretical analysis.

  • Method

    The paper theoretically compares random and hard-negative training, analyzes static hard-negative risks, and proposes STAR with random negatives plus ADORE with dynamic hard negatives.

  • Results

    Experiments on two widely adopted retrieval datasets show significant performance improvements and efficiency gains, with STAR and ADORE together achieving the best retrieval performance.

  • Takeaways & Limitations

    Hard negatives better target top-ranking objectives, while dynamic sampling and random-negative stabilization address risks identified in static hard-negative training.

  • Takeaways & Limitations

    Static hard-negative training is theoretically risky because fixed negatives can be driven to very low ranks without the loss reflecting that change, leaving top-ranking performance unguaranteed.

Abstract

from arXiv · show

Ranking has always been one of the top concerns in information retrieval researches. For decades, the lexical matching signal has dominated the ad-hoc retrieval process, but solely using this signal in retrieval may cause the vocabulary mismatch problem. In recent years, with the development of representation learning techniques, many researchers turn to Dense Retrieval (DR) models for better ranking performance. Although several existing DR models have already obtained promising results, their performance improvement heavily relies on the sampling of training examples. Many effective sampling strategies are not efficient enough for practical usage, and for most of them, there still lacks theoretical analysis in how and why performance improvement happens. To shed light on these research questions, we theoretically investigate different training strategies for DR models and try to explain why hard negative sampling performs better than random sampling. Through the analysis, we also find that there are many potential risks in static hard negative sampling, which is employed by many existing training methods. Therefore, we propose two training strategies named a Stable Training Algorithm for dense Retrieval (STAR) and a query-side training Algorithm for Directly Optimizing Ranking pErformance (ADORE), respectively. STAR improves the stability of DR training process by introducing random negatives. ADORE replaces the widely-adopted static hard negative sampling method with a dynamic one to directly optimize the ranking performance. Experimental results on two publicly available retrieval benchmark datasets show that either strategy gains significant improvements over existing competitive baselines and a combination of them leads to the best performance.

1 INTRODUCTION

Dense Retrieval addresses vocabulary mismatch through semantic query-document matching, but its effectiveness and training efficiency depend strongly on how examples are sampled. The paper analyzes these strategies and proposes STAR and ADORE, whose combination achieves the best retrieval performance.

  • Motivation: Dense Retrieval encodes queries and documents into low-dimension embeddings for semantic matching and efficient similarity search.It constructs a document index offline and searches using query embeddings online.
  • Research gap: Existing DR training methods have conflicting empirical conclusions and can be inefficient because of costly knowledge distillation or periodic index refreshes.These issues motivate a theoretical comparison of training strategies and optimization objectives.
  • Analysis: Random negatives minimize total pairwise errors, whereas hard negatives minimize top-K pairwise errors and better target top-ranking performance.Random-negative training can be dominated by difficult queries whose errors are not reflected by truncated evaluation metrics.
  • Hard-negative risks: Static hard negatives can produce unstable training and fail to guarantee ranking improvement, while dynamic hard negatives change with model parameters and resolve these problems.Static negatives are pre-retrieved and fixed; dynamic negatives are recomputed as the model is updated.
  • Proposed methods: STAR combines static hard negatives with random negatives for stability, while ADORE dynamically trains the query encoder to directly optimize retrieval metrics.STAR also reuses document embeddings within a batch, and ADORE can improve query encoders produced by other methods.
  • Results: Experiments on two retrieval benchmarks show significant improvements for the proposed strategies, with their combination achieving the best retrieval performance.The combined approach is also reported as more efficient than current popular training methods.

2 RELATED WORK

Prior DR work uses random or hard-negative sampling, but the latter often relies on computationally costly static negatives. The paper positions its analysis as a more fundamental alternative to earlier convergence-speed analysis.

  • Training approaches: Dense Retrieval training generally uses negative sampling, while knowledge distillation is a newer alternative.DR encodes documents offline and queries online for similarity-based retrieval.
  • Random negatives: Random-negative methods sample from the corpus or approximate random sampling with in-batch negatives.Prior work also studies increasing the number of random negatives per mini-batch.
  • Hard negatives: Hard-negative methods commonly use BM25 or a warm-up DR model to retrieve top documents, often refreshing indexes periodically at high computational cost.Because these negatives remain fixed for training intervals, the paper calls them static hard negatives.
  • Mixed evidence: Prior studies report that static hard negatives provide no benefit or can worsen ranking performance.The cited findings differ across open-domain question answering and other retrieval settings.
  • Theoretical perspective: The paper argues for optimization-objective analysis rather than convergence-speed analysis, stating that its conclusions agree with experiments.This frames the paper’s theoretical contribution relative to earlier work.

3 TASK FORMULATION

The paper formulates Dense Retrieval as similarity-based ranking of relevant documents using separately encoded query and document embeddings. Because exhaustive corpus optimization is prohibitive, training is expressed through weighted sampling of negative documents.

  • DR model: Given a query and corpus, a DR model encodes the query and documents separately and retrieves relevant documents using a similarity score.The similarity function is often an inner product.
  • Pairwise training: DR training samples pairwise examples containing a query, one negative document, and one positive document.The paper uses a pairwise loss to analyze how training relates to positive-document ranking position.
  • Ranking formulation: The positive document’s ranking position is related to the number of relevant documents ranked above it, with irrelevant documents defined as the corpus excluding relevant documents.This connects ranking positions to the training-loss analysis.
  • Sampling objective: Because optimizing over every corpus sample is too costly, negative-document sampling is represented by assigning different weights to individual negatives.This yields a general learning objective parameterized by the sampling distribution.

4 RANDOM VS. HARD NEGATIVES

Random and hard-negative sampling optimize different ranking objectives. Random sampling can overemphasize difficult queries through an unbounded total-error loss, whereas hard negatives bound and focus optimization on top-K errors.

  • Random negative sampling: Random negative sampling uniformly samples negatives from the corpus and minimizes the positive document’s rank, equivalent to total pairwise-error minimization.Its single-query loss can grow as large as the corpus size.
  • Random negative sampling: Random sampling can let queries with large positive-document ranks dominate training, limiting attention to top-ranking performance.The paper identifies this as a source of serious practical performance loss.
  • Random negative sampling: Random negative sampling is therefore sub-optimal for DR training and motivates alternative sampling strategies focused on retrieval performance.The conclusion follows from its unbounded and top-ranking-insensitive objective.
  • Hard negative sampling: Hard-negative sampling selects the top-K documents as negatives, with K denoting the number of selected hard negatives.The resulting loss is bounded by K for an individual query.
  • Hard negative sampling: Hard negatives minimize top-K rather than total pairwise errors, emphasizing top-ranked pairs and disregarding lower-ranked pairs that have limited evaluation or user-experience impact.This bounded objective produces a more robust optimization of retrieval performance.
  • Comparison: The two strategies share optimal parameters when all relevant documents are already ranked sufficiently highly, but random sampling overweights difficult queries otherwise.Such difficult queries may not affect truncated evaluation metrics proportionally.

5 STATIC VS. DYNAMIC HARD NEGATIVES

Static hard negatives are fixed pre-retrieved documents whose apparent difficulty can deteriorate during training, weakening theoretical ranking guarantees. Dynamic hard negatives track the current model and therefore provide better-supported top-ranking optimization.

  • 5.1 Static Hard Negatives: Static hard negatives are pre-retrieved from a traditional retriever or warm-up dense model and remain unchanged during training.They are denoted D−_s and are much smaller than the corpus.
  • 5.1 Static Hard Negatives: The quality of static hard negatives can grow large because training pushes them toward lower relevance scores, which the loss does not detect.This makes their degradation invisible to optimization.
  • 5.1 Static Hard Negatives: Static hard negative training can achieve zero loss while leaving top-ranking performance theoretically unguaranteed; MRR can approach zero in the worst case.The analysis considers an ideal model whose training loss is already zero.
  • 5.1 Static Hard Negatives: Periodic index refreshing is explored as a partial remedy, but the paper reports that it has limited ability to resolve the analyzed static-negative problems.The theoretical analysis does not include periodic refreshing.
  • 5.2 Dynamic Hard Negatives: Dynamic hard negatives are the top-ranked irrelevant documents under the current model parameters, so they change as training updates the model.Their quality is well bounded because they continually reflect the model’s current ranking.
  • 5.2 Dynamic Hard Negatives: With dynamic hard negatives, zero training loss yields maximum MRR, making dynamic sampling theoretically better than static sampling.The paper concludes that dynamic hard negative sampling is better than the static alternative.

6 TRAINING ALGORITHMS

This section introduces STAR, which stabilizes static hard-negative training with random negatives, and ADORE, which uses dynamic hard negatives to directly optimize ranking performance.

  • 6.1 STAR: STAR combines static hard negatives for top-ranking optimization with random negatives for training stability.It also reuses document embeddings within the same batch to improve efficiency.
  • 6.1 STAR: STAR uses a warm-up model to retrieve fixed top documents as static hard negatives that approximate dynamic hard negatives.These static negatives remain unchanged during training.
  • 6.2 ADORE: ADORE retrieves top documents at each iteration using a fixed pre-trained document index, thereby obtaining dynamic hard negatives for query-side training.It trains the query encoder while keeping the document embeddings and index fixed.
  • 6.2 ADORE: ADORE applies LambdaLoss to weight pairwise losses by changes in the target IR metric and directly optimize ranking performance.The method uses listwise retrieval at each training step and metric-sensitive pair weights.
  • 6.2 ADORE: With a sufficiently large training set, the LambdaLoss method yields the optimal model for the corresponding metric.ADORE can also train with the actual compressed document index used during inference, reducing training–inference discrepancy.
  • 6.2 Combining STAR and ADORE: The two strategies have complementary roles: STAR trains both encoders, whereas ADORE directly optimizes ranking while training only the query encoder.The paper combines them by using STAR for the document encoder and ADORE for further query-encoder training.

7 EXPERIMENTAL SETTINGS

The experiments evaluate dense-retrieval training strategies on TREC 2019 Deep Learning passage and document retrieval tasks, using standardized baselines, encoders, indexing, and optimizer settings.

  • 7.1 Datasets: The TREC 2019 Deep Learning Track includes passage retrieval over 8.8 million passages and document retrieval over 3.2 million documents.Each task has separate training, development, and 43-query test sets.
  • 7.2 Baselines: Representative comparisons include BM25, DeepCT, Rand Neg, In-Batch Neg, BM25 Neg, and ANCE.The dense-retrieval baselines cover random-negative and static-hard-negative sampling strategies.
  • 7.2 Baselines: The study also compares cascade systems using the best LeToR and BERT models, both with BM25 as the first-stage retriever.These comparisons extend beyond standalone dense retrievers.
  • 7.3 Implementation Details: All dense-retrieval models use RoBERTabase encoders, [CLS] embeddings, inner-product scoring, and Faiss similarity search.Documents are truncated to 120 tokens for passages and 512 tokens for documents; the top 200 documents serve as hard negatives.
  • 7.3 Implementation Details: Rand Neg and In-Batch Neg use Lamb with batch size 256 and learning rate 2 × 10^-4 on the passage task, while their trained models are evaluated on documents.LambdaLoss was not used for these baselines because it provided no additional gains.
  • 7.3 Implementation Details: STAR uses Lamb with batch size 256 and learning rate 1 × 10^-4 on passages, then AdamW with batch size 60 and learning rate 2 × 10^-6 on documents.Its warm-up model is BM25 Neg, matching ANCE for direct comparison.
  • 7.3 Implementation Details: ADORE uses AdamW with learning rate 5 × 10^-6 and batch size 32 on both tasks, with MRR@200 for passages and MRR@10 for documents.It further trains the query encoder of an already trained dense-retrieval model.

8 EXPERIMENTAL RESULTS

The experiments test how negative-sampling strategies affect optimization objectives and ranking stability. They show that random sampling can minimize total pairwise errors while hard and dynamic negatives better support top-ranking performance.

  • Random vs. Hard Negatives: 0.2% of difficult queries contribute 60% of total pairwise errors in the Rand Neg model.Difficult queries are defined here as having pairwise errors ≥105.
  • Random vs. Hard Negatives: Random negatives minimize total pairwise errors, whereas hard negatives better minimize top-K pairwise errors and optimize top-ranking performance.The comparison uses top-K with K=200.
  • Random vs. Hard Negatives: Random and hard negative sampling optimize different objectives, contrary to analyses claiming they share the same objective.Random sampling can converge well on total pairwise errors while decreasing top-ranking performance under some initialization conditions.
  • Static vs. Dynamic Hard Negatives: Static hard negatives are quickly ranked low, overlap with dynamic negatives remains below 70%, and their training performance fluctuates wildly.The static negatives also cannot account for dropout noise during training.
  • Static vs. Dynamic Hard Negatives: Dynamic hard negative sampling steadily improves ranking performance, while static hard negative sampling is unstable and often underperforms random sampling.Figure 4 compares dynamic, static, and random negative sampling across training steps.
  • Static vs. Dynamic Hard Negatives: Static hard-negative objectives cannot guarantee performance improvement, although careful hyperparameter tuning may alleviate their problems.The paper therefore motivates training methods designed to improve ranking performance more reliably and efficiently.

8.3 Effectiveness

STAR and ADORE improve dense-retrieval effectiveness across retrieval tasks, with STAR strengthening top-ranking and recall performance and ADORE further improving models through query-side training. Their combination achieves the strongest reported performance, while some comparisons remain mixed.

  • STAR: STAR outperforms Rand Neg by 13% on dev passage MRR@10 and 18% on dev document MRR@100.These gains target top-ranking performance.
  • STAR: STAR outperforms BM25 Neg and ANCE by 40% and 15%, respectively, on TREC DL Doc R@100.STAR requires only one retrieval, whereas ANCE repeatedly rebuilds the index and refreshes static negatives.
  • Comparison with Distillation: STAR outperforms knowledge distillation on the large dev set but underperforms it on the small testing set.The paper states that the necessity of knowledge distillation therefore remains open for further exploration.
  • ADORE: ADORE improves In-Batch Neg top-ranking performance by 20% on dev passage and 22% on dev document datasets.It also improves BM25 Neg recall performance by 19% and 31% on testing passage and document sets.
  • Combined Method: ADORE+STAR achieves the best performance, greatly outperforming competitive baselines and nearly matching BM25-BERT on document retrieval.The comparisons include TCT-ColBERT and ANCE.
  • ADORE Ablation: Dynamic hard negatives greatly improve ADORE’s ranking performance, while LambdaLoss further benefits models such as Rand Neg but not STAR.The ablation results are reported as MRR@10 on MARCO Dev Passage.
  • Compressed Indexes: End-to-end training with the actual compressed document index better optimizes ranking performance for different compression techniques.ADORE is designed to reduce discrepancies between training and inference indexes.

8.4 Training Efficiency

STAR and ADORE improve training efficiency by converging quickly and reducing index-refresh overhead. ADORE also supports compressed indexes that substantially lower GPU memory use, although excessive compression harms retrieval quality.

  • Overall Efficiency: The proposed methods show significant efficiency gains over ANCE in training speed and computational resource use.The paper attributes these gains to fast convergence and avoiding repeated static-negative refreshes.
  • Training Time: ADORE needs 60k steps with batch size 32, while ANCE needs 600k steps with batch size 64 on passage retrieval.The methods are evaluated against ANCE as the efficiency baseline.
  • Index Refresh Overhead: ANCE spends 10.75 hours per corpus encoding and temporary-index build, whereas STAR builds one temporary index and ADORE has no such overhead.The reduced overhead contributes to the reported efficiency gains.
  • Compressed Indexes: PQ=96 reduces GPU memory footprint to 3% with little performance loss, changing MRR@10 from 0.329 to 0.326.ADORE can run on one 11 GB GPU after compression instead of four GPUs in the uncompressed comparison.
  • Compressed Indexes: Over-compressed PQ=6 yields poor search quality, with MRR@10 of 0.05, and nearly degenerates ADORE into random negative sampling.ADORE with PQ=6 has MRR@10 of 0.304 versus 0.301 for Rand Neg.

9 CONCLUSION

The paper explains why hard negatives improve top-ranking optimization, identifies risks in static hard-negative training, and proposes STAR and ADORE as effective and efficient alternatives. Their combination achieves the best retrieval performance, while document-encoder training and broader task applicability remain open questions.

  • Contributions: The paper theoretically formalizes dense-retrieval training and explains why hard negative sampling outperforms random negative sampling for ranking.It also analyzes risks associated with static hard-negative sampling.
  • Contributions: STAR and ADORE achieve significant performance improvements and efficiency gains over other effective methods on two retrieval datasets.Their combination achieves the best retrieval performance.
  • Future Work: Training the document encoder directly from retrieval results remains unexplored.This is identified as a future-work issue.
  • Future Work: The proposed methods are evaluated for ad-hoc search, while application to other retrieval-module tasks such as Open Question Answering remains future work.The paper explicitly identifies broader task evaluation as an open direction.
Loading 2104.08051v1…