PaperlayerView exact revision on arXiv

Source-linked AI summary

Influence Guided Sampling for Domain Adaptation of Text Retrievers

Meet Doshi, Vishwajeet Kumar, Yulong Li, Jaydeep Sen

arXiv:2601.21759v1cs.IRcs.CL

TL;DR

Choosing training-data proportions for domain adaptation of text retrievers remains difficult because dataset size does not reliably indicate informativeness. Inf-DDS learns influence-guided sampling policies that adaptively prioritize datasets, producing stable, competitive retrieval improvements across domains while remaining computationally efficient.

  • Problem

    Finding informative datasets and effective sampling proportions for domain adaptation remains challenging because dataset size does not inherently predict embedding quality and existing adaptive methods can be unstable or computationally costly.

  • Method

    Inf-DDS frames dataset sampling as bilevel reinforcement-learning optimization, using influence-based rewards from domain-specific updates to adapt sampling probabilities toward target development-set performance.

  • Results

    Across diverse benchmarks, Inf-DDS produces more stable sampling trajectories and consistently approaches or outperforms proxy-model and gradient-based methods while remaining computationally efficient.

  • Takeaways & Limitations

    Influence-guided dynamic sampling offers a practical approach for adapting text retrievers across domains, delivering substantial gains from reasonable initial sampling distributions.

  • Takeaways & Limitations

    Inf-DDS is sensitive to its initial sampling distribution and may not reach globally optimal sampling weights; the paper does not explore initialization heuristics.

Abstract

from arXiv · show

General-purpose open-domain dense retrieval systems are usually trained with a large, eclectic mix of corpora and search tasks. How should these diverse corpora and tasks be sampled for training? Conventional approaches sample them uniformly, proportional to their instance population sizes, or depend on human-level expert supervision. It is well known that the training data sampling strategy can greatly impact model performance. However, how to find the optimal strategy has not been adequately studied in the context of embedding models. We propose Inf-DDS, a novel reinforcement learning driven sampling framework that adaptively reweighs training datasets guided by influence-based reward signals and is much more lightweight with respect to GPU consumption. Our technique iteratively refines the sampling policy, prioritizing datasets that maximize model performance on a target development set. We evaluate the efficacy of our sampling strategy on a wide range of text retrieval tasks, demonstrating strong improvements in retrieval performance and better adaptation compared to existing gradient-based sampling methods, while also being 1.5x to 4x cheaper in GPU compute. Our sampling strategy achieves a 5.03 absolute NDCG@10 improvement while training a multilingual bge-m3 model and an absolute NDCG@10 improvement of 0.94 while training all-MiniLM-L6-v2, even when starting from expert-assigned weights on a large pool of training datasets.

1. Introduction

Dense retrievers are trained on diverse datasets, but dataset size does not guarantee embedding quality, making informative dataset selection and proportions important. Inf-DDS addresses this through adaptive, influence-guided sampling that adjusts dataset probabilities according to downstream performance improvements while reducing computational demands.

  • Dataset size alone does not guarantee embedding quality, motivating identification of informative datasets and their optimal training proportions.
  • Random, temperature, proportional, and expert-weight sampling strategies are static or ignore source informativeness, so they may be suboptimal.
  • Existing adaptive methods use gradient-derived rewards, influence estimates, or proxy models, but dynamic sampling methods face practical challenges.
  • Inf-DDS uses small per-domain gradient updates and downstream metric changes to reward domains that produce larger performance improvements.
  • Inf-DDS is an influence-guided reinforcement learning approach that adjusts sampling probabilities across diverse datasets for target-domain retrieval.

2. Related Work

Prior work spans heuristic sampling, task- or domain-specific data selection, influence-based example prioritization, and learned gradient-based reweighting. Inf-DDS builds on MultiDDS and DoGE by using online influence scores instead of noisy gradient-based rewards to adjust dataset-level sampling weights.

  • Heuristic sampling: Heuristic approaches sample languages uniformly or with temperature-scaled distributions interpolating between uniform and sizeproportional sampling.Cooldown demonstrates multilingual-training improvements by oversampling high-resource languages during initial training phases.
  • Domain-specific selection: Domain- and task-specific methods select relevant data using cross-entropy differences, classifiers, hashed-feature importance sampling, or cluster-level importance weights.These approaches include Moore and Lewis, Brown et al., DSIR, and CRISP.
  • Influence-based selection: Influence-based methods prioritize examples estimated to most affect model predictions or target performance, with LESS and Quad using first-order Taylor approximations.Their influence estimates approximate target-loss changes after a gradient update.
  • Learned reweighting: DDS and DoGE learn scorer networks through bi-level optimization, prioritizing examples whose gradients align with a held-out development set; MultiDDS extends DDS to multiple targets.MultiDDS learns per-language scoring functions to optimize performance across development sets.
  • Inf-DDS: Inf-DDS builds on MultiDDS and DoGE by replacing noisy gradient-based rewards with online-computed influence scores from updated model parameters.The method adjusts dataset-level sampling weights and reuses intermediate computations to simplify and stabilize training.

3. Influence Guided Dynamic Data Sampling

Inf-DDS formulates dataset sampling for text-retriever domain adaptation as a bilevel optimization problem, learning an adaptive policy that maximizes performance on target development sets. It uses influence-based rewards to update the sampling policy while efficiently updating the retriever with Reptile-style gradient reuse and dataset subsampling.

  • Adaptive dataset sampling: Inf-DDS learns dataset-level sampling probabilities through a parameterized policy that optimizes retriever performance on development datasets.The policy dynamically adjusts dataset probabilities and assigns each training dataset an importance score.
  • Bilevel optimization: The bilevel procedure alternates gradient-based updates of model parameters θ with REINFORCE updates of scorer parameters ψ.This separates retriever optimization from policy optimization within the dynamic sampling framework.
  • Influence-based rewards: Influence-based rewards measure how training datasets change performance on held-out development data, guiding the scorer to up-sample beneficial datasets.Influences are accumulated across train–development dataset pairs to produce a final impact measure for each training dataset.
  • Efficient optimization: Inf-DDS reuses proxy-model gradients through Reptile-style first-order meta-updates, requiring only one copy of parameter gradients and optimizer states.The method uses weighted Reptile updates to reduce computation and memory overhead.
  • Scalable sampling: For large dataset pools, the scorer can update on a uniform random subsample, reducing per-iteration compute and memory while introducing bias relative to the full-objective gradient.The estimator remains unbiased for the conditional objective over the sampled subset.

4. Experimental Setup

The experiments evaluate sampling strategies for text-retriever domain adaptation across controlled BEIR, multilingual long-document, and diverse-domain embedding settings. They compare Inf-DDS with static, universal-generalization, gradient-based, and task-adaptive importance-sampling baselines.

  • Benchmarks: The study tests domain adaptation under different sampling strategies using three benchmark settings: BEIR, MLDR, and a sentence-transformers embedding dataset.The section introduces the benchmarks and experimental setup used to evaluate text retrievers.
  • BEIR: BEIR experiments train on seven datasets, optimize first on FEVER, and extend evaluation to datasets with available development and test sets.The seven training datasets are MSMarco, NQ, FEVER, FiQA, HotpotQA, SciFact, and NFCorpus; NFCorpus is excluded from target datasets because of noisy relevance labels.
  • Multilingual Long Document Retrieval (MLDR): MLDR experiments treat each language as a domain, sample proportionally within each language, adapt on MLDR-13 development data, and evaluate on its corresponding test set.The biencoder starts from a 568M parameter bge-m3-unsupervised checkpoint and uses the BGE-M3 corpus.
  • Sentence-Transformers Embedding Dataset: The sentence-transformers setting uses a 1 billion-sentence corpus from 32 datasets with Expert initialization, excluding Reddit comments and CodeSearchNet.The exclusions align the training corpus with the target domain by removing a large Reddit dataset and an unrelated code-focused dataset.
  • Baselines: Baselines span static sampling, universal generalization, gradient-based task adaptation, and cluster-level task-adaptive importance sampling.The compared methods are Temperature and Cooldown, DoReMi, MultiDDS and DoGE, and CRISP, respectively.

5. Results And Analysis

Inf-DDS improves retrieval adaptation across BEIR, MLDR, and highly diverse Sentence-Transformers data, outperforming or surpassing competing sampling methods in the reported settings. Its gains depend on initialization but remain substantial with strong expert weights, while dynamic sampling provides the main benefit over meta-learning updates.

  • Domain adaptation on BEIR: Domain similarity alone does not consistently predict retrieval performance, motivating adaptive sampling optimized for downstream results.FEVER, HotpotQA, and FiQA benefit from MS MARCO as well as their corresponding training domains.
  • Domain adaptation on BEIR: Inf-DDS consistently outperforms all baselines on FEVER from τ = 0.3 initialization and remains competitive with CRISP using 32 clusters.The experiments assess robustness to multiple initial sampling distributions because Inf-DDS uses single-shot optimization.
  • Domain adaptation on BEIR: 0.93 points: MultiDDS underperforms Inf-DDS’s best score in joint BEIR-5 optimization, where Inf-DDS beats static sampling for every temperature initialization and MultiDDS in 2 out of 3 scenarios.MultiDDS shows more improvement when initialized with τ = 1, but still trails Inf-DDS’s best result.
  • MLDR: +5.03 points in NDCG@10: Inf-DDS improves bge-m3-dense on MLDR-13, exceeding DoReMi’s +4.76-point gain and achieving the highest individual-language performance in 8 out of 13 languages.The scorer is optimized jointly across 13 development sets, and the model starts from the same initial sampling weights as bge-m3-dense.
  • Sentence-Transformers Embedding Dataset: 1.83 point gain: Inf-DDS improves the uniform baseline on Sentence-Transformers data and reaches within 0.22 points of the off-the-shelf all-MiniLM-L6-v2 Expert model.MultiDDS and DoGE produce no gains from uniform initialization; starting from Expert weights, Inf-DDS adds a further 0.94 point improvement.
  • Computational Overheads and Ablations: Inf-DDS is slightly slower than CRISP, MultiDDS, and static sampling but consistently achieves superior performance, while Reptile updates mainly reduce computational overhead rather than drive the primary gains.Initialization substantially affects performance, although reasonable initial weights followed by consistent updates yield gains.

6. Conclusion … A.3. Multilingual Long Document Retrieval

The paper presents Inf-DDS for adapting text retrievers through influence-guided dynamic data sampling, jointly optimizing model and scorer parameters with influence scores as rewards. The merged sections also specify training configurations for BEIR, sentence-transformer embedding, and multilingual long-document retrieval experiments.

  • 6. Conclusion: Inf-DDS adapts text retrievers to target domains by dynamically sampling training data with influence-guided rewards.The framework uses scorer parameters ψ to parameterize the sampling distribution.
  • 6. Conclusion: Inf-DDS jointly updates model parameters θ and scorer parameters ψ through bi-level optimization.The approach is described as producing more stable sampling trajectories across diverse domains and benchmarks.
  • A.1. BEIR: BEIR experiments initialize the bi-encoder from roberta-base and use a 2e-5 learning rate with BF16 mixed precision.They use in-batch negatives, temperature 0.02, normalized representations, batch size 256, and one NVIDIA-A100 80GB GPU.
  • A. Hyperparameters: The BEIR configuration uses a linear learning-rate decay with 250 warmup steps.The passage also specifies batch size 256 for scorer updates.
  • A.2. Sentence Transformers Embedding: Sentence-transformer embedding experiments use nreimers/MiniLM-L6-H384-uncased, a 2e-5 learning rate, BF16, and in-batch plus cross-device negatives.Training batches contain 2000 examples, while evaluation uses batches of 256 examples per GPU across 8 NVIDIA-A100 80GB GPUs.
  • A.3. Multilingual Long Document Retrieval: Multilingual long-document retrieval experiments initialize BAAI/bge-m3-unsupervised with a 2e-5 learning rate and BF16 mixed precision.Training uses in-batch negatives, cross-device negatives, 8 hard negatives, temperature 0.02, normalized representations, and the bge-m3-kd-distil loss.

B. Datasets · B.1. BEIR

The section presents the BEIR training datasets in Table 1. No further dataset details are provided in the supplied passage.

  • B.1. BEIR: Table 1 is titled “BEIR Train Datasets.”The passage identifies the table but does not list its datasets or statistics.

B.2. Sentence Transformers Embedding Dataset · B.3. BGE-m3 Multilingual

The paper describes diverse training-data mixtures for all-MiniLM-L6-v2 and multilingual bge-m3, spanning user-generated, scientific, question-answering, and language-specific datasets. BGE-m3 uses separate English and Chinese dataset collections, while BEIR development datasets are listed for evaluation.

  • B.2. Sentence Transformers Embedding Dataset: BEIR development datasets are presented in Table 2.The supplied passage provides the table title but not its dataset entries.
  • B.2. Sentence Transformers Embedding Dataset: all-MiniLM-L6-v2 is fine-tuned with a self-supervised contrastive objective over concatenated public sentence-pair corpora.The mixture combines diverse sources rather than a single dataset.
  • B.2. Sentence Transformers Embedding Dataset: The all-MiniLM-L6-v2 mixture includes user-generated Reddit, Stack Exchange, and Yahoo Answers pairs.These sources include paired comments, Q&A threads, and related sentence-pair data.
  • B.2. Sentence Transformers Embedding Dataset: It also includes scientific citation pairs from S2ORC and SPECTER plus question-answering benchmarks such as PAQ, MSMARCO, and Natural-Questions.The passage additionally names SearchQA, SQuAD 2.0, and TriviaQA.
  • B.3. BGE-m3 Multilingual: 8 datasets are included for English bge-m3 fine-tuning, including HotpotQA, TriviaQA, NQ, MS MARCO, COLIEE, PubMedQA, SQuAD, and SimCSE.These datasets cover the English multilingual-model training mixture described in the passage.
  • B.3. BGE-m3 Multilingual: 7 datasets are included for Chinese bge-m3 fine-tuning, including DuReader, mMARCO-ZH, T2-Ranking, LawGPT, and CMedQAv2.The supplied passage continues beyond CMedQAv2 but is truncated.

C. Additional results · C.1. BEIR

On BEIR-5 development sets, Inf-DDS consistently matches or outperforms static sampling and exceeds proportional sampling by an average of 2.24 points. It also outperforms MultiDDS on average, despite not achieving the highest result in every domain.

  • C.1. BEIR: Inf-DDS consistently matches or outperforms static sampling across all BEIR-5 development datasets.Results are reported in Table 7 and Figure 9.
  • C.1. BEIR: 2.24 points is Inf-DDS’s average improvement over proportional sampling across the BEIR-5 datasets.
  • C.1. BEIR: Inf-DDS outperforms MultiDDS on average, although MultiDDS achieves the highest performance in some individual domains.

C.2. Sentence Transformers Embedding Dataset · C.3. MLDR

The sections evaluate adaptive scorer optimization for all-MiniLM-L6-v2 and BGE-M3 using Sentence-Transformers and MLDR data, respectively. Results are reported as Average NDCG@10 on BEIR-5 and MLDR-13 test sets, with optimization jointly performed on corresponding development sets.

  • C.2. Sentence Transformers Embedding Dataset: The all-MiniLM-L6-v2 experiment uses Sentence-Transformers data and jointly optimizes the scorer on BEIR-5 development sets.Evaluation is conducted on the BEIR-5 test collection.
  • C.2. Sentence Transformers Embedding Dataset: Average NDCG@10 is the reported metric for the all-MiniLM-L6-v2 evaluation.The metric is reported on the BEIR-5 test collection.
  • C.2. Sentence Transformers Embedding Dataset: Both experiments pair test-set Average NDCG@10 reporting with joint scorer optimization on the corresponding development sets.The corresponding evaluations are BEIR-5 for all-MiniLM-L6-v2 and MLDR-13 for BGE-M3.
  • C.2. Sentence Transformers Embedding Dataset: Table 3 describes the all-MiniLM-L6-v2 training data supplied by Sentence Transformers.The table includes dataset sizes, hand-picked weights, and normalized sampling percentages.
  • C.3. MLDR: The MLDR-13 experiment trains with BGE-M3 data and jointly optimizes the scorer on MLDR-13 development sets.Evaluation is conducted on the MLDR-13 test set.
  • C.3. MLDR: Average NDCG@10 is the reported metric for the BGE-M3 evaluation on MLDR-13.The reported test collection is MLDR-13.

C.4. Ablation · D. Sampling trajectories

The ablation indicates that dynamic sampling accounts for most performance gains, while Reptile updates contribute only minor additional differences. Sampling trajectories show that Inf-DDS reallocates probability toward datasets associated with target-task performance, sometimes avoiding harmful oversampling by MultiDDS.

  • C.4. Ablation: Reptile updates produce only minor performance differences, suggesting dynamic sampling drives most observed gains.The ablation compares Reptile-enabled and Reptile-disabled updates, while noting that meta-learning may still contribute.
  • D. Sampling trajectories: Inf-DDS and MultiDDS follow distinct sampling probability trajectories under varying initialization temperatures on the FEVER development set.The FEVER training set trajectory is explicitly tracked in Figure 10.
  • D. Sampling trajectories: MultiDDS aggressively upsamples NFCorpus during joint BEIR-5 optimization, degrading overall performance.The trajectory is reported alongside Table 8’s overall performance results.
  • D. Sampling trajectories: Inf-DDS upsamples MSMarco and FEVER for FiQA optimization, whereas MultiDDS upsamples FiQA based on gradient similarity.The passage links Inf-DDS’s trajectory to datasets more relevant for performance gains, as shown in Table 7.
  • D. Sampling trajectories: MultiDDS upsamples HotpotQA more during joint optimization, leading to better HotpotQA performance.The comparison is reported for trajectories optimized on the HotpotQA development set.
  • D. Sampling trajectories: Because DBpedia is absent from training, Inf-DDS upsamples FEVER more during DBpedia optimization.The passage attributes this behavior to the close relationship between DBpedia and FEVER.
  • D. Sampling trajectories: Inf-DDS trajectories are also reported for all-MiniLM-L6-v2 with Expert initialization and τ = 1 initialization on Sentence-Transformers data.Both settings use joint optimization on the BEIR-5 development set.
  • D. Sampling trajectories: For bge-m3-dense, Inf-DDS sampling probabilities are tracked while jointly optimizing across the MLDR-13 development sets.This trajectory complements the multilingual training evaluation described in the figure caption.
Loading 2601.21759v1…