Source-linked AI summary
On the Robustness of LLM-Based Dense Retrievers: A Systematic Analysis of Generalizability and Stability
Yongkang Li, Panagiotis Eustratiadis, Yixing Fan, Evangelos Kanoulas
TL;DR
Robustness of modern decoder-only LLM dense retrievers remains insufficiently characterized beyond leaderboard performance. The paper evaluates generalizability and stability across broad benchmarks with mixed-effects analysis and perturbation tests, finding strong instruction-augmented performance but specialization costs, perturbation-specific robustness, and useful embedding-geometry signals.
Problem
Robustness of decoder-only LLM-based dense retrievers remains underexplored across generalization conditions and perturbations.
Method
The study evaluates retrievers across 30 datasets and uses linear mixed-effects models plus query- and document-side robustness tests.
Results
Instruction-augmented models generalize strongly, reasoning-specialized models show limited broader generalizability, and robustness varies across perturbations.
Takeaways & Limitations
Angular uniformity is the clearest cross-model signal of lexical and surface-level robustness, while larger Qwen3 models tend to be more robust.
Takeaways & Limitations
The study does not evaluate stronger surrogate-based black-box attacks or training-time backdoor attacks.
Abstract
from arXiv · showhide
Decoder-only large language models (LLMs) are increasingly replacing BERT-style architectures as the backbone for dense retrieval, achieving substantial performance gains and broad adoption. However, the robustness of these LLM-based retrievers remains underexplored. In this paper, we present the first systematic study of the robustness of state-of-the-art open-source LLM-based dense retrievers from two complementary perspectives: generalizability and stability. For generalizability, we evaluate retrieval effectiveness across four benchmarks spanning 30 datasets, using linear mixed-effects models to estimate marginal mean performance and disentangle intrinsic model capability from dataset heterogeneity. Our analysis reveals that while instruction-tuned models generally excel, those optimized for complex reasoning often suffer a ``specialization tax,'' exhibiting limited generalizability in broader contexts. For stability, we assess model resilience against both unintentional query variations~(e.g., paraphrasing, typos) and malicious adversarial attacks~(e.g., corpus poisoning). We find that LLM-based retrievers show improved robustness against typos and corpus poisoning compared to encoder-only baselines, yet remain vulnerable to semantic perturbations like synonymizing. Further analysis shows that embedding geometry (e.g., angular uniformity) provides predictive signals for lexical stability and suggests that scaling model size generally improves robustness. These findings inform future robustness-aware retriever design and principled benchmarking. Our code is publicly available at https://github.com/liyongkang123/Robust_LLM_Retriever_Eval.
1 Introduction
The paper studies robustness in open-source decoder-only LLM dense retrievers through generalizability across conditions and stability under perturbations. Across its analyses, instruction-augmented models perform strongly, reasoning-specialized models generalize less broadly, and robustness varies by perturbation type.
- The study presents a systematic robustness analysis of SOTA open-source decoder-only LLM-based dense retrievers.
- Robustness is examined through generalizability across diverse evaluation scenarios and stability under perturbations.
- 30 datasets across 4 benchmarks are analyzed by task type, query type, and corpus source type.
- Linear mixed-effects models estimate marginal mean performance while controlling for dataset-specific random effects.
- Instruction-augmented retrievers achieve the strongest average estimated performance, whereas reasoning-optimized models show a specialization tax across broader retrieval conditions.
- LLM-based retrievers are more robust to misspellings and white-box poisoning, but synonymizing remains challenging and paraphrasing is dataset-dependent.
- Angular uniformity provides the clearest cross-model signal for lexical and surface-level vulnerability, while larger Qwen3 models tend to be more robust.
- Interventions produced no consistent robustness gains, suggesting angular uniformity is more useful diagnostically than as a standalone training target.
2 Related Work
Related work places LLM-based dense retrieval within the evolution from sparse lexical and encoder-based methods. It also identifies open questions about generalization and robustness across heterogeneous tasks, query variations, and adversarial corpus manipulation.
- Prior retrieval systems are organized into sparse lexical, single-vector dense, and multi-vector late-interaction paradigms.
- Sparse lexical methods remain efficient and interpretable but suffer from vocabulary mismatch when relevant texts have limited lexical overlap.
- Decoder-only LLMs have been adapted into dense encoders, while instruction-augmented retrievers improve transfer across retrieval tasks.
- Reasoning-oriented retrievers perform strongly on reasoning benchmarks, raising whether their gains transfer across diverse retrieval conditions.
- Existing robustness research covers domain and task generalization, query perturbations, and adversarial manipulation, but remains fragmented.
- Modern LLM-based retrievers remain insufficiently characterized across heterogeneous tasks, query types, and corpus sources.
- Prior query-variation studies are limited by query-side perturbations and relatively small-scale benchmarks.
- The effectiveness of adversarial attacks developed for BERT-based retrievers against LLM-based encoders remains unclear.
3 Methodology
The methodology uses a shared bi-encoder retrieval pipeline and linear mixed-effects models for factor-wise generalizability analysis. The statistical framework separates fixed effects from dataset- and query-level variability when comparing heterogeneous evaluations.
- Simple averaging can confound model capability with differences in query volume, corpus size, and dataset difficulty.
- A neural bi-encoder maps queries and documents into a shared latent space for dense retrieval.
- Pooling follows each architecture’s default configuration, with decoder-only LLMs typically using last-token or mean pooling.
- Instruction-augmented retrievers prefix queries with task descriptions, and relevance is scored using cosine similarity or dot product.
- Linear mixed-effects models explicitly model hierarchical random variability for unbalanced data with queries nested within datasets.
- Three models analyze Task Type, Query Type, and Corpus Source Type while accounting for dataset- and query-level effects.
- Estimated Marginal Means provide population-level performance estimates averaged over random effects for standardized comparisons.
4 Experimental Setup
The experiments cover 30 datasets from four benchmarks and compare BM25, encoder-only dense baselines, and six decoder-only LLM retrievers. Evaluation uses nDCG@10 alongside perturbation- and poisoning-specific robustness metrics under standardized retrieval settings.
- The evaluation draws on MS MARCO, BEIR, BRIGHT, and BrowseComp-Plus, covering 30 datasets across tasks, domains, and corpus sources.
- The shared model pool includes BM25, Contriever, BGE-M3, and six decoder-only LLM-based retrievers.
- The LLM retrievers comprise three general-purpose instruction-augmented models and three reasoning-oriented models.
- Official developer checkpoints and default encoding configurations are used, including developers’ query prompt templates for instruction-augmented retrievers.
- Documents are truncated to 512 tokens for Contriever and 8192 tokens for other dense retrievers, with cosine similarity used except for Contriever’s dot product.
- nDCG@10 is the primary effectiveness metric, while robustness analyses report relative perturbation drops and ASR@20 for corpus poisoning.
5 RQ1: How generalizable are LLM-based dense retrievers across diverse conditions?
The evaluation examines generalizability across task, query, and corpus-source factors using raw scores and linear mixed-effects models. Instruction-augmented retrievers generalize best overall, while reasoning-focused models trade broad performance for narrow strengths.
- Evaluation design: Nine retrievers are evaluated across 30 datasets from four benchmarks, organized by task type, query type, and corpus source type.The analysis first reports raw per-dataset nDCG@10, then estimates marginal means while controlling for dataset- and query-level confounds.
- Raw results: Raw nDCG@10 varies from above 90 on easier datasets such as FEVER and Quora to below 5 on BrowseComp-Plus, showing substantial dataset heterogeneity.The BEIR-to-BRIGHT average gap exceeds 30 points for most models, making naive aggregation potentially misleading.
- Raw results: ReasonEmbed achieves the best performance on 9 of 12 BRIGHT datasets but scores 48.5 on Quora versus more than 88 for other LLM retrievers.This contrast illustrates strong benchmark-specific performance alongside limited broader generalizability.
- Raw results: Specialized domains remain difficult for every model, with code and theorem retrieval typically producing nDCG@10 scores below 30.BM25 remains competitive in some regimes, including Touché-2020 and BRIGHT, where its average is 14.5 versus 15.0 for Contriever and 12.2 for BGE-M3.
- Statistical analysis: All Model × Factor interactions are highly significant (p < .001), with F-ratios from 166.54 to 468.79, indicating model gaps vary across conditions.Corpus source has no uniform average shift, but its effect differs sharply by model.
- Factor-wise patterns: Instruction-augmented retrievers achieve the strongest estimated marginal means across task, query, and corpus factors, with task EMMs of 47.9–49.1.Their advantage is broad rather than confined to one evaluation factor.
6 RQ2: How stable are LLM-based dense retrievers under perturbations?
LLM-based retriever stability is strongly perturbation- and dataset-dependent: general-purpose instruction-augmented models handle misspellings and some attacks well, but semantic query changes remain difficult. Reasoning-oriented training does not consistently improve stability, while transfer poisoning is largely ineffective.
- Query-side stability: Synonymizing remains challenging across model families, whereas paraphrasing is dataset-dependent and most disruptive for shorter queries.On MS MARCO, synonymizing causes 22–27% drops versus 13–20% for paraphrasing; HotpotQA paraphrasing has an approximately 3% impact.
- Query-side stability: Naturalizing is dataset-dependent, with BGE-M3 especially vulnerable on NQ and HotpotQA, where its drops are about 20.5% and 16%, respectively.MS MARCO remains relatively robust at 3–10%, while domain-specific financial phrasing may contribute to FiQA’s differing profile.
- Query-side stability: Across perturbations, GTE exhibits the most consistently strong stability, whereas reasoning-oriented training does not translate into improved query-side stability.DIVER and ReasonEmbed reach roughly 27% drops on MS MARCO, comparable to BGE-M3 and above their instruction-augmented counterparts.
- Document-side stability: Direct-transfer corpus poisoning is largely ineffective, with transfer ASR@20 below 1% even when Contriever is attacked as the vulnerable target.The near-zero transfer is consistent with weak cross-model transferability, potentially reflecting architectural, training-regime, and embedding-geometry differences.
7 RQ3: What factors are predictive of robustness in LLM-based retrievers?
The analysis evaluates embedding isotropy, model size, and spectral norm as candidate predictors of retriever robustness. Angular uniformity is the clearest cross-model signal for lexical and surface-level vulnerability, while variance uniformity is inconclusive.
- Candidate factors: The study tests embedding isotropy, model size, and spectral norm as candidate robustness factors using correlation and within-family scaling analyses.Isotropy and spectral norm are compared against RQ2 stability metrics, while model size is analyzed within Qwen3.
- Interpretation: The cross-model correlations should be interpreted as small-sample signals rather than definitive estimates.
- Angular uniformity: High average cosine similarity, indicating low angular uniformity, is the clearest cross-model signal of vulnerability to lexical and surface-level perturbations.The relationship is recurrent across datasets and perturbation types, especially for misspelling, reordering, and synonymizing.
- Angular uniformity: On MS MARCO, angular uniformity correlates strongly with Misspelling (r=0.91), Reordering (r=0.92), Synonymizing (r=0.97), and corpus poisoning (r=0.91), all with p<0.01.
- Variance uniformity: Variance uniformity shows weak, directionally inconsistent query-variation correlations and remains a secondary, inconclusive robustness signal.For corpus poisoning, NQ (r=0.65) and HotpotQA (r=0.62) show positive but non-significant trends, while MS MARCO shows r=0.14.
7.2 Model Size and Robustness
Within the Qwen3 family, larger models generally show stronger robustness to query variations and corpus poisoning. The gains are clearest for surface-form perturbations and are not perfectly monotonic or established across architectures.
- Query variations: Larger Qwen3 models generally exhibit smaller nDCG@10 drop rates across datasets, indicating improved stability to input perturbations.
- Query variations: On NQ, the Misspelling drop rate decreases from 17.4% for Qwen3-0.6B to 10.1% for Qwen3-8B.
- Query variations: Scaling more reliably improves robustness to surface-form perturbations than to structural or semantic reformulations.Gains for Reordering and Paraphrasing are weaker and more dataset-dependent.
- Corpus poisoning: HotpotQA corpus-poisoning ASR@20 drops from 25.1% to 11.0% as Qwen3 size increases from 0.6B to 8B.MS MARCO follows the same qualitative pattern, whereas NQ is mildly non-monotonic.
- Scope: The within-family evidence is preliminary: the scaling trend is not perfectly monotonic, and broader cross-family validation remains open.
7.3 Spectral Norm and Robustness
Mean spectral norm is examined as a proxy for Lipschitz smoothness, but it does not provide a stable cross-dataset predictor of robustness. The results favor representation geometry over smoothness as a robustness lens.
- Limitation: The spectral-norm analysis is constrained by its architecture-level approximation, motivating more precise local-sensitivity estimates.
- Overall result: Mean spectral norm exhibits no statistically significant correlations with robustness across the evaluated datasets and perturbation types.
- Query variations: Misspelling shows positive spectral-norm correlations across NQ (r=0.35), MS MARCO (r=0.47), and HotpotQA (r=0.21), but the trends are weak and non-significant.
- Query variations: Reordering, Synonymizing, and Paraphrasing show inconsistent or near-zero spectral-norm correlations.
- Corpus poisoning: For corpus poisoning, spectral norm shows r=0.39 on MS MARCO, r=0.01 on NQ, and r=-0.04 on HotpotQA.
- Interpretation: Overall, representation geometry appears more informative for robustness than the smoothness proxy, whose value is limited and context-dependent.
8 Discussion
The discussion argues that robustness must be evaluated across distinct perturbation and attack axes rather than inferred from scale or architecture alone. It identifies angular uniformity as a diagnostic, Qwen3 scaling as preliminary evidence, and threat-model-specific evaluation as practical priorities.
- Causal interpretation: Correlational evidence identifies angular uniformity as a consistent robustness correlate, but interventions test whether that relationship is causal.
- Training interventions: Reducing average cosine similarity from 0.801 to 0.139 leaves FiQA retrieval effectiveness unchanged at nDCG@10 = 35.5 and yields no consistent robustness improvement.
- Training interventions: Angular uniformity is therefore more useful as a post-hoc diagnostic than as a standalone robustness training target.
- Model interpretation: GTE achieves 0% ASR under white-box corpus poisoning, but the study cannot attribute this robustness to specific training choices because its training details are undocumented.
- Evaluation: Robustness evaluation should separately report at least character-level, semantic, and document-side adversarial perturbations because aggregate scores can mask failure modes.
- Model selection: Within Qwen3, larger models reduce query-variation drop rates and poisoning ASR, but the scaling trend does not generalize cleanly across architectures.
- Threat models: Direct-transfer black-box poisoning has near-zero transfer ASR in the studied no-access setting, whereas white-box poisoning remains a genuine risk at |A| = 50.
- Threat models: Surrogate-based black-box attacks are not evaluated and remain an open threat that could approach white-box attack success rates more closely.
9 Conclusion
This study finds that LLM-based retriever robustness is multidimensional: generalizability and stability vary across models and perturbation types, and no single architecture, scale, or geometric statistic explains them alone.
- The analysis evaluates LLM-based dense-retriever robustness through generalizability across 30 heterogeneous datasets and stability under query perturbations and corpus poisoning.
- LLM-based retrievers show the strongest estimated generalizability, while reasoning-oriented models incur a specialization tax outside targeted reasoning-heavy settings.
- Instruction-augmented models outperform encoder-only baselines on character-level noise, but synonymizing remains broadly challenging and paraphrasing is dataset-dependent.
- Paraphrasing is especially disruptive in shorter-query settings such as MS MARCO and FiQA, while direct-transfer corpus-poisoning attacks remain below 1% ASR across model pairs.
- Embedding isotropy, especially angular uniformity, is the clearest cross-model signal studied, whereas spectral norm associations are weak and context-dependent.
- Directly regularizing angular uniformity reshapes embedding geometry without improving robustness, supporting its use as a post-hoc diagnostic rather than a standalone training objective.
Limitations
The study’s conclusions are bounded by its model, language, attack, geometry, intervention, and dataset coverage, limiting how broadly its robustness patterns should be interpreted.
- The evaluated retrievers are representative but not exhaustive, so newer models may exhibit different robustness patterns.
- All experiments use English datasets, and the findings may not generalize to other languages.
- Document-side stability is evaluated under white-box gradient-based poisoning and direct-transfer no-access attacks, excluding stronger surrogate-based black-box and training-time backdoor attacks.
- Embedding-geometry analysis is primarily correlational, uses a limited cross-model sample and corpus-side proxies, and should be interpreted as indicative rather than definitive.
- The model-size analysis is confined to the Qwen3 embedding family and should not be treated as a general scaling law for LLM-based dense retrievers.
- Computational constraints limited corpus-poisoning experiments to three datasets, so additional domains may reveal further vulnerability patterns.
Ethical Considerations
The paper frames corpus poisoning and query perturbation experiments as robustness research conducted under controlled, responsible-release and licensing practices.
- The experiments study corpus poisoning and query perturbations to diagnose failure modes and support development of more robust retrievers.
- The work follows established protocols and introduces no new attack algorithms, optimization techniques, or deployment guidance beyond publicly available material.
- Public datasets, models, and toolkits are used for research evaluation under their stated licenses and access conditions.
- When redistribution is restricted, the authors release only reproducibility code, configurations, and scripts requiring users to obtain original resources independently.
- Generated adversarial passages, if released, are restricted to controlled, labeled research benchmarks with documentation discouraging misuse.