PaperlayerView exact revision on arXiv

Source-linked AI summary

Generated Query Expansion Still Helps Strong Sparse Retrieval: A Controlled Study with SPLADE-v3

Ryan C. Barron, Cade W. Trotter, Maksim E. Eren, Kim Ø. Rasmussen, Liz D. Miller, Benjamin J. Migliori

arXiv:2609.37911v1cs.IRcs.AI

TL;DR

Scientific queries can be brief while relevant literature uses specialized vocabulary, and prior work leaves uncertain whether expansion helps a strong learned sparse retriever. This study compares four generated formats with SPLADE-v3 under matched retrieval conditions; all four improve aggregate nDCG@10 across three collections, while controls point to added vocabulary as the common benefit.

  • Problem

    Scientific queries may use ordinary wording while relevant papers use specialized terminology, and prior work suggests generated expansion becomes less useful as retrievers grow stronger.

    Source: §I · p. 1

  • Method

    The study adds four generated formats to SPLADE-v3 across three scientific collections and compares them under a shared frozen index and fixed sparse query budget.

    Source: E. Collections, Baselines, and… · p. 4

  • Results

    All four generated formats improve aggregate nDCG@10 across three collections, with best relative gains of 4.81% on NFCorpus, 8.92% on TREC-COVID, and 9.47% on SciDocs.

    Source: §I · p. 1

  • Takeaways & Limitations

    The controls point to added scientific vocabulary as the common benefit, best used as a supplement to a strongly preserved original query.

    Source: §V · p. 6

    The study covers one SPLADE-v3 configuration and three English BEIR collections, and the benchmarks may have appeared in generator training data without a documented-cutoff model or corpus-overlap analysis.

    Source: A. Limitations · p. 7

Abstract

from arXiv · show

Scientific queries are often brief, while relevant papers use specialized vocabulary. Generated query expansion can bridge this mismatch, but earlier work suggests that its value shrinks as the underlying retriever becomes stronger. We test the four generated formats of term lists, a pseudo-document, multiple pseudo-references, and corpus-steered text all together with SPLADE-v3 on NFCorpus, TREC-COVID, and SciDocs. Every condition searches the same frozen document index and follows the same query-side integration rule and 256-dimension budget, isolating the effect of the added content. All twelve method-collection comparisons improve aggregate nDCG@10, with best relative gains of 4.81%, 8.92%, and 9.47%. Eleven remain significant after Holm correction. The gain persists in 103 of 114 interpolation settings, including every setting that assigns at least 30% of the mixture weight to the original query. Shuffled-text and non-contextual lexical-bag controls also remain above baseline in all 24 aggregate comparisons, showing that the added vocabulary carries most of the benefit. A corpus-induced typed concept graph, by contrast, produces no consistent gain, and its relation, depth, validation, random, and gating controls do not rescue it. Generated vocabulary can therefore complement a strong learned sparse retriever, provided that the original query remains strongly represented.

I. INTRODUCTION

The paper tests whether generated vocabulary can still help SPLADE-v3, a strong learned sparse retriever, despite its own contextual lexical expansion. Across three scientific collections, the generated formats improve retrieval, while controls point to added vocabulary as the common benefit.

I. INTRODUCTION

  • Scientific queries may use ordinary words while relevant literature uses specialized entities, methods, and domain terminology, motivating expansion with vocabulary absent from the query.Classical expansion draws terms from judged or initially retrieved documents, while newer methods use generated keywords, hypothetical documents, or pseudo-references.Source: §I · p. 1
  • Because SPLADE already infers contextual vocabulary, the study asks what external generation can add when earlier work found generative expansion less useful for stronger retrievers.SPLADE-v3 is described as a strong checkpoint in the learned sparse retrieval family.Source: §I · p. 1
  • The controlled comparison adds four generated formats and corpus-induced concepts to SPLADE-v3 across NFCorpus, TREC-COVID, and SciDocs, with a shared frozen index and sparse query budget.The comparison is designed to distinguish a benefit specific to generated vocabulary from a generic benefit of adding related terms.Source: §I · p. 1
  • All four generated formats improve aggregate nDCG@10 across three collections, with best relative gains of 4.81% on NFCorpus, 8.92% on TREC-COVID, and 9.47% on SciDocs.The gains persist across mixture weights and controls that shuffle generated text or reduce it to a non-contextual lexical bag; the concept graph shows no comparable pattern.Source: §I · p. 1
  • Weight sweeps, paired tests, and same-content controls indicate that gains are broad and depend more on added vocabulary than exact word order, while graph controls do not reproduce them.The contribution summary identifies relation, depth, random, and gating controls among those tested.Source: §I · p. 2

II. RELATED WORK

Prior query-expansion work ranges from corpus-derived feedback to several language-model-generated formats, but gains are not assured for strong retrievers. SPLADE-v3 already performs contextual lexical expansion, making the source and integration of additional evidence central to this study.

A. Generated Query Expansion

  • Classical expansion derives vocabulary from judged or retrieved documents, whereas language models generate keyword lists, hypothetical passages, multiple pseudo-references, or corpus-steered text.These formats differ in whether they offer compact terms, connect concepts in passages, represent several facets, or constrain vocabulary using retrieved evidence.Source: A. Generated Query Expansion · p. 2
  • Generative expansion may help weaker systems more than stronger ones, may harm the strongest, and can reproduce benchmark evidence encountered during training.The study holds generated texts fixed to measure retrieval integration effects rather than generation variability.Source: A. Generated Query Expansion · p. 2

B. Learned Sparse Retrieval and Integration

  • SPLADE learns contextual lexical expansion while retaining sparse retrieval, so an external reformulator must contribute complementary evidence.The study freezes the document side and changes only the query representation; it uses fixed query-side interpolation for a controlled comparison in a shared sparse space.Source: B. Learned Sparse Retrieval and… · p. 2

C. Structured Expansion and Evaluation

The study compares structured concept expansion with generated evidence and evaluates gains beyond aggregate means. Its shared pipeline changes query-side evidence while keeping the document index and retrieval conditions fixed.

C. Structured Expansion and Evaluation

  • Typed concept edges motivate structured expansion, but graph construction introduces possible failure points from phrase extraction through projection, so the study compares traversal with flat, relation, depth, validation, and random controls.Paired intervals, corrected p-values, standardized effects, and wins/ties/losses supplement aggregate effectiveness measures.Source: C. Structured Expansion and Evaluation · p. 2

III. STUDY DESIGN AND METHODS

  • The study asks whether generated expansion helps frozen SPLADE-v3, whether gains persist across mixture weights and representations, and whether corpus-induced concepts or selective gates can match them.Figure 1 summarizes the shared retrieval pipeline.Source: §III · p. 2

A. Shared Frozen-Index Pipeline

  • A frozen SPLADE-v3 encoder creates the base query and document vectors once; matched conditions change only the query vector and use the same document index.The query combines normalized base and added-evidence vectors under a 256-dimension budget that protects the original query; configured original-query weights are 0.65 for Flat LLM-QE and CSQE and 0.50 for Query2doc and MuGI-style expansion.Source: A. Shared Frozen-Index Pipeline · p. 2, Figure 1 · p. 3

B. Generated Expansion Conditions

The four generated conditions differ in the form of added text, while their retrieval experiments use fixed expansions. The archived materials support rerunning those retrieval experiments, but the formats are not matched for generation efficiency.

B. Generated Expansion Conditions

  • Flat LLM-QE adds up to twelve terms, Query2doc adds one hypothetical passage, MuGI-style expansion adds four pseudo-references, and CSQE generates text conditioned on retrieved evidence.SPLADE encodes each added text, which is combined with the protected base query.Source: B. Generated Expansion Conditions · p. 3
  • The run archive preserves generated texts, prompts, request parameters, and raw responses so retrieval experiments can be rerun from the fixed expansions.The four conditions are not matched for model calls, token use, or latency; the study compares retrieval effectiveness rather than generation efficiency.Source: B. Generated Expansion Conditions · p. 3

C. Interpolation and Same-Content Controls

The section specifies same-content controls for generated queries and a corpus-induced concept-graph alternative, including how concepts are mapped, scored, projected, and selectively gated.

C. Interpolation and Same-Content Controls

  • The same generated content is tested after token shuffling and as an ℓ1-normalized term-frequency bag, with MuGI pseudo-references separately normalized and averaged.The controls use the same tokenizer wordpieces; the bag removes contextual SPLADE expansion.Source: C. Interpolation and Same-Content… · p. 3

D. Corpus-Induced Structured Comparison

  • HIQE derives typed broader, sibling, and related concept edges from phrase relations, dense similarity, shared parents, cooccurrence, and corpus support.Its publication-run edge validation is deterministic and corpus-grounded; the LLM-judgment variant is not used in the main results.Source: D. Corpus-Induced Structured Comparison · p. 3
  • The query mapper uses dense similarity, lexical overlap, and aliases to retain up to four seed concepts, traverse two edges, and keep at most twelve expansion concepts.These limits constrain concept selection and graph traversal.Source: D. Corpus-Induced Structured Comparison · p. 3
  • Before frozen-SPLADE encoding, projection text combines a concept name with optional aliases and representative-document snippets, then mixes the graph vector with the query at λproj = 0.75.Controls vary validation, relation weights or families, traversal depth, concept scores, and random count-matched concepts; gates include hand-set and leave-one-dataset-out logistic-regression variants evaluated against a per-query oracle.Source: D. Corpus-Induced Structured Comparison · pp. 3–4

E. Collections, Baselines, and Statistical Analysis

The study evaluates three scientific-search collections with matched retriever baselines and multiple effectiveness metrics. It uses paired per-query analyses and corrected significance tests, then presents results by research question.

E. Collections, Baselines, and Statistical Analysis

  • NFCorpus, TREC-COVID, and SciDocs contain 3,633, 171,332, and 25,657 documents, with 323, 50, and 1,000 test queries, respectively.Results are reported separately because query counts and relevance structures differ substantially.Source: E. Collections, Baselines, and… · p. 4
  • The matched comparison uses frozen SPLADE-v3 with candidate depth 1,000 and nDCG@10 as the primary metric, alongside lexical, dense, fused, and late-interaction baselines.MRR@10, MAP@10, Precision@10, Recall@10, Recall@100, and candidate recall are secondary measures.Source: E. Collections, Baselines, and… · p. 4
  • Paired nDCG@10 analyses report effect sizes, wins/ties/losses, bootstrap intervals, and sign-randomization tests, with Holm correction for generated methods and Bonferroni correction for graph comparisons.Exact per-query equality is retained as a tie; graph edges receive no manual semantic annotation.Source: E. Collections, Baselines, and… · p. 4

IV. RESULTS

  • Each results subsection answers a research question before presenting its supporting evidence.The questions address generated expansion, robustness to weight and representation, and corpus-induced concepts or gates.Source: §IV · p. 4

A. RQ1: Does Generated Expansion Improve SPLADE-v3?

Generated query expansion improves matched SPLADE-v3 runs across three collections, though the strongest format varies by collection. The gains extend across metrics and queries, and robustness tests point to added vocabulary as the common benefit.

A. RQ1: Does Generated Expansion Improve SPLADE-v3?

  • All four generated formats improve the matched SPLADE-v3 baseline on NFCorpus, TREC-COVID, and SciDocs; best relative nDCG@10 gains are 4.81%, 8.92%, and 9.47%, respectively.Source: A. RQ1: Does Generated Expansion… · p. 4
  • The strongest format varies by collection, and the best system still depends on collection and metric.CSQE is weakest among generated conditions on TREC-COVID, while Query2doc does not lead on the other two collections.Source: A. RQ1: Does Generated Expansion… · p. 4
  • The strongest generated condition on each collection also improves all six secondary measures; Recall@100 rises by 6.20%, 10.44%, and 5.57%.MAP@10 rises by 5.30%, 6.48%, and 12.58%, respectively.Source: A. RQ1: Does Generated Expansion… · p. 4
  • All twelve paired mean differences are positive, and eleven remain significant after Holm correction; the exception is CSQE on 50-query TREC-COVID.For the best method on each collection, wins exceed losses: 99–49 for NFCorpus, 34–11 for TREC-COVID, and 262–127 for SciDocs.Source: A. RQ1: Does Generated Expansion… · p. 4

B. RQ2: What Makes the Gain Robust?

  • 103 of 114 interpolation settings beat SPLADE-v3, including all 90 settings assigning at least 30% mixture weight to the original query.Every failing setting assigns the original query weight α = .1 or .2.Source: B. RQ2: What Makes the Gain Robust? · p. 4
  • Shuffled-text and lexical-bag controls stay above baseline in all 24 aggregate comparisons, pointing to generated vocabulary rather than word order or contextual expansion.The lexical-bag control removes contextual SPLADE expansion; 23 of 24 controls do not significantly differ from their contextual counterparts after Holm correction.Source: B. RQ2: What Makes the Gain Robust? · pp. 4–5

C. RQ3: Can Structured Expansion Match the Gain?

Ungated HIQE does not produce a consistent nDCG@10 gain over SPLADE-v3, and the tested graph ablations do not identify a reliably useful design choice. Although concept coverage is nearly complete, the stored diagnostics do not establish the graph’s topology.

C. RQ3: Can Structured Expansion Match the Gain?

  • Flat equal-weight concepts underperform the full graph on all three collections, but the random count-matched control has a less negative mean and depth two is slightly worse than depth one.Removing corpus validation causes the largest TREC-COVID degradation; parent-only traversal has the least negative mean among named graph variants.Source: C. RQ3: Can Structured Expansion Match… · p. 5
  • Concept coverage reaches 100.0% on NFCorpus, 99.71% on TREC-COVID, and above 99.99% on SciDocs, but connected-component and hierarchy-cycle counts are unavailable.The passage therefore makes no stronger topology claim.Source: C. RQ3: Can Structured Expansion Match… · p. 5
  • The learned gate selects graph expansion for all 50 TREC-COVID topics, preserving its degradation, while a per-query oracle remains 0.0232 nDCG@10 above the learned gate.Exact learned-gate activation counts cannot be recovered for 14 NFCorpus and 13 SciDocs queries.Source: C. RQ3: Can Structured Expansion Match… · pp. 5–6
  • +.0013, −.0126, and +.0006 nDCG@10 are the ungated HIQE changes on NFCorpus, TREC-COVID, and SciDocs; none is significant after Bonferroni correction.All three paired 95% intervals include zero.Source: C. RQ3: Can Structured Expansion Match… · p. 5, Figure 2 · p. 6

V. DISCUSSION

Generated expansion still improves SPLADE-v3, and the controls point to added scientific vocabulary rather than generated prose as the stable source of benefit. The gains favor preserving the original query, while the tested concept graph does not reproduce them.

V. DISCUSSION

  • Four generation formats improve a strong frozen SPLADE-v3 retriever across all three collections, despite its existing learned lexical expansion.The document index and retrieval pipeline are fixed, isolating the difference to query-side evidence.Source: §V · p. 6
  • Shuffled text and a non-contextual lexical bag retain gains, pointing to added scientific vocabulary—not word order or contextual encoding—as the most stable common factor.The vocabulary includes entities, methods, and specialized terms absent from the short query.Source: §V · p. 6
  • Every tested configuration with α ≥.3 beats baseline, while failures occur when the original query receives only 10% or 20% of the mixture weight.The passage supports preserving substantial original-query mass and adapting expansion weight by query, as in QuDAR.Source: §V · p. 6
  • A generated passage can serve internally as a source of candidate terms without being shown to users or treated as a factual answer.This separates retrieval utility from user-facing prose quality.Source: §V · p. 7
  • The tested concept graph does not reproduce the generated gains; corpus support, typed relations, traversal depth, and relation-specific weighting are insufficient in this pipeline.The hierarchy ablation table reports absolute ΔnDCG@10 relative to SPLADE-v3.Source: §V · p. 7, Table V · p. 7

A. Limitations

The evaluation covers one SPLADE-v3 configuration and three English BEIR collections, so its scope does not establish outcomes for other settings. Generator overlap and prospective weight tuning also remain untested.

A. Limitations

  • The study covers one SPLADE-v3 configuration and three English BEIR collections, leaving other retrievers, languages, domains, and interactive settings outside its scope.The public benchmarks may have appeared in generator training data; the study uses no documented-cutoff generator, corpus-overlap analysis, or paraphrased-query control.Source: A. Limitations · p. 7
  • Primary methods use fixed interpolation weights rather than a shared development-set tuning protocol, and the sweep is not prospective tuning.The graph is automatically built and corpus-validated rather than manually labeled, so its negative results may reflect graph construction, graph-guided integration, or both.Source: A. Limitations · p. 7
  • Because the graph uses corpus evidence rather than manually labeled edges, its negative results may reflect graph construction, graph-guided integration, or both.The passage does not isolate which of these factors accounts for the result.Source: A. Limitations · p. 7

VI. CONCLUSION

Generated query expansion improves a strong learned sparse retriever across the three scientific collections, despite SPLADE-v3’s own contextual lexical expansion. The tested concept graph does not show the same benefit, and the conclusion is to preserve the original query while supplementing it with generated vocabulary.

VI. CONCLUSION

  • Across three scientific collections, all four generated formats improve SPLADE-v3 even though it already performs contextual lexical expansion.The tested concept graph does not show the same benefit.Source: §VI · p. 7
  • The clearest interpretation is that generation contributes useful scientific vocabulary absent from the short query.The conclusion identifies this vocabulary as an addition to the original query, not a replacement.Source: §VI · p. 7
  • Generated vocabulary works best when the original query remains strongly represented.The conclusion contrasts this with replacing the user’s wording.Source: §VI · p. 7
Loading 2609.37911v1…