PaperlayerView exact revision on arXiv

Source-linked AI summary

Query Expansion Is More Than Generation: Improving Dense Retrieval through Better Integration

Siyuan Sun, Mihai Surdeanu

arXiv:2608.25521v1cs.IR

TL;DR

LLM-generated query expansions can hurt frozen dense retrieval, and the paper identifies integration—not only generation—as an underexplored factor. AnchorQE separately encodes queries and expansions, while SC-AnchorQE estimates expansion trust online without labels. Across TREC-DL, LoTTE, and BEIR, the approach improves retrieval over conventional baselines and fixed weighting.

  • Problem

    LLM expansions are not reliably beneficial with frozen dense retrievers, while prior work has insufficiently separated generated text from how it is integrated.

  • Method

    AnchorQE separately encodes the original query and expansion and interpolates their normalized vectors; SC-AnchorQE estimates the factor from a short unlabeled stream prefix.

  • Results

    AnchorQE exceeds the dense-retrieval baseline and conventional procedure in all 20 controlled comparisons, with relative gains up to 12.89%.

  • Takeaways & Limitations

    Integration is a critical part of query expansion, and explicit query anchoring can improve retrieval while preserving one-vector serving over the unchanged index.

  • Takeaways & Limitations

    SC-AnchorQE assumes a short unlabeled prefix represents a stable future stream, so rapid distribution shifts can make its factor stale.

Abstract

from arXiv · show

Large language models (LLMs) can generate query expansions without task-specific training, yet the same expansions often make a frozen dense retriever worse. We identify an underexplored factor: prior work has often focused on what text is generated, while how generated text is incorporated into dense retrievers has received less systematic attention. By holding generated expansions fixed, we show that performance degradation can often be attributed to the integration method itself. We introduce AnchorQE, a training-free method that separately encodes the original query and its expansion before interpolating them. The interpolation factor is estimated using an unsupervised online strategy that operates over a small part of the unlabeled test stream. Intuitively, our strategy assigns high expansion trust only when expansions are both retrieval-strong and consistent with the original query's retrieved evidence. We show that AnchorQE improves retrieval effectiveness by up to 12.89% when compared to widely-used expansion-only or text-level concatenation baselines across TREC-DL, LoTTE, and BEIR. Further, we show that our online strategy to estimate the interpolation factor outperforms a fixed weight tuned on a development partition by up to 3.81%.

1 Introduction

The paper argues that dense query expansion often fails because generated text is integrated poorly, not necessarily because it is generated poorly. AnchorQE separates query and expansion representations, while SC-AnchorQE estimates expansion trust online without labels.

  • Motivation: Conventional integration can reduce retrieval below the dense-retrieval baseline even when LLMs generate expansions without task-specific training.The controlled setup changes integration while holding expansions fixed, so degradation cannot be attributed to generation quality alone.
  • AnchorQE: AnchorQE separately encodes the original query and expansion, then interpolates their normalized vectors with an explicit trust factor.The original query remains an anchor, and the method preserves one-vector retrieval over the unchanged index.
  • SC-AnchorQE: SC-AnchorQE estimates the interpolation factor from the first eight unlabeled queries and freezes it for future queries.It assigns higher expansion trust when the expansion retrieves strongly and agrees with evidence retrieved by the original query.
  • Experimental design: The fixed-expansion protocol isolates integration across four LLM generation strategies, showing that QE conclusions can change solely with the integration method.The study compares routes using the same query, saved expansion, and frozen retriever.
  • Contributions: AnchorQE is training-free, index-preserving, and designed to control expansion trust while retaining one-vector serving.The framework is presented alongside equivalence to homogeneous weighted score fusion and a worst-case angular influence bound.

2 Previous Work and Preliminaries

Prior work commonly varies generated content and integration together, confounding their effects. This paper frames integration as a separate decision within dense retrieval and contrasts text-level, score-fusion, and controlled vector-level approaches.

  • Dense Retrieval: A dual-encoder dense retriever maps queries and documents to vectors and ranks documents by their dot product.The query and document encoders may share parameters, but their roles can still differ through instructions or prefixes.
  • Query Expansion: LLM query expansion generates hypothetical documents, pseudo-documents, keywords, subqueries, or reasoning-derived terms before integration into retrieval.The paper represents generation as z = G(q) and integration as v = I(q,z).
  • Controlled Comparison: Existing systems often change generation and integration together, making their separate effects difficult to identify.The fixed-expansion protocol reuses each output across methods and changes only the integration function.
  • Score Fusion: Separate score fusion exposes weighting but conventionally requires multiple index requests and merged candidate lists.AnchorQE recovers the homogeneous linear score objective in one vector when streams share the same document space.
  • Related Work: MuGI and QuDAR address broader integration settings, whereas AnchorQE studies fixed expansions entering one frozen dense retriever space.QuDAR combines heterogeneous sparse–dense and original–expanded ranked lists, while MuGI couples generation multiplicity, retriever choice, and integration.

3 AnchorQE

AnchorQE separately encodes the original query and expansions, then interpolates their normalized vectors so expansion influence is explicit and controllable. SC-AnchorQE estimates this influence from unlabeled retrieval signals, while the framework preserves one-vector serving and has formal ranking and influence properties.

  • 3.1 Definition: AnchorQE separately encodes and normalizes the original query and each expansion, then interpolates the query vector with their weighted mixture.With no expansion or α=0, it returns the original query, while the document encoder and index remain unchanged.
  • 3.1 Definition: The interpolation factor α controls total expansion influence independently of expansion length, while disagreement among expansions weakens their joint correction.The original query remains explicit in the deployed vector.
  • 3.2 Unsupervised Online Factor Estimation: SC-AnchorQE estimates and freezes α from the first eight unlabeled queries, using expansion retrieval strength and agreement with original-query evidence.It uses no training, relevance judgments, future-query effectiveness, fitted parameters, or thresholds.
  • 3.3 Equivalence to Linear Score Fusion: AnchorQE has the same exact document ordering as weighted CombSUM when all streams share a dense space and use raw dot products.This is a ranking equivalence, not equality of calibrated scores, and excludes mixed sparse–dense streams, stream-specific normalization, nonlinear fusion, and independently truncated ANN lists.
  • 3.3 Equivalence to Linear Score Fusion: AnchorQE issues one retrieval request, whereas weighted CombSUM uses N+1 retrieval requests.All representations are compiled into a single query vector for the AnchorQE request.
  • 3.4 Why the Anchor Controls Expansion Influence: At α=.10, no expansion can change the retrieval direction by more than 6.38°, but a wrong expansion can still cross a small ranking margin.The angular bound controls influence rather than relevance; at α=.10, an opposing expansion margin must exceed the query margin by ninefold to reverse a document pair.

4 Experimental Setup

The experiments reuse fixed LLM-generated expansions across integration methods and evaluate frozen retrievers on broad benchmark streams. The setup includes cross-generator and cross-retriever transfer, development-tuned and online-calibrated baselines, and strict no-lookahead evaluation.

  • 4.1 Model Selection: Qwen3-8B generates four expansion types—HyDE, Query2Doc, Q2E keywords, and chain-of-thought-derived terms—whose saved outputs are reused across integration methods.Generator transfer uses Qwen3-1.7B and Llama-3.1-8B-Instruct.
  • 4.1 Model Selection: The study evaluates BGE-large-en-v1.5, GTE large-en-v1.5, and Qwen3-Embedding-0.6B with frozen document embeddings and indexes.The three generators and three retrievers form nine transfer configurations.
  • 4.2 Benchmark Streams: The benchmark suite contains 18 streams spanning TREC-DL, LoTTE, and BEIR, with nDCG@10 for TREC-DL and BEIR and Success@5 for LoTTE.BEIR-14 covers 14 heterogeneous datasets, while LoTTE covers five domains across search and forum queries.
  • 4.3 Baselines and Evaluation Protocol: The published-integration baseline reproduces each source pipeline, while a stronger text baseline selects among 72 concatenation and re-encoding recipes on a separate 6,980-query MS MARCO development set.The same development set selects α=.15 for fixed-factor AnchorQE.
  • 4.3 Baselines and Evaluation Protocol: SC-AnchorQE uses the first eight unlabeled queries of each stream for calibration, removes them from evaluation, and freezes α for the remaining suffix.The strict no-lookahead split scores all methods on the same future suffix.
  • 4.4 Statistical Analysis: Each QE strategy is evaluated independently with paired bootstrap analyses that preserve LoTTE and BEIR nested macro structures.The study uses 10,000 within-collection paired bootstrap replicates and 100,000 collection-level cluster-bootstrap replicates.

5 Main Results

Controlled evaluations show that integration method materially changes dense-retrieval outcomes even when expansions are fixed. SC-AnchorQE improves retrieval without relevance labels and preserves efficient one-vector serving, while generated-evidence quality remains a boundary.

  • 5.2 SC-AnchorQE: Primary Online Results: SC-AnchorQE estimates the interpolation factor from eight unlabeled queries, then freezes it for future queries without training or relevance judgments.Its trust signal combines expansion retrieval strength with agreement with original-query evidence.
  • 5.1 Integration Method Matters: The fixed-expansion comparison isolates integration as the cause of different QE outcomes, with conventional procedures below the DR baseline in 16/20 comparisons.All rows use identical Qwen3-8B expansions and a frozen BGE retriever.
  • 5.1 Integration Method Matters: AnchorQE exceeds the DR baseline and conventional procedure in all 20 comparisons, with gains from .46% to 12.89%.The largest gain is 12.89% for CoT terms on BEIR-14.
  • 5.2 SC-AnchorQE: Primary Online Results: SC-AnchorQE improves over the DR baseline in 19/20 strategy–group comparisons, with positive query-level confidence intervals in 18/20.All four collection-level confidence intervals exclude zero.
  • 5.3 Serving Cost and Linear Fusion: AnchorQE matches weighted CombSUM rankings while compiling the linear objective into one index request instead of merging nine candidate lists.The equivalence holds in a shared dense space; anchored max is slightly stronger but cannot use one linear query vector.
  • 5.5 Stability across Models: Across 45 generator–retriever–benchmark comparisons, SC-AnchorQE beats the DR baseline in 44 and fixed AnchorQE in 41.Performance remains above the DR baseline in a majority of comparisons with the smaller Qwen3-1.7B generator.
  • 5.5 Stability across Models: Generated-evidence quality limits attainable gains even though integration controls expansion influence.The smaller generator’s lower coverage exposes this boundary.

6 Analysis

The analysis favors conservative stream-level calibration: combining expansion strength and agreement provides a deployable balance of effectiveness, trust, and one-request serving. Query anchoring also outperforms the compared non-LLM QuDAR fusion routes.

  • 6.1 Why Conjunctive Calibration?: The conjunctive product has the broadest improvement over fixed AnchorQE among stream-level rules, improving 17/20 comparisons with the largest median gain.Less-conservative combination functions lose coverage, while per-query factors require two probe retrievals for every future query.
  • 6.1 Why Conjunctive Calibration?: The stream product offers the strongest deployable balance of effectiveness, conservative trust, and one-request serving after an eight-query prefix.Embedding and ranked-list agreement are useful but less consistent.
  • 6.2 Qualitative Analysis: How the Query Anchor Changes Retrieval: Anchoring can block wrong answers or add useful details, but cannot repair irrelevant generated text or fully exploit an unusually useful expansion.These examples delimit both the protective and restrictive effects of limiting expansion movement.
  • 6.2 Qualitative Analysis: How the Query Anchor Changes Retrieval: SC-AnchorQE outperforms QuDAR-Simple and QuDAR-Confidence in all 20 QE-strategy–benchmark comparisons.The comparison uses reproduced non-LLM QuDAR routes.

7 Conclusion

AnchorQE improves dense retrieval by separating the original query from generated expansions until vector integration, where explicit trust control limits their influence. SC-AnchorQE estimates this control without supervision while preserving one-vector retrieval over the unchanged index.

  • AnchorQE separately encodes the original query and expansion before interpolating them with an explicit factor controlling expansion trust.The framework keeps both inputs separate until the vector stage.
  • SC-AnchorQE estimates the interpolation factor without supervision while preserving single-vector retrieval over the original, unchanged index.
  • AnchorQE works across generators, retrievers, expansion strategies, and datasets while limiting how generated text can pull retrieval away from the original query.
  • AnchorQE outperforms conventional integration methods and pure dense retrieval without query expansion in nearly all evaluated settings.

8 Limitations

SC-AnchorQE relies on a short unlabeled prefix representing the future stream, so distribution shifts can make its shared calibration factor stale. The evaluation is also a no-lookahead simulation rather than temporal traffic, and broader deployment studies remain necessary.

  • Rapid distribution shifts can make SC-AnchorQE’s stream-level factor stale when the short unlabeled prefix does not represent the future stream.
  • One shared interpolation factor cannot handle every unusually strong or misleading expansion.
  • The benchmark protocol is a strict no-lookahead simulation rather than temporal traffic, so broader deployment studies remain necessary.
  • Table 9 compares unsupervised factor rules against fixed AnchorQE at α= .15 across 20 benchmark-group comparisons.

A Additional Protocol Details

Additional protocol analyses examine BEIR-14 coverage, expansion encoding roles, generation-length regimes, prefix-size sensitivity, and reproduction details. These checks describe how the evaluation varies while retaining frozen dense indexes and controlled comparisons.

  • AnchorQE improves over published integration in 53/56 BEIR-14 QE-strategy–collection comparisons and over development-tuned text re-encoding in all 56.
  • Every BEIR-14 collection improves for at least three of four QE strategies, indicating gains are not concentrated in a few collections.
  • Encoding expansions with the query encoder is better in 6/10 QE-strategy–benchmark comparisons in the reported ablation.
  • SC-AnchorQE remains stable across tested prefix sizes B= 8, 16, 32, with a largest reporting-group spread of only .0027.
  • The protocol reproduces QuDAR using released code while keeping dense streams on the same frozen BGE indexes as AnchorQE.

B Exact Generation Prompts

The appendix specifies exact prompts for HyDE, Query2Doc, Q2E, and CoT terms, alongside table descriptions for diagnostic, ablation, generation-regime, and calibration-prefix evaluations.

  • The appendix tables cover BEIR-14 diagnostics, encoder ablations, generation-length regimes, and calibration-prefix macros.
  • HyDE prompts the LLM to output a brief, factual, professional hypothetical document answering or containing relevant information.
  • Query2Doc prompts one concise pseudo-document containing entities, synonyms, and answer facets without narrow unsupported facts.
  • Q2E prompts compact expansion terms including related terms, synonyms, aliases, acronyms, and key entities.
  • CoT terms internally decomposes intent, entities, constraints, and answer facets, then outputs only final expansion terms.

C Development Tuning of the Text Re-encoding Baseline

The development-tuned text baseline systematically searches how the original query and saved expansion are composed before frozen query encoding. Across held-out test collections, the selected recipes favor short expansions and avoid query repetition, with order depending on expansion type.

  • Candidate-grid construction: 72 candidate recipes combine four expansion lengths, three separators, two text orders, and three query-repetition counts.The grid is defined as 4 × 3 × 2 × 3 = 72.
  • Candidate-grid construction: The grid varies expansion truncation, separators, text order, and repetition of the original query within the encoder’s 512-token limit.Expansion lengths include the original-query length, 32 tokens, 64 tokens, or the complete saved expansion.
  • Development selection: Each strategy selects its recipe independently on the 6,980-query MS MARCO passage development set using nDCG@10, without TREC-DL, LoTTE, or BEIR judgments.This keeps test-collection results strictly held out from recipe selection.
  • Selected recipes: Selected recipes consistently retain short expansions and avoid repeating the query, while preferred order varies by expansion type.HyDE and Query2Doc place the query first; Q2E and CoT place the expansion first.
  • Baseline strength: The tuned text-composition baseline is stronger than directly adopting a single published concatenation format while preserving held-out test evaluation.The baseline changes only the text-composition recipe before frozen query encoding.
Loading 2608.25521v1…