Source-linked AI summary
Tracing and Taming Hallucinated Terms via Subsequence-Association Filtering and Breadth-Aware Selection for LLM Query Expansion
Amit Kumar Jaiswal, Sagar Uprety, Prayag Tiwari
TL;DR
LLM query expansion can introduce unsupported terms, while existing remedies lack a mechanistic account of hallucinations and a principled criterion for selecting terms. SAFE-QE scores candidates using corpus-conditioned reproducibility and grounding, then selects a broad, consistent, stable set. Across collections and retrievers, it matches expansion baselines on nDCG@10 while injecting fewer unsupported terms than Query2doc and HyDE.
Problem
Existing query-expansion remedies lack a mechanistic account of why hallucinated terms appear and a principled criterion for balancing coverage against faithfulness.
Source: §1 · p. 2Method
SAFE-QE scores candidate terms by corpus-conditioned reproducibility and grounding, then applies breadth-aware selection and a stability filter.
Source: §1 · p. 2, §3.1 · p. 4, §3.4 · p. 6Results
Across six collections and three retrievers, SAFE-QE matches generative and corpus-steered expansion on nDCG@10 while injecting 85% fewer unsupported terms than Query2doc and 93% fewer than HyDE.
Source: §6 · p. 12Takeaways & Limitations
SAFE-QE provides comparable ranking quality with fewer unsupported injected terms than Query2doc and HyDE.
Source: §6 · p. 12SAFE-QE’s grounding threshold should be calibrated per retriever; under SPLADE the unsupported-term rate rises relative to BM25, and this calibration is left to future work.
Source: §5.2 · p. 11
Abstract
from arXiv · showhide
Large language models (LLMs) are now a standard source of query expansion terms but they hallucinate. They inject non existent entities and off-topic terms that add ranking noise and can hurt strong retrievers. Existing remedies such as corpus grounding, mutual verification and preference alignment treat hallucinated terms as an undifferentiated nuisance to be filtered by agreement heuristics, without a mechanistic account of why a given term appears or a principled criterion for how many terms to keep. We propose Subsequence-Association Filtered, breadth-aware query expansion (SAFE-QE), a training-free, retriever agnostic method that draws on the subsequence-association account of hallucination. We score each candidate expansion term by its corpus conditioned cross-context reproducibility, terms that reappear under randomised query contexts yet lack support in the pseudo-relevant set are flagged as products of dominant hallucinatory associations and down-weighted. Drawing on the characterisation of language generation as a trade-off between consistency (no hallucination) and breadth (coverage), we cast term selection as constrained coverage maximisation over the pseudo-relevant set, keeping a set that is approximately broad but consistent and stable across resampling. We also introduce the Hallucinated Expansion Term Rate, a query side faithfulness metric that counts injected expansion terms no relevant document supports. We evaluate our approach on benchmarked TREC and BEIR collections. Our SAFE-QE is statistically indistinguishable from generative and corpus-steered expansion on nDCG@10 while injecting fewer unsupported terms 4.3 per query against 27.8 for Query2doc and 57.5 for HyDE and on a strong learned sparse retriever, where every expansion method regresses. Faithfulness is therefore available at no measurable cost in ranking quality.
1 Introduction
LLM query expansion can introduce unsupported terms that add ranking noise, while existing remedies lack a mechanistic account of hallucinations and a principled way to balance coverage and faithfulness. SAFE-QE draws on subsequence associations and the consistency–breadth trade-off to filter terms and select expansions.
1 Introduction
- Existing expansion remedies ground, verify, filter, or align generated terms, but leave the generative mechanism and coverage–faithfulness selection trade-off unaddressed.Prior studies also report expansion noise harming strong retrievers and coverage narrowing for ambiguous queries.Source: §1 · p. 2
- The paper’s schematic links dominant hallucinatory associations to context-free, ungrounded expansion terms that inject ranking false positives and hurt stronger retrievers.The introduction also identifies unfamiliar-query and ambiguous-query failure modes.Source: §1 · p. 2, Figure 1 · p. 3
- SAFE-QE frames hallucination through dominant subsequence associations and term selection through the consistency–breadth–stability characterization of language generation.It combines a corpus-conditioned reproducibility score with breadth-aware, stability-checked selection, and introduces HETR to measure unsupported injected terms.Source: §1 · pp. 2–3
2 Related Work
Prior LLM query expansion work generates pseudo-documents or terms, or grounds generation in corpus feedback. Related research documents expansion failures and theoretical accounts of hallucination and generation breadth that motivate SAFE-QE.
LLM based query expansion
- LLM query expansion includes pseudo-document generation, term generation, and corpus-grounded or feedback-fused methods, with representative systems such as Query2doc, HyDE, CSQE, and ThinkQE.The related-work survey also describes MuGI, GRF, LameR [12], and document-side Doc2Query.Source: §2 · p. 3
- Document-side Doc2Query filters generated queries heuristically before indexing, but does not use a generative-mechanism account.It is presented as the closest prior instance of filtering generated text.Source: §2 · p. 4
Hallucination in expansion, and why it matters
- Prior studies report that expansions can inject noise or non-existent entities, while benefits may depend on generated text containing gold-entailed content.The paper distinguishes its use of memorized associations to separate faithful from hallucinatory associations from Yoon et al.’s diagnosis of knowledge leakage as a confound.Source: §2 · p. 4
Theoretical foundations
- The cited theories connect dominant hallucinatory subsequence associations with a generation trade-off among consistency, breadth, and stability.The paper states that it is the first, to its knowledge, to port either result into information retrieval.Source: §2 · p. 4
3 Our Method
SAFE-QE seeks a broad but consistent subset of generated candidate terms, using corpus-conditioned reproducibility and grounding to identify candidates, then coverage selection and resampling stability to form the expansion.
3.1 Problem Formulation
- The method targets a subset of candidate terms that covers relevant material while excluding hallucinated terms, aiming for approximate breadth without requiring every valid term.Exact breadth is described as as hard as identification and needlessly risky.Source: §3.1 · p. 4
3.2 Candidate generation
- An open instruction-tuned LLM produces N independent samples, and the fraction containing each term serves as a proxy for generator stability.The resulting multiset supplies candidate terms and their sample frequencies.Source: §3.2 · p. 5
3.3 Corpus conditioned reproducibility score
- SAFE-QE measures how often a term reappears across randomized query contexts and how strongly it is supported by the retriever’s top-k pseudo-relevant documents.High cross-context reproducibility signals an association less dependent on query intent; terms that are reproducible yet unsupported are penalized.Source: §3.3 · p. 5
3.4 Breadth-aware, consistency constrained selection
- Selection maximizes coverage of pseudo-relevant documents under a grounding constraint, using a greedy budget-B maximum-coverage objective with a (1−1/e) guarantee.The coverage objective encodes approximate breadth while the grounding constraint enforces consistency; exact breadth is deliberately forgone for a broad, safe subset.Source: §3.4 · pp. 5–6
3.5 Stability filter and expansion
- A stability filter removes selected terms appearing in too few resamples; surviving terms are fused with RM3-style expansion for sparse retrieval or appended before dense encoding.Volatile terms are described as empirically more often hallucinated.Source: §3.5 · p. 6, §3.4 · p. 6
4 Hallucination Metric for Query Expansion
The paper introduces HETR to measure whether injected query-expansion terms are supported by judged relevant documents. It reports this faithfulness metric alongside retrieval effectiveness.
4 Hallucination Metric for Query Expansion
- Retrieval metrics measure ranking effectiveness, not whether injected expansion terms are faithful; a hallucinated term may still help through lucky lexical overlap.HETR adapts HALoGEN’s atomic verification protocol to query expansion.Source: §4 · p. 6
- HETR is the fraction of injected expansion terms unsupported by any relevant document, assessed with a high-precision verifier using lexical matching or entailment.The paper reports HETR alongside effectiveness, defining a useful method as one that lowers HETR without lowering nDCG@10.Source: §4 · p. 7
5 Experimental Studies
Experiments compare SAFE-QE with unexpanded queries, generative and corpus-steered baselines across six collections and three retrievers, using nDCG@10 for effectiveness and HETR for faithfulness. SAFE-QE reduces unsupported expansion terms while maintaining comparable ranking quality, with results dependent on the retriever and expansion-generator scale.
5 Experimental Studies
- The evaluation uses TREC Deep Learning 2019 and 2020 plus four BEIR collections, with BM25, SPLADE, and Contriever as first-stage retrievers.Four open instruction-tuned LLMs generate candidates; reported results use Gemma-2-9B, and the full generation is used for RQ4.Source: §5 · p. 7
5.1 Metrics
- HETR is the fraction of injected terms unsupported by judged relevant documents, while hallucinated-term load is the expected absolute number of unsupported terms per query.Table 3 reports macro-averaged HETR as a micro-rate over injected terms; its load is HETR × |E(q)|.Source: §5.1 · pp. 7–8, Table 3 · p. 10
5.2 Result
- SAFE-QE is statistically indistinguishable from HyDE, CSQE, and MILL on nDCG@10 (all |∆| ≤0.004, p > 0.5) and within 0.008 of Query2doc, with a tenth-of-a-rank-position gap.The Query2doc gap reaches significance (p = 0.021); tests are pooled over four BEIR collections with Gemma-2-9B.Source: §5.2 · p. 8, Table 2 · p. 9
- 4.30 unsupported terms per query for SAFE-QE compares with 27.83 for Query2doc and 57.47 for HyDE, reductions of 85% and 93%, respectively.Against corpus-steered baselines, the reduction is 45% over CSQE and 43% over MILL.Source: §5.2 · p. 8, Table 3 · p. 10
- Table 3 reports SAFE-QE's smallest expansion footprint as 10.3 terms per query, versus 43.0 for Query2doc and 87.2 for HyDE.The table macro-averages across six collections and three retrievers; SAFE-QE uses two sampled generations plus perturbed prompts, incurring generation cost.Source: §5.2 · p. 8, Table 3 · p. 10
- SAFE-QE has the lowest HETR on all five comparable Contriever collections, but is lowest on one of six BM25 collections and two of six SPLADE collections.Under SPLADE, its pseudo-relevant set admits more candidates and its HETR rises; the authors say the grounding threshold should be calibrated per retriever.Source: §5.2 · pp. 10–11
- HETR reductions pooled over BEIR are significant at p < 0.001: -25.9% against HyDE, -24% against Query2doc, -15.1% against MILL, and -11.2% against CSQE.HETR measures unsupported terms as a rate; hallucinated-term load additionally captures how many terms are injected.Source: §5.1 · p. 8, §5.2 · p. 11
6 Conclusion
SAFE-QE matches generative and corpus-steered expansion on nDCG@10 while injecting fewer unsupported terms across six collections and three retrievers. The conclusion also identifies retriever calibration and finite-sample analysis as open issues.
6 Conclusion
- Across six collections and three retrievers, SAFE-QE matches generative and corpus-steered expansion on nDCG@10 while injecting 85% fewer unsupported terms than Query2doc, 93% fewer than HyDE, and 43–45% fewer than corpus-steered baselines.The conclusion characterizes these reductions as 43–45% fewer than the strongest prior mitigation baselines.Source: §6 · p. 12
- On a strong learned sparse retriever, every expansion method regresses, but SAFE-QE halves the regression.Source: §6 · p. 12
- With a 0.5B generator, SAFE-QE is already more faithful than HyDE with a 9B generator, and its faithfulness benefit is attributed to the filter rather than generator size.Source: §6 · p. 12
- SAFE-QE traces and down-weights hallucinatory expansion terms, then selects a set that is broad, consistent, and stable across resampling, following subsequence-association and consistency-breadth-stability accounts.Source: §6 · p. 12
- The paper introduces HETR as a query-side hallucination metric and argues that the rate should be paired with hallucinated-term load, the unsupported-term count the retriever receives.Source: §6 · p. 12
- The grounding threshold τg is fixed and should be calibrated per retriever; component ablation and finite-sample analysis of the breadth-consistency trade-off remain open.Source: §6 · p. 12