Source-linked AI summary
From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism
Jing Liu, Yongxing Qi, Muchen Jiang, Chengnan Hu, Qingqing Peng, Haoming Wang, Yuqing Wang, Yang Yu, Xu Zhang, Ting Wu
TL;DR
Similarity-based RAG retrieval can return keyword-matching documents without the needed information, especially in growing knowledge bases. The paper models terminal retrieval with a causal graph and derives a training-free attention-style re-ranker, which helps in keyword-stuffing settings but is not a general ranking improvement.
Problem
Similarity-based terminal retrieval can return documents sharing query keywords without containing the needed information as knowledge bases grow.
Method
The paper models terminal retrieval with a Reichenbach-grounded causal graph and re-scores candidates using query similarity to the weighted centroid of residual keyword set B.
Results
The method corrects similarity-retrieval failures in an enterprise case and diagnostic experiment, while public-benchmark evaluation delineates its applicability boundary.
Takeaways & Limitations
The method is a guard for keyword-stuffing regimes in growing proprietary knowledge bases and is orthogonal to modern retrievers and rerankers.
Takeaways & Limitations
The method is not a general-purpose ranking improvement and should use hybrid ranking or calibration gating outside clearly identified keyword-stuffing regimes.
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) grounds LLM generation on retrieved documents, but the standard terminal retrieval stage--dense-vector similarity, optionally followed by reranking--often returns documents that share keywords with the query without containing the needed information, a failure mode that grows with the knowledge base. We trace it to a conceptual gap: similarity captures only associational relations, whereas the documents that matter are linked to the query causally. We model the terminal retrieval stage with a causal graph grounded in Reichenbach's common cause principle: the keywords shared by the query and a retrieved document form a latent common cause A, and the document's residual keywords form a latent set B linking the document to the ideal output. Since a retrieved document is a collider (A -> d <- B), retrieval itself opens an associational path between the query and B, which licenses a training-free, attention-style re-scoring rule: the cosine similarity between the query embedding and the weighted centroid embedding of B. Unlike causality-enhanced RAG variants that model causal relations inside the knowledge content, our graph models the causal structure of the retrieval process itself. On a real 471-document enterprise knowledge base, the method promotes a relevant guideline from rank 6 to the top 3; on a controlled diagnostic corpus reproducing the keyword-stuffing regime, it improves the mean target rank from 2.88 to 1.25, while a trained cross-encoder reranker barely helps (2.63). Conversely, on three BEIR benchmarks the score underperforms the similarity baseline, delineating the applicability boundary: the method guards the keyword-stuffing regime of growing proprietary knowledge bases and complements neural rerankers; a corpus-level calibration gate selects the correct regime with >= 95% reliability. A fully local testbed demonstrates deployability.
1 Introduction
The paper targets keyword-based false positives at RAG’s terminal retrieval stage by modeling retrieval causally rather than associationally. It derives a training-free attention-style re-scoring method and evaluates its positioning and scope.
- Motivation: RAG retrieves documents for LLM prompts, but similarity-based terminal retrieval increasingly surfaces keyword-matching documents as knowledge bases grow.A conventional retriever ranked an automotive guideline above the genuinely relevant cross-border data-transfer guideline.
- Causal formulation: The paper formalizes terminal retrieval with a causal graph grounded in Reichenbach’s common cause principle.Shared query-document keywords form latent A, while residual document keywords form B carrying answer-bearing vocabulary.
- Method: The method derives a training-free re-scoring rule using cosine similarity between the query embedding and weighted centroid embedding of latent keyword set B.The paper reports negligible overhead over a conventional retrieval pipeline.
- Positioning: Unlike causality-enhanced RAG variants, the proposed graph models causal structure in the retrieval process rather than inside knowledge content.The paper positions this distinction against adaptive, corrective, graph-structured, and causality-enhanced RAG approaches.
- Evaluation: The evaluation spans a real 471-document proprietary knowledge base, a controlled diagnostic experiment, three BEIR benchmarks, and a fully local reproduction testbed.These settings are used to assess the method’s behavior and applicability boundary.
2 Related Work
Prior RAG retrieval work mainly improves associational scoring, query expansion, adaptivity, or knowledge-content graphs. The paper distinguishes its approach by modeling the causal structure of retrieval itself and deriving a training-free re-scoring rule.
- Associational retrieval: RAG retrieval research improves encoders and rerankers, but these methods optimize associational scores estimated from correlated features.The paper states that such approaches do not model why a document is relevant.
- Adaptive and corrective RAG: Adaptive and corrective methods improve robustness around retrieval while still scoring candidates through learned association.Examples include Self-RAG, FLARE, CRAG, and RA-DIT.
- Query expansion: Query-expansion methods address vocabulary mismatch by augmenting the query with relevance feedback, generated content, or hypothetical documents.The proposed construction instead distills each candidate document into residual answer-bearing vocabulary.
- Sparse retrieval: Term-centric sparse methods learn term weights from relevance supervision, whereas this paper constructs keyword sets from the causal structure of retrieval.The comparison includes DeepCT, doc2query/DocT5Query, and SPLADE.
- Causal and graph-structured RAG: Causality-enhanced RAG systems model causal relations inside knowledge content, while the proposed method models causality in the retrieval process itself.The paper claims this distinction is novel and avoids extra query-time LLM calls.
3 Method
The method refines conventional terminal retrieval by modeling shared and residual keyword sets causally, then re-ranking candidates using an attention-style score over answer-bearing residual vocabulary.
- 3.1 Problem Setting and Notation: Standard retrieval forms S1 with dense embeddings and optional reranking, but its goal is to re-rank candidates that share surface keywords without containing the needed information.The pipeline returns the top-k documents above a similarity threshold before refinement.
- 3.2 A Causal View of Terminal Retrieval: The causal graph defines A as shared query–document keywords and B as residual document keywords linked to the ideal output.A generates the query and document, while B generates the document and ideal answer; d and y are colliders.
- 3.2 A Causal View of Terminal Retrieval: Conditioning on a retrieved document opens the query–B path, making residual answer-bearing vocabulary associated with the query under causal Markov and faithfulness assumptions.Marginally q and B are independent, but q and B become dependent given d.
- 3.4 Scoring Rule and Algorithm: The implementation constructs A and B from weighted keyword sets, absorbing exact and semantically near query matches into A before representing B by a weighted centroid.The experiments use absorption threshold τ=0.6; exact matching can leave near-synonymous distractors in B and degrade the score.
- 3.4 Scoring Rule and Algorithm: The causal-attention score is the cosine similarity between the query embedding and B’s weighted centroid, demoting keyword-only documents and promoting residual answer-bearing vocabulary.Algorithm 1 appends this refinement to the conventional retrieval pipeline.
- 3.4 Scoring Rule and Algorithm: The refinement requires no training or fine-tuning, adds no query-time LLM calls, and has negligible overhead relative to initial retrieval over N ≫ k documents.Its per-query additions are k keyword extractions and k · b̄ term embeddings.
4 Experiments
The experiments evaluate the method on a proprietary deployment, a controlled keyword-stuffing corpus, public BEIR benchmarks, and a local reproduction testbed. Results show strong gains in the stuffing regime but negative performance on BEIR, motivating corpus-level gating.
- 4.1 Experimental Setup: The real deployment used a 471-document proprietary corpus and measured the rank of a known answer-bearing target document.The corpus contained 2.95 GB of local files, embedded with BGE-M3; retrieval performance was reported as target-document rank.
- 4.2 Case Study: Cross-Border Data Transfer Security: Rank 6 → top 3: the proposed method promoted the relevant cross-border data-transfer security guideline above keyword-matching automotive-data documents.The baseline ranked the guideline sixth, whereas the method placed it within the top three.
- 4.4 Controlled Diagnostic Experiment: 2.88 → 1.25 mean target rank: semantic absorption substantially improved retrieval across eight controlled keyword-stuffing themes.The stuffing distractors’ mean rank changed from 2.00 to 4.50; the hybrid achieved mean target rank 1.38 and never ranked below baseline.
- 4.4 Controlled Diagnostic Experiment: 2.63 mean target rank: the trained cross-encoder barely improved over the 2.88 baseline and remained vulnerable to keyword-stuffed distractors.The cross-encoder left stuffing distractors at the top, while exact-match construction was inconsistent on near-synonymous distractor vocabulary.
- 4.5 Public-Benchmark Evaluation: Boundary of Applicability: On three BEIR benchmarks, the causal-attention score underperformed the similarity baseline, while the trained cross-encoder was strongest.The fallback was indistinguishable from the plain score, and the hybrid recovered only part of the gap.
- 4.6 Regime Gating for Deployment: 30 probes kept the re-ranker disabled on BEIR with ≥95% reliability, while 4 probes enabled it on the diagnostic corpus with 100% reliability.The gated system was indistinguishable from baseline on factoid corpora and retained the 2.88 → 1.25 gain in the stuffing regime; calibration requires representative labeled probes and periodic recalibration for drift.
5 Discussion and Limitations
The method’s benefits are concentrated in keyword-stuffing settings, while deployment depends on calibration and several design choices remain open.
- The method helps in keyword-stuffing regimes but hurts on factoid-style public benchmarks where relevant documents share the query’s vocabulary.The authors recommend a hybrid configuration or calibration gate for deployment.
- Deployment should use a representative labeled probe set when relying on the calibration gate to identify the corpus regime.The gate’s reliability depends on the probe set representing the deployment corpus.
- The quality of latent keyword sets A and B depends on the keyword extractor kw(·).Weighted, learned, or LLM-based keyword representations could refine the score.
- Combining the causal score with similarity, shared-keyword counts, and absorption thresholds needs fuller ablation.The paper identifies interpolation weights, thresholds, and τ as unresolved design choices.
- Extending the causal treatment from terminal retrieval to generation remains an open direction.One example is attributing generated claims back to causal keyword sets.
6 Conclusion
The paper reframes terminal RAG retrieval causally and derives a training-free attention-style re-scoring rule. Results support its use in keyword-stuffing regimes, but public benchmarks define a clear boundary against general ranking improvement.
- The method models retrieval with a causal graph in which collider opening links the query to residual keyword set B.This graph is grounded in Reichenbach’s common cause principle.
- It re-scores candidates using cosine similarity between the query embedding and B’s weighted centroid embedding.The rule is training-free and attention-style.
- The method corrects a similarity-retrieval failure on a growing enterprise knowledge base and reproduces the effect in a controlled diagnostic experiment.The experiments distinguish answer-bearing documents from keyword-matching documents.
- Public-benchmark evaluation shows that the method guards the keyword-stuffing regime rather than providing a general ranking improvement.The approach is presented as complementary to modern retrievers, rerankers, query expansion, and graph-based RAG.
- The fully local Qwen3-4B/BGE-M3 testbed demonstrates practical deployability.