PaperlayerView exact revision on arXiv

Source-linked AI summary

Xetrieval: Mechanistically Explaining Dense Retrieval

Zhixin Cai, Jun Bai, Yang Liu, Jiaqi Li, Yichi Zhang, Taichuan Li, Zhuofan Chen, Zixia Jia, Zilong Zheng, Wenge Rong

arXiv:2605.29507v1cs.AIcs.IR

TL;DR

Dense retrieval decisions are difficult to explain because high-dimensional embeddings obscure the factors behind relevance scores. Xetrieval internalizes reasoning in embedding space and decomposes embeddings into interpretable features, producing coherent features and intervention evidence that they capture retrieval mechanisms.

  • Problem

    High-dimensional embeddings make dense-retrieval relevance decisions opaque, limiting understanding in applications requiring accountability, diagnosis, and systematic error analysis.

  • Method

    Xetrieval internalizes reasoning-oriented information in embeddings, then decomposes query and document representations into sparse features whose shared activations explain relevance.

  • Results

    Across multiple retrievers and benchmarks, Xetrieval produces coherent, human-interpretable features, and interventions change retrieval outcomes, evidencing feature-level mechanisms.

  • Takeaways & Limitations

    Xetrieval traces dense-retrieval decisions to latent query-document factors that are locally grounded in similarity computation and globally useful for retrieval behavior.

  • Takeaways & Limitations

    Xetrieval is confined to sentence-embedding outputs and relies on SAE, limiting access to internal network mechanisms and fidelity relative to stronger interpretability frameworks.

Abstract

from arXiv · show

Explaining why dense retrievers assign high relevance scores remains challenging because retrieval decisions are made through opaque high-dimensional embeddings. Existing explanations often focus on surface signals, such as lexical matches, token alignments, or post-hoc textual rationales, and thus provide limited insight into the latent factors that shape dense retrieval behavior at the embedding level. We propose \textit{Xetrieval}, an embedding-level mechanistic framework for explaining dense retrieval. \textit{Xetrieval} first introduces a lightweight reasoning internalizer that approximates Chain-of-Thought reasoning directly in the embedding space with a single forward pass, enriching sentence embeddings with reasoning-oriented information while avoiding expensive autoregressive generation. It then decomposes these reasoning-enhanced embeddings into sparse, human-interpretable features, each associated with a coherent natural language description. By aggregating sparse feature overlaps across multiple document-side views, \textit{Xetrieval} provides feature-level explanations of individual retrieval decisions. Experiments on diverse retrievers and benchmarks show that \textit{Xetrieval} uncovers coherent interpretable features, yields stronger pair-level intervention effects, and supports task-level feature steering. The project page and source code are available at https://hihiczx.github.io/Xetrieval .

1 Introduction

Dense retrieval’s high-dimensional embeddings obscure why documents are retrieved, limiting accountability and error analysis. Xetrieval addresses this opacity by decomposing reasoning-enriched embeddings into interpretable shared features that explain and influence retrieval decisions.

  • Dense retrieval’s opaque high-dimensional relevance computation makes individual retrieval rationales difficult to understand and limits accountability, diagnosis, and systematic error analysis.
  • Prior explanations rely on lexical or token-level evidence, interpretable embedding spaces, or post-hoc analyses of fixed encoders.
  • Xetrieval decomposes query and document embeddings into sparse, interpretable features with coherent natural-language descriptions.
  • For each retrieval decision, Xetrieval attributes the dense relevance score to features jointly activated by the query and retrieved document.
  • Xetrieval enriches sentence embeddings with LLM-generated Chain-of-Thought reasoning that encodes query intent, latent constraints, and evidence requirements.
  • Experiments across retrievers and benchmarks find coherent sparse features whose interventions change retrieval outcomes, supporting feature-level mechanisms underlying dense retrieval decisions.

2 The Xetrieval Framework

Xetrieval explains dense retrieval through a reasoning internalizer and a mechanistic explainer that operate at the embedding level. It enriches embeddings in one step, decomposes them into sparse interpretable features, and attributes relevance to features jointly activated by queries and documents.

  • Framework Overview: Xetrieval combines a reasoning internalizer with a mechanistic explainer to produce embedding-level explanations for dense retrieval.The framework enriches representations with query intent, latent constraints, and evidence requirements before sparse feature decomposition.
  • Mechanistic Explainer: For each query-document pair, Xetrieval attributes relevance to shared sparse features and presents their associated natural-language hypotheses.These feature-level explanations provide model-internal accounts of retrieval decisions and support controllable interventions on retrieval behavior.
  • Reasoning Internalizer: The reasoning internalizer injects reasoning features into sentence embeddings in a single forward step.It approximates LLM-generated Chain-of-Thought reasoning directly in embedding space, avoiding autoregressive generation during retrieval and explanation.
  • Reasoning Internalizer: Three internalizers capture complementary aspects: SUMMARY models core semantics, PURPOSE models retrieval intent and utility, and QA models evidence needs.Each aspect-specific internalizer is implemented as a one-hidden-layer MLP with tanh activation.
  • Mechanistic Explainer: The mechanistic explainer decomposes reasoning-enhanced embeddings into sparse, human-interpretable features for explaining query-document relevance.It uses an SAE to identify latent features activated in both queries and retrieved documents.

3 Experiments

Experiments evaluate Xetrieval across diverse retrieval benchmarks and dense retrievers, testing reasoning internalization, sparse-feature quality, retrieval preservation, and mechanistic validity. Results support TopK-SAE with k = 256 and show that Xetrieval produces efficient, interpretable, retrieval-relevant features with stronger intervention effects.

  • Experimental Setup: Xetrieval is evaluated on 7 retrieval benchmarks spanning reasoning-intensive retrieval, open-domain QA, dialogue, news, argument, and robust ad-hoc retrieval, using NDCG@10.The benchmarks are BRIGHT, NQ, MuTual, TREC-NEWS, Signal-1M, ArguAna, and Robust04.
  • Experimental Setup: The study covers eight dense retrievers across multiple model families and scales, while six LLMs generate aspect-specific reasoning texts for supervision.The retrievers range from approximately 0.1B to 0.6B-plus parameters, including Qwen3-Embedding-4B.
  • SAE Evaluation: Increasing L0 improves reconstruction quality and retrieval retention but generally weakens mono-semanticity, whereas stronger sparsity improves selectivity while harming reconstruction and retention.This establishes a three-way trade-off among reconstruction error, mono-semanticity, and retrieval retention.
  • SAE Evaluation: TopK provides the most favorable trade-off, and at L0 = 256 it maintains strong mono-semanticity, near-baseline retrieval retention, and competitive reconstruction error.The study therefore adopts TopK-SAE with k = 256 as the mechanistic explainer backbone.
  • Retrieval Validation: The reasoning internalizer consistently improves base retrieval in most settings and recovers part of the CoT reasoner’s gain while preserving retrieval-relevant reasoning signals.For stronger Qwen3-Embedding backbones, additional reasoning views still improve BRIGHT, but average gains are smaller because baseline performance is already strong.
  • Intervention Validation: Amplifying Xetrieval’s key features improves retrieval, suppressing them degrades it, and their steering effects exceed those of direct decomposition using raw SAE features.The evaluation intervenes on BRIGHT, ArguAna, and NQ, while non-key feature steering produces smaller and less consistent changes.

4 Related Work

Dense retrieval has progressed through larger models, improved training and data, specialized and LLM-driven embeddings, and reasoning-intensive retrieval. In parallel, explanation research has explored interpretable architectures built around human-understandable features and post-hoc explanations.

  • Dense Retrieval Advances: Dense retrieval advances include specialized embedding models such as E5, GTE, and BGE, alongside LLM-driven retrievers such as Qwen3 Embedding and Jina Embedding.These developments improve representation quality, retrieval performance, and semantic richness.
  • Reasoning-Intensive Retrieval: Reasoning-intensive retrieval uses CoT-enhanced dense retrievers to support complex inference and multi-step reasoning.This line of work reflects growing interest in retrieval requiring reasoning beyond straightforward matching.
  • Explaining Dense Retrieval: Dense retrieval explanations primarily use inherently interpretable architectures or post-hoc explanations.Interpretable architectures reshape embedding spaces around predefined question answers, semantic aspects, sparse lexical weights, or token-level alignments.

5 Conclusion and Future Work

Xetrieval explains dense retrieval by internalizing reasoning and decomposing embeddings into interpretable features that trace decisions to latent query-document factors. Interventions indicate these features are locally grounded in similarity computation and globally useful for retrieval behavior, while future work targets broader retrieval settings and adaptive reasoning.

  • Conclusion: Xetrieval explains dense retrieval beyond opaque similarity scores by internalizing reasoning and decomposing embeddings into interpretable features.The framework traces retrieval decisions to latent query-document factors.
  • Conclusion: Interventions show that Xetrieval’s features are locally grounded in similarity computation and globally useful for retrieval behavior.
  • Future Work: Future work may extend Xetrieval to multi-modal and cross-lingual retrieval, adaptive reasoning pathways, and additional directions.The supplied passage truncates the final future-work item after “and f”.

Limitations

Xetrieval’s analysis is limited to sentence embeddings and does not probe the embedding model’s internal circuits. Its SAE-based decomposition is effective but has limited fidelity and granularity.

  • Xetrieval therefore reveals latent retrieval factors without fully explaining the internal mechanisms that produce them.
  • The analysis is confined to the sentence-embedding output layer and does not investigate internal circuits or representations throughout the embedding network.A deeper understanding would require examining internal representations and interactions across the full embedding network.
  • SAE decomposition provides effective attribution of retrieval decisions but offers limited fidelity and granularity.

Ethical Considerations · A Details of Reasoning Internalizer

Xetrieval is intended to improve transparency in dense retrieval by exposing sparse, human-readable factors behind retrieval decisions. The authors recommend treating it as an analysis tool, not a standalone decision-making mechanism, because its explanations may be imperfect.

  • Ethical Considerations: Xetrieval exposes sparse, human-readable factors behind dense retrieval decisions.This transparency goal may support auditing, debugging, and failure analysis.
  • Ethical Considerations: The framework may benefit auditing of retrieval applications.Its explanations are positioned as aids for examining system behavior.
  • Ethical Considerations: The framework may support debugging retrieval applications.Exposed factors can help investigate system behavior and errors.
  • Ethical Considerations: The framework may assist failure analysis for retrieval applications.The intended use includes analyzing failures rather than replacing decision-making.
  • Ethical Considerations: Users may over-interpret imperfect explanations as definitive justifications.The authors identify this as a potential risk, especially in high-stakes settings.
  • Ethical Considerations: The authors recommend using Xetrieval as an analysis tool rather than a standalone decision-making mechanism.This recommendation responds to the risk of treating imperfect explanations as definitive.

A.1 Training Details

Xetrieval trains aspect-specific reasoning internalizers on paired document and LLM-generated reasoning embeddings from a multi-domain StackExchange corpus. The lightweight models use single-pass embedding transformations and train and infer efficiently.

  • Data construction: Training pairs combine each StackExchange document with LLM-generated reasoning text for SUMMARY, PURPOSE, or QA, embedded as paired representations.A separate reasoning internalizer R_t learns to map the original embedding z to the aspect-specific embedding z^(t).
  • Data construction: The corpus samples multiple StackExchange communities to improve coverage and reduce domain bias, totaling 11,796 documents.The QA teacher output is treated as one question–answer text block before embedding.
  • Time cost: Each internalizer typically trains within 1–2 minutes, while inference uses one feed-forward pass over cached embeddings and finishes in seconds.This efficiency follows from the lightweight architecture and avoids autoregressive reasoning at inference time.

A.2 Evaluation Details · B Mechanistic Explainer Details

The evaluation samples benchmark subsets to reduce the cost of generating LLM-based chain-of-thought supervision, while preserving ground-truth documents and standardized corpus targets. Additional results show that reasoning internalizers consistently improve dense retrievers and closely approach CoT reasoners across teacher-data sources.

  • A.2 Evaluation Details: Sampling benchmark subsets reduces the cost of generating LLM-based CoT reasoning content for large-scale corpora.A subset is sampled from each benchmark rather than generating reasoning content for every corpus item.
  • A.2 Evaluation Details: For BRIGHT, each domain subset includes all ground-truth documents plus randomly sampled corpus documents to reach 1,000 documents per subset.Queries and documents from all subsets are then aggregated into one unified BRIGHT evaluation set.
  • A.2 Evaluation Details: For NQ, MuTual, TREC-NEWS, Signal-1M, Robust04, and ArguAna, corpora below 10,000 documents receive randomly sampled additions to reach that target.All ground-truth documents are retained, and Table 4 summarizes the resulting benchmark corpus statistics.
  • A.2 Evaluation Details: Table 4 reports the statistics of the sampled benchmarks used in evaluation.The table summarizes final corpus statistics after benchmark-specific sampling procedures.
  • A.2 Evaluation Details: Across dense retrievers, reasoning internalizers consistently outperform base retrievers on NDCG@10 when trained with supervision from different LLM teachers.Tables 6–11 report these comparisons across teacher-generated training sources.
  • A.2 Evaluation Details: Reasoning internalizers closely approach CoT reasoners across teacher-data sources, indicating that they preserve LLM reasoning content within the embedding space.This qualitative trend holds across all evaluated dense retrievers.

B.1 SAE Details … C.1 Local Attribution

The paper details SAE training and evaluation, feature explanation procedures, case studies, and local attribution interventions. Xetrieval uses reasoning-oriented document views and decoder-direction edits to test whether explanation features are tied to retrieval decisions.

  • B.1 SAE Details: The SAE training corpus contains 84,860 documents drawn from the reasoning internalizer dataset and additional retrieval- and reasoning-relevant StackExchange domains.
  • B.1 SAE Details: SAE quality is evaluated through Reconstruction Error, Mono-Semanticity, and Retrieval Retention using BRIGHT-based reconstruction and retrieval tests.Reconstruction Error is the mean squared error between original and reconstructed embeddings over 100 BRIGHT Biology documents, while Mono-Semanticity uses intruder detection across the training corpus.
  • B.2 Explaining Details: Qwen3-32B generates feature explanations and performs the associated evaluation using prompts provided in Table 12.
  • B.3 Case Studies: Four additional case studies show how the reasoning internalizer bridges semantic gaps between raw document and query embeddings by uncovering deeper reasoning.
  • C.1 Local Attribution: Pair-level intervention tests local attribution by fixing the query representation and intervening only on the original document embedding.
  • C.1 Local Attribution: Pairs are sampled from the top-K original-retriever results with K=32, distinguishing annotated true positives from retrieved false positives.
  • C.1 Local Attribution: Interventions operate in embedding space through decoder-direction spans: one edit erases the selected span and another retains it, followed by normalization and cosine-similarity change measurement.The reported comparisons include direct decomposition, Xetrieval, and the non-overlap active-feature control.
  • C.1 Local Attribution: Xetrieval constructs explanation features by combining the original document embedding with QA, summary, and purpose reasoning-oriented views, then taking query-document overlap unions.The reasoning internalizer exposes additional candidate features, but the intervention target remains the original document embedding; non-overlap active features provide a control.

C.2 Task-Level Steering Details · D LLM Usage

Task-level steering applies RUS-identified top-k features with dataset-specific k values, while LLMs were used only for drafting and refinement under author review. The appendix also documents configuration comparisons, prompts, and case studies involving multiple language models.

  • C.2 Task-Level Steering Details: Task-level feature steering is performed on the top-k features identified by RUS.
  • C.2 Task-Level Steering Details: For BRIGHT, the steering budget is k=256, whereas ArguAna and NQ use k=1024.The values were chosen according to each dataset’s domain breadth.
  • D LLM Usage: ChatGPT and Gemini assisted with drafting and refining the text.
  • D LLM Usage: All LLM-assisted content was reviewed, revised, and verified by the authors.The authors take full responsibility for the paper’s content.
  • D LLM Usage: The LLMs contributed wording and phrasing suggestions but not independent research ideation, experimental design, or result analysis.
  • D LLM Usage: Additional appendix material provides prompts for feature explanation and evaluation.
  • D LLM Usage: Case studies cover geometric problem-solving, coordinate geometry and algorithmic reasoning, economic and philosophical reasoning, and hardware interface programming and system design.
Loading 2605.29507v1…