PaperlayerView exact revision on arXiv

Source-linked AI summary

ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers

Effrosyni Sokli, Isaac Roberts, Alexander Schulz, Barbara Hammer, Gabriella Pasi

arXiv:2609.06155v1cs.IRcs.AI

TL;DR

Dense retrievers are black boxes, and feature-level attributions may not capture the complexity of their high-dimensional representations. ExpertLens applies routing-aware visualizations and concept attribution to MoE-enhanced retrievers; across five IR benchmarks and two models, routing improves query neighborhood structure.

  • Problem

    Dense retrievers are black boxes, while high-dimensional representations challenge traditional feature-attribution explanations of MoE-enhanced retrieval.

    Source: §1 · p. 1

  • Method

    ExpertLens combines routing-aware discriminative embedding visualizations with automatically extracted Concept Activation Vectors to explain MoE retrievers at the representation level.

    Source: §1 · pp. 1–2, §3.1 · p. 3

  • Results

    Across five IR benchmarks and two MoE-enhanced dense retrievers, MoE routing improves the relative neighborhood structure around queries in embedding space.

    Source: §6 · p. 10

  • Takeaways & Limitations

    ExpertLens provides global post-hoc explainability that complements feature-level attribution for MoE-enhanced dense retrievers.

    Source: §1 · p. 2

Abstract

from arXiv · show

Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art performance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Existing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative embedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Experiments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves embedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Analysis of expert subspaces further reveals general-purpose dominant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is publicly available.

1 Introduction

Dense retrievers capture semantic relationships but remain difficult to interpret, while feature-level attributions do not explain how MoE expert specialization structures representations. ExpertLens addresses this gap with global, representation-level post-hoc analysis of MoE-enhanced dense retrievers.

1 Introduction

  • Dense retrievers encode semantic relationships beyond lexical matching but remain black boxes, and feature-level attributions do not explain how expert specialization shapes embedding spaces.The paper motivates moving beyond scalar feature importance to interpretable embedding-space structure.Source: §1 · p. 1
  • ExpertLens uses DeepView visualizations to examine how expert routing affects embedding geometry and document organization in MoE-enhanced dense retrievers.The framework is evaluated on five IR benchmarks and two retrievers with contrasting gating mechanisms.Source: §1 · pp. 1–2
  • ExpertLens analyzes expert-induced subspaces qualitatively and quantitatively to characterize specialization through linguistic properties and semantic themes.Source: §1 · p. 2
  • Automatically extracted Concept Activation Vectors associate semantic attributes with embedding-space structure, complementing feature-level attribution with global explanations of retrievers and expert specialization.Source: §1 · p. 2

2 Related Work and Background

The section motivates representation-level explanations for MoE dense retrieval because feature attribution and generic visualization leave routing-induced embedding geometry underexplored. ExpertLens connects gate-conditioned DeepView projections with concept-level CAV explanations.

Explainability in IR

  • ExIR uses SHAP [29], LIME [37], LIRME, and RankingSHAP for feature-level ranking explanations; neural representation work [13] [50] leaves MoE embedding geometry underexplored.MoE gating dynamically routes inputs through experts, partitioning the embedding space according to routing decisions rather than input features.Source: §2 · p. 2

Concept-level explanations

  • CAVs [10] [25] with sparse dictionary learning [15] support concept-level explanations, while labor-intensive manual labeling motivates LLM-generated labels [12].Their application to IR remains underexplored, with a notable supervised-CAV exception in recommender systems.Source: §2 · p. 2

MoE-enhanced IR models

  • SB-MoE [44] learns task-driven routing unsupervised with noisy Top-1 training; DenseC3 uses Bloom complexity labels and externally pretrained multilabel routing.Both place six experts atop a dense bi-encoder and fuse their outputs by gating-weighted sum.Source: §2 · p. 2

Embedding-space visualization

  • Unlike unsupervised t-SNE and UMAP [32], DeepView [40] combines discriminative distances with UMAP to emphasize class-relevant structure; DiDi balances this with unsupervised regularization.The discriminative distances are equivalent to the Fisher information metric and use Jensen-Shannon divergence locally.Source: §2 · pp. 2–3

ExpertLens

  • ExpertLens derives DeepView’s discriminative classifier from each model’s gate, making projections reflect expert-routing boundaries and the induced embedding-space geometry.This lens is intended to reveal subspace geometry that generic dimensionality reduction cannot.Source: §2 · p. 3

3 The Proposed Framework

ExpertLens uses routing-aware visualizations and analyses to explain how MoE-enhanced retrievers organize embedding spaces, specialize experts, and align semantic concepts with routing. It evaluates geometric neighborhoods, linguistic properties, and concept coherence across supervised and unsupervised MoE variants.

3.1 Motivation

  • DeepView incorporates each model’s gating mechanism into its projection, while SB-MoE [44] and DenseC3 isolate gating strategy as the primary architectural difference.Both retrievers share a single-layer, six-expert MoE architecture with gate-weighted summed output; SB-MoE uses unsupervised gating and DenseC3 supervised gating.Source: §3.1 · p. 3

3.2 ExpertLens

  • ExpertLens has three phases: embedding-space structure analysis, expert-subspace disentanglement and specialization, and concept extraction with expert attribution.The phases use discriminative projections, characterize expert-associated subspaces, and map automatically extracted CAVs to experts.Source: §3.2 · p. 3

3.2.1 Phase I.

  • DeepView projections show MoE variants progressively repositioning relevant documents closer to their queries, assessed through Precision@k and query-document cosine distance.The gating-derived classifier makes projection boundaries reflect expert assignments; the two measures assess relevant-document placement in the query neighborhood and absolute proximity, respectively.Source: §3.2.1 · pp. 4–5, Figure 2 · p. 4

3.2.2 Phase II.

  • Phase II selects queries with expert-aligned document clusters by weighting DBSCAN–expert-label Adjusted Rand Index by the non-noise core ratio.Queries with fewer than two DBSCAN clusters are excluded; the top 100 queries per dataset and model are retained for linguistic profiling.Source: §3.2.2 · p. 5
  • Phase II profiles each active expert’s subspace using five linguistic metrics and compares it with the top-1000 retrieval pool and full-corpus baseline.The metrics are readability, token entropy, relative vocabulary size, corpus predictability, and average sentence length.Source: §3.2.2 · p. 5

3.2.3 Phase III.

  • Phase III decomposes document embeddings into sparse concept activations and dictionary directions, labels concepts from highly activated queries with an LLM, and aggregates concept activations by expert.Query-based labels may be cleaner but narrower than document-based meanings, so they are treated as indicative summaries; expert-concept associations are visualized with a Sankey diagram.Source: §3.2.3 · p. 5
  • Phase III’s Q_concept(n) error measures whether documents share their top-n active concepts with cosine-nearest neighbors, with low error indicating semantically coherent embedding structure.The measure is evaluated across supervised and unsupervised MoE variants.Source: §3.2.3 · p. 6

4 Experimental Setup

The evaluation examines how MoE affects embedding-space neighborhoods, what textual properties characterize expert subspaces, and whether Concept Activation Vectors (CAVs) reveal interpretable expert concepts.

4 Experimental Setup

  • The evaluation is organized into three phases addressing embedding-space neighborhoods, expert specialization, and interpretable semantic concepts associated with experts.The stated research questions ask how MoE affects embedding structure, which textual characteristics define expert specialization, and whether CAVs reveal expert-associated concepts.Source: §4 · p. 6

4.1 Datasets, Models & Baselines

  • Five benchmarks cover passage retrieval and search-based Q&A; two MoE retrievers are compared with matched non-MoE baselines across four backbones.Phase I uses all four backbones, while Phases II and III use TinyBERT to manage computational costs; the baseline shares training data and settings with the MoE variants.Source: §4.1 · p. 6

4.2 Phase I Setup

  • At λ=0.6, Q_data peaked at 97.4%, while Q¬data was 95.3% and Q_knn was 0.096; this setting was used uniformly across evaluations.DeepView distills a gating-mechanism classifier, and λ=1.0 reduces to regular UMAP, where Q_knn falls to 0.455. Precision@k cutoffs reflect judgment density; cosine distance measures absolute query–relevant-document proximity.Source: §4.2 · pp. 6–7

4.3 Phase II Setup

  • Phase II applies DBSCAN to top-1000-document DeepView projections, using ε=0.25 and min_samples=7 to retain at least 100 well-structured queries per dataset and MoE variant.The top 100 queries are profiled by expert subspace; active experts handle at least 10 retrieved documents on average across analyzed queries.Source: §4.3 · p. 7

4.4 Phase III Setup

  • Phase III trains a Top-k SAE on TinyBERT document embeddings from both MoE variants, using a 624-atom dictionary, 90% sparsity, and 50 epochs.Concept labels come from Gemma-3-27b using the top 20 activating queries; Q_concept is averaged over 40 runs sampling 20,000 points, and expert-concept associations use corpus-wide activations.Source: §4.4 · p. 7

5 Results & Discussion

Expert routing improves retrieval and reshapes embedding neighborhoods, while expert subspaces show linguistic specialization rather than isolated semantic-topic specialization. Concept-neighborhood results indicate that multiple concepts are needed to explain document relationships.

5.1 RQ1: Embedding Space Structure

  • MoE variants consistently achieve higher P@k than Fine-Tuned across four backbones and five benchmarks; supervised DenseC3 gains 14.78% over Fine-Tuned on NQ with TinyBERT at P@1.Relevant documents move closer to queries relative to the retrieved pool, even as MoE variants increase overall cosine distance and embedding-space spread.Source: §5.1 · p. 8

5.2 RQ2: Expert Specialization

  • Across both MoE variants and three datasets, dominant experts handle most retrieved documents and have lower lexical richness, while minority experts score highest on lexical diversity, entropy, and corpus predictability.Supervised routing distributes documents more evenly across experts, but the inverse relationship between routing load and lexical richness persists.Source: §5.2 · pp. 8–9

5.3 RQ3: Concept Extraction and Expert Attribution

  • In both MoE variants, concepts activate across multiple experts rather than mapping to isolated semantic topics; supervised routing concentrates about two-thirds of positive activations in its dominant expert.In the unsupervised variant, two primary experts share most concept activations, consistent with expert assignment depending on broader document characteristics.Source: §5.3 · p. 9, Figure 5 · p. 9
  • At the 10% threshold, unsupervised Q_concept error improves by 0.04 to 0.05 over its supervised counterpart, while supervised error is lower when fewer concepts are evaluated.For NQ, MSMARCO, and HotpotQA, the variant error differences are approximately 0.11, 0.08, and 0.05, respectively; with 1% (6) of concepts, supervised neighboring documents share over two-thirds of them on average.Source: Figure 7 · p. 9, §5.3 · p. 10

6 Conclusions

ExpertLens provides a representation-level view of how MoE routing shapes dense-retriever embedding geometry and expert specialization. Across benchmarks and retrievers, routing improves query-relative neighborhoods, while expert subspaces reveal consistent linguistic specialization.

6 Conclusions

  • Across five IR benchmarks and two MoE-enhanced dense retrievers with contrasting gating mechanisms, routing improves query-relative neighborhoods, while expert partitioning increases overall embedding-space spread.The evaluation links these geometric changes to expert routing across all tested datasets and retrievers.Source: §6 · p. 10
  • Expert specialization shows consistent linguistic profiles: general-purpose dominant experts absorb structurally average documents, while minority experts form lexically rich, concentrated subspaces.These specialization patterns hold across all datasets.Source: §6 · p. 10
  • Concept activations are broadly distributed across experts, with dominant experts accounting for most; document neighborhoods reflect broad semantic concepts, suggesting routing follows linguistic properties rather than semantic isolation.The findings connect expert assignment to document linguistic properties rather than isolated semantic groupings.Source: §6 · p. 10
  • ExpertLens combines DeepView discriminative embedding visualizations for expert-subspace characterization with concept-level attribution to explain how routing shapes embedding geometry.It operates at the representation level, complementing feature-level attribution methods in Information Retrieval explainability.Source: §6 · p. 10

A Prompt for Concept Labeling

The section presents the zero-shot Gemma-3-27b prompt used to label discovered concepts, with Figure 8 showing semantic-label generation for SAE concepts from top-activating HotpotQA queries.

A Prompt for Concept Labeling

Related papers

Loading 2609.06155v1…