Source-linked AI summary
ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers
Effrosyni Sokli, Isaac Roberts, Alexander Schulz, Barbara Hammer, Gabriella Pasi
TL;DR
Dense retrievers are effective but difficult to interpret, and feature-level explanations do not fully capture their learned embedding spaces. ExpertLens addresses this gap with routing-aware DeepView visualizations, expert-subspace analysis, and automatically extracted CAVs. Across five IR benchmarks and two MoE retrievers, the framework reports improved relative neighborhood structure and semantically organized expert subspaces.
Problem
Dense retrievers are effective but behave as black boxes, while feature-level post-hoc attributions inadequately capture high-dimensional embedding-space structure.
Method
ExpertLens uses routing-aware DeepView projections, expert-subspace characterization, and automatically extracted CAVs for representation-level post-hoc explanation.
Results
Across five IR benchmarks and two MoE-enhanced dense retrievers, MoE routing improves relative neighborhood structure around queries in the embedding space.
Takeaways & Limitations
ExpertLens reveals how routing shapes embedding geometry, with dominant general-purpose experts and minority experts showing distinct linguistic specialization.
Abstract
from arXiv · showhide
Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art performance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Existing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative embedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Experiments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves embedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Analysis of expert subspaces further reveals general-purpose dominant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is publicly available.
1 Introduction
ExpertLens addresses the limited interpretability of MoE-enhanced dense retrievers by explaining global embedding-space structure and expert specialization rather than only local feature contributions. It combines discriminative visualization, expert-subspace analysis, and automatically extracted CAVs across IR tasks and domains.
- Dense retrievers capture semantic relationships beyond lexical matching but remain black boxes for ranking decisions.
- MoE routing improves retrieval effectiveness while making high-dimensional representations harder for traditional feature-attribution methods to explain adequately.
- ExpertLens provides global, representation-level post-hoc explanations tailored to MoE-enhanced dense retrievers.
- DeepView projections expose how expert routing shapes embedding-space geometry and organizes document representations.
- The framework analyzes expert-induced subspaces and uses automatically extracted CAVs to associate semantic attributes with embedding-space structure.
- ExpertLens complements feature-level attribution by providing explainable insights into embedding-space structure and expert specialization.
2 Related Work and Background
The paper builds on feature attribution, concept-level explanation, MoE retrievers, and discriminative visualization. ExpertLens combines these strands by using model-specific routing information to reveal expert-induced embedding subspaces and their semantic structure.
- Prior IR explainability methods mainly estimate the contribution of individual query-document features, including listwise feature interdependencies.
- CAVs and sparse dictionary learning support concept-level NLP explanations, while automated labeling reduces the labor of assigning semantic names to many concepts.
- SB-MoE and DenseC3 augment dense bi-encoders with six experts whose outputs are fused through gating-weighted sums.
- SB-MoE learns routing unsupervised, whereas DenseC3 uses Cognitive Complexity labels as a weakly supervised routing signal.
- Unlike unsupervised t-SNE or UMAP, DeepView uses discriminative information to emphasize user-relevant structure in embedding projections.
- ExpertLens derives DeepView’s classifier from each model’s internal gating mechanism, making projection boundaries correspond directly to expert routing decisions.
3 The Proposed Framework
ExpertLens analyzes MoE-induced embedding-space structure through discriminative visualizations, expert-subspace profiling, and concept extraction. Its three phases connect routing boundaries to geometric organization, linguistic specialization, and semantic coherence.
- Phase I: Embedding-space analysis: ExpertLens compares MoE-induced embedding structure using discriminative projections tied to model routing decisions.DeepView combines featurewise similarity with classifier-induced distances derived from each model’s internal gating mechanism.
- Phase I: Embedding-space analysis: The framework uses a controlled comparison of SB-MoE and DenseC3, which share a six-expert, single-layer architecture but differ in gating mechanism.SB-MoE uses unsupervised routing, whereas DenseC3 uses supervised routing.
- Phase I: Embedding-space analysis: ExpertLens evaluates whether relevant documents occupy better-defined query neighborhoods using Precision@k and cosine distance.These measures compare the relative neighborhood position and absolute proximity of relevant documents across retriever variants.
- Phase II: Expert specialization: Phase II selects queries whose retrieved documents form compact, expert-aligned clusters and profiles the resulting expert subspaces with corpus-level linguistic metrics.The composite score combines adjusted Rand Index with the proportion of non-noise documents; queries with fewer than two clusters are excluded.
- Phase III: Concept extraction and attribution: Phase III extracts sparse concept directions from document embeddings with an autoencoder, labels them using top-activating queries, and tests their neighborhood consistency.The Q_concept(n) error measures whether geometrically proximate documents activate similar top-n concepts, with lower error indicating stronger semantic coherence.
4 Experimental Setup
The evaluation spans three phases, five IR benchmarks, two retrieval tasks, and multiple MoE-enhanced retriever configurations. It combines discriminative visualization, neighborhood metrics, clustering, linguistic profiling, and concept extraction to study expert routing.
- The study addresses embedding-space structure, expert specialization, and interpretable semantic concepts across three evaluation phases.
- Experiments use five public benchmarks spanning passage retrieval and open-domain question answering formulated as search.
- The comparison includes Fine-Tuned, Unsupervised SB-MoE, and Supervised DenseC3 variants across four backbone bi-encoders.
- DeepView visualizations distill each gating mechanism into a six-expert classifier trained with KL divergence for 30 epochs.
- Precision@k uses k∈{20,100} for TREC DL and k∈{1,2} for sparse-judgment datasets, reflecting their differing relevance densities.
- DBSCAN partitions two-dimensional projections with ε=0.25 and min_samples=7, retaining queries with expert-aligned structure for linguistic profiling.
5 Results & Discussion
The evaluations show that MoE routing improves embedding-space organization and retrieval neighborhoods, while expert subspaces exhibit consistent linguistic specialization and multi-concept structure.
- Embedding Space Structure: MoE-enhanced variants consistently place relevant documents closer to queries than Fine-Tuned models across five benchmarks and multiple bi-encoders.The Supervised variant gains 14.78% over Fine-Tuned on NQ with TinyBERT at k=1.
- Embedding Space Structure: Expert-induced subspace partitioning improves relative neighborhood structure while increasing the overall geometric spread of embedding representations.These effects co-occur across domains, tasks, and bi-encoders of varying scale and pre-training paradigms.
- Expert Specialization: Dominant experts absorb most retrieved documents and form general-purpose subspaces, whereas minority experts capture lexically richer text across datasets and MoE variants.This inverse relationship holds for lexical diversity, Shannon’s entropy, and corpus predictability.
- Concept Extraction and Expert Attribution: Semantic concepts are broadly distributed across experts rather than isolated within individual experts, with dominant experts carrying most activations.The findings suggest expert assignment reflects broader document characteristics because documents contain multiple semantic themes.
- Concept Extraction and Expert Attribution: Neighborhoods in the supervised embedding space are explained well by a small concept set, whereas the unsupervised space benefits from broader concept coverage.At 10% concept coverage, the unsupervised variant improves over the supervised counterpart by 0.04 to 0.05 in Q_concept error.
6 Conclusions
ExpertLens provides representation-level post-hoc explanations for MoE-enhanced dense retrievers by connecting embedding geometry, expert subspaces, and concept-level attributions.
- Conclusions: ExpertLens uses DeepView visualizations and concept-level attribution to explain how expert routing shapes embedding-space geometry.The framework complements feature-level attribution methods in Information Retrieval explainability.
- Conclusions: Across five IR benchmarks and two retrievers, MoE routing improves query-neighborhood structure and increases overall geometric spread.Dominant experts form general-purpose subspaces, while minority experts form lexically rich and concentrated subspaces.
- Conclusions: Concept activations are broadly distributed across experts, indicating that expert specialization is not limited to isolated semantic topics.
A Prompt for Concept Labeling
The concept-labeling prompt instructs Gemma-3-27b to infer shared semantic properties from groups of search queries activating the same sparse-autoencoder feature.
- Prompt: Gemma-3-27b is prompted to analyze groups of search queries and assign each group a semantic label.
- Prompt: Each query group is treated as activating a specific latent feature whose shared intent, topic, or linguistic property should be identified.
- Prompt: The displayed example applies the labeling prompt to concepts discovered from HotpotQA data.