PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals

Joao Fonseca, Rodrigo Rodrigues, Paolo Romano

arXiv:2608.17687v1cs.AIcs.LG

TL;DR

Most hallucination detectors work at answer or sentence level, leaving per-token localization underexplored. InnerExpert uses MoE-specific routing signals for single-pass per-token detection and achieves up to 0.91 answer-level and 0.76 token-level AUROC across five datasets and two MoE architectures.

  • Problem

    Most hallucination detectors operate at answer or sentence level, while reliable per-token detection for localizing hallucinated spans remains underexplored.

  • Method

    InnerExpert combines MoE-specific routing signals with standard transformer signals in a single-pass per-token detector trained using LLM-generated labels.

  • Results

    Up to 0.91 answer-level and 0.76 token-level AUROC across five datasets and two MoE architectures, outperforming the evaluated baselines with a single forward pass.

  • Takeaways & Limitations

    Combining multiple MoE-specific signals yields consistent gains, indicating that their combination provides information unavailable to any individual signal.

  • Takeaways & Limitations

    Human validation was small-scale, and the evaluation was limited to English.

Abstract

from arXiv · show

Despite their widespread use, Large Language Models (LLMs) remain limited by a fundamental problem: the generation of plausible but false content, known as hallucinations. Most existing detection methods operate at the answer or sentence level, yet per-token detection is essential for localizing hallucinated spans and enabling fine-grained interventions. In this paper, we explore the use of the Mixture-of-Experts (MoE) paradigm to address this gap. In MoE architectures, a single forward pass activates a sparse subset of experts (i.e., distinct feedforward networks per layer) via a routing mechanism, producing internal signals (e.g., router entropy, expert disagreement, and expert usage patterns) that are unavailable in dense architectures and have not been previously exploited for hallucination detection. To this end, we introduce InnerExpert, the first method to leverage these MoE-specific signals for per-token hallucination detection. InnerExpert combines routing-level and standard transformer signals into compact per-token feature vectors, classified by a lightweight detector trained on labels produced by an LLM-as-a-judge pipeline, which enables continuous model updates without manual annotation. Our results show that InnerExpert outperforms existing methods across five datasets and two MoE architectures, achieving up to 0.91 answer-level and 0.76 token-level AUROC, while requiring only a single forward pass.

1 Introduction

InnerExpert addresses hallucination detection by exploiting MoE-specific internal signals that are naturally aligned with per-token localization and unavailable in dense architectures. It combines these signals with standard transformer features in a single-pass detector trained through LLM-based labeling, achieving up to 0.91 answer-level and 0.76 token-level AUROC across five datasets and two open-weight MoE architectures.

  • Motivation: MoE routing exposes token-level signals unavailable in dense models, including router entropy, expert hidden-state disagreement, and expert-usage distributions.A routing function selects a sparse subset of feedforward experts per layer, producing these internal signals during a forward pass.
  • Method: InnerExpert is a single-pass, per-token hallucination detector that uses MoE-specific internal signals without modifying the host LLM or adding sampling.It combines the efficiency of internal-signal methods with a trainable detector and leverages router entropy, expert disagreement, and expert-usage patterns.
  • Training: InnerExpert uses an unsupervised training procedure that labels generated answers with an LLM against reference evidence.This provides the training labels for the per-token hallucination detector without relying on manual annotation.
  • Results: 0.91 answer-level AUROC and 0.76 token-level AUROC are achieved across five datasets and two open-weight MoE architectures.The evaluation compares InnerExpert with sampling-based, internal-signal, and trainable baselines.
  • Analysis: InnerExpert formalizes six MoE-specific signals alongside standard hidden-state and attention scores for systematic analysis of hallucination-discriminative information.The signal inventory supports studying which routing features distinguish hallucinated content.

2 Background

The section defines hallucinations as fluent, coherent tokens that are factually incorrect or unsupported, and describes MoE routing as a source of internal signals unavailable in dense transformers. It also outlines InnerExpert’s per-token scoring and aggregation framework for identifying hallucinated content.

  • Hallucination in LLMs: A hallucination is a subset of fluent, internally coherent generated tokens that are factually incorrect or unsupported.The model generates tokens autoregressively from pθ(yt | x, y<t).
  • Mixture-of-Experts and Its Internal Signals: MoE layers replace standard feedforward sublayers with N expert networks and a router that selects k experts per token.The selected experts’ weighted outputs are added to the residual stream for the next layer.
  • Mixture-of-Experts and Its Internal Signals: A single MoE forward pass exposes internal signals unavailable in dense architectures.These signals arise from routing and expert computation within the sparse MoE layers.
  • Hallucination in LLMs: Some incorrect outputs faithfully propagate erroneous training or retrieved content rather than reflecting a generation failure.The passage states that such cases are treated as low-uncertainty by sampling-based and internal-signal detectors, while temporal out-of-distribution evaluation mitigates the conflation.
  • Hallucination detection: InnerExpert assigns each generated token a hallucination score s(yt) ∈ [0, 1] and derives an answer-level score by aggregation.The score uses hidden states, attention weights, logits, routing distributions, and expert activations collected during one forward pass.
  • Hallucination detection: Tokens are flagged as hallucinated when their score exceeds threshold τ, which can be tuned to trade precision for recall.Answer-level prediction is obtained by aggregating token-level scores.

3 InnerExpert

InnerExpert combines cumulative standard-transformer and MoE-specific internal signals into compact per-token features, then uses a lightweight classifier to score hallucination probability. Its routing signals capture uncertainty, expert representation quality, and routing dynamics, while supervision comes from LLM-judge-generated hallucinated spans mapped to tokens.

  • Signal collection: InnerExpert collects cumulative internal signals at every layer during each decoding step, distinguishing standard transformer signals from MoE-specific routing signals.Signals are computed over the generated prefix y≤t and produce a per-token value at each step.
  • MoE-specific signals: Six MoE-specific signals measure router uncertainty, expert representation quality, and routing distribution dynamics unavailable in dense architectures.They include router entropy, expert hidden score, expert similarity, expert usage distribution, Gini impurity, and inverse Herfindahl index.
  • MoE-specific signals: Low pairwise similarity among activated experts indicates disagreement and inconsistent representations for the current token.Expert similarity is computed as a routing-weighted average of pairwise cosine similarities.
  • Feature assembly and scoring: Per-layer signals are flattened into fixed-order feature vectors, and a lightweight classifier predicts the probability that each generated token is hallucinated.The classifier can use standard machine-learning families, while generated-token features are standardized except for bounded expert usage distributions.
  • Training labels: LLM-judge hallucinated answer spans are mapped to binary token labels according to character-span overlap, producing supervision aligned with each feature vector.The judge compares answers against reference evidence and returns both an answer-level label and hallucinated span strings.

4 Experiments

The experiments evaluate InnerExpert across multiple hallucination-detection baselines, datasets, and MoE host models using answer- and token-level AUROC. Results show strong predictive performance, cross-dataset robustness, single-pass efficiency, and complementary value from combining MoE signals.

  • Experimental questions: Experiments address InnerExpert’s reliability, baseline comparisons, signal contributions, and the quality of LLM-as-a-judge labels.The evaluation is organized around four research questions, with all metrics computed from LLM-as-a-judge labels.
  • Datasets: Evaluation uses temporally out-of-distribution RealtimeQA data plus SQuAD, TruthfulQA, NQ-Open, and FreshQA for cross-dataset testing.Training uses RealtimeQA data from January 2024 to December 2025; the temporal test subset contains 365 questions, while each additional dataset contributes 200 questions.
  • Predictive performance: 0.762 token-level AUROC is achieved by InnerExpert with LR on OLMoE, while XGBoost reaches 0.753 on Gemma, improving over the best single-signal baselines by 0.12 and 0.20.All five InnerExpert classifier variants exceed the best single-signal baseline in average token-level AUROC on both host models, although the strongest classifier varies by host model.
  • Cross-dataset generalization: InnerExpert maintains an advantage over single-signal baselines across all five evaluation datasets, including FreshQA, NQ-Open, and TruthfulQA.The reported advantage covers datasets that differ substantially from the RealtimeQA training distribution.
  • Inference cost: InnerExpert reaches the Pareto frontier as a single-pass method, while extracting the full MoE signal set costs approximately 2.5× vanilla generation.Sampling-based methods occupy a higher-cost region without proportional predictive gains; standard internal-signal baselines add less than 10% overhead.
  • Signal contributions: 0.912 answer-level AUROC is achieved by IE XGBoost on Gemma, compared with 0.882 on OLMoE and a 0.197 gain over OLMoE’s best individual MoE signal.Combining all signals improves over the best individual signal on both host models, with larger relative gains at token level.

5 Conclusion · A Hallucination Detection via Epistemic Uncertainty · B Related Work

InnerExpert is a lightweight, single-pass per-token hallucination detector that combines MoE-specific and standard transformer signals, achieving strong answer- and token-level AUROC while exposing a largely unexplored source of hallucination-detection information. The paper motivates this approach through epistemic uncertainty, surveys existing detection paradigms, and notes limitations in validation scale and language coverage.

  • 5 Conclusion: 0.91 answer-level and 0.76 token-level AUROC are achieved by InnerExpert while outperforming sampling-based, internal-signal, and trainable-detector baselines in a single forward pass.The detector combines MoE-specific with standard transformer signals and has modest computational overhead.
  • 5 Conclusion: The human validation study is small, evaluation is limited to English, and future work will expand annotator diversity, sample size, and language coverage.The classifiers are naturally lightweight because they leverage MoE-specific signals.
  • A Hallucination Detection via Epistemic Uncertainty: Epistemic uncertainty reflects reducible uncertainty about the model’s knowledge, whereas aleatoric uncertainty reflects irreducible ambiguity in the data-generating process.Aleatoric uncertainty is reflected in output-distribution entropy, while epistemic uncertainty can be reduced with additional evidence.
  • A Hallucination Detection via Epistemic Uncertainty: A token is likely hallucinated when epistemic uncertainty is high relative to aleatoric uncertainty but the model still emits a confident-looking prediction.The theoretical link uses expected disagreement among posterior samples; it tracks divergence from the data-generating distribution only in the well-specified Bayesian limit.
  • B Related Work: Sampling-based detectors assess consistency across multiple stochastic generations, including SelfCheckGPT, Semantic Energy, and INSIDE.These methods use agreement, semantic-cluster energy, or internal embedding-space consistency to estimate hallucination risk.
  • B Related Work: Internal-signal methods detect hallucinations from hidden states, attention patterns, or output distributions collected during a single forward pass.Related methods include classifiers over hidden states and LLM-Check, while other approaches modify representations at inference time to mitigate hallucinations.
  • B Related Work: Trainable detectors typically apply lightweight classifiers to internal signals, including token-level log-likelihoods, entropy scores, and hidden-state embeddings.Examples include HaluNet and a joint framework evaluating answer and reasoning consistency.
  • B Related Work: MoE routing selects a sparse subset of expert networks per token and exposes router distributions, per-expert hidden states, and expert usage patterns that dense models do not provide.Despite the growing prominence of MoE models, these signals remain largely unexplored for hallucination detection.

C Additional Method Details · C.1 Signal Summary · C.2 Inference Procedure

The appendix documents InnerExpert’s signal inventory and end-to-end inference procedure. It distinguishes standard from MoE-specific per-token signals and describes how generated-token features become token- and answer-level hallucination scores.

  • C Additional Method Details: The appendix provides reference material for InnerExpert’s signals, inference procedure, and training pipeline.
  • C.1 Signal Summary: InnerExpert groups extracted signals into standard transformer signals and MoE-specific signals derived from MoE blocks.
  • C.1 Signal Summary: All per-layer signals accumulate over the generated prefix y≤t, producing a per-token value at each generation step t.
  • C.1 Signal Summary: Table 5 defines host-model feature dimensionality using L, H, N, |Φt|, Params, and Act.L denotes layers, H attention heads, N experts, |Φt| per-token feature dimensionality, Params total parameters, and Act. active parameters per token.
  • C.1 Signal Summary: For Gemma, N=129 consists of 128 routed experts and one shared expert that is always active.
  • C.2 Inference Procedure: During greedy decoding, InnerExpert collects internal signals at each token position and assembles them into Φt.Given question x and optional evidence e, the host model generates y=(y1,...,yT).
  • C.2 Inference Procedure: The trained classifier outputs s(Φt)∈[0,1] per-token hallucination scores, while max aggregation yields the answer-level score S(y).

C.3 Training Procedure

InnerExpert is trained from automatically labeled token-level hallucination data generated under base and RAG conditions, using extracted per-token feature vectors and grouped stratified validation. After model selection, the best family model is refit on combined training and validation data, with its decision threshold optimized for F1.

  • Data and labeling: Training uses RealtimeQA questions with answers generated under base and RAG conditions, while an LLM judge supplies answer-level labels and hallucinated spans mapped to binary token labels.Feature vectors for all generated tokens are assembled into a training matrix.
  • Data and labeling: A stratified train/validation split is grouped by question identifier to prevent related answers from crossing partitions.
  • Model selection: The best model from each family is refit on the combined train+validation data.
  • Model selection: The decision threshold τ is F1-optimized on the combined set through a precision-recall curve sweep.

D Implementation Details

InnerExpert requires modified MoE forward passes to expose per-expert hidden states and incurs modest memory overhead during generation. The implementation also requires substantial storage and includes a re-implementation of HaluNet from its published architecture.

  • Expert hidden state extraction: InnerExpert monkey-patches each MoE block to save prerouting per-expert outputs, since standard implementations expose only routing-weighted combinations.These saved states support signals such as expert hidden score and expert similarity.
  • Memory usage: Full instrumentation raises OLMoE peak GPU memory from 12.7 GB to 13.1 GB, an overhead of approximately 3%.For Gemma, peak memory increases from 46.8 GB to 48.1 GB (2.8%).
  • Storage requirements: The project uses approximately 1.7 TB of disk space, largely because raw batch files are saved during generation.Pretrained models account for 119 GB, trained InnerExpert detectors for 311 MB, and the datasets require 0.9 TB, 0.17 TB, and 0.48 TB respectively.
  • HaluNet re-implementation: HaluNet is re-implemented with log-likelihood, entropy, and hidden-state embedding branches whose outputs are fused via attention.The first two branches use mean pooling and 2-layer MLPs; the embedding branch uses two 1D convolutional layers with kernel size 3, padding 1, ReLU, and adaptive average pooling.

E Datasets Description · F Additional Experiment Details

The experiments use RealtimeQA for training and temporal out-of-distribution testing, while four additional datasets assess cross-dataset generalization. Additional details specify the InnerExpert training procedure and classifier hyperparameter grids.

  • E Datasets Description: RealtimeQA uses January 2024–December 2025 questions for training and 365 questions from January–June 2026 for temporally out-of-distribution testing.Each question includes retrieved reference-evidence passages, enabling evaluation on post-training events.
  • E Datasets Description: SQuAD evaluation samples 200 validation questions, each paired with Wikipedia context serving as reference evidence.SQuAD contains over 100,000 crowdworker questions overall.
  • E Datasets Description: TruthfulQA evaluation samples 200 train-split questions and uses each provided answer as evidence despite lacking reference passages.TruthfulQA contains 817 questions across 38 categories.
  • E Datasets Description: NQ-Open evaluation samples 200 validation questions and uses the provided answer as evidence because its open-domain setting omits accompanying documents.NQ-Open retains questions with short answers after discarding evidence documents.
  • E Datasets Description: InnerExpert training generates base and retrieval-augmented answers, obtains binary labels and hallucinated spans from an LLM judge, and maps spans to token labels.Feature extraction follows Algorithm 1; classifier families are tuned with grouped train/validation splits, then threshold τ is optimized for F1.
  • E Datasets Description: FreshQA evaluation samples 200 questions from a November 2025 snapshot, including items requiring changing-world knowledge and false-premise debunking.The benchmark is dynamic and includes questions with false premises.
  • F Additional Experiment Details: Table 7 reports the hyperparameter grids used for each classifier family.The passage identifies the table’s purpose but does not provide the grid values.

F.1 Training Dataset Analysis

The training dataset is generated from host-model answers with and without evidence, then labeled by an LLM judge for answer-level grounding and exact hallucinated spans. Examples illustrate that evidence can correct unsupported answers, though OLMoE still produces hallucinations in some evidence-supported cases.

  • Data generation: Training prompts elicit concise answers about recent events both with and without reference evidence.The templates distinguish a base condition without evidence from an evidence-supported condition.
  • Labeling protocol: The LLM-as-a-judge labels each answer as hallucinated or grounded and identifies hallucinated spans as exact answer substrings.A grounded answer receives label 0 with an empty hallucinated_spans list.
  • Training examples: OLMoE is shown hallucinating both without evidence and with evidence, including unsupported answers about a film award, a restaurant opening, darts, and a cruise-ship outbreak.The examples contrast unsupported no-evidence responses with evidence-conditioned responses that sometimes remain incorrect.

F.2 LLM-as-a-Judge Label Validation (RQ4) · F.3 Per-token hallucination detection (RQ1 - Cont.) · F.4 Comparison to baselines (RQ2 - Cont.)

LLM-as-a-judge labels align substantially better with human annotations than weak overlap-based labels, while confidence and class distributions expose limitations in the weak heuristic. For detection, InnerExpert variants show threshold-sensitive F1 performance, with MLP and logistic regression generally most consistent across host models.

  • F.2 LLM-as-a-Judge Label Validation (RQ4): Lexical and semantic overlap metrics often fail to assess factuality and faithfulness robustly, motivating comparison with LLM-as-a-judge labels.The evaluated weak heuristic used metrics such as BLEU, ROUGE, and BERTScore.
  • F.2 LLM-as-a-Judge Label Validation (RQ4): 86.5 % accuracy: the LLM judge made 27 misclassifications out of 200 human-validated samples, versus 59.5 % accuracy for the weak heuristic.The LLM judge produced 20 false hallucinated labels and 7 false grounded labels.
  • F.2 LLM-as-a-Judge Label Validation (RQ4): 74.4 % for OLMoE and 82.0 % for Gemma: hallucinated LLM-judge labels concentrated in the weak heuristic’s [0.8, 1.0] confidence bin.High disagreement still occurred when the weak heuristic assigned high confidence, highlighting missed hallucinations.
  • F.2 LLM-as-a-Judge Label Validation (RQ4): 83.0 %: OLMoE answers were labeled hallucinated overall, compared with 49.4 % for Gemma, with stronger imbalance in base than evidence conditions.OLMoE had 99.7 % hallucinated base answers and 66.2 % with evidence; Gemma had 85.4 % and 13.3 %, respectively.
  • F.3 Per-token hallucination detection (RQ1 - Cont.): IE (MLP) achieved the highest average token-level F1 on OLMoE, while IE (LR) achieved it on Gemma.IE (XGBoost), despite being strongest under AUROC, degraded sharply on Gemma, showing that F1-optimized thresholds may not generalize.
  • F.4 Comparison to baselines (RQ2 - Cont.): IE (MLP) achieved the highest average answer-level F1 on both OLMoE and Gemma.IE (XGBoost) again degraded sharply on Gemma, while OLMoE F1 values were often inflated by extreme label imbalance.
  • F.4 Comparison to baselines (RQ2 - Cont.): F1 rankings were less informative than AUROC on OLMoE because many methods exceeded 0.94 amid extreme label imbalance.The imbalance was documented in Table 11, where OLMoE had 83.0 % hallucinated answers overall.

F.5 Signal contribution analysis (RQ3 - Cont.)

At F1, IE (MLP) preserves the combination advantage and achieves the strongest results across both models and evaluation levels. IE (XGBoost) instead trails several individual signals, reflecting threshold sensitivity rather than weaker discriminative power.

  • F.5 Signal contribution analysis (RQ3 - Cont.): IE (MLP) achieves the highest F1 on both models at both evaluation levels.The combination benefit observed under AUROC is preserved for IE (MLP).
  • F.5 Signal contribution analysis (RQ3 - Cont.): IE (XGBoost) underperforms several individual signals at F1 on both models.This reverses the AUROC finding.
  • F.5 Signal contribution analysis (RQ3 - Cont.): The F1 reversal reflects threshold sensitivity rather than reduced discriminative power.The passage contrasts F1 behavior with the AUROC finding across both models.

G Future Work

Future work includes certifying model τ-lucidity, adapting existing hallucination-detection approaches, extending InnerExpert to controlled text generation, and improving scalability across domains, tasks, languages, and MoE architectures without retraining.

  • Future Work: Future work could certify a model’s τ-lucidity, guaranteeing no hallucination with probability 1 −τ within a given domain.This direction builds on the threshold discussion in Section 3.1.
  • Future Work: Researchers could adapt existing hallucination-detection approaches or extend InnerExpert as a Controlled Text Generation method.The passage cites Fonseca, Bell, and Stoyanovich (2025) as an example of an approach to adapt.
  • Future Work: Scaling InnerExpert across domains, tasks, languages, and MoE architectures without retraining is identified as an important direction.This would address scalability across the listed settings.
Loading 2608.17687v1…