PaperlayerView exact revision on arXiv

Source-linked AI summary

LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals

Samuel Yeh, Sharon Li, Tanwi Mallick

arXiv:2509.21875v3cs.CL

TL;DR

RAG can still hallucinate despite sufficient, relevant retrieved documents, while prior utilization-based detectors depend on model- and dataset-specific tuning. LUMINA measures external-context and internal-knowledge signals and statistically validates them; across common benchmarks and four LLMs, it reports consistently high performance, including up to +13% AUROC over prior utilization-based methods on HalluRAG.

  • Problem

    RAG systems can hallucinate despite sufficient relevant retrieved context, and existing utilization-based detection methods require tuning that limits generalizability.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    LUMINA measures external-context utilization through distributional distance and internal-knowledge utilization through changes in token predictions across transformer layers, with statistical validation of the measurements.

    Source: §1 · p. 1

  • Results

    Across common RAG benchmarks and four LLMs, LUMINA performs consistently well and achieves up to +13% AUROC over prior utilization-based methods on HalluRAG.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The signals provide interpretable utilization insights, and reported robustness to retrieval noise and proxy LLMs supports the framework’s practicality.

    Source: §1 · p. 2, §5 · p. 10

    Section 3 assumes retrieved documents are correct, sufficient, and relevant, and that scoring and response generation use the same LLM, though the paper tests relaxing these assumptions.

    Source: §4.3 · p. 8

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) aims to mitigate hallucinations in large language models (LLMs) by grounding responses in retrieved documents. Yet, RAG-based LLMs still hallucinate even when provided with correct and sufficient context. A growing line of work suggests that this stems from an imbalance between how models use external context and their internal knowledge, and several approaches have attempted to quantify these signals for hallucination detection. However, existing methods require extensive hyperparameter tuning, limiting their generalizability. We propose LUMINA, a novel framework that detects hallucinations in RAG systems through context--knowledge signals: external context utilization is quantified via distributional distance, while internal knowledge utilization is measured by tracking how predicted tokens evolve across transformer layers. We further introduce a framework for statistically validating these measurements. Experiments on common RAG hallucination benchmarks and four open-source LLMs show that LUMINA achieves consistently high AUROC and AUPRC scores, outperforming prior utilization-based methods by up to +13% AUROC on HalluRAG. Moreover, LUMINA remains robust under relaxed assumptions about retrieval quality and model matching, offering both effectiveness and practicality. LUMINA: https://github.com/deeplearning-wisc/LUMINA

1 INTRODUCTION

RAG systems can still hallucinate despite sufficient, relevant retrieved documents, while prior utilization-based detectors rely on tuning choices that may not generalize. LUMINA measures external-context and internal-knowledge signals and reports strong detection results across benchmarks and models.

1 INTRODUCTION

  • Prior utilization-based methods rely on selected attention heads or layers, requiring extensive tuning that can limit generalization across models and datasets.LUMINA instead measures the signals in a layer-agnostic manner and introduces statistical hypothesis testing to validate them against verifiable implications.Source: §1 · pp. 1–2
  • LUMINA quantifies external-context and internal-knowledge utilization as signals for detecting RAG hallucinations.External utilization uses distributional discrepancy across document-conditioned next-token predictions; internal utilization tracks how token predictions evolve across transformer layers.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • More than 0.9 AUROC on HalluRAG across models, with up to +13% over prior state-of-the-art, is reported for LUMINA.The experiments span common RAG hallucination benchmarks and four LLMs.Source: §1 · p. 2
  • LUMINA outperforms score-based and learning-based hallucination-detection methods, according to the paper’s stated contributions.The paper also describes its framework for statistically validating the utilization measurements.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries motivate hallucination detection through the balance between retrieved evidence and internal knowledge, then define utilization-based scores. They also review earlier measures and identify tuning and validation gaps.

2.1 PROBLEM FORMULATION AND MOTIVATION

  • A response is conjectured to be more likely hallucinated when internal-knowledge utilization greatly exceeds external-context utilization, assuming relevant, correct, sufficient retrieval.The motivation is that RAG tasks requiring external knowledge should ground reasoning in that evidence.Source: §2.1 · pp. 2–3
  • The paper’s core question is how to quantify external-context and internal-knowledge utilization for generated tokens and responses.Token-level hallucination scores combine the two utilization signals with a weighting hyperparameter, and response-level scores average token scores.Source: §2.1 · p. 3

2.2 RELATED WORK

  • Sun et al. (2025b) measure context use with attention-weighted token similarity and internal use with hidden-state JS divergence; Wang (2025) uses semantic-entropy-based measures.The cited approaches select particular attention heads and transformer layers.Source: §2.2 · p. 3
  • Earlier utilization measures require nontrivial head and layer selection, with dataset- and model-specific tuning that limits generalizability.The paper also notes that prior work did not validate whether its scores truly reflect context and internal-knowledge utilization.Source: §2.2 · p. 3

3 METHODOLOGY

LUMINA quantifies external-context and internal-knowledge utilization, then statistically tests whether its measurements capture those signals.

3.1 QUANTIFYING EXTERNAL CONTEXT UTILIZATION

  • LUMINA measures external-context utilization as the distance between predictive distributions conditioned on retrieved versus random documents, using token-embedding distributions and MMD.The measure is non-parametric and LLM-agnostic, making it applicable across models and datasets.Source: §3.1 · p. 4

3.2 QUANTIFYING INTERNAL KNOWLEDGE UTILIZATION

  • LUMINA measures internal-knowledge utilization with an information processing rate that tracks how actively next-token predictions change across transformer layers.The logit lens projects layer hidden states to vocabulary distributions; the rate weights prediction differences by layer depth and normalizes by prediction uncertainty.Source: §3.2 · p. 5

3.3 STATISTICAL VALIDATION OF THE MEASUREMENT

  • All four validation implications reject their null hypotheses, supporting the measurements across four instruction-tuned LLMs.The tests use more than 65k tokens.Source: §3.3 · p. 6, Table 1 · p. 6

4 EXPERIMENTS

LUMINA is evaluated across RAG hallucination benchmarks and open-source LLMs, with comparisons covering main results, relaxed assumptions, and component ablations.

4.1 EXPERIMENTAL SETTINGS

  • Evaluation covers RAGTruth and HalluRAG, four instruction-tuned open LLMs, and AUROC, AUPRC, and PCC metrics.The compared baselines include uncertainty, cross-sample consistency, verbalization, and utilization-based strategies.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • >0.9 AUROC on HalluRAG across models, with LUMINA almost always outperforming ReDeEP across datasets and LLMs.Table 2 summarizes the cross-dataset and cross-model comparison; the gap is particularly large on HalluRAG.Source: Table 2 · p. 7, §4.2 · p. 7
  • LUMINA is competitive with a supervised SAPLMA classifier and sometimes outperforms it, despite requiring no labeled training data.The paper treats the supervised baseline as a performance upper bound.Source: §4.2 · p. 8

4.3 RELAXING ASSUMPTIONS

  • RAGTruth experiments show stable performance for most models under 0–30% context noise, while Llama2-13B drops by more than 0.1 after noise injection.Removing retrieved-document sentences even raises AUPRC for Llama3-8B.Source: §4.3 · p. 8, Figure 2 · p. 8
  • Llama2-7B can match or exceed Llama3-8B on Llama3-8B responses, and Llama2-7B and Llama3-8B remain stable across generation models.Qwen3-4B and Llama4-17B also perform well across generation LLMs.Source: §4.3 · p. 8, Figure 3 · p. 9

4.4 ABLATION STUDY

  • The optimal RBF kernel performs similarly to cosine, and cosine is the default because it depends less on hyperparameters.The study compares cosine with several RBF kernel settings on RAGTruth.Source: §4.4 · p. 9, Figure 4 · p. 9
  • Combining external-context and internal-knowledge scores yields the highest AUPRC for every LLM, improving Llama2-13B by more than 10%.For Llama2-13B, a high internal-knowledge score with a low external-context score is associated with greater likelihood of hallucination.Source: §4.4 · p. 9

5 CONCLUSION

LUMINA quantifies how RAG generations use external context and internal knowledge. Across benchmarks and four LLMs, it performs consistently well and is robust to context noise and proxy-LLM detection.

  • LUMINA quantifies external-context and internal-knowledge utilization to assess how LLMs balance retrieved evidence against parametric knowledge.Source: §5 · p. 9
  • Across common benchmarks and four LLMs, LUMINA consistently performs well, outperforming prior utilization methods and competing with supervised hallucination detectors.Source: §5 · pp. 9–10
  • Combining external-context and internal-knowledge scores boosts hallucination-detection performance.Source: Figure 5 · p. 10
  • Analyses find LUMINA robust to retrieved-document noise and usable with proxy LLMs for generation-LLM-agnostic detection.Source: §5 · p. 10

ETHICS STATEMENT

The paper frames LUMINA as a way to estimate context and knowledge utilization in RAG responses, with the goal of improving reliability and informing future hallucination-mitigation work.

  • The ethics statement presents LUMINA as a way to estimate external-context and internal-knowledge utilization in RAG responses.Source: ETHICS STATEMENT · p. 10
  • The authors state that improved hallucination detection may help increase RAG reliability and reduce misinformation risk in deployment.Source: ETHICS STATEMENT · p. 10
  • Further analysis of context and knowledge use may inform approaches to mitigate hallucinations and build more reliable AI systems.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper documents implementation details and releases code to support reproducibility, describes possible broader impacts, and outlines the comparison baselines.

REPRODUCIBILITY STATEMENT

  • Implementation details, experimental settings, and code are reported to help future studies reproduce the experiments.The codebase is publicly available.Source: REPRODUCIBILITY STATEMENT · p. 10

A BROADER IMPACTS

  • LUMINA's score validation framework may help empirically validate mechanistic-interpretability hypotheses, while its information processing rate offers a lens on LLM internal states.The authors suggest deeper study could help characterize how LLMs reason and leverage internal knowledge, and could inform training objectives or decoding algorithms.Source: A BROADER IMPACTS · p. 14

B DETAILS OF BASELINES

  • Perplexity measures response uncertainty, while LN-Entropy uses sequence-length-normalized entropy and Focus scores informative tokens using entropy, token probability, and attention weights.Higher LN-Entropy indicates greater uncertainty and a higher likelihood of hallucinations.Source: B DETAILS OF BASELINES · p. 14
  • SelfCKGPT checks consistency between a target answer and sampled responses with an NLI model, while EigenScore measures semantic consistency through response-covariance eigenvalues.Both methods use a sample size of 20 in these experiments; the SelfCKGPT setting follows Manakul et al. (2023).Source: B DETAILS OF BASELINES · p. 15
  • P(True) estimates whether an LLM judges a generated answer true, whereas RefChecker extracts claims and checks their consistency with reference documents.The experiments use a claim-extraction model fine-tuned by Hu et al. (2024) for RefChecker.Source: B DETAILS OF BASELINES · p. 15
  • ReDeEP combines external-context and internal-knowledge signals, using attended-token similarity and changes in vocabulary distributions across feed-forward layers.It weights and sums the two scores to produce a hallucination score.Source: B DETAILS OF BASELINES · p. 15

C DETAILS OF DATASETS

The experiments use RAGTruth and HalluRAG, datasets with different annotation sources, tasks, and evaluation scopes.

  • RAGTruth is a human-annotated dataset with 15,090 training and 2,700 test examples spanning summarization, data-to-text generation, and question answering.It includes answers from six LLMs and an extended test set with Llama3-8B responses.Source: C DETAILS OF DATASETS · p. 15
  • HalluRAG is an LLM-annotated question-answering dataset with GPT-4o labels verified by humans, and evaluation uses only its answerable questions.Questions are generated from Wikipedia sentences, and answers are produced using the relevant article.Source: C DETAILS OF DATASETS · p. 15

D IMPLEMENTATION DETAILS OF LUMINA

LUMINA approximates its external-context score using top-token distributions and calibrates its internal-knowledge score to account for sampled generation.

D IMPLEMENTATION DETAILS OF LUMINA

  • External-context MMD is approximated using the top 100 tokens, and the comparison distribution comes from another data point's retrieved documents.In a real RAG system, the comparison documents can instead be random or less relevant retrieved documents.Source: D IMPLEMENTATION DETAILS OF LUMINA · p. 15
  • Table 3 reports PrecOpt, RecallOpt, and F1Opt for LUMINA and baselines on each dataset.Its caption characterizes LUMINA as achieving a balanced precision-recall trade-off and high F1 score.Source: Table 3 · p. 16
  • The internal-knowledge score is calibrated by the ratio between the generated token's probability and the highest-probability token's probability.This adjustment accounts for generation sampling, since the generated token may not be the highest-probability token.Source: D IMPLEMENTATION DETAILS OF LUMINA · pp. 15–16

E.1 EVALUATION WITH OTHER METRICS

LUMINA shows a balanced precision-recall trade-off, competes with a supervised baseline, and performs well under multiple λ settings. Its component combination is also evaluated.

E.1 EVALUATION WITH OTHER METRICS

  • 0.9 PrecOpt and 0.9 RecallOpt on HalluRAG with Mistral-7B; LUMINA shows a balanced precision-recall trade-off across datasets and LLMs.The authors state that this balance means LUMINA does not over-predict hallucinations to achieve a high F1Opt score.Source: §E.1 · p. 16

E.2 COMPARE WITH SUPERVISED BASELINES

  • LUMINA performs competitively against SAPLMA and sometimes outperforms it, although Table 4 omits Llama3-8B because it is absent from SAPLMA's training set.Table 4 reports AUROC, Pearson's correlation coefficient, and AUPRC.Source: §E.2 · p. 17, Table 4 · p. 17

E.3 PERFORMANCE WITH HYPERPARAMETER TUNING

  • For Llama2-13B and Mistral-7B, λ = 0.5 is optimal; for Llama2-7B and Llama3-8B, λ = 0.2 is optimal.For the latter two models, performance drops by less than 0.025 at λ = 0.5; the figure reports RAGTruth AUPRC. The component study reports that both information-processing-rate components are important.Source: §E.3 · p. 17, Figure 6 · p. 17, Table 5 · p. 18

E.4 ADDITIONAL ABLATION STUDY

The ablations find LUMINA’s results are stable across tested MMD approximation sizes and random documents. Both components of its information processing rate contribute to overall performance.

E.4 ADDITIONAL ABLATION STUDY

  • < 0.02% AUROC difference across k ∈{50, 100, 500} on RAGTruth indicates insensitivity to the tested MMD approximation sizes.The authors suggest k = 50 when computational power is limited.Source: §E.4 · p. 18
  • < 0.0025 standard deviation across five random-document choices on RAGTruth indicates robustness to the choice of random document.Source: §E.4 · p. 18
  • Both layer weighting probability ratio and entropy normalization contribute to overall AUROC, supporting the information processing rate design.Table 5 reports the AUROC of ablating these two components.Source: §E.4 · p. 18

F ERROR ANALYSIS

The error analysis attributes many errors to label problems and low-quality retrieval, while also identifying task- and score-dependent edge cases. LUMINA requires two forward passes and is reported as more efficient than ReDeEP.

F ERROR ANALYSIS

  • Some apparent errors are annotation mistakes: annotators may miss fabricated content or label semantically equivalent content as hallucination, cases LUMINA detects correctly.Source: F ERROR ANALYSIS · p. 18
  • Summarization produces many false negatives when contradictory content has a high internal knowledge score but remains mostly grounded, yielding a relatively low hallucination score.The authors note that hallucination-score distributions may differ by task and recommend evaluating detection separately for each task.Source: F ERROR ANALYSIS · p. 18
  • Irrelevant or vague retrieved documents can lead to false positives when the model relies on internal knowledge to answer or says it cannot answer.Source: F ERROR ANALYSIS · p. 18
  • Many LUMINA errors reflect incorrect labels or low-quality retrieved documents, while edge cases also arise from task-dependent biases.The error analysis samples 50 false positives and 50 false negatives for GPT-5 classification; Table 6 reports proportions by error type.Source: F ERROR ANALYSIS · p. 19, Table 6 · p. 19
  • Errors can occur when internal and external context scores are close, particularly on reasoning-intensive tasks; task-specific score balance may improve performance.Source: F ERROR ANALYSIS · p. 19

G COMPUTATIONAL RESOURCES

  • Two forward passes suffice; for responses around 150 tokens, scoring averages less than 1 second, and LUMINA is reported as more efficient than ReDeEP.The authors attribute the efficiency to output-layer MMD approximation and applying the logit lens once per layer for internal knowledge scoring.Source: G COMPUTATIONAL RESOURCES · p. 19
Loading 2509.21875v3…