PaperlayerView exact revision on arXiv

Source-linked AI summary

Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation

Runsong Jia, Zhen Fang, Mengjia Wu, Jie Lu, Yi Zhang

arXiv:2609.08267v1cs.AIcs.IR

TL;DR

Hallucination detection remains difficult when model knowledge is outdated or incomplete, motivating evidence-grounded verification in RAG systems. EAEV aligns generated entities with retrieved contexts across multiple dimensions and tests stability under perturbations. It reports consistent improvements, including 87.89% AUROC on LLaMA2-13B, with generalization across benchmarks and architectures.

  • Problem

    Existing hallucination detectors rely heavily on internal signals, while outdated or incomplete model knowledge limits reliable detection of recent and specialized information.

  • Method

    EAEV performs entity-level verification using identity, semantic, and consistency alignment plus counterfactual stability analysis over retrieved evidence.

  • Results

    87.89% AUROC on LLaMA2-13B is reported, alongside consistent improvements and strong generalization across RAG benchmarks and model architectures.

  • Takeaways & Limitations

    Evidence-traceable entity verification provides a practical direction for improving hallucination detection in evidence-grounded generation systems.

  • Takeaways & Limitations

    EAEV depends on the quality and coverage of retrieved evidence, and it does not directly mitigate hallucinations during generation.

Abstract

from arXiv · show

Hallucination detection is crucial for large language models (LLMs), as hallucinated content creates significant barriers in applications requiring factual accuracy. Current detection methods mainly depend on internal signals like uncertainty and self-consistency checks, using the model's pre-trained knowledge to identify unreliable outputs. However, pre-trained knowledge may become outdated and has coverage limitations, especially for specialized or recent information. To address these limitations, retrieval-augmented generation (RAG) has emerged as a promising solution by retrieving relevant evidence at inference time, grounding outputs beyond the model's parametric knowledge. In this paper, we target a critical and practical learning problem RAG-based hallucination detection (RHD), where RAG is employed to enhance hallucination detection by addressing information updating challenges. To address RHD, we propose a novel method Evidence-Aligned Entity Verification (EAEV), which detects entity-level hallucinations by leveraging RAG to align generated entities with retrieved evidence contexts. Specifically, EAEV evaluates entity-evidence alignment through three complementary dimensions and introduces counterfactual stability analysis to ensure robust alignments under evidence perturbations. Experiments across multiple RAG benchmarks demonstrate that EAEV achieves consistent improvements over existing methods with strong generalization capabilities.

1 Introduction

RAG-based hallucination detection addresses the limits of internal-signal methods by aligning generated entities with retrieved evidence. EAEV combines complementary alignment dimensions and counterfactual stability analysis, achieving strong performance and generalization across RAG benchmarks.

  • Motivation: Hallucinations create accuracy risks in practical LLM applications, especially in domains such as medical diagnosis, education, and financial advice.Undetected errors can propagate misinformation and contribute to decision-making failures.
  • Limitations of Existing Methods: Traditional detectors rely on uncertainty, consistency, or external verification, but often provide limited entity-level traceability or incur additional overhead in RAG pipelines.Their internal knowledge may also be outdated or insufficient for recent and specialized information.
  • Research Problem: RAG-based hallucination detection asks how retrieved contexts can support direct entity–evidence alignment for fine-grained factual verification.Models may fabricate entities even when correct information is present in the retrieved context.
  • Proposed Method: EAEV verifies entity mentions through identity, semantic, and consistency alignment, then applies counterfactual stability analysis to distinguish robust support from fragile matches.The method is designed to provide evidence-traceable, fine-grained decisions beyond answer- or sentence-level detection.
  • Results: 87.89% AUROC on LLaMA2-13B demonstrates EAEV’s reported effectiveness, alongside strong generalization across multiple RAG benchmarks and model architectures.The authors also report state-of-the-art results while maintaining practical deployability.

2 Learning Setups

The paper formulates RAG-based hallucination detection as evidence-alignment verification over queries, retrieved contexts, and generated answers. Its entity-centric framework extracts factual mentions, evaluates multiple alignment signals, and aggregates them into interpretable decisions.

  • 2.2 Traditional Hallucination Detection: Traditional hallucination detection maps a query and answer to a binary hallucination decision, but evidence may remain underutilized in RAG settings.The formulation is represented as ŷ = D(Q, A), where ŷ indicates hallucination.
  • 2.3 RAG-based Hallucination Detection: RAG-based hallucination detection determines whether a generated answer is supported by retrieved contexts rather than relying solely on model internals.The setup uses a query Q, retrieved contexts P, and generated answer A as inputs to detector D.
  • 2.4 Entity-Centric Verification Framework: Factual errors in RAG settings are treated as primarily entity-level, involving named entities, temporal expressions, and quantities that affect response reliability.This motivates verification at a finer granularity than whole-answer classification.
  • 2.4 Entity-Centric Verification Framework: The framework extracts candidate mentions of types ENT, NUM, and NP, retrieves evidence windows, and selects primary evidence for each mention.Mention types correspond to named entities, numerical values, and noun phrases.
  • 2.4 Entity-Centric Verification Framework: Verification combines identity, semantic, and consistency alignment with counterfactual stability analysis before aggregating mention signals into entity-level decisions.The resulting scores are intended to remain interpretable and directly traceable to evidence.

3 Methodology

EAEV frames RAG-based hallucination detection as entity-level evidence alignment, using retrieved contexts to localize factual inconsistencies. It combines identity, semantic, and consistency signals with counterfactual stability analysis, then aggregates them for entity-level verification.

  • RAG-based hallucination detection: EAEV treats hallucination detection as entity-level verification against retrieved evidence, improving localization beyond sentence- or paragraph-level decisions.The framework extracts factual commitments, selects primary evidence windows, and maintains evidence traceability throughout verification.
  • Multi-dimensional alignment: The method evaluates each mention–evidence pair through identity, semantic, and consistency alignment because factual support can fail in different ways.Identity captures lexical matches and aliases, semantic alignment captures paraphrases, and consistency alignment checks numerical overlap and contradictions.
  • Multi-dimensional alignment: Identity alignment prioritizes exact lexical matches while using fuzzy token-level matching to handle orthographic variations and aliases.The normalized token set ratio measures lexical overlap between the mention and selected evidence.
  • Multi-dimensional alignment: Consistency alignment combines numerical overlap with lightweight contradiction detection for temporal, numerical, and relational conflicts.Extracted numerical values are compared using normalized intersection over union, while negative signals identify explicit inconsistencies.
  • Entity-centric aggregation: The final verification score combines consistency and stability, producing entity risk scores that can be max-pooled for answer-level assessment while preserving evidence-window traceability.The resulting signals supervise a single-model verifier used for inference.
  • Counterfactual stability analysis: Counterfactual stability analysis tests whether evidence support persists when surface characteristics change, filtering fragile matches caused by spurious correlations.The perturbations include evidence removal, whitespace compression, and alphanumeric-only filtering; stability is summarized using minimum support and stability gaps.

4 Experiment

EAEV is evaluated across multiple RAG benchmarks, model backbones, baselines, and metrics, showing strong cross-model performance and generalization. Ablations identify counterfactual stability as especially important, while sensitivity analysis supports 30-token windows and robustness across 25–35 tokens.

  • Evaluation setup: EAEV is evaluated on RAGTruth, HotpotQA, and DelucionQA using Qwen2.5-7B, LLaMA2-7B, and LLaMA2-13B against multiple hallucination-detection baselines.Evaluation uses AUROC, Accuracy, and F1 score.
  • Main results: 84.72% average AUROC on Qwen2.5-7B, 79.63% on LLaMA2-7B, and 87.55% on LLaMA2-13B exceed the strongest TSV baseline by 2.29, 2.59, and 3.34 percentage points, respectively.The reported averages span the evaluated datasets.
  • Generalization: Cross-dataset results indicate generalization across general-knowledge, multi-hop reasoning, and precision-critical technical scenarios.RAGTruth, HotpotQA, and DelucionQA represent distinct reasoning and domain requirements.
  • Scalability: EAEV’s scaling behavior supports use across deployment regimes, with larger models benefiting from greater capacity and smaller models still achieving meaningful improvements.The authors characterize the framework as practical across varied model sizes and task demands.
  • Ablation analysis: Counterfactual stability analysis provides the largest ablation contribution, while all alignment dimensions contribute meaningfully and the full framework performs best.Balanced degradation patterns indicate that the components address distinct verification challenges.
  • Window sensitivity: 30-token windows yield optimal performance, while the framework remains reasonably robust within the 25–35 token range.Smaller windows limit contextual information, whereas larger windows introduce noise that dilutes alignment signals.

5 Conclusion

The paper concludes that EAEV performs entity-level verification through multidimensional evidence alignment and counterfactual stability analysis. Its reported improvements across benchmarks and architectures, together with context-only operation and interpretable signals, support practical RAG hallucination detection.

  • Method: EAEV performs entity-level verification through multi-dimensional evidence alignment and counterfactual stability analysis.The framework distinguishes genuine factual support from spurious correlations.
  • Implication: The findings support robust entity-level verification as a practical direction for hallucination detection in evidence-grounded generation systems.The stated scope is real-world RAG deployment in factual applications.
  • Conclusion: EAEV reports substantial improvements across benchmarks and model architectures while operating entirely within retrieved contexts.The conclusion also describes its verification signals as interpretable.

Limitations

The authors identify limitations involving output instability, dependence on retrieved-evidence quality and coverage, and the absence of direct hallucination mitigation during generation.

  • Output stability: LLM outputs can be unstable across runs, particularly for complex queries requiring multi-step reasoning.This is identified as a limitation of the approach’s operating setting.
  • Retrieval dependence: Because EAEV verifies within context, missing or incomplete retrieval may limit detection performance.The limitation concerns dependence on retrieved evidence quality and coverage.
  • Mitigation scope: EAEV does not directly mitigate hallucinations during generation, leaving integration with decoding or training-time mechanisms for future work.The paper focuses on accurate and interpretable detection rather than generation-time mitigation.

A.1.1 Traditional Hallucination Detection

Traditional hallucination detectors primarily use uncertainty estimation and self-consistency mechanisms derived from model outputs, with representative methods operating over semantic consistency or meaning-level divergence.

  • Internal-signal detection: Traditional detectors primarily leverage uncertainty estimation and self-consistency mechanisms within model outputs.The section presents these as representative internal-signal approaches.
  • Representative methods: SelfCheckGPT measures semantic consistency across multiple generations, whereas Semantic Entropy operates on meaning-level divergences.These methods exemplify distinct internal-signal detection strategies.
  • Recent directions: Recent approaches also explore attention-level interpretability and representational analysis of knowledge.The supplied passage names NoVo and representational analysis as examples.

A.1.2 Evidence-Based Hallucination Detection in RAG

RAG hallucination detection builds on evidence-based verification and complementary self-verification approaches. The cited methods include research-and-revision, atomic factual evaluation, and systematic self-verification.

  • RARR uses research and revision stages for evidence attribution and consistency-based correction.
  • FActScore evaluates factuality at the atomic level by decomposing generated text into verifiable claims.
  • CoVe performs self-verification through question generation and independent answering.

A.2 Experimental Details

Experiments evaluate EAEV across three RAG datasets, three backbone models, multiple detection metrics, ablations, sensitivity settings, newer backbones, entity-level detection, and resource usage. Results show broad generalization, robust configuration behavior, strong entity-level separability, and improved efficiency in the reported setup.

  • Datasets: RAGTruth contains approximately 18,000 annotated responses with word-level and instance-level hallucination labels across three task categories.Metrics are computed at the answer level.
  • Datasets: RAGBench contains approximately 100,000 examples, with HotpotQA testing multi-hop reasoning and DelucionQA covering automotive-manual question answering.Both components provide retrieved contexts and answer annotations for hallucination detection.
  • Models and evaluation: Experiments use Qwen2.5-7B, LLaMA2-7B, and LLaMA2-13B, comparing EAEV with eleven representative hallucination detection baselines.The reported metrics are AUROC, Accuracy, and F1 score.
  • Ablation analysis: Counterfactual stability analysis produces the largest ablation effect, with removal causing 5.52 average AUROC points of degradation across datasets.The results support distinguishing genuine evidence support from spurious correlations.
  • Sensitivity analysis: EAEV achieves best performance at 30-token answer-side windows, while performance degrades below 25 tokens and above 35 tokens.The framework remains reasonably stable within the 25-35 token range.
  • Generalization and efficiency: EAEV generalizes to Qwen3-8B, yields meaningful entity-level separability, and is faster and more memory efficient than SelfCheckGPT in the reported setup.SelfCheckGPT requires multiple additional LLM forward passes and similarity computations, whereas EAEV uses a single generation with evidence-based processing.
Loading 2609.08267v1…