A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection

• Passage-level hallucination detection is needed because users typically receive comprehensive responses, while sentence-by-sentence checking is inefficient and costly.Source: §1 · p. 1
• The paper introduces PHD, a benchmark for evaluating passage-level hallucination detection methods, and RV, a zero-resource method that detects factual errors without token probabilities or external knowledge.Source: §1 · p. 2
• RV significantly outperforms existing methods and baselines on the PHD benchmark and WikiBio-GPT3 while achieving competitive performance, lower token costs, and faster response times.Source: §1 · p. 2
• The study also evaluates robustness across different LLMs and manually analyzes failures, revealing a shared limitation of zero-resource detection methods.Source: §1 · p. 2

2.1 Hallucinations of LLMs

• Hallucinations vary by generation task; the paper distinguishes intrinsic contradictions from extrinsic statements unsupported by the source and primarily studies extrinsic hallucinations.Source: §2.1 · p. 2
• Extrinsic hallucinations can contain correct external information, so the paper uses hallucination mainly for non-factual statements or fabricated unverifiable knowledge.Source: §2.1 · p. 2

2.2 Mitigate Hallucinations

• Retrieval-based mitigation addresses knowledge gaps but depends on external databases, motivating zero-resource detection that can invoke retrieval only when hallucinations occur.Source: §2.2 · p. 2
• Token probability and perplexity provide intrinsic uncertainty signals, but token-level probability distributions are inaccessible for black-box models such as ChatGPT.Source: §2.2 · p. 2
• The paper focuses on passage-level zero-resource detection rather than the sentence-level setting emphasized by related work.Source: §2.2 · pp. 2–3

3.1 Data Generation

• PHD uses Wikipedia entities and articles as source material, sampling entities from low, medium, and high proxy data-volume domains.Source: §3.1 · p. 3
• The benchmark seeks natural hallucinations by selecting entities with different Google-derived proxies for training-data exposure, then prompting ChatGPT to generate Wikipedia-style passages.Source: §3.1 · p. 3

3.2 Human Annotation

• Passage-level annotation is challenging because workers must verify claims across long documents and websites, so the authors use strict worker selection and a two-stage process.Source: §3.2 · pp. 3–4
• The labels are factual when every claim has evidence, non-factual when any non-factual or unverifiable information remains, and unverifiable only during the first stage when Wikipedia evidence is insufficient.Source: §3.2 · pp. 3–4
• Quality-control examples produced average Cohen’s Kappa k=0.848 (0.80<=k<=1.00), indicating perfect consistency, while Fleiss’ Kappa achieved 0.77.Source: §3.2 · p. 4

3.3 Statistics

• PHD-Low contains nearly half hallucinated passages, whereas PHD-High rarely contains hallucinations, supporting a relationship between knowledge gaps and hallucination frequency.Source: §3.3 · p. 4, Figure 1 · p. 4, Table 1 · p. 4

4.1 Reverse Validation Method

• A matching returned entity indicates factual information, whereas a mismatch indicates hallucinated content because incorrect retrieval conditions produce wrong or missing entities.Source: §4.1 · p. 5
• RV reconstructs a response as a reverse query: it extracts an entity, uses the remaining information to query the LLM, and validates the returned answer against the entity.Source: §4.1 · p. 5
• The method formalizes forward response generation as P(R|Q) and P(I|E), then reverses the process as P(E|I) before entity-answer matching.Source: §4.1 · p. 5

4.2 Implementation of Variants

• The question-generation variant asks for a question about the entity without naming it, while the entity-matching variant rewrites information as requirements and returns a matching entity.Source: §4.2 · p. 5
• Entity matching reports the percentage of satisfied requirements, recognizes only matches exceeding 90 percent as factual, and exposes unmet requirements for interpretability.Source: §4.2 · p. 5
• Both variants use exact string matching for the final entity-answer comparison.Source: §4.2 · p. 5

5 Experimental Setup

• The experiments use a stable ChatGPT version with temperature 0.0, and Table 2 lists the prompts used at each Reverse Validation stage.The prompt table indicates that Question and Requirements are generated in the first stage.Source: §5 · p. 5, Table 2 · p. 6
• The study evaluates methods on the PHD benchmark and WikiBio-GPT3, a 200-passage subset labeled non-factual through aggregated sentence annotations.WikiBio-GPT3 is derived from 238 GPT3-generated passages using sentence-level labels and contains passages determined to be non-factual.Source: §5 · pp. 5–6
• PHD performance is measured with Precision, Recall, and F1, while WikiBio-GPT3 is evaluated with Accuracy.Precision measures the share of rejected passages that are non-factual, Recall measures rejected non-factual passages recovered, and F1 is their harmonic mean.Source: §5 · p. 6
• The evaluation compares replicated zero-resource detectors, two Reverse Validation variants, and an All False baseline for passage-level detection.The compared methods include LMvsLM revised, SelfCheckGPT via BERTScore, Zero-shot Detection, Reverse Validation via Question Generation, and Reverse Validation via Entity Matching.Source: §5 · pp. 6–7
• The two Reverse Validation variants construct queries through Question Generation or Entity Matching to access the language model’s parametric knowledge base.The variants differ in how they construct the reverse-validation query.Source: §5 · p. 7

6.1 Main Results

• Across all PHD domains, both Reverse Validation variants outperform the baselines, with Entity Matching performing better than Question Generation.The Entity-Answer Match stage benefits from assessing the percentage of requirements fulfilled by an entity.Source: §6.1 · p. 7, Table 3 · p. 8
• Both Reverse Validation variants perform well on PHD-LOW and PHD-Medium but struggle on PHD-High, suggesting difficulty may relate to entities’ training-data volume.This pattern is presented as a possible implicit relationship rather than a confirmed causal explanation.Source: §6.1 · p. 7
• The Reverse Validation variants detect fine-grained hallucinations in WikiBio-GPT3 passages containing only one non-factual sentence, whereas revised LMvsLM misses passages below a 60% hallucination ratio.Table 4 reports accuracy and the minimum hallucination ratio among rejected passages.Source: §6.1 · p. 7
• Reverse Validation has lower token costs and response latency than the compared methods on PHD, supporting its use in production-level systems.The efficiency comparison is reported in Table 5.Source: §6.1 · p. 7

6.2 Ablation Study Results

• Changing the backbone causes SelfCheckGPT via BERTScore to devolve into an All False baseline, while LMvsLM revised classifies most claims as non-factual.These effects arise because cross-model generated samples or answers diverge from the ChatGPT-generated claims; both ChatGPT and Llama-2-7b-chat-hf also fail in zero-shot passage-level detection.Source: §6.2 · pp. 7–8
• Reverse Validation via Question Generation remains relatively stable when switching from ChatGPT to Llama-2-7b-chat-hf, while Entity Matching degrades as expected with the smaller backbone.The ablation uses Llama-2-7b-chat-hf as an open-source backbone and reports results on PHD.Source: §6.2 · pp. 7–8

7 Case Study and Analysis

• The analysis includes manually examined false negatives and false positives, with Tables 7 and 8 presenting two cases of each error type.Source: §7 · p. 8, §7.1 · p. 8, §7.2 · p. 9

7.1 False Negative

• Exact string matching produces false negatives when answers use abbreviations or broader entities instead of the expected full or specific entity.Examples include an abbreviation for “Maximilian II Emanuel” and “Jordan” instead of “History of Jordan.”Source: §7.1 · p. 8
• LLM-based fuzzy matching addresses entity discrepancies but significantly improves precision while impairing recall.Source: §7.1 · pp. 8–9

7.2 False Positive

• Reverse Validation can mistake outdated information for factual content when its query matches stale knowledge in the model’s parametric memory.This occurs when real-world changes are absent from training data, producing confident responses with few internal inconsistencies.Source: §7.2 · p. 9
• Outdated-data hallucinations expose a shared limitation of zero-resource methods because confident stale knowledge creates few detectable inconsistencies.Source: §7.2 · p. 9

• The paper introduces PHD, a dataset for evaluating passage-level hallucination detection.Source: §8 · p. 9
• It proposes a zero-resource hallucination detection method and demonstrates effectiveness and robustness against existing methods on two datasets.Source: §8 · p. 9
• Manual failure analysis reveals a shared limitation of zero-resource hallucination detection methods.Source: §8 · p. 9

• Reverse Validation cannot identify hallucinations caused by outdated data, a limitation also shared by current zero-resource detection methods.Source: Limitations · pp. 9–10
• The approach is specifically tailored for passage-level hallucination detection and may not be suitable for sentence-level detection.Source: Limitations · p. 10

• ChatGPT generated the data, and hired workers annotated it while following the relevant code of ethics.Source: Ethics Statement · p. 10
• Annotators were paid approximately $8 per hour, above the local minimum wage standard.Source: Ethics Statement · p. 10