The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations

Motivation

• LLM hallucinations create risks of false information and misuse alongside the benefits of generative AI.The paper connects these concerns to calls for AI-training moratoria, copyright questions, and regulatory proposals.Source: §1 · pp. 1–2

Taxonomy

• The paper distinguishes factual mirage and silver lining orientations, each divided into intrinsic and extrinsic forms with mild, moderate, and alarming severity.These distinctions profile hallucination by prompt factuality, orientation, and degree.Source: §1 · p. 3
• It categorizes hallucination into six types: acronym ambiguity, numeric nuisance, generated golem, virtual voice, geographic erratum, and time wrap.The categories provide a finer-grained vocabulary for analyzing hallucination behavior.Source: §1 · p. 3

Resources

• HILT is a publicly available dataset containing 75,000 text snippets generated by 15 contemporary LLMs with human annotations.The annotations cover the proposed hallucination categories.Source: §1 · p. 3

Measurement

• HVI provides a quantitative spectrum for evaluating and ranking LLMs according to their vulnerability to hallucination.The paper positions HVI as a resource for hallucination research, mitigation assessment, and potentially AI policy-making.Source: §1 · p. 3

Mitigation

• The proposed mitigation approach identifies high-entropy words in outputs from high-HVI LLMs and replaces them using a lower-HVI LLM.The paper presents this as one of two strategies for alleviating hallucination.Source: §1 · p. 3

Scope and motivation

• Prior work studied hallucination notions loosely and in isolation, whereas this paper examines it across NLG tasks using fundamental text-generation principles.The authors state that hallucination can occur in any NLG task and that their findings can extend across NLP tasks.Source: §2 · p. 3

Orientations

• Factual Mirage concerns factually correct prompts, while Silver Lining concerns factually incorrect prompts; each has intrinsic and extrinsic subcategories.FM includes distortion or hallucination of correct prompts, whereas SL involves elaborate narratives based on incorrect prompts.Source: §2.1 · p. 3, §2.1.1 · pp. 3–4, §2.1.2 · p. 4

Categories of hallucination

• The six hallucination types are Acronym Ambiguity, Numeric Nuisance, Generated Golem, Virtual Voice, Geographic Erratum, and Time Wrap.These categories cover imprecise acronym expansions, inconsistent numerical values, fabricated personalities, unverifiable quotations, incorrect locations, and fused timelines.Source: §2.2 · pp. 4–5

Degrees of hallucination

• Hallucination severity is annotated as mild, moderate, or alarming, labeled 0, 1, and 2 respectively.Mild hallucinations are superficial; moderate hallucinations introduce fictitious or tangential facts; alarming hallucinations radically diverge from the prompt.Source: §2.3 · p. 5

Dataset construction

• HILT uses NYTimes tweets and Politifact headlines as prompts for 15 LLMs, generating 75,000 text passages categorized into factual mirage and silver lining.Each model produces 5,000 entries, divided into 2,500 FM and 2,500 SL passages; the table caption reports 129K annotated sentences overall.Source: : HallucInation eLiciTation dataset · p. 5, Table 1 · p. 5

• HVI quantifies and ranks LLMs according to their vulnerability to producing hallucinations.The measure accounts for hallucinated-sentence ratios, factual-mirage and silver-lining tendencies, and is scaled from 0–100 for interpretability.Source: §4 · p. 6
• Larger LLMs without RLHF are prone to both orientations of hallucination.Source: §4 · p. 6
• Numeric Nuisance and Acronym Ambiguity are mild categories whose silver-lining orientation decreases as LLM size grows.Source: §4 · p. 6
• Time Wrap and Geographic Erratum become more prevalent with larger models, while Virtual Voice increases markedly from GPT-3.5 to GPT-4.Source: §4 · p. 6
• Generated Golem, Virtual Voice, and Geographic Erratum are rarely observed in smaller LLMs such as T5 and Dolly.Source: §4 · p. 6
• The relationship between LLM size and hallucination is not necessarily direct because training-data quality, factual training, overconfidence, and other factors also contribute.Source: §5 · pp. 7–8

• The paper proposes automatic high-entropy replacement and human-in-the-loop sentence-level fact-checking as two mitigation directions.The automatic method replaces high-entropy words from high-HVI outputs using a lower-HVI LLM; the human-in-the-loop method flags susceptible sentences for review.Source: §6 · p. 8

Automatic mitigation

• ENTROPYBB identifies high-entropy words and replaces them with predictions from a lower-HVI LLM.Consecutive high-entropy words are masked as one unit before replacement, which is especially effective for Generated Golem and Acronym Ambiguity.Source: §6 · p. 8
• For GPT-3 content, albert-large-v2 performs best at detecting high-entropy words, while distilroberta-base performs best at replacing them.Source: §6 · p. 8

Human-in-the-loop mitigation

• FACTUALITYGB retrieves 20 search results, selects relevant sentences, and uses textual entailment to classify generated sentences as support, refute, or not enough information.Source: §6.2 · p. 9, §6.2.1 · p. 9

Comparative performance

• ENTROPYBB primarily addresses Acronym Ambiguity and Numeric Nuisance, whereas FACTUALITYGB is more applicable to complex hallucinations.The reported comparison motivates combining black-box and gray-box approaches.Source: §6.2.1 · p. 9

• The benchmark and HVI are intended to serve the wider scientific community and aid policy-makers.Source: §7 · p. 9
• The benchmark and HVI will remain publicly open for further collaborative updates.Source: §7 · p. 9
• The two proposed mitigation techniques can serve as baselines.Source: §7 · p. 9

• HVI is proposed as a framework for assessing hallucination risk, mitigation, and machine-generated content.Source: §8 · p. 10
• The paper presents HILT and HVI as publicly accessible resources intended to support future research, policy-making, and collaborative updates.Source: §8 · p. 10
• The annotation scheme assigns one hallucination category per sentence despite acknowledged multi-class and multi-label instances.Source: §8 · p. 10
• The defined hallucination categories may become incomplete as new categories and combinations emerge with advances in LLMs.Source: §8 · p. 10
• The study’s selection of 15 contemporary LLMs may not encompass all available models in the rapidly changing field.Source: §8 · p. 10
• FACTUALITYGB distinguishes different entities through entailment, whereas ENTROPYBB handles token-level distinctions such as 1789 versus 1889.Source: §8 · p. 10

• The authors discourage malicious use of their findings to create AI-generated fake news indistinguishable from human-written content.Source: §9 · p. 11

Scope and motivation

• Hallucinations can sometimes have beneficial effects in computer vision, such as in-painting occluded image content, but are usually undesirable in text.Source: Frequently Asked Questions (FAQs) · p. 15
• The study focuses on text hallucination because downstream impacts are worsened by the lack of a programmatic way to distinguish hallucinated from factual output.Source: Frequently Asked Questions (FAQs) · p. 15
• Hallucinations are examined for both factually correct and factually incorrect prompts, motivating the Factual Mirage and Silver Lining data.Source: Frequently Asked Questions (FAQs) · p. 15

HVI and mitigation

• The mitigation approach detects and replaces high-entropy points because they are more uncertain and more likely locations for hallucination.Source: Frequently Asked Questions (FAQs) · p. 15
• HVI is intended as a uniform score specifically measuring LLM hallucination tendency, unlike general downstream-task metrics.Source: Frequently Asked Questions (FAQs) · p. 15
• The choice between black-box and gray-box mitigation methods is described as dependent on the LLM and task.Source: Frequently Asked Questions (FAQs) · p. 15

Annotation procedure

• Annotation uses three layers—orientation, category, and degree—applied through a sentence-wise procedure.Source: §B.2 · pp. 16–17
• The HILT dataset was constructed with a web interface displaying the prompt and generated text to annotators.Source: §B.3 · p. 17, Figure 9 · p. 17

Annotator quality control

• A 2,000-sample in-house annotation process supported guideline development and annotator selection before crowdsourcing.Source: §B.4 · p. 17
• Only AMT annotators exceeding 90% accuracy on the in-house dataset were selected for the pilot task.Source: §B.4 · p. 18
• Compensation was iteratively adjusted because both insufficient pay and excessive wages could undermine annotation quality.Source: §B.4 · p. 18

Label scheme

• The annotation scheme labels orientation, category, and degree in three successive levels, including miscellaneous category 7 when needed.Source: §B.4 · p. 18

Outcome

• Qualified annotators and suitable compensation rates improved the quality of crowdsourced annotations.Source: §B.4 · p. 18

Agreement measures

• Fleiss’s kappa and Krippendorff’s alpha measured agreement among three annotators across the NYT and Politifact datasets.Fleiss’s kappa handles multiple annotators, while Krippendorff’s alpha accommodates nominal and ordinal data.Source: §B.5 · p. 18

Agreement results

• Nearly or more than 80% agreement was obtained for orientation and category annotations in both datasets.The passage reports substantial to almost perfect agreement across all three annotation types.Source: §B.5 · p. 18
• Degree annotation showed slight variation because annotators differed in their subjective assessment of the percentage of hallucination in a sentence.The passage attributes this variation to differing interpretations of percentage.Source: §B.5 · p. 18

Datasets

• HILT used NYT for factually correct prompts and Politifact for factually incorrect prompts.These two datasets supported curation of the hallucination dataset.Source: - Prompt sources · p. 19

Illustrative examples

• Tables 7 and 8 provide pre- and post-mitigation examples identifying hallucinated, high-entropy, replaced, uninformative, and refuted text fragments.The labels distinguish hallucinated fragments, high-entropy fragments, replacements, highlighted text where no information was found, and textual-entailment refutations.Source: Table 7 · p. 21, Table 8 · p. 22
• Tables 9 to 23 illustrate overall hallucination drops across 16 combinations of four LLMs for multiple generating models.Rows represent models detecting high-entropy words, and columns represent models replacing those words.Source: §F.1 · p. 22, Table 10 · p. 23, Table 11 · p. 23

Overall hallucination drops

• Table 9 reports a 10.66 maximum overall hallucination drop for GPT-3, using albert-large-v2 for detection and distilroberta-base for replacement.The tables vary the detector and replacer across 16 combinations of four LLMs.Source: Table 9 · p. 22
• Maximum drops are also reported for Dolly, OPT, Alpaca, T0, XLNet, and T5 across detector–replacer combinations.The listed maxima are 8.20, 8.58, 9.80, 8.40, 9.00, and 7.90, respectively.Source: Table 16 · p. 24, Table 17 · p. 24, Table 19 · p. 25

Sampling and evaluation

• 2,000 data points were manually evaluated, comprising 500 instances each of IFM, EFM, ISL, and ESL.The sample was designed to maintain a balanced distribution across the six hallucination categories.Source: §F.2 · p. 26
• FACTUALITYGB treated hallucination as absent when highlighted sentences were rewritten by humans.This evaluation rested on the assumption that human rewriting removes hallucination.Source: §F.2 · p. 26

Comparative findings

• ENTROPYBB primarily tackles less complex categories such as acronym ambiguity and numeric issues.The result is reported from the relative analysis in Fig. 5 and Table 24.Source: §F.3 · p. 26
• FACTUALITYGB is more applicable to complex hallucination cases than ENTROPYBB.The passage presents the two techniques as complementary in their category coverage.Source: §F.3 · p. 26
• Table 24 compares Silver Lining and Factual Mirage HVI scores before and after ENTROPYBB and FACTUALITYGB across all six categories.The table reports the comparative HVI analysis for both mitigation techniques.Source: Table 24 · p. 27