Real-Time Detection of Hallucinated Entities in Long-Form Generation

• The annotated legal example marks supported entities in green, hallucinated entities in red, and token-level probe scores with yellow intensity.Scores below 0.4 are omitted; examples include fabricated names, products, and broader details.Source: Figure 2 · p. 2
• The pipeline and probes can also flag broader fabricated claims that do not correspond cleanly to a single entity.The example includes the fabricated detail “at his residence in San Diego.”Source: Figure 2 · p. 2

• Long-form generation requires segment-level detection because factual and hallucinated claims can be intertwined across multi-paragraph responses.Short-form QA often contains one brief atomic claim, whereas long-form outputs contain numerous interconnected claims.Source: §1 · p. 2
• Entity-level token labeling enables streaming detection because entities have token boundaries, unlike claims that require post-hoc extraction after complete sentences.The approach targets fabricated names, dates, and citations as token-aligned indicators of factual errors.Source: §1 · p. 3
• 0.87 AUC versus 0.71 for semantic entropy on Llama-3.3-70B shows that linear probes outperform an uncertainty baseline in long-form hallucination detection.The probes use entity labels derived from a frontier LLM with web search and run during the model’s forward pass.Source: §1 · p. 3
• Training on long-form text transfers well to short-form QA and across model families, whereas short-form training does not recover long-form performance.Cross-model transfer suggests the probes capture patterns not limited to one model’s outputs.Source: §1 · p. 3
• LoRA probes reach 0.90 AUC on Llama-3.3-70B and can support real-time abstention when hallucination risk is detected.The paper uses KL regularization to limit changes to model behavior while improving probe performance.Source: §1 · p. 3
• The approach remains insufficient for robust deployment, despite suggesting a path toward scalable real-time monitoring.Further work is explicitly required before practical deployment.Source: §1 · p. 3

• Internal-state probes predict hallucinations from hidden representations, but their generalization to open-ended long-form generation remains unproven.Related work includes linear and MLP-based probes applied before or during generation.Source: §2 · p. 3
• Streaming token-level classifiers have been developed for context-grounded inconsistency, whereas this paper targets factual incorrectness against broader world knowledge.CH-Wang et al. [2024] focus on inconsistency with provided source documents; this work detects incorrect entities more broadly.Source: §2 · p. 3
• Semantic entropy measures uncertainty over clustered meanings, but sampling-based estimation is computationally intensive.Semantic entropy probes reduce cost by predicting semantic entropy from hidden states, with lower performance than the sampling-based method.Source: §2 · p. 4
• External verification systems extract claims, retrieve evidence, and evaluate each claim, causing latency and computational costs that impede real-time detection.A single sentence may generate tens of claims, each requiring multiple searches and LLM calls.Source: §2 · p. 4
• Entity-level detection preserves token alignment for fine-grained analysis when supported and fabricated information coexist within one response.Yeh et al. [2025] introduced HalluEntity, and this paper builds on that entity-level framing for supervised probes.Source: §2 · p. 4

3.1 Dataset construction and entity-level annotation

• LongFact++ expands LongFact with 10 times more prompts across topical, biographical, citation-focused, and legal domains.The prompts are designed to elicit entity-dense completions containing factual and hallucinated content.Source: §3.1 · p. 4
• A frontier LLM with web search extracts entity spans from original completions and labels them Supported, Not Supported, or Insufficient Information.The labels preserve original token boundaries for token-level training.Source: §3.1 · p. 5
• Human labels matched the LLM labels in 84% of sampled spans, while controlled-error evaluation found 80.6% recall and a 15.8% false positive rate.The controlled set contained 904 injected errors across 100 samples.Source: §3.1 · p. 5

3.2 Problem formulation and token-level probe training

• The detector assigns each token a hallucination probability and aggregates token scores by the maximum within each annotated entity span for evaluation.Entity labels are binary, with 1 denoting hallucinated and 0 supported entities.Source: §3.2 · p. 5
• The probe combines token-wise BCE with a span-max objective, annealing from dense token gradients toward focus on the most informative token.Entity-token weighting uses α=10 to prevent abundant background tokens from dominating the loss.Source: §3.2 · p. 6
• LoRA probes combine probe loss with language-modeling or KL regularization to constrain changes to the underlying model.Most experiments use language-modeling regularization with λ_reg=0.01, while linear probes use no regularization.Source: §3.2 · p. 6

3.3 Baselines

• Baselines include token-level entropy, token-level perplexity, semantic entropy, and black-box self-evaluation.These methods provide uncertainty-based and self-evaluation comparisons for hallucination detection.Source: §3.3 · p. 6

4.1 Experimental setup

• The study evaluates Llama-3.1-8B and Llama-3.3-70B primarily, with replication on Gemma-2-9B, Qwen-2.5-7B, and Mistral-Small-24B.Tests cover LongFact, LongFact++, TriviaQA, HealthBench, and MATH, using roughly 25,000 labeled training samples.Source: §4.1 · p. 7
• Evaluation uses entity-level classification with span-max scoring, reporting AUC and recall at a 10% false-positive rate.For MATH, the entire completion is scored by its maximum token score.Source: §4.1 · p. 7

4.2 Results

• Long-form probes achieve AUCs above 0.85 for linear probes and above 0.89 for LoRA, while uncertainty baselines remain below 0.76.These results hold on LongFact and HealthBench for both primary models.Source: §4.2 · p. 7
• Short-form TriviaQA results remain strong despite stronger baselines, with LoRA probes consistently exceeding 0.96 AUC.Linear probes also perform well in this setting.Source: §4.2 · p. 7
• Probes also perform strongly on MATH, suggesting that signals of correctness generalize beyond fabricated-entity detection.MATH is an out-of-distribution mathematical-reasoning task without discrete entities.Source: §4.2 · p. 7, §4.1 · p. 7
• Across three secondary models, LoRA probes again outperform linear probes, reaching 0.87–0.90 AUC on LongFact with only 2,000 annotated samples per model.Full secondary-model results are reported in Table 4.Source: §4.2 · p. 7
• Training on mixed long-form and short-form data yields the best overall performance, while long-form-only training also works well.The comparison is referenced in Section 5.1.Source: §4.2 · p. 7
• LoRA probes are strongest across LongFact, HealthBench, TriviaQA, and MATH, outperforming linear probes and uncertainty-based baselines.Table 1 reports AUC and R@0.1 for the two primary models across all four test sets.Source: §4.2 · p. 7, Table 1 · p. 8

5.1 Generalization between short- and long-form generation settings

• ∼0.10 AUC separates short-form-trained from long-form-trained probes on long-form evaluation across training-set sizes.Short-form-trained probes improve with more data but remain behind long-form-trained probes, whereas long-form-trained probes transfer to short-form tests with a gap below 0.05 AUC.Source: §5.1 · pp. 8–9, Figure 4 · p. 9
• Long-form supervision is emphasized because long-form, multi-claim outputs are where real-world hallucinations commonly occur.The asymmetric transfer results motivate including long-form data in training rather than relying only on short-form annotation.Source: §5.1 · p. 9

5.2 Cross-model generalization

• Off-diagonal cross-model AUC scores are typically within 0.02–0.04 of same-model performance, indicating strong transfer across model outputs.The results support model-agnostic factuality signals rather than detector-specific internal signals.Source: §5.2 · p. 9, Figure 5 · p. 10
• Mistral-Small-24B probes differ by at most 0.02 AUC when trained on their own data versus Llama-3.1-8B data.This extends transferability to the choice of training-data source.Source: §5.2 · p. 10, Figure 5 · p. 10

5.3 Impact on model outputs and behavior

• KL regularization traces a smooth Pareto frontier in which stronger regularization lowers KL divergence while slightly reducing detection AUC.Linear probes preserve behavior perfectly but limit detection, whereas unregularized LoRA has high detection performance with substantial behavioral changes.Source: §5.3 · pp. 10–11, Figure 6 · p. 11
• 0.8898 AUC accompanies near-zero KL divergence of 0.0046, 52.8% win rate, and 71.2% MMLU at λKL=0.50.KL regularization outperforms unregularized LoRA and LM regularization while balancing detection with behavioral preservation.Source: §5.3 · p. 11

5.4 Hallucination monitoring enables selective answering

• Selective answering uses token-level probe scores to halt generation and return an abstention when risk exceeds threshold t.The proof-of-concept is evaluated on SimpleQA with KL-regularized probes.Source: §5.4 · p. 11, Figure 7 · p. 12
• Lowering the monitoring threshold increases conditional accuracy while reducing the fraction of questions attempted.With no monitoring, the system attempts approximately 80% of questions but has conditional accuracy below 30%; more aggressive monitoring makes answering increasingly selective.Source: §5.4 · pp. 11–12, Figure 7 · p. 12

• Entity-level detection misses relational errors involving real entities, where no single hallucinated entity can be isolated.The method also does not explicitly target reasoning errors or context-dependent contradictions.Source: §6 · p. 12
• Flexible span labeling captures otherwise missed errors but can label broader phrases or clauses, introducing ambiguity about what constitutes an entity.Longer spans commonly arise when errors cannot be localized to one atomic entity.Source: §6 · p. 12
• Annotation noise from LLM judges, retrieval failures, and ambiguous entity mapping constrains both training effectiveness and evaluation confidence.The pipeline achieves 80.6% recall on synthetic hallucinations with a 15.8% false positive rate.Source: §6 · p. 12
• Current reliability remains insufficient for production deployment: the best LoRA probes achieve approximately 70% recall at a 10% false positive rate on long-form text.Reducing hallucinations in selective answering requires sacrificing approximately 50% of correct answers.Source: §6 · p. 13

7 Discussion

• The approach frames hallucination detection as token-level sequence labeling, enabling streaming monitoring with 0.89+ AUC on long-form detection.Automated web-search annotation supplies labels distinguishing grounded and fabricated entities, and LoRA probes outperform uncertainty-based baselines.Source: §7 · p. 13
• Current performance remains insufficient for high-stakes deployment, with approximately 70% recall at a 10% false positive rate and entity-level coverage of only part of problematic outputs.Future work targets better annotation, generation-time interventions, and reasoning or relational hallucinations.Source: §7 · p. 13

B.1 Example from HealthBench

• HealthBench and MATH examples visualize token-level probe scores, with yellow intensity indicating scores and thresholds of 0.25 and 0.30 respectively.HealthBench additionally marks supported entities green and hallucinated entities red.Source: Figure 8 · p. 21, Figure 9 · p. 22

C.1 LongFact++

• LongFact++ is ten times larger than LongFact and expands query structure and verifiable fact-rich domain coverage.Its prompts include topic-focused, biography, citation-focused, and legal questions; explicit refusals are filtered for biography prompts.Source: §C.1 · p. 23

C.2 Dataset splits

• All models share a 2,000-prompt long-form test set, while training reservoirs and sample counts vary by model.Primary models use 8,000 long-form prompts and secondary models use 2,000; HealthBench is evaluation-only.Source: §C.2 · pp. 23–24
• Within each model, TriviaQA and HealthBench avoid train-test overlap, but corresponding splits are not guaranteed to be disjoint across models.The shared long-form prompts are identical across models, although their generations differ.Source: §C.2 · pp. 23–24

C.3 Analysis of entity categories

• DATE, PERSON, and ORG dominate the named-entity spans in LongFact++ generations, with additional representation from CARDINAL, GPE, and PERCENT.This distribution is skewed toward temporal, human-related, and organizational entities.Source: §C.3 · p. 24, Figure 10 · p. 24

C.4 Analysis of entity span length

• A median span length of 5 tokens and a 90th percentile of 13 tokens show that the pipeline usually identifies short, fine-grained entities.The distribution is shown in Figure 11.Source: §C.4 · p. 25, Figure 11 · p. 25
• Longer spans often encode relational errors involving real entities, so the pipeline labels the broader span containing the erroneous relationship.Examples include claims about Senate ratification or election to the National Academy of Sciences.Source: §C.4 · p. 25
• The flexible entity definition catches errors a rigid taxonomy would miss, but means some labels are not atomic noun-like entities.This is a minority of cases but a meaningful departure from the idealized framing.Source: §C.4 · p. 25

D.1 Long-form evaluation (LongFact, LongFact++, and HealthBench)

• Long-form token-level methods score an entity span by its maximum token score, while semantic entropy samples ten continuations and measures semantic-cluster uncertainty.The evaluation covers LongFact, LongFact++, and HealthBench.Source: §D.1 · p. 28

D.2 Short-form evaluation (TriviaQA)

• TriviaQA assigns a binary label to one answer entity span after retaining only questions with unanimously correct or incorrect completions across five generations.Token-level methods use span-max scoring for that answer span.Source: §D.2 · p. 28

D.3 Mathematical reasoning evaluation (MATH)

• MATH labels each greedily generated response by final-answer correctness, scoring token-level methods by the maximum score across the entire response.Semantic entropy instead clusters ten sampled extracted answers.Source: §D.3 · p. 28

E Label quality validation

• Exact matching discards annotations whose spans do not occur in the original completion, preventing hallucinated annotations from contaminating training data.This safeguard does not ensure that labels on valid spans are accurate.Source: E Label quality validation · p. 29
• Human annotations matched LLM labels in 84% of 50 randomly sampled spans, validating precision but not recall.The human annotators independently searched for supporting evidence.Source: E Label quality validation · p. 29
• On 100 synthetic examples containing 904 injected hallucinations, the pipeline detected 80.6% of them, or 729/904.The controlled set was created by rephrasing factual Wikipedia content and injecting tracked errors.Source: E Label quality validation · p. 29
• The pipeline correctly labels every injected hallucination span that it overlaps, while labeling factual content as supported in 84.2% of cases.The latter implies a 15.8% false positive rate.Source: E Label quality validation · p. 30
• Conservative labeling tends to over-flag factual content and may overstate real-world performance because synthetic Wikipedia-derived examples are easier to verify.These errors constrain the ultimate performance ceiling of the detectors.Source: E Label quality validation · p. 30
• Annotations from Claude Opus 4 improve AUC by only approximately 0.01 over Claude Sonnet 4, indicating strong cross-annotator consistency.The comparison uses 224 Llama-3.1-8B test completions.Source: E Label quality validation · p. 30, Table 2 · p. 30

F.1 Token-level uncertainty metrics

• Token-level entropy and perplexity use maximum aggregation over a span, whereas semantic entropy clusters sampled completions by semantic equivalence before computing uncertainty.Semantic clusters are formed using pairwise bidirectional entailment judgments.Source: §F.1 · p. 31, §F.2 · p. 31

F.3 Black-box self-evaluation: can we just ask the model whether it’s hallucinating?

• The evaluation selects sentences containing exclusively supported or exclusively hallucinated annotated spans, discarding mixed or unannotated sentences for clean labels.Models receive a follow-up Yes/No question that references the selected sentence within the preceding conversation.Source: §F.3 · p. 32
• Sentence-level evaluation is preferred because it outperforms direct span verification and preserves context needed for pronouns, statistics, and claims dependent on earlier information.The same sentence-selection procedure is applied to LoRA probes for comparison.Source: §F.3 · p. 32
• AUCs of 0.58–0.68 on long-form versus 0.81–0.89 on TriviaQA show that self-evaluation degrades sharply as content becomes multi-factual and context-dependent.The larger Llama-3.3-70B model performs substantially better than Llama-3.1-8B in self-evaluation.Source: Figure 14 · p. 32, §F.3 · p. 33
• Supervised LoRA fine-tuning yields only marginal gains over prompting, although alternative prompts, reasoning traces, or dataset designs might improve the baseline.The baseline is included both for comparison with trained probes and to test whether poor performance results from suboptimal prompting.Source: §F.3 · p. 33

G.3 Probe performance per entity type

• Linear and LoRA probes show broadly similar AUC performance across NER categories, indicating probe failures are relatively uniform across entity types.The analysis uses span-max aggregation on Llama-3.1-8B LongFact++ generations and includes categories with at least 200 samples.Source: §G.3 · p. 34, Figure 15 · p. 34

H Extended results: selective answering

• Selective answering improves conditional accuracy while decreasing the total number of questions attempted at a probe threshold of t=0.5.The reported results cover all models.Source: Table 5 · p. 35

I.1 Quantitative analysis

• The quantitative evaluation measures distributional change with token-wise KL divergence, generation quality with GPT-4.1 win rates, and knowledge retention with zero-shot chain-of-thought MMLU accuracy.KL divergence compares the original and LoRA-adapted model distributions across token positions.Source: §I.1 · p. 36
• λ values of 0.50 or higher tend to preserve model quality in Arena-Hard-Auto win-rate results for Llama-3.1-8B variants.Table 7 reports win rates judged by GPT-4.1 with bootstrap confidence intervals.Source: §I.1 · p. 36, Table 7 · p. 37
• KL-regularized probes provide smoother trade-offs between hallucination detection and distributional shift, while LM regularization behaves erratically and can increase KL divergence through overfitting.Figure 16 varies regularization strength λ; LM loss is not treated as the primary metric because minimizing it can cause overfitting and distribution shift.Source: §I.1 · p. 36, Figure 16 · p. 38, Figure 17 · p. 38

I.2 Qualitative analysis

• Minimal-regularization LoRA probes are observed to make outputs more conservative, more willing to acknowledge uncertainty, and more likely to recognize potentially unreliable information.The qualitative examples use λLM=0.01, a final-layer linear probe head, and temperature 0.Source: §I.2 · p. 39
• The examples show the LoRA-augmented model qualifying an incorrect referee name, refusing an invalid anagram guess, and providing more cautious guidance about lithium and breastfeeding.The citation example likewise shows the model acknowledging that specific references cannot be verified.Source: Figure 18 · p. 39, Figure 19 · p. 40, Figure 20 · p. 40
• These qualitative behavioral changes are observed only when probes are optimized on the last layer, rather than the layer used elsewhere in the paper.The note distinguishes the final-layer setup from attaching the probe at ⌊0.95 × num layers⌋.Source: §I.2 · p. 39

J Use of existing assets

• The work uses existing models and datasets listed in Tables 8 and 9, while AI tools were used only to refine writing and assist with code development.The authors state that the research, analysis, and conclusions remain original.Source: Table 8 · p. 42, Table 9 · p. 42, L Statement on AI-assisted tool usage · p. 42

K Compute statement

• Training a Llama-3.1-8B LoRA probe on the full annotated dataset takes less than 2 hours on one H100 GPU.The annotation pipeline costs approximately $0.70 and takes about 45 seconds per 1–2 thousand-token sample, with samples processed in parallel.Source: K Compute statement · p. 42