Source-linked AI summary
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs
Urja Pawar, Rajitha Ramanayake, Owen O'Neill, Nabeel Kemal, Abhishek Mandal, Houssem Chatbri, Christopher Martin
TL;DR
The paper addresses hallucination detection when trusted reference evidence and model internals are unavailable, using semantic disagreement and token-probability signals from black-box APIs. It introduces aggregation and combination methods, then finds that Stacked often leads while TopK and CoCoA remain competitive without labels, with performance depending on calibration and setting.
Problem
The paper asks how to detect hallucinated responses across settings without trusted reference evidence or model-internal access, when false alarms and missed fabrications both matter.
Method
The paper samples responses, measures semantic entropy and token-log-probability uncertainty, and evaluates TopK, CoCoA, Gated, and Stacked combinations.
Results
Stacked gave the best performance in nearly half of evaluated cases, while TopK and CoCoA remained competitive without supervised labels; no method was universally strongest.
Takeaways & Limitations
Method choice, response generation, and operating thresholds should be selected jointly for the target dataset, model, labels, false-positive cost, and review capacity.
Takeaways & Limitations
Gated and Stacked require labelled target-domain hallucination data, zero-shot transfer is not evaluated, and Long-Text QA contains only 30 questions.
Abstract
from arXiv · showhide
When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic entropy, which measures disagreement among sampled response meanings, and uncertainty derived from token log-probabilities. Their failure modes can be complementary: semantic entropy becomes uninformative when responses form one semantic cluster, while token uncertainty can miss consistently confident errors. We extend token-based uncertainty detection by aggregating token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader token features. We evaluate seven benchmarks, including five public benchmarks (four text datasets and multimodal handwritten-cheque extraction) and two constructed benchmarks (Financial Summaries and Long-Text QA), using four language models. In our evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across dataset characteristics.
1 Introduction
The paper studies hallucination detection when trusted reference evidence and model internals are unavailable, using semantic and token-based signals exposed by black-box APIs. It develops complementary methods and evaluates their consistency across datasets, models, and operating conditions.
- Black-box APIs may expose generated text and token log-probabilities but not trusted reference context or hidden states.
- Semantic entropy measures disagreement among sampled response meanings, while token methods estimate confidence from sequence and token-level probabilities.The paper aggregates token signals across the same sampled responses used for semantic entropy.
- TopK aggregates token uncertainty across responses, while Gated routes single-cluster cases to token features and Stacked learns jointly from both signal families.
- Semantic entropy can become uninformative for one semantic cluster, whereas token uncertainty can miss consistently confident hallucinations.The two signal families therefore expose complementary failure modes.
- Across seven benchmarks and four language models, Stacked performed best in nearly half the cases, while TopK and CoCoA remained competitive without supervised labels.Threshold calibration remained important, and no method was universally strongest.
- The evaluation examines false-positive-rate budgets from 1% to 15%, generation and calibration sensitivity, and dataset-characteristic variation.The research questions explicitly target failure patterns, complementary information, and consistency across operating conditions.
2 Complementary Black-Box Hallucination Signals
The paper frames semantic and token uncertainty as complementary black-box signals and combines them through unsupervised and supervised methods. Its methods reuse sampled responses to aggregate token evidence and integrate it with semantic information.
- Semantic uncertainty: Semantic entropy groups sampled responses into semantic equivalence classes and measures uncertainty over their empirical class distribution.
- Semantic uncertainty: Standard semantic entropy may miss plausible classes through finite sampling and discard graded similarities through hard clustering.The related work discusses kernel, nearest-neighbour, and spectral alternatives that retain graded similarity information.
- Token uncertainty: Token-level methods assess confidence within a generation using likelihood, predictive entropy, self-evaluation, relevance weighting, or temporal uncertainty.
- Hybrid methods: CoCoA combines target-response confidence with semantic dissimilarity, whereas the proposed methods aggregate broader token features across responses and add query-level semantic signals.
- Proposed methods: TopK aggregates token uncertainty, Gated routes by semantic-cluster count, and Stacked jointly learns from semantic and token features without reference or hidden-state access.
- Proposed methods: The multi-response token representation summarizes confidence, response dynamics, cross-sample variation, and semantic diversity using distributional features.
- Proposed methods: Gated uses semantic entropy when K ≥2 and a logistic-regression classifier over aggregated token features when K = 1.Its variants differ in the semantic entropy score while sharing the token classifier.
3 Methodology and Experimental Setup
The study evaluates 13 hallucination-detection methods across seven benchmark types and four language models using held-out cross-validation and false-positive-rate operating points. It also varies generation, clustering, and regularisation settings to assess sensitivity.
- Training and preprocessing: Semantic similarity thresholds are selected on validation data and fixed before five-fold cross-validation, while similarity-matrix methods operate directly on continuous similarities.
- Datasets: The evaluation covers seven benchmarks spanning factual, ambiguous, multi-hop, extractive, long-context, financial, and multimodal settings.Five benchmarks are public and Financial Summaries and Long-Text QA are author-constructed.
- Models: Four language models are evaluated where supported, yielding 26 usable dataset–model pairs with both hallucination-label classes.Llama 3.3 70B lacks both classes for Long-Text QA, and Cheque Generation requires vision capability.
- Generation: The default generation setting uses N = 10 responses, temperature 1.0, and k = 5 requested log-probability candidates per token.GPT-5.4 returned one candidate per token, producing effective k = 1 results for token-based methods.
- Methods: The 13 methods include semantic-entropy baselines, TopK, Spectral Epistemic, CoCoA variants, and supervised Gated and Stacked variants.
- Metrics: AUROC is averaged across held-out folds, and true-positive rates are reported at false-positive-rate budgets from 1% to 15%.These operating points expose the trade-off between hallucination coverage, false positives, review capacity, and error costs.
- Sensitivity analysis: Sensitivity analyses vary response count, temperature, returned candidates, clustering threshold, and regularisation strength.
4 Results and Discussion
Semantic disagreement and token uncertainty exhibit complementary failure modes, and no detection method is uniformly strongest across datasets, models, and operating points. Stacked is generally close to the best method, while threshold and method choices remain setting-dependent.
- Complementary signal failure modes: 39–99% of hallucinated queries form one semantic cluster, forcing standard semantic entropy to zero; token uncertainty remains higher in hallucinated cases on six of seven datasets.Among single-cluster hallucinations, 21–56% have lower TopK uncertainty than the median non-hallucinated query, so overlap remains substantial.
- Performance patterns across methods, datasets, and models: At strict 1–3% FPR budgets, TopK led AmbigQA and HotpotQA, Stacked led AA Omni Finance and SQuAD, and Gated led Cheque Generation.At higher budgets, the leading family changed on AmbigQA, HotpotQA, and Cheque Generation, while Stacked remained strongest on AA Omni Finance and SQuAD.
- Performance patterns across methods, datasets, and models: 11 of 26 comparisons were led or shared by Stacked, versus seven by CoCoA, five by TopK, three by Gated, and one by semantic or spectral scores.The counts sum to 27 because Stacked and CoCoA tied in one setting.
- Performance patterns across methods, datasets, and models: Stacked was within 0.05 AUROC of the best method in 20 of 26 comparisons and within 0.02 in 16.TopK and CoCoA had the next-smallest median shortfalls, remaining competitive in more specific settings even when they did not lead.
- Conditional method and threshold selection: Token-confidence or response-dynamics features provided the strongest univariate separation in 121 of 148 ablation settings.Figure 4 compares feature groups using the largest absolute Cohen’s d within each group across 23 ablation settings.
- Conditional method and threshold selection: Increasing retained token candidates from one to three raised TopK AUROC from 0.621 to 0.721 on AA Omni Finance but left average Stacked AUROC nearly unchanged on SQuAD, from 0.767 to 0.768.Sampling and temperature preferences also varied by dataset, and calibration was more variable on the small Long-Text QA dataset.
5 Limitations
The evaluation has important scope and access constraints: supervised methods need labelled target-domain data, and some benchmark and API settings limit interpretation.
- Gated and Stacked require labelled target-domain hallucination data, and zero-shot transfer is not evaluated.
- Long-Text QA contains only 30 questions, and leading confidence intervals overlap across all datasets.
- Token methods require API-exposed log-probabilities, while GPT-5.4 returned only one candidate per token.
6 Conclusion
Across the evaluated black-box hallucination-detection setting, semantic and token signals were complementary but neither method was universally strongest. Stacked led or shared the lead in 11 comparisons, while performance remained dependent on dataset, model, and operating conditions.
- 11 comparisons were led or shared by Stacked across seven datasets and four models.
- Stacked remained within 0.05 AUROC of the best method in 20 comparisons when labelled target-domain data were available.
- TopK and CoCoA were competitive alternatives without labels, but their thresholds required careful calibration.
- Performance varied across datasets, generating models, and false-positive-rate budgets, while response count, temperature, and token-candidate count showed no uniform improvements.
- The appendix documents uncertainty methods, implementation details, datasets, token-level features, and hallucination-label construction procedures.
A Uncertainty detection methods
The methods combine semantic disagreement and token-level confidence signals available from sampled black-box responses. They include corrected semantic measures, TopK, CoCoA, and supervised Gated and Stacked combinations.
- A.1 Semantic and spectral uncertainty measures: Semantic uncertainty methods use sampled responses, semantic clusters, or graded response similarities to quantify variation in meanings.Standard semantic entropy uses the empirical semantic-cluster distribution; alternatives correct finite-sample effects or retain graded similarities.
- A.1 Semantic and spectral uncertainty measures: The Good–Turing correction estimates semantic alphabet size from observed clusters K and singleton clusters n1.
- A.1 Semantic and spectral uncertainty measures: Hybrid uses the larger of Good–Turing and UEigV estimates, while Von Neumann entropy computes uncertainty from the similarity-matrix spectrum.
- A.2 TopK: TopK combines each response’s mean token entropy with its mean chosen-token log-probability and across-response variation.Higher scores indicate greater uncertainty; with one returned candidate, the token-entropy term is zero.
- A.3 CoCoA: CoCoA combines target-response confidence with semantic disagreement from sampled alternatives, using sequence-level or length-normalised uncertainty.Neither variant uses supervised training labels.
- A.4 Gated and Stacked: Gated routes multi-cluster cases to a semantic score and single-cluster cases to an aggregated-token classifier, whereas Stacked learns jointly from both signal families.
B Additional experimental details
The evaluation fixes shared embedding, clustering, and classifier procedures within datasets while documenting the data and label-construction pipeline. Supervised preprocessing is fitted within training folds to avoid information leakage.
- B.1 Shared implementation: Responses are embedded with text-embedding-3-large into 3,072-dimensional vectors, with no dimensionality reduction before clustering.
- B.1 Shared implementation: Greedy representative-based clustering assigns each response to the first representative meeting threshold τ, with thresholds selected on held-out validation data.UEigV uses ϵ = 0.1.
- B.2 Supervised classifiers: Both supervised classifiers use L2-regularised logistic regression; Gated trains on single-cluster examples, while Stacked uses combined features reduced to at most 15 principal components.Scaling, principal-component analysis, and logistic regression are fitted within each training fold.
- B.3 Evaluation data and labels: The supporting data section documents public and custom datasets, token-level features, dataset construction, and prompts used for hallucination labels.
C.1 Public datasets
The evaluation covers five public benchmarks spanning factual, ambiguous, multi-hop, extractive, and multimodal tasks, alongside two constructed finance-focused benchmarks. It also documents the aggregated token features and classifier inputs used for detection.
- Public datasets: Five public benchmarks cover factual knowledge, ambiguous questions, multi-hop reasoning, extractive comprehension, and handwritten-cheque extraction.The public set comprises AA Omni Finance, AmbigQA, HotpotQA, SQuAD, and Cheque Generation.
- Constructed datasets: Two constructed benchmarks test hallucinations in financial summarisation and document-grounded question answering.Financial Summaries probes confusable names and figures, while Long-Text QA includes deliberately unanswerable sub-questions.
- Constructed datasets: Long-Text QA contains 30 five-question sets over regulatory and financial documents, with one deliberately unanswerable sub-question per set.The controlled unanswerable item is always Q5.
- Feature construction: Query-level token representations aggregate confidence, uncertainty, candidate margins, positional changes, and response length across N sampled responses.Gated and Stacked use broader aggregated features than TopK.
- Reproducibility boundary: The code is not released under an unrestricted public licence because of institutional data-governance constraints.The paper therefore specifies the exact feature subsets used by trained components.
D.4 Cheque Generation robustness
Cheque Generation robustness is assessed across response counts for vision-capable models using externally scored sweeps separate from the primary pooled evaluation. Increasing response counts generally helps the strongest semantic method for two models, but combined methods vary comparatively little.
- Response-count robustness: Increasing N generally improves the strongest semantic method for GPT-5.1 and GPT-5.4, while TopK, Gated, and Stacked vary comparatively little.The matched sweeps cover N ∈ {3, 5, 7, 10, 15, 20}.
- Response-count robustness: Additional responses do not produce a consistent gain for the combined methods.The externally scored sweeps are reported separately from canonical pooled out-of-fold results.
- Evaluation scope: CoCoA is omitted from the cheque-generation sweep because it contains no CoCoA scores.The figure reports the strongest semantic method, TopK, and averaged Stacked and Gated variants.
E.1 Model-family ablation
The model-family analysis reports pooled out-of-fold results across available text model–dataset settings, with Cheque Generation evaluated separately for vision-capable models. Close AUROC differences are common, and TopK only marginally exceeds Gated Hybrid on two datasets.
- Evaluation scope: Cheque Generation is reported separately because text-only model families cannot perform the multimodal task.The primary GPT-4.1-mini result and externally scored response-count sweep are separate from text-only tables.
- Cross-model results: Pooled out-of-fold AUROC differs by 0.01 to 0.02 on most cells.The comparison covers GPT-5.1, GPT-5.4, Llama 3.3 70B, and a regenerated GPT-4.1-mini run.
- Cross-model results: TopK ranks marginally above Gated Hybrid on HotpotQA and Long Text QA.Appendix F.2 reports pooled estimates and 95% bootstrap intervals for every cell.
E.2 Response-count ablation
Response-count, temperature, token-candidate, and calibration analyses show dataset- and method-dependent sensitivity rather than uniformly improving detection. More samples or diversity can expose information, but neither guarantees better AUROC.
- Response-count ablation: Increasing N does not produce uniformly positive AUROC changes across datasets or method categories.Additional samples alone do not resolve the dataset-dependent weaknesses of either signal family.
- Sampling-temperature ablation: Higher temperatures generally reduce the single-cluster rate but do not consistently improve AUROC.The preferred temperature is dataset-dependent because added variation may be unrelated to hallucination.
- Top-k candidate ablation: Most token-candidate improvements occur when increasing k from 1 to 2 or 3; later candidates yield smaller and less consistent changes.Semantic-only methods and CoCoA SP/PPL remain flat across k.
- Calibration: Calibration sensitivity is assessed across five clustering thresholds and three regularisation strengths.The 15 settings are summarised using method-level minimum–maximum AUROC ranges and interval widths.
F.1 Significance tests
Significance tests and uncertainty analyses support broad cross-dataset patterns but do not establish universally or individually superior methods. Close rankings can change with aggregation and should be interpreted using confidence intervals.
- Significance tests: All three Stacked variants significantly separated labels on AA Omni Finance, Cheque Generation, and SQuAD, but not on AmbigQA, HotpotQA, Long-Text QA, or Financial Summaries at p < 0.05.
- Significance tests: Gated Hybrid was significant on five datasets, with Long-Text QA and Financial Summaries as the exceptions.
- Significance tests: CoCoA SP and CoCoA PPL were significant on several datasets, but neither reached significance on SQuAD or Financial Summaries.
- Significance tests: Exploratory Mann–Whitney U tests should be interpreted alongside effect sizes and confidence intervals rather than as evidence of universal superiority.
- Confidence intervals and aggregation: Long-Text QA changed winners across aggregation procedures: mean-fold AUROC ranked Gated Hybrid first at 0.731, while pooled AUROC ranked CoCoA SP first at 0.728.
- Confidence intervals and aggregation: The leading estimate overlapped at least one competitor on every dataset, so results do not establish statistically distinct single-benchmark superiority.