Gray-Box Hallucination Detection in LLMs with Token Identities

1 INTRODUCTION

• Probability-only detectors discard token identities, mapping distinct decoding states with the same probability profile to the same representation.This removes information about which alternatives the model considered.Source: §1 · pp. 1–2
• At identical ranked probabilities, candidate identities distinguish competing factual answers from semantically equivalent alternatives that may indicate lower hallucination risk.Figure 1 illustrates the contrast between the two candidate structures.Source: §1 · p. 2, Figure 1 · p. 2
• GURI combines token identity codes with probabilities and aggregates the resulting representations across an answer for jointly trained answer-level detection.It retokenizes candidate strings with a shared external tokenizer and aggregates subtoken vectors into identity codes.Source: §1 · p. 2
• Mean test ROC-AUC rises from 72.09% for LOS-Net to 79.09% for GURI across three open-weight models and three question-answering benchmarks.GURI also outperforms transferred LOS-Net in 17 of 18 directed cross-model transfers without target fine-tuning.Source: §1 · p. 2
• The paper identifies token identity as a missing signal and presents GURI as a transferable gray-box detector for open- and closed-source models.Its contributions also include empirical gains and evidence that identities provide predictive information beyond token probabilities.Source: §1 · p. 2

2 PREVIOUS WORK.

• The GURI pipeline aggregates top-K candidate subtoken codes, fuses them with probabilities, and processes the sequence with a response-level Transformer.Both fusion and prediction components are trained jointly for answer-level hallucination detection.Source: Figure 2 · p. 3
• The paper uses “hallucination” to encompass factual errors, reasoning mistakes, and inconsistencies in model outputs.This follows the broad usage in error-detection research.Source: §2 · p. 3
• Black-box detectors use prompts and generated text, while white-box approaches use internal signals such as hidden representations, activations, or attention patterns.Examples include sampled-response comparisons, semantic uncertainty, probing classifiers, and activation-based architectures.Source: §2 · p. 3
• Gray-box methods use output-token probabilities without hidden states or attention matrices, including confidence scores, semantic-aware methods, and learned probability-based detectors.LOS-Net processes sequences of actual-token probabilities and ranked candidate distributions.Source: §2 · p. 3
• GURI combines candidate-token identities and probabilities in a learned detector, with fixed random identity codes retaining information absent from probability-only representations.Unlike prior semantics-based methods described here, it does not require explicit semantic similarity or entailment computations.Source: §2 · p. 3

3.1 SETUP, MOTIVATION AND OVERVIEW OF THE APPRAOCH

• The setup predicts an answer-level factuality label from generation-time signals: top-K candidate probabilities and the generated token's probability across response steps.These sequences are called TDS-K and ATP and form the restricted output-signature representation used for the deployment setting where only a limited number of candidate probabilities are exposed.Source: §3.1 · p. 4
• Probability-only projection discards token identities and their assignment to probability values, so distinct generation traces can become indistinguishable to downstream detectors.The ambiguity follows from the representation: a deterministic detector receiving the same projected input must produce the same output.Source: §3.1 · p. 4

3.2 TOKEN-IDENTITY CODES

• GURI maps observed token strings through a shared external tokenizer and aggregates subtoken codes, giving the same string a common representation across models and occurrences.The default configuration uses a fixed random code table; pretrained embeddings are another option, but do not outperform fixed random codes in the reported experiments.Source: §3.2 · pp. 4–5

3.3 ARCHITECTURE

• GURI combines each token's identity code with its probability and uses a Transformer to aggregate the fused sequence across the answer for factuality prediction.The candidate-fusion MLP, actual-token-fusion MLP, and Transformer are jointly optimized under an answer-level objective, while the external code table remains fixed.Source: §3.3 · p. 5

4.1 EXPERIMENTAL SETUP

• Evaluation covers three open-weight models and three QA benchmarks, comparing GURI with Confidence and LOS-Net using test ROC-AUC.HotpotQA and TriviaQA are closed-book, while NQ-WC supplies context; the main comparison reports mean and standard deviation across 36 hyperparameter–seed combinations.Source: §4.1 · p. 6

4.2 MAIN RESULTS

• GURI gains approximately 7.00 ROC-AUC points over LOS-Net across nine open-weight model–dataset combinations; on GPT-4.1, mean ROC-AUC rises from 67.39 to 71.84.The open-weight gains range from 2.11 points on Ministral/TriviaQA to 12.30 points on Gemma/NQ-WC.Source: §4.2 · pp. 6–7, Table 2 · p. 7

4.3 TRANSFER ACROSS MODELS AND DATASETS

• Among three open-weight models, GURI outperformed transferred LOS-Net in 17 of 18 directed cross-model transfers; between HotpotQA and TriviaQA, its average gain was 3.76 ROC-AUC points with the target model fixed.For transfers involving NQ-WC, dataset changes are confounded with context availability: NQ-WC supplied context, whereas HotpotQA and TriviaQA did not.Source: §4.3 · pp. 7–8

4.4.1 DOES THE SOURCE OF THE CODES MATTER?

• With scalar fusion, fixed random codes achieve higher mean ROC-AUC than pretrained Phi embeddings in all nine settings.Pretrained embeddings are therefore not required for the gains observed with this architecture.Source: §4.4.1 · p. 8, Table 3 · p. 8

4.4.2 DOES EARLY SCALAR COMPRESSION LIMIT GAINS?

• Retaining 256-dimensional token representations yields no consistent improvement over scalar fusion, and random codes outperform Phi embeddings in seven of nine alternating-Transformer settings.The experiment tests whether early scalar compression limits the usefulness of code representations.Source: §4.4.2 · p. 8

4.4.3 THE ROLE OF ACTUAL AND NON-ACTUAL TOKEN IDENTITIES

• With random codes, non-actual identities alone match or exceed both identity channels in all nine settings, while actual identities alone remain 0.5 to 3 points below the full model.With pretrained codes, non-actual-only performance drops by 2 to 7 points in every cell; the two channels become complementary, but the full model remains below its random-code counterpart in every cell.Source: §4.4.3 · pp. 8–9, Figure 5 · p. 9

• Token identity provides useful predictive information beyond next-token probabilities.Source: §5 · p. 9
• Fixed random codes retain external-tokenizer structure, so the experiments show pretrained semantic geometry is unnecessary, not that all lexical or tokenization structure is irrelevant.Source: §5 · p. 9
• Channel analyses cover nine model–dataset combinations under one training configuration, and channel ordering depends on the code table.Evaluation-time permutations measure reliance on shifted inputs, whereas retraining tests what a channel can learn when available from the beginning.Source: §5 · p. 9
• Automatic string-match labels may reward correct substrings in otherwise incorrect responses and penalize appropriate abstention.Open-weight experiments also included continuation after end tokens, leaving first-end-token stopping unevaluated; the method assumes access to candidate strings and probabilities, which not every commercial API exposes.Source: §5 · p. 9
• Future evaluation should use human factuality labels, additional generation protocols, and broader model and task families.Source: §5 · p. 10

• GURI adds fixed token-identity codes to probability-only gray-box detection while preserving access and efficiency advantages, and consistently improves over LOS-Net across the evaluation suite.Fixed random codes perform comparably to pretrained embeddings; with random codes, non-actual candidate identities alone preserve full performance.Source: §6 · p. 10

• HotpotQA and TriviaQA each use 10,000 training and validation examples sampled with replacement or subsampled from specified splits.HotpotQA used seed 0; TriviaQA used random state 42.Source: §A.1 · p. 13
• NQ-WC retains 20,770 rows before generation, but the generated data excludes the longest-context cases that failed generation.The resulting data contained 9,674 training-pool and 9,694 test examples; 326 and 306 examples failed, respectively.Source: §A.1 · p. 13
• Detector training and validation use a label-stratified five-fold split, with only the first fold held out for validation rather than full five-fold cross-validation.HotpotQA and TriviaQA use 8,000/2,000 splits, and assignments may differ across models because correctness labels depend on the target model.Source: §A.1 · p. 13

• Open-weight models receive one user message using their chat template; HotpotQA and TriviaQA prompts contain only the question, while NQ-WC includes context and question.Source: §A.2 · p. 13
• All nine open-weight model–dataset combinations produce exactly 100 generated tokens because end-token stopping is disabled and decoding continues to the cap.Detector inputs include the full recorded sequences, including continuation after the first end token.Source: §A.2 · p. 13
• Detector inputs retain the top 10 candidates and use standardized log-probabilities rather than probabilities, requiring full-vocabulary statistics in open-weight experiments.Source: §A.2 · p. 13

A.3 CORRECTNESS LABELS AND CLASS BALANCE

• A response was labeled factual when its lowercased text contained a lowercased reference answer; TriviaQA accepted any official alias, while HotpotQA and NQ-WC used one reference string.No punctuation removal, whitespace normalization, or stemming was applied; the detector output represented P(factual).Source: §A.3 · p. 13
• Open-weight matching covered all 100 generated tokens, including continuation after the first end token, and a match only in that continuation could earn a factual label.A refusal without a reference match received label 0; matching was over the complete generated response.Source: §A.3 · p. 13
• These labels did not verify every claim in a response; the supplied table caption identifies factual-label percentages but provides no values.The train pool is data before the training/validation split.Source: §A.3 · p. 14, Table 4 · p. 14

A.4 DETECTOR ARCHITECTURE AND TRAINING

• The response encoder used a 256-dimensional hidden state, with ten candidate features projected to 128 dimensions and actual-token features and rank supplying the other 128.Classification used a learned classification token followed by a linear layer and sigmoid; the identity-code tables remained fixed.Source: §A.4 · p. 14

A.5 HYPERPARAMETER SEARCH

• The main comparison averaged test ROC-AUC across 36 configurations crossing three seeds, three encoder depths, two learning rates, and two weight decays.Each run’s checkpoint was selected using validation ROC-AUC; the reported mean and standard deviation reflected both configuration and seed variation.Source: §A.5 · p. 14

A.6 IDENTITY-CODE TABLES

• Only one random codebook was evaluated, and the pretrained-versus-random comparison also differed in normalization, so it did not isolate pretrained semantics.Tokens that could not be decoded or retokenized received the zero vector.Source: §A.6 · p. 15

A.7 GPT-4.1 GENERATION SETUP

• GPT-4.1 generation used top_logprobs=10 and a 50-token cap, versus 100 tokens in the open-weight experiments.The API supplied returned candidate log-probabilities rather than a full-vocabulary distribution; missing candidate slots were padded with −109.Source: §A.7 · p. 15

A.8 ALTERNATING TRANSFORMER ARCHITECTURE

• The alternating architecture applied attention across generation positions within each slot, then across slots at each position, before pooling and classification.It used no candidate-position embeddings; input codes stayed fixed while the other components were jointly trained.Source: §A.8 · p. 15

A.9 CHANNEL EXPERIMENTS

• Permuting either identity channel lowered test ROC-AUC, and joint permutation losses were non-additive; these interventions measure reliance by a frozen detector, not channel value after retraining.With random codes, the actual-token channel had the larger permutation loss in eight of nine cells, while retraining with non-actual candidates alone matched the full model; pretrained-code results differed.Source: §A.9 · p. 16, Figure 6 · p. 16, Figure 7 · p. 17

A.10 TRANSFER EVALUATION DETAILS

• Transfer evaluation used one source-validation-selected checkpoint per source pair across all nine targets, without target fine-tuning or target-validation selection.The 81 evaluations included 9 unshifted, 18 model-only, 18 dataset-only, and 36 combined shifts; NQ-WC transfers also changed context availability.Source: §A.10 · pp. 16–17, Table 5 · p. 17
• For HotpotQA transfer, Ministral→Llama achieved 49.74 ROC-AUC, compared with 68.87 in the reverse direction.The low-scoring evaluation loaded the target identity codes, which ruled out missing codes as its explanation but did not explain the result.Source: §A.10 · p. 17