The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models

• LLMs may encode whether a query is answerable even when they generate a hallucinatory answer.Source: §1 · pp. 1–2
• Up to 80% performance gains occur on (un)answerable questions when prompts explicitly mention the possibility of (un)answerability.Source: §1 · p. 2
• Answerability is decodable without prompt guidance through alternative beam responses and linear separation in hidden representations.Source: §1 · p. 2
• The answerability representation generalizes across QA datasets, as classifiers trained on one dataset can classify questions from others.Source: §1 · p. 2
• The findings motivate decoding methods that use latent answerability information to improve factual generation.Source: §1 · p. 2

• SQuAD v2, Natural Questions, and MuSiQue provide benchmarks containing answerable and (un)answerable questions for evaluating this capability.Source: §2 · p. 2
• Prior studies use few-shot prompting, prompt tuning, and larger instruction-tuning exposure to improve models’ handling of (un)answerable questions.Source: §2 · p. 2
• Other approaches manipulate predictions through data augmentation or analyze how the supplied data affects answerability detection.Source: §2 · p. 2
• Interpretability and probing methods analyze or manipulate latent information, while beam search methods use multiple candidate outputs to influence decoding.Source: §2 · p. 2
• Early exiting and prediction calibration are related efforts to analyze and improve model predictions and output distributions.Source: §2 · p. 2
• The paper’s prompt variants combine basic instructions with an answerability hint and two exemplars.Source: Figure 2 · p. 3

3 Method

• The paper tests answerability encoding through prompt manipulation, beam scrutiny, and probing with subspace identification and erasure.Source: §3 · p. 3

3.1 Prompt Manipulation

• Prompt manipulation compares guiding prompts with more instructive prompts that explicitly hint at answerability and recommend how to respond.Source: §3.1 · p. 3
• The setup includes zero-shot and few-shot prompts, with two exemplars and an (un)answerable exemplar in the hinting condition.Source: §3.1 · p. 3

3.2 Beam Relaxation

• Beam relaxation increases beam size and replaces the highest-probability response with “unanswerable” when another beam signals answerability.Source: §3.2 · p. 3

3.3 Identifying an Answerability Subspace

• A logistic-regression probe uses first-generated-token embeddings from the last hidden layer to classify answerable versus (un)answerable queries.Source: §3.3 · p. 3
• The probing setup compares contextual last-layer representations with initial embeddings that should not encode answerability.Source: §3.3 · p. 3

3.4 Erasing the Answerability Subspace

• The study focuses on first-token representations because averaging embeddings across generated tokens performed worse on a development set.Source: §3.4 · p. 3
• Concept erasure projects representations to minimally change them while preventing linear prediction of answerability, then applies the projection at first generation.Source: §3.4 · p. 4

• For probing, 1000 training instances per benchmark are balanced between answerable and unanswerable cases, with 800 used to train linear classifiers and the remainder reserved for development.Source: §4 · p. 4
• The benchmarks are SQuAD 2.0, NATURAL QUESTIONS, and MuSiQue, each containing answerable and unanswerable reading-comprehension instances.NATURAL QUESTIONS uses the annotated long answer as context for answerable cases and a semantically closest non-answer paragraph for unanswerable cases; MuSiQue constructs contexts from aligned or closely linked paragraphs across sub-questions.Source: §4 · pp. 4–5
• The study evaluates zero-shot and few-shot prompting with regular prompts versus prompts hinting at answerability; few-shot results average three in-context-example variants with standard deviations.Source: Table 2 · p. 5
• The primary evaluation classifies whether models answer or abstain, using F1 with “unanswerable” as the positive label rather than judging answer correctness.Source: §4 · p. 5
• Exact match and token-wise F1 on the QA task are also reported to check that the methods do not impair the models’ primary task.Source: §4 · p. 5
• Three instruction-finetuned models—Flan-T5xxl, Flan-UL2, and OPT-IML—are evaluated across three benchmarks.Source: §4 · pp. 4–5

5.1.1 Zero-Shot Scenario

• Up to 80 points of detection improvement follows from hinting at (un)answerability in zero-shot prompts, with hints also reducing model-size effects.Flan-T5xxl surpasses larger counterparts on two of three benchmarks after hinting.Source: §5.1.1 · p. 5
• Over 50-point gains in exact match and token-wise F1 occur on the combined QA task when prompts include an (un)answerability hint.The overall improvement largely comes from correctly responding to (un)answerable questions, despite average answerable-question drops of 8.3% in F1 and 7.1% in exact match.Source: §5.1.1 · pp. 5–6, Table 3 · p. 6

5.1.2 Few-Shot Scenario

• Few-shot hints significantly improve (un)answerability identification and QA scores across averaged prompt-example variants.Results average three in-context-example variants and report standard deviations.Source: §5.1.2 · p. 6

5.2 Beam Relaxation

• Larger beams increasingly contain responses acknowledging (un)answerability, while the most probable response changes negligibly and answerable-question scores degrade only slightly.Figure 3 distinguishes recognition anywhere in the beam from the usual top-beam metric.Source: §5.2 · pp. 6–7, Figure 3 · p. 6
• Performance decreases on NQ and MuSiQue relative to SQuAD are attributed to distribution shift and greater task difficulty, including MuSiQue’s multi-hop reasoning demand.The authors note that NQ queries are not always conventional questions and that limited multi-hop datasets with (un)answerable questions constrain training.Source: §5.2 · p. 7

5.3 Identifying an Answertability Subspace

• Linear probes achieve above 75% F1 across models and datasets from first-token last-layer representations, supporting an (un)answerability linear subspace.PCA visualizations separate answerable, correctly recognized (un)answerable, and hallucinated (un)answerable queries; the latter appear between the first two groups.Source: Figure 4 · p. 7, Figure 5 · p. 7, §5.3 · p. 8
• Cross-dataset transfer remains above uncontextualized first-layer performance, suggesting probes capture an abstract subspace beyond shallow dataset-specific features.Figure 6 reports training datasets by columns and test datasets by rows.Source: §5.3 · p. 8, Figure 6 · p. 8

5.4 Erasing the Answerability Subspace

• Erasing the answerability subspace lowers Flan-UL2 classification F1 from 50.1 to 31.2 with regular beams and from 65.4 to 32.7 with beam relaxation.The authors interpret this degradation as evidence that the subspace influences answerability-task behavior; Table 5 evaluates QA effects before and after erasure.Source: §5.4 · p. 9, Table 5 · p. 9

• Models encode question (un)answerability in hidden states despite often generating hallucinatory answers.Source: §6 · p. 9
• A beam often contains an answerability-acknowledging response even when the highest-ranked answer is hallucinatory.Source: §6 · p. 9
• Answerable and (un)answerable questions are linearly separable in the embedding space before answer decoding.Source: §6 · p. 9, Figure 7 · p. 9
• Better prompting, decoding, or simple auxiliary classification models may mitigate answering (un)answerable questions.Source: §6 · p. 9

• The experiments cover only a few datasets and models, leaving model-scale relationships for future work.Source: §7 · p. 9
• Because the study uses linear probing and erasure, its results should be interpreted as a lower bound for identifying (un)answerability.Source: §7 · p. 9
• The experiments focus on (un)answerability within a given context rather than the open-domain setting.Source: §7 · p. 9

8 Ethics Statement

• Model hallucination can have real-world implications in applications such as search engines, but this study is preliminary and not a general QA solution.Source: §8 · p. 9

A (Un)answerability-Recognizing Responses

• The study identifies abstention through responses such as “Unanswerable,” “N/A,” “I don’t know,” and “No answer.”Source: A (Un)answerability-Recognizing… · p. 14

B Performance on the Answerable Instances

• Adding an answerability hint slightly reduces answerable-question exact match and F1 for the most part, but overall performance improves because unanswerable detection improves more.Source: B Performance on the Answerable… · p. 14

C Prompt Variant Tuning

• Three prompt variants use “unanswerable,” “IDK,” or “N/A” as the instructed response, selected per dataset from a separate development set.Source: C Prompt Variant Tuning · p. 14

D Impact of Hint Placement

• Instruction-level answerability hints are considerably more advantageous than exemplar-level hints in few-shot prompting.Source: D Impact of Hint Placement · pp. 14–15, Table 8 · p. 15
• Once instructions contain a hint, adding the hint to exemplars has minimal additional impact, except for OPT-IML.Source: D Impact of Hint Placement · p. 15

• Relaxed beam search reduced answerable-question exact match by at most 7.7 points and F1 by at most 8.7 points versus regular beam search.The comparison uses conventional beam search and a relaxed variant that substitutes an (un)answerability-recognizing response when one appears in the beam.Source: E Impact of the Relaxed Beam Search… · p. 15
• The adapted beam-search variant therefore had only a marginal effect on answerable queries.Source: E Impact of the Relaxed Beam Search… · p. 15
• The results indicate that the improvement in Section §5.2 comes from better treatment of (un)answerable questions rather than gains on answerable questions.Source: E Impact of the Relaxed Beam Search… · p. 15

• Scores are averaged across three in-context-example variants to mitigate the effect of example selection in the few-shot setting.Source: F In-Context-Learning Variants · p. 15
• For regular prompts, the variants use two answerable examples, whereas prompts signaling possible (un)answerability use one answerable and (un)answerable example.Source: Figure 8 · p. 17, Figure 9 · p. 18, Figure 10 · p. 19
• The alternative prompt variants replace the unanswerable example’s answer with “IDK” or “N/A,” and the answerable-question beam experiments compare regular with relaxed beam search across beam sizes.Source: Table 9 · p. 16, Figure 8 · p. 17, Figure 9 · p. 18