Source-linked AI summary
Can a Model Catch Its Own Hallucinations for Free?: Label-Free Doubt Signals Hold Their Own Against a Labelled Dataset for Abstention
Ali Asaria, Tony Salomone, Deep Gandhi
TL;DR
Teaching models to abstain usually requires correctness labels, so this paper tests whether their own confidence can guide abstention without labels. Across six open-weights models, label-free confidence-gated abstention matches label-supervised abstention-tuning at matched coverage, with no statistically detectable difference.
Problem
Teaching language models to abstain usually requires correctness labels, motivating whether their own free confidence can provide a label-free alternative.
Method
The paper fine-tunes models to natively abstain on low-confidence answers, using frozen token-probability confidence without correctness labels.
Results
At matched coverage, no statistically detectable hallucination difference appears between label-free and label-supervised abstention-tuning across all six models.
Takeaways & Limitations
A model’s own confidence is a promising near-free substitute for correctness labels when teaching short-form abstention.
Takeaways & Limitations
Confidently wrong answers remain a dominant failure mode because the single-pass confidence signal cannot flag them.
Abstract
from arXiv · showhide
Large language models state false facts as fluently as true ones, yet a model often "knows" internally when it is on shaky ground: the probability it assigns to its own answer tends to dip on the facts it gets wrong. The usual way to act on this, teaching a model to abstain rather than guess, requires a labelled dataset of right and wrong answers. We ask whether the model's own confidence, which is free and needs no labels, can do that job instead. We fine-tune each model (with LoRA) to answer when its frozen confidence is high and to say "I'm not sure" when it is low, using the signal alone and no correctness labels. Across six open-weights models (1B-8B, two families) on short-form factual question answering, with correctness adjudicated by an independent judge model, this label-free recipe holds its own against label-supervised abstention-tuning: at matched coverage we find no statistically detectable difference between the two. A control that drills hard examples instead of abstaining does not help, indicating the gain comes from calibration, not rote memorization. The signal's one blind spot is confidently wrong facts, which it cannot flag. A model's own doubt is thus a near-free substitute for a labelled dataset when teaching it when to abstain. Code and artifacts are available on request.
1 Introduction
The paper introduces label-free signal-gated abstention fine-tuning, using a model’s frozen confidence to teach native abstention without correctness labels. Across six models, it matches label-supervised abstention-tuning at matched coverage, with gains attributed to calibration and a known blind spot for confidently wrong facts.
- Method: The method retargets low-confidence training questions to “I’m not sure” while preserving high-confidence answers, using frozen token-probability confidence without correctness labels.Confidence is computed once over each answer span before fine-tuning.
- Evidence: Across six models and two families, frozen confidence discriminates the models’ hallucinations, including at 1B scale.The signal is higher on facts the models get right than on facts they get wrong.
- Results: At matched coverage, the label-free recipe shows no detectable difference from label-supervised abstention-tuning on any of six models.Correctness is adjudicated by an independent judge.
- Mechanism: The up-weighting control does not reduce hallucination, indicating that the improvement comes from calibration rather than memorization.The control drills the same hard questions without abstaining.
- Limitations: The method cannot flag confidently-wrong facts, and hallucination concentrates among rare entities.These limitations are mapped in the paper’s analysis of failure cases.
2 Related Work
Related work detects hallucinations using internal confidence and teaches abstention using correctness supervision, while other methods train confidence into model weights. This work differs by using a frozen, single-pass token-probability signal thresholded into binary abstain/answer targets.
- Internal confidence signals: Internal-confidence research uses token probabilities and activation- or verbalization-based alternatives, while cautioning that raw sequence probability can be weak for question answering.Kumaran et al. identify intrinsic blind spots in first-order token log-probabilities.
- Teaching abstention and refusal: Abstention and refusal methods commonly rely on correctness signals, labels, or rewards to determine when models should abstain.Examples include refusal-aware tuning, reinforcement-learning-based knowledge-boundary abstention, and training-free conformal abstention.
- Internalizing confidence by training: Confidence-training methods incorporate confidence or abstention behavior through confidence tokens, routing, cascade tuning, calibration losses, or proper-scoring rewards.These approaches bake confidence or abstention behavior into model weights.
- Internalizing confidence by training: The closest self-supervised methods distill sampling-, self-evaluation-, or self-distillation-based uncertainty, whereas this work uses a frozen, single-pass token-probability signal.The signal is thresholded into a binary abstain/answer target using a cheap recipe.
3 Method
The method uses the frozen base model’s own token-level confidence to gate abstention during LoRA fine-tuning, without correctness labels. It compares this label-free recipe with standard self-distillation, label-supervised abstention, and a hard-example up-weighting control.
- The confidence signal: Confidence is the mean log-probability assigned by the frozen base model to tokens in its own greedily decoded answer.Higher confidence indicates concentrated probability; lower confidence indicates internal hedging despite fluent output.
- Signal-gated abstention fine-tuning: C3 ranks training questions by confidence, teaches “I’m not sure.” on the lowest-confidence fraction, and self-distills the model’s answer elsewhere.The abstention threshold controls coverage; fine-tuning uses LoRA with r=16 and no correctness labels.
- Comparators: C1 applies LoRA to the model’s greedy answers everywhere, providing no abstention and no confidence signal.The base comparator is the untrained instruction model.
- Comparators: C2 uses correctness labels to make abstain-or-answer decisions, while C4 up-weights low-confidence questions toward answering instead of abstaining.C2 follows an R-Tuning-style recipe; C4 tests whether gains reflect memorization rather than calibration.
4 Experimental Setup
The experiments evaluate six LoRA-tuned open-weight models on entity-disjoint factual QA splits, using coverage–hallucination tradeoffs and AURC to assess selective answering. An independent judge adjudicates test correctness, with matched-coverage comparisons and bootstrap confidence intervals.
- Models: Six open-weight instruction models span two families and three scales: Llama-3.2-1B/3B, Llama-3.1-8B, and Qwen3-1.7B/4B/8B.All models use LoRA with rank r=16, with one training run per family–scale cell.
- Data: The dataset contains 700 development and 700 test questions from PopQA and TriviaQA, with 300 TriviaQA and 400 PopQA questions per split.Development and test sets share zero entities, preventing memorized-fact leakage; each model generates its own on-policy answers and confidence signal.
- Metrics and protocol: Coverage measures answered questions, hallucination rate measures incorrect answered items, and AURC summarizes the risk–coverage curve, with lower AURC better.The risk–coverage curve varies the abstention threshold, capturing the tradeoff that greater abstention generally leaves fewer, easier questions answered.
- Metrics and protocol: Test correctness is adjudicated by the independent open-weight gemma-2-27b-it judge, while matched-coverage hallucination comparisons use 1,000-bootstrap percentile intervals.The judge is outside the Llama and Qwen families, and the intervals capture test-set sampling variance rather than training-seed variance.
5 Results
The frozen mean-log-probability signal distinguishes correct from incorrect answers across six models and supports label-free abstention. At matched coverage, label-free abstention matches label-supervised abstention, while selective prediction and controls indicate calibrated doubt rather than memorization.
- Signal discrimination: AUROC reaches 0.778/0.821/0.858 for Llama 1B/3B/8B and 0.755/0.778/0.725 for Qwen3 1.7B/4B/8B, separating correct from incorrect answers.All values exceed 0.65; performance rises monotonically with Llama scale but not with Qwen3 scale.
- Abstention performance: At matched coverage, overlapping C2-vs-C3 95% bootstrap confidence intervals show no detected hallucination difference between label-supervised and label-free abstention.The C3/C2 reduction ratio is 0.95–1.10, interpreted as parity within noise rather than superiority.
- Calibration versus memorization: The up-weighting control remains at base hallucination with near-full coverage, whereas retargeting low-signal questions to abstention reduces hallucination.This supports calibration rather than memorization; the control uses string-match evaluation at n=250, while the stronger selective-prediction result uses judge-based evaluation at n=700.
- Selective prediction: 7/25/28% and 17/24/26% are C3’s area-under-risk–coverage values for Llama 1B/3B/8B and Qwen3 1.7B/4B/8B, respectively.The largest gains occur for larger Llama models, and useful corrections outnumber wasted abstentions by roughly 7:1.
- Robustness: The parity result holds across correctness metrics, model families, and n=250 to n=700, while replacing mean log-probability with energy yields no gain.An apparent 69% point estimate at n=250 disappears under the n=700 bootstrap, indicating sampling noise.
6 Discussion and Limitations
Across six models, label-free abstention matches label-supervised abstention at approximately matched coverage, supporting confidence as a near-free substitute for correctness labels. Its limits are rare-entity hallucinations and especially confidently wrong answers that evade the signal, while equivalence testing remains a needed refinement.
- Failure modes: 2–3×: hallucination rates are roughly this much higher for the worst popularity quartile than the best, including Llama-8B at 0.89 versus 0.32.The method addresses rare facts by abstaining rather than knowing them.
- Failure modes: 36% of all items are answered-but-wrong with sufficiently high confidence that the signal fails to flag them, comprising most of the 39% answered-but-wrong residual.C3 retains and self-distills these confidently wrong answers, so the calibration claim applies specifically to low-signal abstention behavior.
- Practical takeaway: Across six models, label-free abstention shows no detected difference from label-supervised abstention at matched coverage, while both outperform the untrained base.The comparison uses overlapping 95% bootstrap confidence intervals, and the label-free method requires no correctness labels.
- Future refinements: A paired equivalence test at exactly pinned coverage and multiple-comparison adjustment across six models are proposed to strengthen the parity finding.The reported result is failure to detect a difference rather than formal equivalence; the small residual coverage gap, if anything, favors label-free abstention.
8 Conclusion and Future Work
On English short-form factual QA with models up to 8B, frozen token-probability confidence can enable label-free native abstention competitive with label-supervised abstention-tuning across six open-weights models, with no detected difference and no labeling cost. Future work includes stricter equivalence testing, additional baselines and self-supervised signals, and extension to long-form factuality.
- Conclusion: Frozen token-probability confidence enabled label-free native abstention competitive with label-supervised abstention-tuning across six open-weights models up to 8B.The setting was English short-form factual question answering.
- Conclusion: No difference was detected between label-free and label-supervised abstention-tuning, and the label-free method incurred no labeling cost.The reported gain is consistent with calibration rather than memorization.
- Future Work: Future work proposes a formal equivalence test at exactly pinned coverage and both judge-trained and second label-supervised baselines.These additions would strengthen comparisons between abstention approaches.
- Future Work: Future work also includes richer self-supervised signals, such as verbalized confidence and activation probes, plus extension to long-form factuality.These directions broaden the confidence-signal approach beyond the current setting.