Alleviating Hallucination in Reasoning Tasks with Training-Free Uncertainty-Guided Steering

1 Introduction

• Hallucination detection commonly uses inference-time confidence estimates to signal when a model should abstain, but these post-hoc mechanisms are reactive.The paper asks whether uncertainty signals can instead steer generation toward more reliable regions.Source: §1 · p. 1
• Prior supervised-probe steering applies static truthfulness directions uniformly, despite intermediate representations changing across multi-step reasoning.These methods also require extra training and are sensitive to shifts between probe-training data and inference-time distributions.Source: §1 · pp. 1–2
• USteer uses intermediate-layer uncertainty signals, including NCI scores under the logit lens, which emerge before the final layer and provide real-time feedback on inference reliability.The paper describes intermediate layers as an internal ensemble.Source: §1 · p. 2
• At each decoding step, USteer uses the gradient of an uncertainty score with respect to intermediate hidden representations to nudge them toward higher-confidence regions.The on-the-fly intervention adapts to the evolving reasoning chain without supervised training or parameter updates.Source: §1 · p. 2
• USteer consistently improves reasoning accuracy while reducing hallucinations across CommonsenseQA, StrategyQA, GSM8K, and AQuA and across Llama-3.2, Qwen-2.5, and Qwen-3, with negligible latency overhead.The paper presents this as a practical and robust framework for inference-time control.Source: §1 · p. 2

2 Related Work

• Prior activation-steering work spans sentiment transfer, cross-lingual understanding, and safety-related steering against jailbreaking and toxic generation.The cited examples include Subramani et al. (2022), Turner et al. (2023), Mahmoud et al. (2025), and Wang et al. (2025).Source: §2 · pp. 2–3
• Closest to USteer, hallucination-mitigation methods use supervised probes to derive a static truthfulness direction applied across decoding steps, whereas USteer steers dynamically using internal uncertainty signals.The passage attributes the dynamic steering approach to Liu et al. (2026) uncertainty signals.Source: §2 · p. 3
• Contrastive decoding compares expert and amateur predictions, while USteer uses early-layer uncertainty to steer hidden representations rather than select tokens contrastively.DoLa treats final-layer logits as expert predictions and earlier-layer logits as amateur predictions.Source: §2 · p. 3

3.1 Setup

• The NCI-derived confidence guidance score is a squared cosine similarity between a hidden representation and the weight vector of its most-likely token.The paper uses the score to quantify uncertainty associated with a prediction.Source: §3.1 · p. 3

3.2 Observation: Uncertainty Signal Emerges at Intermediate Layers.

• The logit lens normalizes intermediate layer outputs using final-layer gains, selects each layer’s likely token, and computes its guidance score.The normalization uses the learned gains from the model’s final normalization layer and the feature-dimension RMS.Source: §3.2 · p. 4

3.3 USteer: Uncertainty-guided Steering at Intermediate Layers.

• USteer performs single-step gradient ascent on the confidence score, propagates the gradient through RMSNorm, and applies a normalized update to the pre-normalization residual state.The steering-strength hyperparameter is α.Source: §3.3 · pp. 4–5

3.2 Observation: Uncertainty Signal Emerges at Intermediate Layers.

• Correct intermediate embeddings tend to have higher guidance scores than incorrect ones; the distinction is strongest at Layer 15 but appears at Layers 13 and 14.The paper also reports discriminative AUROC results across multiple models, datasets, and layers.Source: §3.2 · p. 4, Figure 2 · p. 5

4.1 Main Results

• USteer improves reasoning accuracy across evaluated datasets and model architectures with negligible latency overhead, while ITI and TruthX sometimes underperform the base model.Table 1 reports accuracy and latency; the authors attribute USteer's stronger results to dynamic, on-the-fly uncertainty signals rather than static directions.Source: §4.1 · pp. 6–7, Table 1 · p. 7

4.2 Performance across Confidence Regions

• USteer yields larger net accuracy gains than existing methods across pooled confidence regions, especially in the lowest-confidence tail, but can reinforce confidently incorrect predictions.The confidence regions pool examples across four datasets; gains are measured against greedy decoding.Source: §4.2 · p. 7, Table 2 · p. 8

4.3 Effect of Steering Layers

• Steering improves accuracy at lower, middle, and upper layers, with larger gains at upper layers where uncertainty signals distinguish correct from hallucinated generations more strongly.This pattern supports restricting candidate steering layers to the final intermediate layers.Source: §4.3 · p. 8, Table 3 · p. 9
• 68.56 accuracy is the standard-decoding baseline; all tested steering depths outperform it, with an approximately bell-shaped trend reflecting stronger influence versus less reliable lower-layer guidance.The best-performing depth varies across model families.Source: §4.3 · pp. 8–9, Figure 3 · p. 8

4.4 Effect of Steering Strength

• Across CSQA and GSM8K, increasing steering strength initially improves accuracy, but excessive strength such as α = 0.5 degrades performance; a broad range beats standard decoding.Performance varies smoothly with α on both tasks.Source: Figure 4 · p. 8, §4.4 · p. 9

4.5 Effect of Stochastic Decoding

• On CSQA, USteer consistently improves over standard stochastic decoding across tested temperatures, with non-overlapping 95% confidence intervals in most settings.The evaluation uses five runs with different random seeds per temperature.Source: §4.5 · p. 9, Table 4 · p. 9

4.6 Effect of Alternative Confidence Guidance Scores

• Softmax confidence and fDBD both improve accuracy over standard inference across datasets, although softmax yields smaller gains than NCI and fDBD.The alternative-score evaluation uses Qwen-2.5-3B-Instruct.Source: §4.6 · p. 10, Table 5 · p. 10
• USteer requires intermediate hidden states and gradients, limiting use with black-box proprietary APIs; it targets hallucination, not bias, fairness, or harmful-content concerns.The authors also note that uncertainty signals may be less reliable in future architectures without residual-connection ensembles, potentially reducing effectiveness.Source: §4.6 · p. 10

5 Conclusion

• Across multiple tasks and settings, USteer guides generation toward more reliable outputs with negligible overhead.The conclusion characterizes this as proactive hallucination mitigation integrated into decoding.Source: §5 · p. 10
• USteer dynamically nudges intermediate representations toward high-confidence regions using a geometric uncertainty guidance score.The intervention is training-free and integrated directly into decoding.Source: §5 · p. 10
• Unlike prior steering based on static supervised-probe directions, USteer uses uncertainty signals proactively during generation.The conclusion frames this as moving beyond post-hoc filtering toward more adaptive and controllable language models.Source: §5 · p. 10

A.1.1 CSQA Example

• The CSQA example answers that a bank is the likely security measure served by a revolving door.The response selects choice (a).Source: §A.1.1 · p. 14

A.1.2 StrategyQA Example

• The example answers no to whether Albany, Georgia, will reach 100,000 occupants before Albany, New York, reasoning that Georgia’s Albany is unlikely to surpass New York’s population.It notes that Albany, New York, already has a population exceeding 100,000.Source: §A.1.2 · p. 14

A.1.3 GSM8K Example

• The GSM8K example calculates that Janet sells nine eggs daily and earns $18.The response subtracts seven eggs used for breakfast and muffins from 16, then multiplies the remainder by $2.Source: §A.1.3 · p. 14

A.1.4 AQuA Example

• The AQuA example counts 676 letter pairs and 1,000 digit triples, yielding 676,000 passwords.The response uses repetition being allowed and multiplies the two counts.Source: §A.1.4 · p. 14

B.1 Evaluation on Qwen-2.5-14B-Instruct.

• USteer improves reasoning accuracy over standard decoding on all four datasets, performs best on CSQA and AQuA, and is competitive on StrategyQA and GSM8K.Table 6 reports reasoning-task accuracy for Qwen-2.5-14B-Instruct.Source: Table 6 · p. 15, §B.1 · p. 15

A.2 Chain-of-Thought Few-shot Prompting

• The prompting setup uses canonical chain-of-thought few-shot examples and extracts task-specific answers from designated response phrases.Extracted answers are counted correct when they match the ground truth; steering evaluation on Qwen-2.5-14B-Instruct uses its last three intermediate layers.Source: §A.2 · p. 15, §B.1 · p. 15

B.2 Evaluation on Qwen-3-32B.

• USteer improves reasoning accuracy on CSQA, StrategyQA, and GSM8K while maintaining accuracy on AQuA.The passage describes AQuA as a challenging task.Source: §B.2 · p. 15
• Qwen-3-32B has substantially higher accuracy across datasets, leaving less headroom for further improvement than smaller models.The passage presents the results as further evidence that uncertainty-based steering remains effective as model scale increases.Source: §B.2 · p. 15
• USteer consistently improves standard decoding across datasets on Qwen-3-32B.Table 7 reports task accuracy and states that USteer consistently improves standard decoding across datasets.Source: Table 7 · p. 15, §B.2 · p. 15

C Statistical Analysis under Greedy Decoding

• USteer’s point estimates improve over standard greedy decoding across all datasets and both models, but the intervals preclude conclusive significance claims.The analysis computes 95% bootstrap confidence intervals by resampling finite evaluation sets.Source: C Statistical Analysis under Greedy… · p. 16
• StrategyQA and AQuA have approximately ±6 percentage-point intervals, versus approximately ±3 points or less for CSQA and GSM8K.The smaller StrategyQA and AQuA sets contain 229 and 254 examples; CSQA and GSM8K contain 1,221 and 1,319.Source: C Statistical Analysis under Greedy… · p. 16
• Benchmark evaluation-set sizes primarily limit the statistical power of these comparisons.Greedy decoding is deterministic, so the reported accuracy uncertainty reflects finite test examples rather than variation across repeated runs.Source: C Statistical Analysis under Greedy… · p. 16

D Proof of Lemma 3.2

• Table 9 reports AUROC for uncertainty guidance scores across four reasoning datasets, three models, and each model’s final three layers.Higher AUROC indicates stronger discrimination between correct and incorrect generations; entries are ordered from the final layer to the two preceding layers.Source: Table 9 · p. 17

E fDBD Confidence Guidance Score.

• The fDBD score is presented as an alternative to NCI for confidence guidance, using the simplified, squared formulation from Definition 3.1.The passage defines Ck(z) as the k highest-scoring alternative tokens, excluding the most likely token.Source: E fDBD Confidence Guidance Score · p. 17
• All fDBD experiments use k = 1000, and a higher score indicates greater average separation from competing-token decision boundaries and thus higher confidence.Source: E fDBD Confidence Guidance Score · p. 17

• AUROC evaluates g(zl) as a predictor of generation correctness across four reasoning datasets and the final three layers of three Llama and Qwen models.Correctness is the binary target; higher AUROC indicates stronger discrimination between correct and incorrect generations, with 50% corresponding to chance.Source: F Quantitative Evaluation of… · p. 17
• The predictive signal varies across models, datasets, and layers.Source: F Quantitative Evaluation of… · p. 17
• The AUROC findings support the observed distributional trend that uncertainty signals predictive of correctness can emerge before the final layer.The passage identifies this as further motivation for using intermediate uncertainty signals to guide inference.Source: F Quantitative Evaluation of… · p. 17

• USteer’s example identifies the implicit constraint behind receiving only the puppy and selects the correct answer, unlike standard decoding.The standard response associates pets with kennels; the steered response infers that the person was limited in the number of pets.Source: G Qualitative Analysis of Generated… · pp. 17–18
• Standard decoding focuses on the association between pets and kennels, leading to the incorrect answer (C).Source: G Qualitative Analysis of Generated… · p. 17
• USteer reasons that the person wanted both pets but was limited by her circumstances, choosing answer (A).Source: G Qualitative Analysis of Generated… · pp. 17–18

H Effect of USteer on Stochastic Generation Behavior

• USteer modestly increases most-likely-token selection and average predictive probabilities, while reducing distinct answers across five sampling runs.The reported measures are the percentage of steps selecting the highest-probability token, average predictive probability, and average distinct answers per question.Source: H Effect of USteer on Stochastic… · p. 18, Table 10 · p. 18, Table 11 · p. 18
• Average generation length shows a similar temperature-dependent trend with and without USteer, so steering is not simply greedy or lower-temperature decoding.Source: H Effect of USteer on Stochastic… · p. 18, Table 13 · p. 18

NeurIPS Paper Checklist

• The checklist reports that code was not released at submission time because release was subject to legal and compliance review.The paper uses publicly available datasets and models, but full open access was not provided at submission.Source: NeurIPS Paper Checklist · p. 21