PaperlayerView exact revision on arXiv

Source-linked AI summary

Look Before You Leap: Factual Decoding with Internal Attribution Signals

Hayeong Ryu, JungMin Yun, Byeonggeuk Lim, Sunhee Jo, YoungBin Kim

arXiv:2609.15745v1cs.CLcs.AI

TL;DR

Early factual errors can compound through autoregressive generation, and post-hoc correction or weight-level interventions cannot effectively preempt this snowballing. DESCAPE estimates an internal factual-attribution signal and uses it in candidate scoring to steer decoding. Across five factuality benchmarks, the paper reports improvements over decoding-time baselines in multiple settings.

  • Problem

    Early factual errors can compound through autoregressive generation, while post-hoc correction and weight-level interventions cannot effectively preempt hallucination snowballing.

    Source: §1 · p. 1

  • Method

    DESCAPE identifies a factual-salient MLP layer span through ablation, then uses a lightweight probe to estimate its attribution signal for candidate scoring during decoding.

    Source: §1 · p. 2, §3.2 · p. 3

  • Results

    Across five factuality benchmarks, DESCAPE improves factuality over decoding-time baselines in multiple settings, with less consistent gains on strict short-answer QA under EM/F1.

    Source: §6 · p. 9, Limitations · p. 9

  • Takeaways & Limitations

    The results highlight the potential of internal attribution signals as practical decoding-time signals for preventing hallucination-prone generation trajectories.

    Source: §6 · p. 9

    The probe tracks hallucination-prone steps at the zone level but does not guarantee token-level precision, so false positives may penalize factually correct continuations.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Hallucination remains a critical challenge in large language models (LLMs), where early factual errors compound through autoregressive generation in a snowballing effect that neither post-hoc correction nor weight-level intervention can effectively preempt. We propose DescaPE (DEcoding Signal Control Against Path Error-snowballing), a decoding framework that leverages internal model signals to suppress hallucination-prone trajectories at inference time. Through sliding-window MLP ablation, we identify a factual-salient layer span within LLMs whose derived signal is selectively elevated for factual tokens and exhibits anomalous spikes at hallucination-prone steps. We train a lightweight probe to approximate this signal from a single forward pass and integrate it into candidate scoring to penalize high-risk continuations while rewarding factually grounded ones. Experiments across five factuality benchmarks on three LLMs demonstrate that DescaPE achieves factuality improvements over decoding-time baselines in multiple settings, while incurring only 1.10x latency overhead in our efficiency evaluation. Our code is available at https://github.com/hayeonggg/DESCAPE.

1 Introduction

Early factual errors can compound during generation, while post-hoc correction and weight-level interventions cannot effectively preempt that snowballing. DESCAPE uses an internal factual-attribution signal to steer decoding away from hallucination-prone continuations, and the paper reports factuality improvements across multiple benchmark settings.

1 Introduction

  • Early factual errors can compound and propagate during autoregressive generation, a failure mode that post-hoc corrections and weight-level interventions cannot effectively preempt.This motivates a lightweight intervention during decoding that does not modify model parameters or rely on external modules.Source: §1 · p. 1
  • The authors ask whether an internal factual signal, decoupled from the final output distribution, can guide decoding beyond intrinsic steering, external verification, and self-verification approaches.Prior approaches are described as limited by initiated hallucinated trajectories, inference overhead and distribution shift, or reliance on the generator’s own distributional space.Source: §1 · pp. 1–2
  • A factual-salient layer span yields a signal elevated for knowledge-intensive tokens and unusually spiky at hallucinated-token steps.The signal tracks factual content during generation and provides a basis for preemptive intervention.Source: §1 · p. 2
  • DESCAPE uses a lightweight probe to estimate the attribution signal and integrates it into candidate scoring to penalize high-risk continuations and favor grounded ones.Across five factuality benchmarks, the paper reports improvements in several settings with minimal inference overhead.Source: §1 · p. 2

2 Related Work

Prior decoding-time approaches steer representations, rescore candidates with external signals, or use self-feedback. The paper distinguishes DESCAPE by deriving a candidate-level signal from the generator’s factual-salient layer span to intervene selectively at risky steps.

2.1 Intrinsic Signal-Based Intervention

  • Intrinsic steering and layer-wise logit adjustments promote factual predictions, but the cited approaches apply adjustments across generation steps without explicitly identifying hallucination-prone ones.DESCAPE derives a candidate-level attribution signal from a causally identified factual-salient span for selective intervention.Source: §2.1 · p. 2

2.2 Extrinsic Verification Decoding

  • External-model methods rescore or filter continuations, but can incur distribution mismatch and nontrivial inference overhead.Monitoring Decoding uses a large external monitor to resample high-risk tokens in real time; DESCAPE derives its signal from the generator’s own factual-salient span.Source: §2.2 · pp. 2–3

2.3 Self-Verification Decoding

  • Self-verification uses sampling agreement, self-consistency, or rollback, but remains within the generator’s representation space and may intervene only after an erroneous trajectory is detected.DESCAPE instead derives a signal from a factual-salient span distinct from the final output distribution for preventive trajectory control.Source: §2.3 · p. 3

3 Identifying Factual-Salient Layer Spans

Sliding-window MLP ablation identifies a factual-salient layer span whose signal tracks factual-token generation and differs between hallucinated and non-hallucinated tokens. The authors use this span as a basis for decoding-time risk assessment.

3.1 Background and Motivation

  • The method focuses on contiguous MLP spans because prior causal tracing localizes factual recall in mid-layer MLPs, which also show stronger factual attribution and factual–function separation than attention modules in component-wise ablation.The selected span’s candidate-level signal is defined using actual ablation rather than a factuality label.Source: §3.1 · p. 3, §3.2 · p. 4

3.2 Sliding-Window MLP Ablation

  • Sliding-window MLP ablation zeroes the outputs of MLPs within a contiguous layer span and scores the span by the original-versus-ablated probability gap for the target answer token.The score is averaged across answer tokens for multi-token answers, and the span with the maximum score is selected as factual-salient.Source: §3.2 · pp. 3–4

3.3 Empirical Observations on Factual Signal

  • All three models show a pronounced Factual Attribution Score peak at a specific span: layers 12–18 for Llama and Qwen, and 20–26 for Mistral.For Qwen, the signal is more diffuse across spans, although the highest-scoring span is designated and its validity is further corroborated in Section 5.2.Source: §3.3 · p. 4
  • Across the three models, δt is elevated for factual tokens such as answers, entities, and numbers, while remaining near zero for function words.The authors interpret this pattern as selective activation during factual knowledge retrieval.Source: §3.3 · p. 4
  • Hallucinated tokens exhibit abnormal δt spikes more frequently than non-hallucinated tokens across the models.The empirical CDF for non-hallucinated tokens lies above that for hallucinated tokens, even while δt remains broadly elevated during factual generation.Source: §3.3 · pp. 4–5

4 Methodology

DESCAPE uses internal signals from a factual-salient layer span to control hallucination-prone trajectories during decoding. A lightweight probe estimates the attribution signal, which is incorporated into beam scoring to penalize high-risk continuations and reward grounded ones.

4 Methodology

  • DESCAPE combines the probe with signal-integrated beam scoring to penalize high-risk continuations and reward factually grounded ones at inference time.The framework leverages internal signals from the factual-salient layer span.Source: §4 · p. 5

4.1 Factual Attribution Signal Approximation

  • A lightweight probe approximates the factual attribution signal from a single forward pass, avoiding the substantial overhead of dual forward passes at every decoding step.Training targets are collected for candidate tokens during autoregressive generation on up to 3,000 Databricks Dolly-15k samples.Source: §4.1 · p. 5
  • The probe training extends Huber loss with spike-sensitive weights that dynamically up-weight samples with large positive supervision targets.The stated design is intended to focus the probe on high-spike regions.Source: §4.1 · p. 5

4.2 Factual Signal-Based Decoding

  • The scoring strategy penalizes candidate tokens with anomalously high attribution signals and rewards tokens in the stable factual zone.The signal is combined with language-model generation probability; cumulative beam scores guide selection, with duplicate sequences removed and final length normalization applied.Source: §4.2 · pp. 5–6

5 Experiments

Across three LLMs, DESCAPE improves open-ended factual QA and long-form generation in several settings, with less consistent gains on strict short-answer EM/F1; further analyses examine its efficiency, components, probe, and prompt sensitivity.

5.1 Experimental Setting

  • The evaluation spans five benchmarks covering open-ended factual QA, short-answer QA, and long-form generation, using three base models and seven decoding or refinement baselines.TruthfulQA and FreshQA use LLM-as-Judge; FACTSCORE measures factual precision, while NQ and TriviaQA use EM, token-level F1, and SoftEM.Source: §5.1 · pp. 6–7

5.2 Main Results

  • DESCAPE achieves the highest TruthfulQA T*I(%) across all three models and scores 67.20 on Llama-3.1 FACTSCORE, while gains on NQ and TriviaQA are less consistent under strict EM/F1.On TruthfulQA, Llama-3.1 also gains +8.3 percentage points in Info(%) over greedy decoding.Source: §5.2 · p. 7
  • 1.10× latency overhead over greedy decoding is lower than Self-Refine’s 7.24× and END’s 1.83× on TruthfulQA with Llama-3.1.The paper characterizes this as a favorable trade-off between generation quality and computational cost.Source: Table 2 · p. 6, §5.2 · p. 7

5.3 Further Analysis

  • On 100-sample Llama-3.1 pilot subsets, penalty plus bonus performs best across benchmarks, while early stopping cuts latency by about 51.0% with comparable generation quality.For TruthfulQA, early stopping reduces latency from 56.1 to 27.5 ms/tok; the penalty contributes more than the bonus alone.Source: §5.3 · p. 7, Table 3 · p. 7
  • The learned probe agrees with the real signal on 75.8% of safe-zone and 69.6% of risk-zone assignments, while achieving comparable T*I(%) at roughly half the latency.Figure 4 presents zone-level agreement, and Table 4 compares probe-based decoding with the real-signal variant across three models.Source: Figure 4 · p. 7, §5.3 · p. 8, Table 4 · p. 8
  • The tested settings favor α=0.5 and τ=3 for competitive T*I(%) with low latency, γ=0.3 and τfact=0.5 near the best T*I score, and beam width B=5 for the best efficiency-performance trade-off.The moderate bonus setting scores 0.54 versus 0.56 at the best observed setting, and performance is largely insensitive to K in the tested range.Source: §5.3 · p. 8, Figure 5 · p. 8
  • With the decoding budget unchanged, an answer-only prompt shortens responses and improves EM and F1 across all tested NQ and TriviaQA model–dataset combinations.The analysis identifies response-format mismatch and strict lexical matching as contributing factors to the observed short-answer gap.Source: §5.3 · pp. 8–9, Table 5 · p. 8
  • In a TruthfulQA case study, DESCAPE gives a factually accurate answer identifying Estonia as the top-performing country, unlike most baselines’ responses containing hallucinated claims.The example is presented as qualitative evidence of suppressing hallucination-prone continuations at the token level.Source: §5.3 · p. 9, Table 6 · p. 9

6 Conclusion

DESCAPE uses an internal attribution signal from a factual-salient layer span to control decoding before hallucination-prone continuations are selected. Its probe supports efficient inference-time scoring, and experiments report factuality gains in multiple settings across five benchmarks.

6 Conclusion

  • Sliding-window MLP ablation identifies a factual-salient layer span whose signal is selectively activated for factual tokens and serves as a precursor to hallucination.The signal is incorporated into candidate scoring to intervene before hallucination-prone continuations are selected and propagate.Source: §6 · p. 9
  • A lightweight probe enables real-time candidate scoring without modifying base-model parameters or relying on an external verifier LLM.The conclusion describes the probe as an efficient approximation of the internal signal.Source: §6 · p. 9
  • Across five factuality benchmarks, DESCAPE improves factuality over decoding-time baselines in multiple settings with minimal latency overhead.The paper presents internal attribution signals as practical decoding-time signals for preventing hallucination-prone generation trajectories.Source: §6 · p. 9

Limitations

DESCAPE is scoped to fact-seeking generation, with several limitations around model-specific span identification, token-level risk classification, and the kinds of errors it addresses. The appendices describe how knowledge-available samples and candidate layer spans are selected for analysis.

Limitations

  • Offline layer-span identification requires extra pre-deployment computation and may need to be repeated for new architectures; risk classification can also penalize correct continuations.Strict short-answer QA gains are less consistent under EM/F1, partly because of response-format sensitivity and lexical matching.Source: Limitations · p. 9
  • DESCAPE is designed for fact-seeking generation, and its behavior on creative or other non-factual tasks remains unevaluated systematically.The paper recommends activating it selectively for factuality-critical prompts.Source: Limitations · pp. 9–10
  • The analysis targets decoding failures when knowledge is internally available, not distinguishing them from missing parametric knowledge or triggering abstention for genuine knowledge gaps.Detecting such gaps and using the signal to trigger abstention or refusal are identified as future directions.Source: Limitations · p. 10
  • The framework does not explicitly model cross-step logical consistency when individually plausible claims accumulate into a contradictory multi-hop answer.Source: Limitations · p. 10

A.1 Dataset and Sample Selection Criteria

  • For offline attribution analysis, TriviaQA samples are retained only when greedy responses partially match gold answers or aliases, reducing noise from knowledge gaps.The filter is not used during decoding; inference scores candidates without gold answers or known/unknown labels.Source: §A.1 · p. 14

A.2 Layer Span Search Strategy and Ablation Setup

  • The layer-span search uses seven-layer windows with stride four from layer 8 onward, selecting the span with the highest mean attribution signal.Span-level rather than single-layer ablation is used because adjacent layers may play complementary roles.Source: §A.2 · p. 14

A.3 Component-Wise Ablation Comparison

The component ablation supports MLP spans as the source of the strongest factual attribution and factual–function separation. The appendices describe token-level analyses, probe supervision and scoring, and evaluation measures.

A.3 Component-Wise Ablation Comparison

  • MLP ablation yields the strongest factual attribution and factual–function separation; adding attention does not further increase the signal.Table 7 compares MLP, attention, and joint MLP+attention under the same sliding-window setup.Source: Table 7 · p. 14, §A.3 · p. 14

B.1 Signal Sensitivity by Token Type

  • Token sensitivity is examined across answers, entities, numbers, and function words, while hallucinated and non-hallucinated token signals are compared by empirical CDFs.The token-type analysis uses correct predictions; the hallucination comparison uses incorrect TruthfulQA samples containing both token types.Source: §B.1 · p. 14, §B.2 · p. 14

C Probe Training Data Collection

F.1 Benchmark Evaluation Protocols

  • Evaluation combines benchmark-specific judging for TruthfulQA, FreshQA, and FACTSCORE with SoftEM substring matching and Spearman probe correlations, including triggered steps.The supplied protocols specify GPT-4o-mini judging for TruthfulQA and FreshQA and retrieval-plus-ChatGPT evaluation for FACTSCORE.Source: §F.1 · p. 16, §F.2 · p. 16, §F.3 · p. 16

G Experimental Implementation Details

The implementation details specify probe optimization and the decoding baselines used in experiments, along with the hardware platform.

G Experimental Implementation Details

  • The probe is trained for 30 epochs with AdamW at learning rate 3 × 10−4, batch size 512, and an 80/20 train-validation split.The best checkpoint is selected by validation Spearman ρ; gradient clipping, Huber threshold, and spike-weight settings are also specified.Source: G Experimental Implementation Details · p. 16
  • Baselines include greedy decoding, beam search with width 5, default DoLa, a two-stage Self-Refine pipeline, and official implementations of SLED, END, and ActLCD.Experiments ran on a single NVIDIA GeForce RTX 3090 24GB GPU.Source: G Experimental Implementation Details · pp. 16–17
  • Table 9 defines Global as the fixed (0.5, 3.0) setting and compares it with the best configuration for each model–benchmark pair.Source: Table 9 · p. 17

H Hyperparameter Transfer and Label-Free Calibration

The fixed hyperparameter setting transfers reasonably well across the evaluated model–benchmark pairs, and label-free threshold calibration stays close to label-tuned alternatives. The section describes both the transfer comparison and unlabeled-prompt calibration.

H Hyperparameter Transfer and Label-Free Calibration

  • The transfer sweep tests α ∈ {0.3, 0.5, 0.7} and τ ∈ {2.5, 3.0, 3.5} on subsets of TruthfulQA, FreshQA, and TriviaQA.The quantile notation Qq denotes the q-quantile of the corresponding empirical distribution.Source: H Hyperparameter Transfer and… · p. 17
  • The global setting remains within 1.5 percentage points of the individually best configuration in seven of nine model–benchmark combinations, with larger gaps for Llama-3.1 on FreshQA and TriviaQA.Source: H Hyperparameter Transfer and… · p. 17
  • Across all three models, the label-free configuration remains within 1.0 percentage point of both global and oracle settings, showing model-specific threshold scaling can be calibrated without held-out factual labels.Only scale-sensitive thresholds are calibrated from the predicted attribution distribution ˆδ on unlabeled prompts; α = 0.5 and γ = 0.3 remain fixed.Source: H Hyperparameter Transfer and… · p. 17

I Analysis on Short-Answer QA

On short-answer QA, DESCAPE’s gains look more modest under standard EM/F1, while its Llama-3.1 results improve under SoftEM; output length and gold-token preservation provide context for interpreting these findings.

I Analysis on Short-Answer QA

  • DESCAPE generates longer responses, especially for Llama-3.1 on NQ (43.85 tokens) and TriviaQA (15.70 tokens), while standard EM/F1 gains appear more modest.The passage notes that standard EM-based evaluation can underestimate factual accuracy when correct information is embedded in longer responses; the short-answer underperformance may therefore be partly attributable to metric sensitivity.Source: I Analysis on Short-Answer QA · p. 18
  • On Llama-3.1, DESCAPE reaches 75.77 on NQ and 49.56 on TriviaQA under SoftEM, narrowing the gap with other methods.For Mistral-v0.3 and Qwen2.5, where output lengths are more comparable across methods, SoftEM gains are relatively modest.Source: I Analysis on Short-Answer QA · p. 18
  • Gold Preservation measures whether the gold token stays top-ranked after DESCAPE scoring when the base model ranked it top-1; preservation is 97.91–99.87% across model–dataset combinations.These results provide little evidence of broad candidate-level degradation on short-answer QA.Source: I Analysis on Short-Answer QA · p. 18

J Probe Quality Analysis

Probe approximation remains stable across tested labeling scales, with low risk false positives; supplementary analyses report preserved multi-hop performance and provide dataset prompts and case studies.

J Probe Quality Analysis

  • Overall probe correlation remains high across labeling scales (Llama-3.1: 0.874 for both), while 3k labels modestly improve triggered-step ρ for Mistral-v0.3 and Qwen2.5.The main experiments use 1k labels for Llama-3.1 and 3k for Mistral-v0.3 and Qwen2.5.Source: J Probe Quality Analysis · pp. 18–19
  • TruthfulQA performance is relatively stable across labeling scales: Llama-3.1 has its highest T*I(%) with 1k labels (48.5), while 3k labels yield a marginal drop (47.7).Mistral-v0.3 and Qwen2.5 show slight improvements with 3k labels.Source: J Probe Quality Analysis · p. 19
  • Risk FP—the share of real-signal safe or factual candidates misclassified as risk—is below 6% across all three models.The probe does not perfectly recover the real signal, but candidates the real signal deems non-risk are rarely penalized solely due to approximation error.Source: J Probe Quality Analysis · p. 19

K Multi-Hop Reasoning Preservation

  • DESCAPE has comparable multi-hop performance to Beam Search while preserving most Beam-correct answers and their supporting facts.The reported results suggest the intervention does not broadly suppress correct multi-hop reasoning.Source: K Multi-Hop Reasoning Preservation · p. 19, Table 17 · p. 19

L Prompt Templates

M Case Studies

Related papers

Loading 2609.15745v1…