PaperlayerView exact revision on arXiv

Source-linked AI summary

Internal Flow Signatures for Self-Checking and Refinement in LLMs

Sungheon Jeong, Sanggeon Yun, Ryozo Masukawa, Wenjun Haung, Hanning Chen, Mohsen Imani

arXiv:2602.01897v1cs.LG

TL;DR

LLMs can generate fluent but context-unfaithful answers, while external safeguards add cost and act after generation. The paper introduces internal flow signatures from stabilized depthwise dynamics, then uses a lightweight validator for self-checking and targeted single-block refinement. The reported outcome is consistent hallucination separability across QA and General, with AUROC typically above 0.65, alongside localized correction without changing the base model.

  • Problem

    LLMs can produce fluent, globally incorrect answers, while external verification and post-generation judges add latency and cost and leave decision formation largely unobserved.

  • Method

    The method bias-centers states at a fixed boundary, constructs moving readout-aligned competitor subspaces, transports adjacent frames orthogonally, and trains a lightweight validator on the resulting flow signatures.

  • Results

    AUROC typically above 0.65 on QA and General, where internal flow signatures provide a consistently separable signal for hallucination detection.

  • Takeaways & Limitations

    Internal monitoring can support self-checking, culprit localization, and a lightweight single-point correction without modifying the base model.

  • Takeaways & Limitations

    Refinement is a first-step single-block clamp whose gains are heterogeneous because long-form deviations can be distributed across tokens and depths.

Abstract

from arXiv · show

Large language models can generate fluent answers that are unfaithful to the provided context, while many safeguards rely on external verification or a separate judge after generation. We introduce \emph{internal flow signatures} that audit decision formation from depthwise dynamics at a fixed inter-block monitoring boundary. The method stabilizes token-wise motion via bias-centered monitoring, then summarizes trajectories in compact \emph{moving} readout-aligned subspaces constructed from the top token and its close competitors within each depth window. Neighboring window frames are aligned by an orthogonal transport, yielding depth-comparable transported step lengths, turning angles, and subspace drift summaries that are invariant to within-window basis choices. A lightweight GRU validator trained on these signatures performs self-checking without modifying the base model. Beyond detection, the validator localizes a culprit depth event and enables a targeted refinement: the model rolls back to the culprit token and clamps an abnormal transported step at the identified block while preserving the orthogonal residual. The resulting pipeline provides actionable localization and low-overhead self-checking from internal decision dynamics. \emph{Code is available at} \texttt{github.com/EavnJeong/Internal-Flow-Signatures-for-Self-Checking-and-Refinement-in-LLMs}.

1. Introduction

The paper addresses fluent but context-unfaithful LLM answers and the cost of post-generation safeguards by auditing depthwise decision formation internally. It stabilizes internal measurements with bias-centered monitoring and moving readout-aligned frames, enabling lightweight validation and targeted refinement.

  • LLMs can produce fluent answers that are locally consistent yet globally incorrect, while users also need insight into whether confidence reflects appropriate reasons.
  • External verification, retrieval, and additional LLM judges add latency and cost while observing the decision process only after generation.
  • Depthwise logit competition between the top token and close competitors provides a measurable trace of how token decisions form.
  • The method removes depth-dependent token-shared offsets and uses window-specific readout-aligned frames with orthogonal alignment for stable depthwise signatures.
  • A lightweight validator detects atypical transported motion and localizes a culprit event, enabling single-block refinement without modifying the base LLM.

2. Related Work

Prior work addresses hallucination through external checks, intermediate readouts, representation analysis, internal reliability signals, and inference-time interventions. This paper connects these lines by using moving flow signatures to localize and refine sample-specific failures.

  • External verification, retrieval augmentation, post-hoc language-model checking, and faithfulness benchmarks mitigate factual errors but add cost and treat the base model as a black box.
  • Logit Lens, Tuned Lens, and related probes decode intermediate logits to study how token predictions evolve across depth.
  • Depth-dependent representation geometry and shifting task-relevant subspaces limit the usefulness of a single global coordinate system.
  • The paper models local readout-aligned subspaces and their drift as meaningful depthwise computation rather than nuisance variation.
  • Internal reliability methods predict errors or hallucinations without modifying the base model using generation statistics, entropy, logit dynamics, or activations.
  • Unlike prior internal interventions, this refinement localizes a sample-specific token-depth culprit and alters only its update while preserving the orthogonal component.

3. Depthwise Flow Signatures

The method represents depthwise residual-stream dynamics at a fixed monitoring boundary using bias-centered, moving readout-aligned subspaces. Orthogonal transport produces basis-invariant signatures of motion, turning, component updates, and drift for validation and localization.

  • 3.1. Setup, monitoring boundary, and objective: All measurements use residual-stream states and block contributions observed at one fixed inter-block monitoring boundary.
  • 3.2. Competitor Directions and Moving Subspaces: Window-specific subspaces are fitted from top-token and competitor directions obtained through logit readouts, avoiding a single global readout subspace.
  • 3.1. Setup, monitoring boundary, and objective: Bias-centered monitoring removes depth-dependent token-shared normalization offsets from token-wise motion.
  • 3.2. Competitor Directions and Moving Subspaces: Sampled competitor directions are stacked and decomposed by SVD, with the top k right singular vectors forming each window basis and deterministic fallback handling degenerate windows.
  • 3.3. Transported Flow Signatures: Adjacent window frames are aligned by orthogonal transport, producing transported increments, step lengths, turning angles, centered increments, and component summaries.
  • 3.3.2. Component Contributions Under Boundary Normalization: Boundary-normalized attention and MLP injections are projected into the target frame using path-integrated effective updates and Jacobian-vector products.
  • 3.3.3. Subspace Drift and Gauge Invariance: Orthogonal coordinate changes preserve Euclidean norms and angles, establishing invariance of transported step length and turning angle to within-window basis rotations.
  • 3.5. Flow-Guided Refinement: The resulting flow signatures support validator-based detection and depth-localized culprit events for downstream self-checking and refinement.

4. Experiments

Experiments evaluate internal flow signatures for hallucination detection and flow-guided refinement across HaluEval tasks and LLM families. Detection is strongest for localized depthwise deviations, while refinement benefits depend on task locality and incurs bounded regeneration overhead.

  • Experimental setup: The validator is evaluated on HaluEval across QA, Dialogue, Summarization, and General tasks and five LLM families.Each task-model split uses an 8:2 train:test prompt split.
  • Detection results: AUROC is typically above 0.65 on QA and General across model families, showing consistent separability from internal flow signatures.The reported metrics are classification Accuracy and AUROC.
  • Flow patterns: QA hallucinations often produce narrow depth-localized bursts with increased transported step length, turning, effective update magnitude, and concentrated drift.General shows a related localized regime change, whereas late diffuse accumulation weakens separability.
  • Flow patterns: Summarization and Dialogue show task-specific flow regimes, with separability weakening when hallucination remains diffuse rather than concentrated.Summarization more often remains diffuse and low-energy at later depths during non hallucination; Dialogue includes both burst and diffuse regimes.
  • Refinement results: Relative hallucination reductions reach 48.99% on QA and 26.86% on General in the best cases.Summarization changes range from 1.71%–6.78%, while Dialogue reductions range from 5.59%–21.33%.
  • Runtime cost: Refinement preserves a final continuation length of T tokens, with total decoded tokens T +(tcur−t0) and worst-case overhead bounded by 2×.Without cached restoration, replaying the prefix can add up to another t0 tokens of forward passes per intervention.

5. Discussion

The discussion frames refinement as a task-dependent control primitive whose effectiveness depends on base-model competence and the locality of trajectory deviations.

  • Scope and limitations: Separability is stronger when the base model handles a task reliably and hallucinations align with consistent depth-local regime changes.QA and General are the stronger settings in the reported experiments.
  • Scope and limitations: Dialogue and Summarization are harder settings where diffuse trajectories and ambiguous labels weaken alignment with internal signatures and degrade validator performance.This limits the consistency of detection across tasks.
  • Refinement scope: The single-block clamp is presented as a first refinement mechanism, with less headroom when deviations are distributed across tokens and depths.The discussion identifies multiple sites or staged corrections as directions for extending the intervention.

6. Conclusion

The paper presents internal monitoring as a practical basis for detecting hallucinations, localizing their emergence, and applying lightweight targeted correction without changing the base model.

  • Conclusion: A small validator predicts likely hallucinations from internal generation signals without modifying the base model and indicates where failures emerge.A single-point refinement then reduces hallucinations with minimal additional computation.
  • Conclusion: The overall framework combines detection and correction through actionable internal monitoring of answer formation.The conclusion motivates stronger and more general interventions.

Impact Statement

The paper frames internal flow signatures as a model-agnostic approach for auditing and improving reliability, while specifying detailed constructions for normalization, windows, token competitors, and degenerate subspaces.

  • Impact Statement: Internal flow signatures aim to improve reliability, auditing, and safety by enabling earlier detection and correction of unsupported generations.The paper also identifies risks involving deceptive-generation optimization, jailbreak tuning, and surveillance-like deployment.
  • Method: The construction uses deterministic depth-window assignment, including overlap handling and a forced final window aligned to the last block.The assignment selects the latest started window containing each block and supports overlapping windows when stride is smaller than window length.
  • Method: The monitored pipeline uses fixed-boundary states, bias-centered logits, moving depth windows, and readout-aligned competitor directions to characterize token-wise decision dynamics.Token masks exclude positions whose logits are not semantically meaningful or are dominated by formatting and control tokens.
  • Method: Window-specific subspaces are fitted from sampled competitor-difference directions, with deterministic fallback bases when masked direction matrices are empty or rank-deficient.This fallback preserves a defined basis even when competitor geometry supplies no usable direction signal.

A.8. Drift metric properties and simple bounds

The appendix establishes that projector drift is governed by principal angles and that anchor-coupled drift is bounded by geometry-only drift, while documenting localized refinement and robust aggregation choices.

  • Drift metric properties: Projector spectral drift equals sin(θmax), lies between 0 and 1, and is zero exactly when the compared subspaces coincide.The result follows from the principal-angle decomposition of orthogonal projectors.
  • Drift metric properties: Anchor-coupled drift measures subspace change along the visited anchor state and cannot exceed geometry-only drift.The bound is χt,j ≤ dG(Uj, Uj+1).
  • Refinement: Refinement localizes one culprit token-depth event, regenerates the suffix, and intervenes at one block while preserving the orthogonal component.Only the component inside the readout-aligned subspace is rewritten.
  • Aggregation: Per-step ratio features are reduced to token-level summaries with masked medians over valid depth steps, stabilizing aggregation under outliers and variable effective depths.A summary is set to 0 when a token has no valid depth step.
  • Centering: Coordinate-wise median centering removes token-shared shifts but is not rotation-equivariant under general within-window orthogonal basis changes.A Euclidean mean or geometric median is identified as a drop-in alternative when strict equivariance is required.

B.9. Validator architecture and masked pooling

The validator converts masked depth-token flow events into a sequence, processes them with normalization, an MLP, and a GRU, then pools valid-event logits for classification and refinement.

  • Architecture: The validator applies feature-wise LayerNorm, embeds events with a lightweight MLP, processes the sequence with a GRU, and aggregates logits using mask-aware pooling.Invalid padded events are ignored during the final aggregation.
  • Architecture: Max pooling or logsumexp pooling is applied after the GRU so event ordering can influence recurrent hidden dynamics before aggregation.Both pooling choices operate only over valid event positions.
  • Training: Validators are trained separately for each task and base-LLM pair using an 8:2 train-test split and binary labels retained from the evaluation data.The reported numbers use held-out test splits and AdamW optimization.
  • Refinement: Refinement selects the highest-scoring valid event and uses its token-depth coordinates to target one transformer block while keeping the prefix fixed.The suffix is regenerated after the localized intervention.
  • Refinement: A prefix-calibrated reference step scale triggers shrink-only updates when suffix transported steps exceed the upper band.The default calibration uses up to 64 prefix steps and an upper-band clamp ratio of 1.05.

C.1. QA Task

QA results show that validators often rely on depth-localized gates and coupled group mass, but missed hallucinations and false positives can share diffuse or gated patterns across model families.

  • Overall QA pattern: Across QA models, predicted positives frequently form a single depth gate with strong multi-group co-activation, while missed hallucinations remain late and diffuse.Depth gating is more consistent than module top1 dominance as a cue.
  • QWEN2.5: 15.5% hallucination prevalence, 23.4% predicted positive rate, 38.8% precision, and 58.4% recall characterize one reported QA operating point.The associated early gate places 100.0% of TP and 99.6% of FP at b = 1.
  • QA model pattern: 15.0% hallucination prevalence, 41.1% predicted positive rate, 22.2% precision, and 60.8% recall accompany an early b = 0 gate shared by TP and FP.The gate accounts for 67.9% of y = 1 and 44.0% of y = 0.
  • Phi 3: 13.0% hallucination prevalence, 63.2% predicted positive rate, 14.8% precision, and 71.9% recall accompany depth-and-token gating concentrated at early positions.Missed hallucinations retain a later-depth tail, while predicted positives cluster at tokens 0 and 1.
  • Late-gate model: 15.1% hallucination prevalence, 36.6% predicted positive rate, 24.2% precision, and 58.6% recall accompany a sharp b = 21 spike shared substantially by TP and FP.The spike accounts for 42.1% of y = 1 and 23.7% of y = 0.
  • Mistral: 14.1% hallucination prevalence, 51.6% predicted positive rate, 16.9% precision, and 61.9% recall accompany a late b = 20–22 spike, with missed hallucinations remaining diffuse and TN-like.False positives mirror the same late locations as true positives.

C.2. General Task

General hallucination-related predictions are associated with localized composite flow events, but models also exhibit depth-gate and diffuse regimes that limit reliable separation.

  • Overall pattern: General predicted positives commonly show a narrow-depth composite burst with jointly increasing transported motion, group co-activation, and drift concentration.The recurring event spans motion, attention, MLP, competitor, and drift signals rather than a single top1 group.
  • Depth gate: Some models use a fixed early depth gate, making hotspot location more predictive than module decomposition.For Gemma2, every predicted positive has its hotspot at b = 1, while top1 dominance remains motion-saturated.
  • Failure modes: Diffuse late or accumulation regimes weaken separability because transported steps, turning, and drift spread across deeper bands and overlap.Missed hallucinations often follow this late pattern, resembling non-hallucinated samples.
  • Operating points: 13.8% precision and 63.8% recall accompany a depth-shifted operating point in which predicted positives concentrate at depth 0 while missed cases skew late.Hallucination prevalence is 7.7%, and 62.8% of predicted positives occur at depth 0.
  • Model contrast: Mistral shows an early, drift-dominant, high-mass hallucination-like event versus late, motion-dominant, low-mass non-hallucination behavior.False positives inherit the hallucination-like signature, whereas false negatives inherit the non-hallucination-like signature.
  • Caveat: Occlusion deltas are all zero in the current run, so conclusions rely on top1 fractions, group mass, and hotspot depth.This limits interpretation of the occlusion-based signal.

C.3. Summarization Task

Summarization flow patterns divide into gate or collapse events and diffuse trajectories. Gate events drive predictions but frequently overlap between true and false positives, while diffuse hallucinations become missed cases.

  • Overall pattern: Summarization predictions are dominated by two geometric regimes: localized gate or collapse events and low-energy diffuse trajectories.The gate regime has locally larger, more curved transported steps and joint component rises; the diffuse regime spreads across depth.
  • Gate regime: Predicted positives enter a narrow-depth collapse with joint multi-group activation, making true and false positives difficult to distinguish.For Qwen2.5, the collapse concentrates near depth 14 with weight ≈0.94.
  • Diffuse regime: Diffuse trajectories remain low-energy and distributed across depth, causing missed hallucinations to resemble true negatives.For Qwen2.5, representative motion is ≈0.044 for FN and ≈0.028 for TN.
  • Failure mode: The no-gate or diffuse subset explains why missed hallucinations often remain close to non-hallucinated samples.This pattern is explicitly reported for Phi 3 and generalized across the task discussion.
  • Model patterns: Gemma2, Phi 3, LLaMA3, and Mistral each exhibit gate-versus-diffuse mixtures, with localized deformation driving predicted positives and diffuse cases increasing ambiguity.Across models, the gate reflects internal dynamics rather than correctness, producing TP-to-FP overlap.

C.4. Dialogue Task

Dialogue is best characterized by burst and diffuse regimes whose proportions and depth locations vary across models. Burst events drive predictions, while diffuse cases overlap across labels and sustain ambiguity.

  • Overall pattern: Dialogue contains a narrow-depth burst regime with larger transported steps, sharper bends, and joint component rises, alongside a low-mass diffuse regime.The diffuse regime spreads across later depths and overlaps strongly across labels.
  • Interpretation: Label differences generally reflect regime proportions rather than clean structural separation.Module dominance is often saturated and therefore unreliable for interpretation.
  • Qwen2.5: Qwen2.5 has a 12.9% early-burst hallucination subset and an 87.1% late-diffuse subset that closely matches true-negative motion patterns.The early burst concentrates at depths 0 to 6, with b = 4 alone accounting for 65.6%.
  • Gemma2: Gemma2 mixes a 46.8% high-mass, very-late explosion regime with a 53.2% low-mass diffuse regime, and both regimes appear in both labels.Top1 dominance is motion 100% across all splits, so dominance does not explain label differences.
  • Phi 3: Phi 3 separates primarily by depth band and magnitude regime, with both labels mixing an early-to-mid co-activated regime and a late low-mass regime.The early-to-mid hallucination regime accounts for 46.8%, while the late regime accounts for 53.2%.
  • LLaMA3: LLaMA3 is dominated by a shared depth-0 burst regime, so label differences are primarily proportional rather than structural.For hallucinations, the burst regime accounts for 87.0% and fixes hotspots at depth 0.
  • Detection outcomes: The detector exhibits task-dependent class asymmetries, including missed hallucinations in Dialogue when high normal accuracy accompanies low hallucination accuracy.Training curves show rapid early gains followed by saturation for General, while Dialogue errors reflect ambiguity and history conditioning.

D.2. Refinement Comparison

Table 4 compares regeneration, random-depth clamping, and flow-guided single-block refinement across tasks and model families. Flow guidance achieves the lowest hallucination ratios, with depth targeting contributing beyond regeneration or generic intervention.

  • Flow Guided applies the same single-block clamping intervention at the culprit depth localized by the flow-signature validator.
  • Across all tasks and model families, flow guidance yields the lowest hallucination ratio.
  • QA: 15.25→10.95 for Qwen 2.5 and 15.10→7.70 for LLaMA3 illustrate QA gains over the compared settings.
  • General: General-task reductions are smaller but consistent, including Qwen 2.5 at 10.12→8.75, while flow guidance remains ahead of random depth.
  • Summarization and Dialogue: Summarization and Dialogue show modest absolute reductions, but flow-guided refinement remains best, with Dialogue examples of 43.60→34.30 and 39.95→34.95.
  • The comparison indicates that regeneration alone does not explain the gains and that selecting the flow-localized depth is essential for effective refinement.
Loading 2602.01897v1…