PaperlayerView exact revision on arXiv

Source-linked AI summary

The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining

Vy Nguyen, Ziqi Xu, Jeffrey Chan, Estrid He, Feng Xia, Renqiang Luo, Erik Cambria, Xiuzhen Zhang

arXiv:2609.32964v2cs.AI

TL;DR

Prior work mitigates hallucination but leaves open how models internally decide to commit despite signals of unanswerability. This paper uses causal gating to identify a sparse Commit-Abstain Circuit and finds that a CAC-informed policy improves abstention decisions. The circuit’s commitment signals accumulate earlier than later abstention corrections, which often fail to overturn them.

  • Problem

    Prior work detects or mitigates hallucination, but leaves unexplained how models internally arrive at commitment despite signals of unanswerability.

    Source: §1 · p. 2

  • Method

    The paper analyzes the pre-generation commit-abstain margin and applies causal gating to identify a sparse circuit of attention heads and MLP sublayers.

    Source: §1 · p. 2

  • Results

    12.2 points: the CAC-activation policy improves decision accuracy over the model’s intrinsic margin and reduces false abstentions by 2.5× across ten models and three benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Commitment-promoting components accumulate earlier, while later abstention-promoting components often fail to overturn that accumulated commitment.

    Source: §1 · p. 2, §6 · p. 10

    The analysis is limited to unsupported commitment on unanswerable inputs in instruction-tuned models, not incorrect answers to answerable questions.

    Source: §6 · p. 10

Abstract

from arXiv · show

Language models (LMs) often hallucinate by committing to confident answers rather than abstaining, even when they do not have enough information to answer reliably. A large body of existing work mitigates hallucination through detection or abstention mechanisms, but leaves open how models internally arrive at the decision to commit or abstain in the first place. We study this decision through mechanistic analysis, framing hallucination as unsupported commitment: the model commits despite exhibiting signals of unanswerability. Using causal gating, we identify a Commit-Abstain Circuit (CAC), a sparse, causally localised subset of attention heads and MLP sublayers underlying this decision. Across ten LMs (3B-14B) from five families and three benchmarks, the CAC exhibits a recurring accumulate-yet-undercorrect pattern: commitment-promoting components build up commitment in earlier layers, while abstention-promoting components act later as corrective signals that are often insufficient to overturn the accumulated commitment. Building on this finding, a lightweight policy trained on CAC activations improves decision accuracy by 12.2 points over the model's intrinsic commit-abstain margin, reduces false abstentions by 2.5 times, transfers to unseen benchmarks, and extends to larger models (27B-35B). The CAC is both diagnostic, clarifying how models overcommit, and practical, enabling improved abstention decisions.

1 Introduction

Prior work mitigates hallucination or detects unanswerability, but leaves unexplained how models decide to commit despite internal signals that they should abstain. The paper frames this as unsupported commitment and identifies a sparse circuit whose activations also support improved abstention decisions.

1 Introduction

  • Existing detection and abstention approaches mitigate hallucination behaviorally but do not explain how models internally arrive at hallucination.Prior work also finds unanswerability information in internal states without localizing the components underlying the commit-abstain decision.Source: §1 · pp. 1–2
  • The paper asks whether sparse model components causally contribute to commit-abstain decisions and whether understanding them can improve on default abstention behavior.Unsupported commitment is defined as failing to abstain despite signals of unanswerability.Source: §1 · p. 2
  • The paper analyzes pre-generation logits with a commit-abstain margin and causal gating to identify a sparse circuit across ten LMs and three benchmarks.The circuit shows early commitment-promoting components accumulating commitment while later abstention-promoting signals often fail to overturn it.Source: §1 · p. 2
  • 12.2 points: a lightweight policy trained on CAC activations improves decision accuracy and reduces false abstentions by 2.5× across ten models and three benchmarks.The reported gains transfer to unseen benchmarks and are reproduced on larger models.Source: §1 · p. 2

2 Related Work

Prior work detects or mitigates hallucination and identifies internal representations related to model behavior, but does not localize the components underlying commit-abstain decisions. This paper takes a mechanism-first approach to epistemic abstention.

Hallucination Detection and Abstention

  • Hallucination research includes output detection and abstention methods, but generally treats hallucination as a phenomenon to mitigate rather than explaining its internal emergence.Detection includes uncertainty estimates, evidence verification, and internal probing; abstention methods include training-free and training-based approaches.Source: §2 · p. 3

Mechanistic Interpretability

  • Mechanistic studies identify sparse circuits for other behaviors, while this work adapts causal gating to epistemic abstention rather than safety refusal.The method gates MLP sublayers as well as attention heads under a margin-based objective.Source: §2 · p. 3

The Representation-Behaviour Gap

  • Prior representation studies find internal signals relevant to answering, but do not characterize the components behind commit-abstain decisions.The paper identifies attention heads and MLP sublayers whose component-level contributions retain information partially lost in the aggregated margin.Source: §2 · p. 3

3 The Commit-Abstain Circuit

The paper measures commit-abstain preference before generation and decomposes the resulting margin into signed component contributions. It defines the CAC as the causally localized components mediating that preference.

3 The Commit-Abstain Circuit

  • The commit-abstain margin compares the strongest immediate commitment-token logit with the strongest abstention-token logit before generation.The vocabulary is partitioned into abstention tokens that initiate deferral and all remaining commitment tokens.Source: §3 · p. 3
  • A nonnegative margin indicates a preference to commit, while a negative margin indicates a preference to abstain.The margin is computed from the final input-position residual state and token logits.Source: §3 · p. 3
  • The margin decomposes into signed contributions from attention heads and MLP sublayers, with positive contributions pushing toward commitment and negative contributions toward abstention.The contribution projection uses the direction between the highest-logit commitment and abstention tokens; the final normalization is fixed to the ungated forward pass.Source: §3 · p. 4
  • The Commit-Abstain Circuit is the causally localized subset of attention heads and MLP sublayers that mediates the margin-based commit-abstain preference.The paper localizes this subset empirically and uses it to design an improved abstention policy.Source: §3 · p. 4

4 Localisation of the Commit-Abstain Circuit

Across ten instruction-tuned models and three benchmarks, the margin carries unanswerability information, and causal gating identifies a sparse circuit with distinct commitment- and abstention-promoting components. Functional tests show that commitment components sustain discrimination while abstention components act as corrections that are often insufficient to reverse commitment.

4.1 Setup for CAC Localisation

  • The localization study uses 1,000 instances each from KUQ, SQuAD 2.0, and MuSiQue, with supervised experiments split evenly into training and test sets.Ten instruction-tuned models span five families; Figure 2 reports margin AUROC across the 30 model-dataset configurations.Source: §4.1 · p. 4, Figure 2 · p. 5

4.2 Unanswerability Signals in the Commit-Abstain Margin

  • The margin’s sign agrees with full-generation commit-abstain behavior in 97.6% of instances on average, supporting the pre-generation readout used in the localization analysis.The range is 95.8%–98.7%.Source: §4.2 · p. 5
  • All 30 margin AUROCs exceed chance, averaging 0.811, yet 27% of unanswerable instances still have a nonnegative margin; Figure 3 shows these false-commitment regions.The range is 0.640 (Llama 3B on SQuAD 2.0) to 0.913 (Qwen 9B on MuSiQue).Source: §4.2 · p. 5, Figure 3 · p. 5

4.3 Localising the CAC via Causal Component Gating

  • Causal gating adapts joint component optimization to the margin by gating attention heads and MLP sublayers, classifying components by their responses to opposing retention and removal pressures.The task loss increases the margin for answerable inputs and decreases it for unanswerable inputs.Source: §4.3 · pp. 5–6

4.4 Identifying the CAC

  • CACs occur in every tested model and cover a median 5.2% of components, with MLP sublayers comprising 25% of CAC components on average; Figure 4 maps their locations.The identified components causally influence the margin under the three-phase gating protocol; Figure 5 illustrates gate-score separation for Llama 3B.Source: Figure 4 · p. 6, Figure 5 · p. 6, §4.4 · p. 7
  • Commitment generally accumulates earlier than abstention: both trajectories build positive contributions until 80% depth, then abstention reverses sharply; Figure 6 plots this pattern.C-components are more than twice as numerous per A-component at 60%–70% depth across models.Source: §4.4 · p. 7, Figure 6 · p. 7

4.5 Functional Analysis of the CAC

  • Amplification shifts decisions in opposite directions: C-components reduce false abstention while increasing false commitment, whereas A-components reduce false commitment while increasing false abstention at moderate scaling.Figure 8 shows C-component amplification driving false abstention near zero at α = 10, while A-component amplification leaves false commitment largely unchanged even at α = 1000.Source: Figure 8 · p. 7, §4.5 · p. 8
  • C-components sustain the margin’s discriminative structure, while A-components shift decisions toward abstention but cannot independently eliminate false commitment.Ablating C-components lowers AUROC across models, whereas A-component amplification cannot eliminate false commitment even under extreme scaling.Source: Figure 8 · p. 7, §4.5 · p. 8

5 Learning an Abstention Policy from the CAC

The section tests whether CAC activations can support an abstention policy beyond characterising commit-abstain decisions. The CAC-informed policy improves test performance over intrinsic and established baselines, including on unseen datasets and larger models.

5 Learning an Abstention Policy from the CAC

  • The section frames policy evaluation as a test of whether the CAC is actionable beyond characterising the commit-abstain decision.The policy is trained on CAC activations.Source: §5 · p. 8

5.1 CAC-Informed Abstention Policy

  • CAC-informed features combine each component’s contribution to the commit-abstain margin with aggregate statistics, and a lightweight MLP predicts answerability for threshold-based abstention.The features retain per-component contributions that the scalar margin collapses; the MLP’s gains suggest additional nonlinear interactions, while a logistic regressor still gains +10.6 accuracy points over Zero-Threshold.Source: §5.1 · p. 8, §5.2 · pp. 9–10

5.2 Experiments & Results

  • The evaluation compares the CAC-informed MLP with Zero-Threshold, a size-matched Non-CAC Random MLP, and six established abstention methods, using the same training split.HaMI uses full-dimensional activations and hallucination-indicative token selection, unlike the CAC policy’s 16–64 localized components.Source: §5.2 · p. 9
  • The policy achieves mean AUROC 0.864 and decision accuracy 0.814 across 30 configurations, with +8.2 AUROC points and +12.2 accuracy points over Zero-Threshold.Mean false abstention falls from 0.320 to 0.127 (2.5× lower); it also outperforms HaMI in AUROC on 23 of 30 configurations and accuracy on 20 of 30.Source: §5.2 · p. 9, Table 2 · p. 9
  • Across 24 OOD configurations, the policy improves accuracy over Zero-Threshold in every case, with mean gains of +6.9 points; gains average +6.3 points on two larger models.It outperforms HaMI in 19 of 24 configurations and transfers to unseen datasets without fine-tuning.Source: §5.2 · pp. 9–10, Table 3 · p. 10

6 Conclusion

The paper presents unsupported commitment as a gap between internal unanswerability signals and model behavior, explained by a recurring pattern of early commitment accumulation and later, often insufficient correction. Its scope is limited to unsupported commitment on unanswerable inputs in instruction-tuned models.

6 Conclusion

  • Commitment-promoting components act earlier and accumulate momentum, while later abstention-promoting components often fail to overturn it.The authors identify this recurring pattern in a sparse Commit-Abstain Circuit using causal gating.Source: §6 · p. 10
  • The CAC offers a mechanistic account of unsupported commitment and a foundation for targeted inference-time control over when models commit or defer.The authors contrast this direction with only detecting hallucinations after they emerge or training models to abstain more often.Source: §6 · p. 10
  • The findings cover unsupported commitment on unanswerable inputs in instruction-tuned models, not incorrect answers to otherwise answerable questions.The main analysis uses a pre-generation margin that agrees with full-generation behavior in 97.6% of cases, while generation-time dynamics remain only partially characterised.Source: §6 · p. 10
  • The abstention readout and CAC are partial operationalisations; the CAC is a group-level causal account, not a complete characterisation of every component or interaction.The abstention vocabulary may not cover all abstention behaviors.Source: §6 · p. 10

A Datasets

The five datasets were selected to stress-test abstention across knowledge source, reasoning depth, and unanswerability mechanism.

A Datasets

  • The five datasets vary along three orthogonal difficulty axes: knowledge source, reasoning depth, and unanswerability mechanism.Knowledge source ranges from parametric recall to in-context retrieval, and reasoning depth from single-hop to multihop.Source: A Datasets · p. 20, Table 4 · p. 20

B Construction and Validation of the Abstention Token Set A

The appendix defines a fixed abstention-token set from held-aside unanswerable instances and checks whether CAC findings remain stable under alternative constructions. The margin formulation tracks full-generation behavior with 97.6% mean agreement.

B Construction and Validation of the Abstention Token Set A

  • The 17-token abstention set A is built from a disjoint pool of 1,000 unanswerable instances, and the commitment set C contains all other tokens.Tokens are pooled across ten models, and A is restricted to tokens that unambiguously initiate epistemic deferral.Source: B Construction and Validation of the… · p. 20, Table 5 · p. 21
  • Two annotators classified candidate tokens with 10-token continuations as deferral or substantive answer, achieving 98.3% agreement (Cohen’s κ = 0.96).Disagreements were resolved by majority vote among three additional annotators; the interface is shown in Fig. 9.Source: B Construction and Validation of the… · p. 20, Figure 9 · p. 21
  • The zero-threshold decision from the commit-abstain margin agrees with full-generation labels in 97.6% of test configurations on average.Most disagreements involve pivot constructions where a substantive opening token is followed by a mid-generation reversal.Source: B Construction and Validation of the… · p. 20
  • CAC sparsity and ablation AUROC remain stable across three alternative abstention-set constructions, with the largest AUROC deviation at 0.008.The authors attribute stability to high-frequency tokens present in all four constructions.Source: B Construction and Validation of the… · p. 20

C Per-Configuration Margin Analysis

The section describes how gating uses components’ differential contributions to identify commitment- and abstention-promoting roles, then reports the localization protocol and analysis scope.

D.1 Gate Dynamics

  • Across 30 model-dataset configurations, first-token decisions agree with full-generation behavioral labels 97.6% of the time.Table 6 reports this agreement rate.Source: Table 6 · p. 22
  • The gradient-direction analysis does not yield a closed-form fixed point because optimization is non-convex, gradients vanish near saturation, and component contributions shift as other gates change.The final classification uses jointly optimized gate values to capture interactions.Source: §D.1 · p. 22

D.2 Hyperparameter Details

  • Gate optimization pre-screens components by training-set differential contribution, gates the top K per model, and leaves all other components ungated.The protocol uses Adam for 100 steps and averages results over 10 seeds.Source: §D.2 · pp. 22–23

D.3 CAC Localisation Results

  • The localization summary reports CAC sparsity, the MLP share, and commitment-to-abstention component ratios by normalized depth; a separate table summarizes sensitivity to alternative A constructions.The supplied passages describe these tables’ scope but do not provide their cell values.Source: Table 7 · p. 22, §D.3 · p. 23

D.4 CAC Validation and Sensitivity Analyses

The validation analyses test whether the localized circuit is reproducible, captures the commit-abstain margin, and remains stable under screening and regularization changes.

Cross-seed Stability

  • Across models and seeds, 80% of positive gates and 78% of negative gates are near-binary, and CAC membership requires consistent role assignment in at least 8 of 10 seeds.The passage reports that the resulting CAC topology is reproducible across reruns.Source: §D.4 · p. 23

CAC Validation

  • The CAC reproduces the margin’s sign in 85–92% of test cases (mean 88.4%), with mean agreement higher on unanswerable than answerable instances, 90% versus 86%.Table 10 reports sign agreement on 1,500 test instances per model.Source: §D.4 · p. 23, Table 10 · p. 24

Sign Agreement

  • Sign disagreements cluster near the decision boundary: mean absolute margin is 1.3 for disagreements versus 6.1 for agreements.The passage characterizes the CAC as reproducing decisions except in the most marginal cases.Source: §D.4 · p. 24

Screening Threshold Sensitivity

  • Doubling the screened candidates changes component counts by at most one, while the minimum reported Jaccard similarity across component types at 2K is 0.94.At 4K the minimum falls to 0.88; the passage says the base threshold recovers the core CAC.Source: §D.4 · p. 24

Regularisation Sensitivity

  • Across five tested regularization strengths, components entering or leaving the CAC have mean cross-seed identification rates no higher than 0.64, below the 0.80 membership threshold.The passage concludes that the high-confidence core is preserved across the tested values.Source: §D.4 · p. 24

D.5 Causal Intervention

The section describes interventions on CAC components and reports amplification outcomes, while outlining the ablation comparison and its baselines.

D.5 Causal Intervention

  • The ablation protocol progressively removes top-ranked CAC components and compares directional sign flips with 20 size-matched random non-CAC ablations.Table 13 reports full-ablation sign flips; Figure 7 contains progressive ablation curves against random baselines.Source: §D.5 · pp. 24–25, Table 13 · p. 25
  • Amplification scales one component’s output before residual addition, testing A-components on false commitments and C-components on false abstentions across a sweep of α values.The two sweeps mirror the component roles proposed by the CAC.Source: §D.5 · p. 25, Table 14 · p. 26, Table 15 · p. 26
  • At moderate amplification (α ≤ 3), Qwen 4B and Phi-4 mini reach negative mean margin by α = 3.0, while FC rates at α = 100 and 1000 are nearly identical across models.The mean absolute difference between those extreme settings is 0.004.Source: §D.5 · p. 25, Table 15 · p. 26

E.2 Main Results

The section compares CAC-feature classifiers with the zero-threshold margin and describes case studies where component patterns support abstention despite a committing margin.

E.3 Logistic Regression Variant

  • CAC logistic regression improves over Zero-Threshold in all 30 configurations by a mean 10.6 accuracy points, while the MLP exceeds logistic regression in 20 of 30 configurations by a mean 1.6 points.The MLP advantage is concentrated on KUQ; logistic regression and MLP are nearly tied on SQuAD 2.0 and MuSiQue.Source: §E.3 · p. 27, §E.2 · p. 27

F Case Studies

  • The case studies show instances where the raw margin favors commitment but component-level decomposition reveals a pattern the MLP uses to abstain correctly.The passage presents these as examples of the failure mode the CAC-informed policy is designed to correct.Source: F Case Studies · p. 27

F.1 Case 1: Unsolved Research Problem (KUQ, Llama 3B)

The section examines two unsupported-commitment cases—an unsolved research question and a false-premise question—in which positive surface margins conceal CAC abstention signals while a CAC-informed MLP abstains correctly. It also reports that the abstention set was human-validated, the annotation task did not require IRB review under institutional policy, and LLMs were objects of study rather than research tools.

F.1 Case 1: Unsolved Research Problem (KUQ, Llama 3B)

  • In Case 1, L19.MLP contributes +8.27, inflating the surface margin and making ∆CAC positive despite the remaining components leaning negative.The remaining contributions have P− = −8.09 and P+ = +12.62; the MLP’s zMLP is 0.089.Source: §F.1 · p. 28

F.2 Case 2: False Premise Question (SQuAD 2.0, Ministral 3B)

  • In Case 2, multiple negative contributions, especially L24.MLP (−2.44) and L22.H6 (−1.39), produce a strong CAC abstention signal despite Zero-Threshold’s positive surface margin.The MLP reads the CAC consensus and correctly abstains with zMLP = 0.167.Source: §F.2 · p. 29

F.1 Case 1: Unsolved Research Problem (KUQ, Llama 3B)

  • Case 1’s ∆CAC is +4.53 despite ∆(x) = +7.94, while Case 2’s ∆CAC is −5.22 despite ∆(x) = +1.37; the MLP abstains correctly in both.Zero-Threshold commits in both cases because the surface margin is positive.Source: §F.1 · p. 28, §F.2 · p. 29

G Computational Cost

  • The paper reports estimated wall-clock times for each pipeline stage across twelve models on a single NVIDIA H100 80GB.Table 21 covers 1,500 data instances and 10 seeds for CAC localisation.Source: G Computational Cost · p. 29, Table 21 · p. 31

4. Experimental result reproducibility

  • The authors state that the experimental results are reproducible, citing documented datasets, models, splits, decoding settings, hardware, MLP training details, baseline configurations, and supplementary code.The stated setup also includes full token lists and threshold-calibration procedures.Source: §4 · p. 32

7. Experiment statistical significance

  • The paper reports means, per-configuration breakdowns, ranges across 30 in-distribution and 24 out-of-distribution configurations, and sign-test p-values for headline claims.Source: §7 · p. 33

10. Broader impacts

  • The authors describe the work’s intended positive impact as improving LLM safety, reliability, and interpretability through abstention, and report no significant direct negative impacts foreseen.Source: §10 · p. 35

14. Crowdsourcing and research with human subjects

  • Two workers independently labeled each candidate token with its 10-token continuation to validate the abstention set, and were paid $20/hour.The stated rate was above the US federal minimum wage.Source: §14 · p. 36

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

  • The authors answered that IRB approval was not obtained, because the short-text annotation involved no personal data or sensitive content and institutional policy did not require review.Source: §15 · p. 37

16. Declaration of LLM usage

  • The authors answered N/A for LLM usage because LLMs were the objects of study, not components of the research methodology.Source: §16 · p. 37

Related papers

Loading 2609.32964v2…