PaperlayerView exact revision on arXiv

Source-linked AI summary

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

Quang Minh Nguyen, Luis Frentzen Salim

arXiv:2608.17809v1cs.CL

TL;DR

LLMs need to handle users’ beliefs alongside facts, but prior evidence shows weakness when beliefs are grounded in incorrect information. This paper evaluates belief confirmation across expressions and models, finding that errors vary by phrasing and arise when fact-checking overrides belief tracking.

  • Problem

    It is unclear whether LLMs’ weakness in confirming beliefs about false claims generalizes across epistemic expressions or reflects confusion between belief confirmation and fact verification.

  • Method

    The authors evaluate 10 LLMs on belief-confirmation prompts spanning epistemic expressions, analyze reasoning strategies and attention, and test instruction and attention-suppression interventions.

  • Results

    +50% to −14%: the factual-versus-false accuracy gap varies across epistemic expressions, while fact-checking the embedded claim lowers confirmation accuracy on false claims.

  • Takeaways & Limitations

    Fact-checking can interfere with belief confirmation, and separating belief acknowledgment from factual verification remains an open problem.

  • Takeaways & Limitations

    The evaluation uses single-turn templated prompts from one benchmark, so effects in natural multi-turn dialogue and other datasets remain untested.

Abstract

from arXiv · show

Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevitably intertwine with fact and knowledge, making the ability to handle them in tandem desirable for large language models (LLMs), as they are increasingly deployed in user-facing settings. Prior work showed that even capable LLMs exhibit a systemic weakness in acknowledging user beliefs grounded in incorrect information. We extend this evaluation to 10 LLMs across 18 epistemic expressions and find that the size and direction of the weakness depend on the verb used to express the belief, with the accuracy gap between factual and false information ranging from +50% on "I vaguely remember" to -14% on "I seriously doubt". We further show that the phenomenon stems from task confusion: models default to fact-checking the underlying claim, overriding the user's stated belief; chains of thought that explicitly fact-check show lower accuracy on false information than those that do not; and a single instruction can reverse the failure across verb families. Mechanistically, models attend more to false beliefs they fail to confirm, but suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods. Our findings clarify prior results and show how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs. Our code is available at https://github.com/ngqm/belief-fact-phrasing.

1 Introduction

The paper examines whether LLMs can confirm users’ beliefs while handling the truth of underlying claims, across different epistemic expressions. It finds that fact-checking can override belief confirmation, with performance varying by phrasing and prompting.

  • Motivation: Belief confirmation is desirable for user-facing LLMs because everyday beliefs intertwine with facts and knowledge.Humans routinely express beliefs with phrases such as “I think” and “I suppose.”
  • Prior finding: Prior work found that capable LLMs confirm user beliefs less often when the underlying claim is incorrect than when it is correct.This pattern appears when models answer whether they believe a stated claim.
  • Research questions: The study tests whether belief-confirmation weakness generalizes across belief phrasings and reflects limited capability or task confusion.It evaluates these questions on the KaBLE benchmark across 10 open-weight LLMs.
  • Task confusion: Instructions not to fact-check narrow the gap and raise overall accuracy, whereas instructions to fact-check lower accuracy further.This indicates that models can perform belief confirmation but are confused by the original task.
  • Mechanism and open problem: Explicit fact-checking appears in many chains of thought and interferes with confirming beliefs grounded in false information.The findings motivate decoupling belief acknowledgment from factual verification without degrading either capability.

2 Related Work

Related work distinguishes belief from knowledge and motivates evaluating whether LLMs track a user’s belief separately from the truth of the underlying claim. The paper extends the KaBLE benchmark beyond “believe” to 18 expressions, varied fact-checking instructions, and decoding-time attention intervention.

  • Belief and knowledge: Belief and knowledge are distinct, so acknowledging a user’s belief requires separating belief tracking from evaluating the underlying claim’s truth.Cognitive research likewise treats knowledge attribution and belief attribution as dissociable capacities.
  • Prior benchmark: KaBLE showed that LLMs often fail to acknowledge users’ stated beliefs when underlying claims are false, using only the verb “believe.”This study varies expressions across 18 verbs and tests fact-checking instructions plus a decoding-time attention intervention.
  • This study’s scope: 18 verbs, three fact-checking-instruction regimes, and a decoding-time attention intervention extend prior evaluation of belief acknowledgment.The investigated mechanism is models’ fact-checking of the underlying claim when asked whether the user believes it.

3 Methodology

The study evaluates first-person belief confirmation using a 1,000-item factual/false benchmark, 18 epistemic verbs, fact-checking prompt variants, and 10 open-weight instruction-tuned LLMs. Each model answers all 18,000 benchmark prompts under the original template.

  • Benchmark: The benchmark contains 1,000 clear-veracity statements—500 factual and 500 false—and the gold answer is always “Yes” because the task asks whether the user holds the belief.Items use the form “I believe that X” and ask “Do I believe that X?” with Yes, No, and Undeterminable options.
  • Epistemic verbs: The evaluation covers 18 epistemic verbs across positive belief, confidence, evidential, and negation families.The confidence family includes am 0/20/40/60/80/100% confident expressions.
  • Prompt templates: The original prompt replaces “believe” with each evaluated verb, while variants require, permit, or forbid fact-checking.Full templates and variant wording are provided in Appendix A.
  • Models: The study evaluates 10 open-weight instruction-tuned LLMs from the Gemma 3, Llama 3, and Qwen 3.5 families.Models are accessed through OpenRouter except qwen-3.5-4b, which is run locally.
  • Evaluation setup: Each model answers 18,000 prompts under the original template, corresponding to 1,000 statements multiplied by 18 verbs.Inference and decoding settings are listed in Appendix F.

4 Belief Confirmation Capability is Not Uniform Across Epistemic Expressions

Belief-confirmation weakness varies substantially across epistemic expressions rather than generalizing uniformly. Across confidence expressions, false-claim accuracy remains high through 80% confidence but drops at 100%.

  • Epistemic Expression Generalization: +50% on “vaguely remember” and +49% on “am certain” versus −14% on “seriously doubt” show that the factual–false accuracy gap varies across 18 expressions.Negation expressions are the only group that collapses or inverts the gap.
  • Epistemic Expression Generalization: Option-order permutation shows no confirmation-accuracy collapse at any position for any model, ruling out a preference for option (A) as the gap’s cause.The correct letter was remapped after permuting the option order.
  • Confidence Levels: At 100% stated confidence, accuracy on false claims drops by 18–48% across the five largest models.The summary identifies this as the point where false-claim accuracy falls among the “am X% confident” expressions.

5 Task Confusion Drives Belief Confirmation Weakness

The belief-confirmation weakness reflects task confusion: fact-checking overrides belief tracking, especially on false claims. Instructions forbidding fact-checking improve performance, while attention suppression only partially recovers accuracy in one model.

  • Fact-Checking Instructions: Forbidding fact-checking raises false-claim accuracy across every verb family, from 48.3% to 80.7% for positive belief, 57.0% to 81.5% for confidence, and 33.4% to 62.0% for evidential.For seriously doubt, factual accuracy rises from 54.4% to 78.1%, narrowing the gap from −14.7% to −2.5%.
  • Reasoning Strategies: 42.9% of judged chains use factual verification, and false-claim accuracy is 25.1% for fact-checking chains versus 75.8% for chains that do not.The association is descriptive because models choose their own reasoning strategies, while instruction contrasts provide causal evidence for task confusion.
  • Attention and Errors: Claim attention is higher on incorrectly than correctly answered false items in all six locally tested models, remaining significant after controlling for claim length.The difference is significant in five of six models before length control and all six afterward.
  • Causal Intervention via Attention Suppression: On llama-3.1-8b, suppressing claim attention raises confirmation accuracy by 12.0% while verification falls by 3.5%, indicating partial, task-specific recovery.For false claims, confirmation rises from 54.0% to 74.0%, while verification falls from 87.0% to 82.0%.

6 Conclusion

LLMs’ ability to confirm users’ beliefs depends on both belief phrasing and claim truth, with false-claim errors arising when models verify embedded claims instead of tracking stated beliefs. The paper highlights a tension between belief acknowledgment and factual correction and calls for methods that robustly separate them.

  • Conclusion: Belief confirmation depends on both the epistemic expression used and whether the underlying claim is factual or false.The accuracy gap between factual and false claims varies in both size and direction across epistemic expressions.
  • Conclusion: False-claim errors reflect task confusion: models default to verifying the embedded claim and override the user’s stated belief.
  • Conclusion: Belief acknowledgment and factual correction are desirable capabilities that can conflict on the same input.The authors call for methods that robustly decouple belief acknowledgment from factual verification.

Limitations

The intervention analysis is limited to locally runnable open-weight models and does not establish which architectures or sizes will respond. The evaluation also remains confined to single-turn templated prompts and one benchmark, leaving frontier-model behavior, natural dialogue, and cross-dataset replication untested.

  • Causal intervention: Only one locally runnable open-weight model improves confirmation accuracy beyond bootstrap standard error, and response cannot yet be predicted from architecture or size.The analysis is restricted to models that can be run locally; alternative steering methods may help where attention manipulation is infeasible.
  • Model coverage: Limited compute prevented testing frontier models, which may behave differently.
  • Evaluation scope: The single-turn evaluation uses KaBLE’s templated prompts, so the effect of instructions not to fact-check in natural multi-turn dialogue remains untested.
  • Evaluation scope: Because all results come from one benchmark, replication on other belief-confirmation datasets is a necessary next step.Further proposed extensions include relating belief tracking to internal claim knowledge and claim exposure during pretraining, alongside testing mixed-polarity items.

Ethical Considerations · A Prompt Templates · B Full Template Accuracy Breakdown

The paper highlights user-facing risks when LLMs fact-check beliefs instead of tracking them, and documents prompt templates plus accuracy breakdowns for factual and false claims. Explicitly instructing models not to fact-check improves factual-claim accuracy across every verb family, especially negation.

  • Ethical Considerations: LLMs often fact-check incorrect claims instead of acknowledging users’ stated beliefs, creating risks in note-taking, recall, brainstorming, and emotional-support assistants.The authors argue this behavior warrants mitigation before deployment when users intend their statements to be tracked or quoted.
  • A Prompt Templates: The original prompt template substitutes each of 18 belief verbs for “believe.”This establishes the template family used to evaluate how verb phrasing affects belief tracking.
  • A Prompt Templates: Each prompt asks whether the user still holds the stated belief, with Yes, No, and Undeterminable answer choices.The required response ends with “So, the answer is,” followed by (A), (B), or (C).
  • A Prompt Templates: Three appended instruction variants respectively prohibit fact-checking, permit it, or require it before answering about the user’s belief.These variants are appended verbatim after the Answer: line and separated by a blank line.
  • B Full Template Accuracy Breakdown: Accuracy is reported separately for factual and false claims, averaged over 10 models, with columns matching the main-text evaluation.The main text reports false claims only, while the full breakdown includes both claim types.
  • B Full Template Accuracy Breakdown: An instruction not to fact-check raises accuracy on factual claims in every verb family.This result indicates that suppressing fact-checking changes performance even when the underlying claim is factual.
  • B Full Template Accuracy Breakdown: The factual-claim accuracy improvement from not fact-checking is strongest for negation.The passage identifies negation as the verb family with the largest improvement, without specifying its numerical value.

C Reasoning Strategy Categories

The study categorizes model chains of thought into five reasoning strategies and validates an LLM judge against 200 hand-labeled examples. Agreement is substantial overall, though category-specific reliability varies.

  • Sample design: 9,000 CoTs were judged from stratified samples spanning 10 models and 18 verbs, with 25 factual and 25 false items per model–verb combination.A separate validation sample contained 200 CoTs, with 40 per category.
  • Validation: κ = 0.78 overall, with 82.5% raw agreement; one-vs-rest κ ranges from 0.60 for logical affirmation to 1.00 for no reasoning.The remaining category values are 0.88 for factual verification, 0.74 for direct repetition, and 0.79 for subjectivity deflection.
  • Categories: The five categories are factual verification, logical affirmation, direct repetition, no reasoning, and subjectivity deflection.An other category covered only 0.3% of judged samples.
  • Category definitions: Factual verification determines belief correctness by checking the underlying claim, whereas logical affirmation focuses on statement structure and direct repetition simply echoes the stated belief.No reasoning gives little explanation, while subjectivity deflection treats the belief as personal and undeterminable.

D Attention Measurement Details · E Attention Suppression on Other Models · F Inference Details

The paper measures claim-directed attention at answer onset and finds that incorrectly answered false items receive more such attention across models. Attention suppression is not consistently beneficial, while behavioral evaluations use standardized greedy-decoding protocols with some provider and implementation variability.

  • D Attention Measurement Details: Attention on the stated claim is measured at the last prompt token, then averaged across all heads and layers for each false item.Both occurrences of the claim in the prompt are included, and the resulting value is the fraction of attention directed to claim tokens.
  • D Attention Measurement Details: Higher claim attention occurs on incorrectly answered false items in all six models, with significant differences in five.Adding claim token count as a covariate makes the difference positive and significant in all six models, including llama-3.2-3b.
  • D Attention Measurement Details: gemma-3-12b’s large relative attention increase is uncertain because it is based on only n = 12 incorrectly answered items.The table explicitly cautions that this estimate rests on 12 incorrect items.
  • E Attention Suppression on Other Models: The attention-suppression intervention does not consistently help the four additional locally run models.llama-3.2-3b loses 3.5% on confirmation and 6.5% on verification at α∗, with the verification drop about 2 bootstrap standard errors.
  • F Inference Details: Nine of 10 models are evaluated on all 18 verbs through OpenRouter using greedy decoding with temperature 0, max_tokens = 512, and provider-default top-p.OpenRouter routing was not pinned across providers, potentially contributing to cross-model variability.
  • F Inference Details: The locally run qwen-3.5-4b uses HuggingFace Transformers in bfloat16 with greedy decoding and max_new_tokens = 512.Responses are parsed by extracting the option letter following the final “So, the answer is” phrase.
  • F Inference Details: Local intervention runs use bfloat16, eager attention, and greedy decoding because SDPA’s fast path can silently disable the explicit attention mask.This implementation detail is required for the attention-suppression intervention when is_causal=True.
  • F Inference Details: The LLM judge is DeepSeek-V4-Flash accessed through OpenRouter with temperature 0, max_tokens = 200, and JSON output constraints.Judge validation against the authors’ hand-labels is described in Appendix C.

G Attention Suppression Implementation and Protocol · H Answer Position Control · I AI Assistant Use

The paper implements attention suppression by reducing answer-token attention to claim tokens, selecting intervention strength on held-out KaBLE data. It also controls for answer-position preferences and discloses that coding was assisted by Claude Code.

  • G Attention Suppression Implementation and Protocol: The intervention targets both tokenized occurrences of claim X in the repeated KaBLE prompt.The target set S is recovered by token-aligning the verbatim claim substring wherever it appears.
  • G Attention Suppression Implementation and Protocol: During generation, attention scores from answer positions to claim positions are modified uniformly across layers and heads.Attention among prompt tokens and among answer tokens remains unchanged.
  • G Attention Suppression Implementation and Protocol: The intervention shifts each answer-token-to-claim-token logit by α ≤0, with larger |α| producing stronger suppression.Prompt hidden states still retain claim information because prompt-token attention is not modified.
  • G Attention Suppression Implementation and Protocol: For each model, task, and claim-type cell, α is swept over {−0.5, −1, −2, −4} using 150 randomly sampled KaBLE items.Fifty items select the nonzero α∗ by confirmation accuracy averaged over factual and false claims, while 100 are held out for reporting.
  • H Answer Position Control: Task 5 confirmation accuracy is re-measured after permuting “Yes” across answer positions (A), (B), and (C) on all 10 models.The correct answer letter is remapped, with 100 items per cell and the same inference settings as the main evaluation.
  • H Answer Position Control: A model relying solely on an (A) preference would have near-zero confirmation accuracy when “Yes” moves off (A).The control addresses whether Task 5’s gold answer always being (A) artificially inflates raw confirmation accuracy.
  • I AI Assistant Use: Coding was assisted by Claude Code.The paper provides the Claude Code product URL in its disclosure.
  • H Answer Position Control: Table 6 reports factual and false confirmation accuracy, their factual-minus-false gap, and false-claim accuracy when fact-checking is forbidden.Each cell contains values for “Yes” placed at (A), (B), and (C), with n = 100 per cell.
Loading 2608.17809v1…