PaperlayerView exact revision on arXiv

Source-linked AI summary

DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark

Jayanta Sadhu, Sayem Shahad, Kenneth Marino

arXiv:2608.30413v1cs.AI

TL;DR

Existing defeasible-reasoning evaluations rely on static datasets with limited structural coverage and limited insight into why models fail. DERELAB addresses this gap by generating formally verified, multi-turn belief-updating conversations across default and inheritance reasoning, and its experiments show widespread confirmation bias and belief-revision failures in LLMs.

  • Problem

    Existing defeasible-reasoning datasets are static, vulnerable to benchmark saturation, and limited in explaining whether model failures reflect reasoning deficits or surface pattern matching.

  • Method

    DERELAB generates parameterized multi-turn belief-updating conversations across default and inheritance reasoning with formally verified ground truth, enabling controlled diagnostic and cognitive-bias experiments.

  • Results

    Nine base models under ten inference configurations show no strong performance across all conditions, near-universal confirmation bias, and failures to revise conclusions despite recognizing weakening updates.

  • Takeaways & Limitations

    DERELAB provides a framework for scalable defeasible-reasoning evaluation and reveals that current LLM failures include belief revision, not only factual understanding.

  • Takeaways & Limitations

    The framework currently evaluates only default and inheritance topologies, and the evaluation includes only two explicitly reasoning-enabled configurations.

Abstract

from arXiv · show

Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of newer evidence. Although recent studies have examined language-model behaviors in defeasible reasoning, the datasets have been static and lack wide coverage of non-monotonic reasoning categories. We introduce DeReLab, a generative framework that produces multi-turn belief-updating conversations from parameterized graph structures across default and inheritance reasoning, with formally verified ground truth at every turn, enabling controlled measurement of how models respond to confirming and disconfirming evidence. This controlled generation process creates a testbed for experimental designs that isolate specific reasoning demands. Applying this capability to the study of confirmation bias, we evaluate nine open and proprietary large language models and find that nearly all exhibit a systematic tendency to accept congruent evidence while resisting incongruent updates, with several models correctly identifying a weakening update yet failing to revise their conclusion. We believe our work and findings will facilitate future research on evaluating language models in defeasible reasoning.

1 Introduction

DERELAB addresses limitations of static defeasible-reasoning benchmarks with a generative, formally verified framework for multi-turn belief updating. Its evaluation finds broad difficulty with defeasible reasoning and near-universal confirmation bias, including failures to revise conclusions after recognizing weakening evidence.

  • Static defeasible-reasoning datasets risk benchmark saturation and cannot reveal whether failures reflect reasoning deficits or surface-level pattern matching.
  • DERELAB generates multi-turn belief-updating conversations from parameterized graphs spanning default and inheritance reasoning, with formally verified ground truth at every turn.
  • Nine base models evaluated under ten inference configurations show no strong performance across all conditions, with accuracy degrading as reasoning complexity increases.
  • Confirmation bias is near-universal on default reasoning: models accept evidence confirming prior beliefs while resisting contradictory evidence.
  • Several models identify that an update weakens a hypothesis yet fail to revise their conclusion, implicating belief revision rather than factual understanding as the failure locus.
  • The framework supports scalable evaluation across difficulty tiers and cognitive-science-inspired probes of LLM defeasible reasoning.

2 Related Work

Prior work established formal and dataset-based approaches to non-monotonic and defeasible reasoning, while later efforts introduced incremental updates and methods for extending evaluations beyond static pattern matching. Related research also motivates studying cognitive biases in LLM judgments.

  • Early research studied defeasible and non-monotonic reasoning through formal logics, proofs, default-reasoning datasets, and natural-language inference.
  • Subsequent datasets introduced additional information that can alter inferences, making defeasibility a more nuanced target for evaluating non-monotonic reasoning.
  • Research on cognitive biases in LLMs has expanded beyond ethical and social concerns to biases and heuristics affecting the rationality of model judgments.
  • Sandbox-style evaluations moved beyond simple pattern matching through explicit rules, iterative proof generation, human annotation, and tunable dataset generation.

3 The DERELAB Framework

DERELAB is a configurable framework that generates incremental default- and inheritance-reasoning conversations with deterministic ground truth and controlled cognitive-bias experiments. Its structures, update types, and entity design support diagnosis of belief revision while limiting contamination from memorized world knowledge.

  • Framework design: DERELAB generates defeasible-reasoning problems on demand across varied structures and fine-grained parameters, avoiding static benchmark obsolescence and enabling controlled diagnostic ablations.
  • Conversation structure: Incremental conversations reveal facts one update at a time, allowing conclusions to be reinforced, defeated, reinstated, or left unchanged as premises evolve.
  • Default reasoning: Default reasoning uses parameterized chains with confirmations, defeaters, distractors, and source-priority conflicts; difficulty varies through chain length, objects, distractors, and defeat edges.
  • Inheritance reasoning: Inheritance reasoning propagates properties through linear or branching taxonomies, while node-specific blocks require models to track whether the subject’s own path is interrupted.
  • Default reasoning: Default-reasoning questions test direct chain manipulation, irrelevant information, and source priority, including cases where higher-priority rules bypass prior defeats.
  • Inheritance reasoning: Inheritance conversations use entailment answers of yes, no, or unknown, with transitions revealing whether updates weaken the hypothesis and testing scope across branches.
  • Ground truth: A path-based resolver deterministically labels each turn as ENTAILED, DEFEATED, or UNDETERMINED from the accumulated premise set.
  • Entity design: Pseudoword entities reduce contamination risk because they lack prior semantics, making correct responses dependent on premises supplied in context.

4 Experimental Setup

The canonical DERELAB evaluation set covers multiple reasoning topologies and difficulty levels, using turn-level accuracy under diverse model configurations. The experiments include closed and open-weight models, with Qwen3-32B evaluated in thinking and non-thinking modes.

  • Dataset: Figure 2 reports turn-level accuracy for each model under every topology–difficulty condition.An asterisk marks reasoning-enabled inference.
  • Dataset: The canonical evaluation set contains 900 conversations spanning three reasoning topologies and two difficulty levels.Each topology–difficulty combination contains 150 conversations.
  • Models: Qwen3-32B is evaluated in both thinking and non-thinking modes for direct comparison.The asterisk identifies the extended-thinking configuration.
  • Models: The evaluation compares closed and open-weight models across families, scales, architectures, and inference modes.The model set includes dense and mixture-of-experts architectures.

5 Results and Discussion

Model performance varies systematically with reasoning type, task difficulty, and inference configuration, while belief updating exposes widespread confirmation bias and dissociation between recognizing and applying weakening evidence. Results also show distance-related confidence changes and high reproducibility across seeds.

  • Overall Reasoning Accuracy: Reasoning-enabled configurations outperform standard instruction-tuned configurations, while default reasoning is harder than inheritance reasoning.The belief-update task is the most demanding condition.
  • Confirmation Bias in Belief Updating: Confirmation bias is statistically significant in 17 of 19 model–reasoning-type cells after Holm–Bonferroni correction.BiasGap ranges from 0.082 for Gemma-4-12B to 0.935 for GPT-5.1.
  • Confirmation Bias in Belief Updating: BiasGap is generally larger for default reasoning than inheritance reasoning, with especially large contrasts for GPT-5-mini and GPT-5.1.The paper attributes this pattern to default reasoning’s richer update vocabulary and exception structure.
  • Confirmation Bias in Belief Updating: Several models identify weakening updates correctly yet retain an affirmative entailment answer on the same turn.GPT-5-mini has C2-eff 95.4% versus C2 accuracy 20.3%, while Llama-3.1-70B has 85.1% versus 14.9%.
  • Confirmation Bias in Belief Updating: Reversing question order substantially reduces C2-effect accuracy across tested models, indicating dependence on the preceding entailment judgment.For example, Gemma-4-31B declines from 0.970 to 0.264 and Qwen3-32B from 0.803 to 0.414.
  • Confirmation Bias in Belief Updating: Qwen3-32B’s standard configuration shows a default BiasGap of 0.670, rising to 0.816 when question order is reversed, whereas its thinking configuration has near-zero default bias.The thinking configuration also has low anchoring rates on both topologies.
  • Confidence and Inference Path Length: On-path confidence declines with distance for most models, while GPT-5.1 shows accumulation and Llama-3.1-8B changes answers incorrectly on 36% of irrelevant turns.GPT-5.1 and Gemma-4-31B make such off-path changes on fewer than 1% of turns.
  • Robustness: Seeded evaluations are highly reproducible, with ICC 0.949 and no significant differences across 20 seeds for any model–condition pair.The coefficient of variation is below 4% for five of eight models on both default and linear-inheritance conditions.

6 Conclusion

DERELAB provides a formally verified, multi-turn structural framework for evaluating defeasible reasoning in language models. The experiments indicate brittle belief updating, including near-universal confirmation bias, distance-related confidence degradation, and unreliable tracking of structural relevance.

  • Conclusion: DERELAB evaluates defeasible reasoning through formally verified, multi-turn benchmarks generated from structural configurations.The framework supports analysis of belief updating and related cognitive behaviors.
  • Conclusion: The reported failures include near-universal confirmation bias, confidence degradation with inference-path length, and unreliable structural-relevance tracking.The authors propose extending DERELAB with additional reasoning structures and cognitive science-inspired probes.

Limitations

The paper’s limitations include restricted topology coverage and no comparison with human performance. The section also describes DERELAB’s graph structures, logical edges, and resolution process.

  • Limitations: The current framework evaluates default and inheritance topologies, leaving broader defeasible reasoning structures for future expansion.The authors also note that human-model consistency was not evaluated.
  • Limitations: DERELAB does not evaluate whether model reasoning is consistent with human performance.Whether human reasoning diverges from algorithmic conclusions remains open.
  • Limitations: The evaluation includes only two explicitly reasoning-enabled configurations: GPT-5-mini and Qwen3-32B in extended-thinking mode.The authors state that evaluating more such models could reveal additional patterns.
  • Framework Structure: DERELAB generates default and inheritance graphs with entity and property nodes connected by factual, inheritance, defeasible, hypothesis, and irrelevant-property edges.Difficulty is controlled through graph size, depth, branching, distractors, and defeat-edge injection.
  • Data Generation: Pseudoword entities and domain-aware attributes populate generated graphs while reducing reliance on memorized real-world facts.Attributes distant from the reasoning chain create irrelevant updates that should not affect entailment.
  • Ground-Truth Resolution: The resolver trims query-relevant paths, propagates states topologically, and resolves conflicts using direct preemption, specificity, or skeptical inference.Positive, defeating, and ignored links are treated differently during logical propagation.

D Data Creation Process

The data creation process combines parameterized reasoning graphs with nonce entities, sourced and augmented domain properties, and templated natural-language realizations. It supports default and multiple inheritance structures for controlled defeasible-reasoning conversations.

  • Entity Generation: Nonce entities prevent models from relying on memorized real-world associations, directly testing reasoning over conversation premises.The entities are generated as artificial words rather than meaningful real-world terms.
  • Entity Generation: Character-level Markov n-gram models generate nonce words from varied seed corpora, with positional transition buckets, smoothing, legality checks, and filters.Filtering removes real English words, repeated characters, and unpronounceable consonant strings.
  • Reasoning Structures: The framework includes default reasoning, linear inheritance, and tree inheritance graph examples for constructing the reasoning structures.These structures are illustrated in Figures 7 and 12–14.
  • Property Construction: Domain properties are collected from ConceptNet, Wikidata, and Datamuse, then categorized and expanded with a large language model.The process enriches both property groups and domain-attribute mappings.
  • Property Construction: Attributes group related values, which are inserted into sentence templates such as “[Subject] can fly”; Table 10 reports domain attribute counts.The predicate template stores the variable value slot while domains provide the corresponding capabilities or properties.

F Seeded Sample Variance: Statistical Details

The seeded-variance analysis evaluates reproducibility across models and configurations while defining confidence, path, and intrusion metrics for inheritance reasoning. Results show highly consistent accuracy across random seeds and formalize how confidence and errors are analyzed along reasoning paths.

  • Experimental Design: 20 independent seeds generate 30 conversations per seed across eight models and two easy-tier configurations: Default Reasoning and Linear Inheritance.Accuracy is evaluated at the turn level against formal ground truth.
  • Seed Reproducibility: The pooled ICC is 0.949 (95 % CI [0.9382, 0.9631]), indicating excellent consistency across seeds under all tested configurations.The confidence interval’s upper bound does not fall below 0.93, supporting robustness to bootstrap sampling variability.
  • Seed Reproducibility: All 16 model–configuration cells are non-significant under Kruskal–Wallis tests, providing no evidence of systematic seed-level accuracy differences.The tests assess whether the 20 per-seed accuracy scores within each cell come from the same distribution.
  • Seed Reproducibility: Together, the ICC and Kruskal–Wallis results indicate that reseeding produces statistically equivalent datasets without systematically advantaging or disadvantaging any seed.This conclusion concerns the tested model–configuration cells and evaluation setup.
  • Inference-Path Metrics: On-path updates affect the hypothesis, whereas off-path updates lie on sibling branches; intrusion rate measures wrong answer changes after causally irrelevant evidence.The analyses treat on-path and off-path turns as separate, nonoverlapping conditions.
  • Inference-Path Metrics: Path-length analysis combines Spearman ρ, mixed-effects per-hop coefficient β̂, and short-versus-long-chain ΔBrier to assess confidence and calibration changes.The regression controls for conversation-level baseline difficulty, while positive ΔBrier indicates worsening calibration at greater depth.

G.1 On-Path Results

Inheritance reasoning reveals separable weaknesses: some models lose calibrated confidence as relevant chains lengthen, while others incorrectly react to irrelevant off-path updates. These failure modes do not systematically co-occur, so improving one does not imply improving the other.

  • On-path results: Five of nine models show declining confidence and calibration as inheritance distance increases.The affected models are Llama-3.1-8B, Llama-3.1-70B, Gemma-4-12B, Gemma-4-31B, and Gemma-4-MoE.
  • On-path results: Each additional inferential hop costs affected models up to 6–7 percentage points of correct-label confidence.The strongest reported coefficients are β̂ = −0.061 for Gemma-4-31B and β̂ = −0.069 and −0.056 for the Llama models.
  • On-path results: GPT-5.1, Qwen3, and Gemma-4-E4B show no significant distance effect, while GPT-5.1 improves on longer hard chains with ΔBrier = −0.170.Gemma-4-E4B remains poorly calibrated in absolute terms despite its distance robustness.
  • On-path results: Hard chains amplify calibration degradation, with Gemma-4-31B reaching ΔBrier = +0.157 on long hard chains.Its long-chain hard Brier score rises to 0.201, despite near-zero Brier on easy chains.
  • Off-path results: Off-path intrusion separates models: Llama-3.1-8B changes answers incorrectly on 36.2% of easy irrelevant turns, whereas most others remain below 6%.Gemma-4-31B, Qwen3-32B*, and GPT-5.1 have the lowest reported intrusion rates.
  • Off-path results: Only GPT-5.1 and Qwen3-32B consistently weight relevant evidence more than irrelevant evidence, with signal ratios above 1× across conditions.Gemma-4-E4B instead often preserves wrong answers, showing low intrusion but only 31–45% off-path accuracy.
  • Discussion: The failure modes are empirically dissociable: Gemma-4-31B is distance-sensitive but structurally selective, while Llama-3.1-8B combines distance degradation with structural blindness.GPT-5.1 shows neither failure, whereas Gemma-4-E4B is distance-robust and intrusion-resistant but poorly calibrated and inaccurate off-path.

H Confirmation Bias Experiment

The experiment measures confirmation bias and related belief-updating failures across default and inheritance reasoning. Models often resist incongruent updates, with errors varying by topology and including failures to revise conclusions despite recognizing weakening.

  • Measurement: Confirmation bias is assessed by comparing responses to congruent and incongruent updates relative to the model’s own prior belief.The prior is the model’s preceding prediction, not the ground truth, and resets when the hypothesis subject changes.
  • Measurement: C3 and C3inh isolate semantic override: models may treat logically inert negative updates as weakening or sibling-branch negations as affecting the hypothesis.C3 applies to already-defeated default chains, whereas C3inh concerns irrelevant sibling branches in inheritance reasoning.
  • Findings: 17 of 19 valid model–reasoning-type tests show significant bias after Holm–Bonferroni correction.Gemma-4-MoE on default reasoning and Qwen3-32B* on inheritance reasoning are the exceptions.
  • Findings: GPT-5.1 has the largest default-reasoning bias, with Gap = 0.935 and OR = 891.The odds of correct updating on congruent turns are nearly 900 times those on incongruent turns.
  • Findings: Qwen3-32B* reduces default bias from Gap = 0.670 to 0.008, while its inheritance gap remains 0.065 and is non-significant.The extended-thinking variant nearly eliminates the default-topology effect but provides only partial inheritance improvement.
  • Findings: Qwen3-32B and GPT-5.1 have the highest default anchoring rates, whereas Qwen3-32B* and GPT-5-mini show near-zero anchoring.Anchoring measures preserving the initial prediction when the correct answer changes.
  • Findings: Llama-3.1-70B shows default C3 override of 0.839 but inheritance C3inh override of 0.062, making the error topology-specific.Qwen3-32B shows the same pattern, while Gemma-4-E4B has default C3 override of exactly 0.000 and high anchoring.
  • Findings: GPT-5-mini labels C2 updates as weakening with C2-eff = 0.954 but has only C2 acc = 0.203 on the same turns.Llama-3.1-70B and Qwen3-32B show the same know-but-don’t-output dissociation at different magnitudes.

I.1 Reversed Question Order: Addressing the Metacognitive Confound

The reversed-order experiment tests whether metacognitive accuracy and confirmation-bias results depend on asking entailment before effect-of-update. Bias remains robust, while some apparent metacognitive knowledge declines when question order changes.

  • Motivation: The standard format asks the entailment question before the belief-update question, potentially allowing post-hoc rationalization.Reversing the order tests whether effect-label accuracy reflects knowledge available before the entailment commitment.
  • Design: The reversed-order experiment evaluates six models on default reasoning across easy and hard conditions.The models span a range of original effect-label accuracies.
  • Results: BiasGap is unchanged or larger for all six models under reversed ordering.Gemma-4-MoE changes from Gap = −0.007 to Gap = 0.259, OR = 24.8, while Qwen3-32B changes from 0.670 to 0.816.
  • Results: C2-eff drops under reversed ordering for Gemma-4-31B, Gemma-4-12B, and Qwen3-32B.Their values change from 0.970 to 0.264, 0.887 to 0.209, and 0.803 to 0.414, respectively.
  • Results: Gemma-4-E4B’s C2-eff rises from near-zero to 0.349 when the question order is reversed.The reversed order removes interference from its strongly biased entailment responses.
  • Implications: The authors qualify the know-but-don’t-output interpretation: confirmation bias is robust, but some original C2-eff values overstated knowledge during entailment decisions.The qualification applies especially to Gemma-4-31B, Gemma-4-12B, and Qwen3-32B.

J Real-Entity Robustness Check

The robustness check compares pseudoword and real-entity names while holding the underlying property chains and graph structures constant. Naming effects are small and inconsistent for default and linear inheritance, but tree inheritance improves across all seven models without reliably changing overall conclusions.

  • Design: The comparison replaces pseudoword entities with real-world objects while keeping the same real-world property chains and graph structure.The test examines whether entity naming affects reasoning quality.
  • Results: Default and linear inheritance show small, inconsistent differences between pseudoword and real-entity conditions.Most default-reasoning changes are below 0.05, and no consistent naming advantage appears.
  • Results: Tree inheritance improves with real entities across all seven models.Gemma-4-12B improves by 0.24, while Qwen3-32B* improves by 0.02.
  • Conclusion: Overall, real-world names do not reliably improve or degrade reasoning accuracy when graph structure is held constant.The results support the validity of the pseudoword design as not being an artifact of unfamiliar naming.
Loading 2608.30413v1…