PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation

Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski

arXiv:2610.31639v1cs.LGcs.AI

TL;DR

Chain-of-thought rationales may be plausible without causally driving predictions, since dense attention allows reasoning to use the broader context. CausalCoT decomposes queries into a DAG and evaluates nodes using isolated, parent-routed contexts. Across the reported benchmarks, it maintains competitive accuracy while improving faithfulness and supporting interventions.

  • Problem

    CoT rationales are not guaranteed to causally drive final answers because dense attention can let models use context directly while producing post-hoc explanations.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    CausalCoT builds a DAG of sub-questions and evaluates each node in an isolated context containing only its direct parents’ answers or the input for roots.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    CausalCoT maintains competitive accuracy and improves faithfulness, with sensitivity increases of +4.81 to +21.19 p.p. against the best baselines on average.

    Source: §5.2 · p. 8, §7 · p. 10

  • Takeaways & Limitations

    The dependency graph supports test-time interventions, allowing local corrections to propagate through downstream reasoning.

    Source: §3.3 · p. 5, §5.3 · p. 8

    CausalCoT supports context-isolated controllability and intervention-based faithfulness, but does not guarantee mechanistic faithfulness of the model’s internal computation.

    Source: §7 · p. 10

Abstract

from arXiv · show

Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage. Across a diverse suite of reasoning benchmarks, CausalCoT preserves competitive predictive performance while improving explanation faithfulness, increasing causal sensitivity of intermediate reasoning steps by an average of 12 and up to 21 percentage points over the strongest baseline. The structured dependency graph additionally enables test-time interventions in which local corrections propagate predictably through downstream reasoning.

1 INTRODUCTION

Standard CoT can produce plausible rationales that do not causally drive predictions because attention permits information to bypass stated reasoning steps. CausalCoT instead routes reasoning through isolated, parent-dependent sub-questions and supports downstream interventions.

1 INTRODUCTION

  • Standard CoT’s dense attention can let models use context biases or hints while omitting them from plausible rationales, and interventions on early steps may have little effect.Prior work cited in the introduction reports these patterns; shared-context decomposition methods remain vulnerable to attention leakage.Source: §1 · pp. 1–2
  • CausalCoT decomposes a query into a DAG of interdependent sub-questions, evaluating each node using only its question and routed parent answers (or the input for roots).This procedural Markovian constraint is intended to prevent non-parent context from influencing downstream predictions.Source: §1 · p. 2
  • CausalCoT supports test-time interventions on reasoning nodes, with local corrections intended to propagate downstream through the dependency graph.The introduction describes this as counterfactual simulatability.Source: §1 · pp. 2–3
  • The paper uses “causal” for strict procedural dependence between reasoning steps, not formal statistical causal inference or structural causal models.This scope is stated in the paper’s footnote.Source: §1 · p. 2

2 BACKGROUND

The background distinguishes plausible, human-like rationales from explanations whose intermediate steps genuinely influence predictions. It frames post-hoc rationalization as the focus and describes perturbing reasoning steps as a way to assess causal reliance.

Chain-of-Thought and LLM Explanations

  • CoT and related prompting strategies elicit step-by-step rationales, but optimize coherent, persuasive text rather than strict dependencies between reasoning steps.The background also notes that some training paradigms bake step-by-step generation into the model policy.Source: §2 · p. 3

The Illusion of Faithfulness

  • Post-hoc rationalization occurs when intermediate steps act as superficial justifications without causal influence; perturbing or corrupting them tests whether predictions rely on those steps.The paper distinguishes this focus from counterfactual inconsistency, where behavior under context perturbations contradicts stated rules.Source: §2 · p. 3

3 METHOD: CAUSAL CHAIN-OF-THOUGHT (CAUSALCOT)

The method attributes unfaithful reasoning to attention leakage and reward-driven generation, then separates planning from execution in a DAG whose nodes use isolated contexts. Its exposed structure supports auditing and test-time interventions.

3.1 RATIONALE: THE ROOTS OF UNFAITHFUL REASONING

  • In autoregressive reasoning, attention can carry information from the full context into each step, letting predictions bypass the stated logical chain.Token-level optimization can also favor plausible, human-appealing text over faithful representation of internal processes.Source: §3.1 · pp. 3–4

3.2 THE CAUSALCOT FRAMEWORK

  • CausalCoT separates plan generation from execution: a planner builds a DAG of atomic sub-questions and their answer dependencies before the model evaluates them.This decoupling is intended to reduce entanglement between planning and execution.Source: §3.2 · p. 4
  • Nodes are evaluated topologically in isolated contexts: roots receive the input, while descendants receive only their question and direct parents’ answers.The resulting Markovian procedure removes direct paths to non-parent reasoning content.Source: §3.2 · p. 5

3.3 INFERENCE PROPERTIES AND HUMAN-IN-THE-LOOP INTERVENTIONS

  • Experts can inspect or revise the DAG and correct intermediate nodes; only downstream descendants need recomputation after a local edit.The framework also supports interventions on node values or input information while keeping the graph fixed.Source: §3.3 · p. 5
  • With the graph fixed, interventions can propagate changed node values downstream, and the final answer reflects those interventions under the same generated graph.The paper frames this as faithful counterfactual simulation under its forced Markovian procedure.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments compare CausalCoT with standard CoT and Factored Decomposition across accuracy, faithfulness, and intervenability. They use diverse benchmarks and assess both perturbations to intermediate reasoning and simulated expert corrections.

4 EXPERIMENTS

  • The experiments ask whether DAG constraints affect accuracy, whether intermediate steps causally influence predictions, and whether test-time corrections reliably improve reasoning.The evaluation is organized around performance, faithfulness, and intervenability.Source: §4 · p. 5

4.1 EXPERIMENTAL SETUP

  • Evaluation spans GPQA, MedQA, and the Counterfactual Simulatability Suite across health, business, and ethics, using Gemma 3, GPT-5.4, and Claude 4.5/4.6.Baselines are standard CoT and Factored Decomposition; the latter concatenates sub-answers at final prediction, allowing renewed attention leakage.Source: §4.1 · p. 6
  • Faithfulness testing injects errors into intermediate nodes and uses an LLM judge to assess whether corrupted conclusions match the logical downstream implication.Intervenability testing simulates expert corrections to planning or execution steps and measures predictive improvement across k ∈{1, 2, 3}.Source: §4.1 · p. 6

5 RESULTS

CausalCoT maintains competitive predictive accuracy while improving the causal faithfulness of intermediate reasoning and enabling local test-time corrections to improve downstream answers.

5.1 RQ1: PREDICTIVE ACCURACY

  • CausalCoT's average accuracy is 1.05 to 6.38 p.p. below the best-performing baselines across datasets, while remaining competitive overall.The authors attribute the decline to enforcing the DAG and removing dense attention pathways.Source: §5.1 · pp. 6–7, Table 1 · p. 7
  • Repeated structural attempts improve CausalCoT's Pass@3 by helping overcome initial graph formulation errors, although initial planning remains a bottleneck.Standard CoT gains less on several settings, consistent with generating more correlated variations.Source: §5.1 · p. 7

5.2 RQ2: EXPLANATION FAITHFULNESS

  • +4.81 to +21.19 p.p. in average sensitivity over the best baselines; CausalCoT has the highest average Answer Change on every model and highest Pred. Change on all but two.Pred. Change measures whether the altered prediction follows the logical implication of the injected error.Source: §5.2 · pp. 7–8, Table 2 · p. 8

5.3 RQ3: TEST-TIME INTERVENABILITY

  • At k = 3, GPT-5.4 Mini with CausalCoT achieves an approximately 7 p.p. advantage over CoT on GPQA.Across up to three edits, local corrections are routed to downstream descendants for recomputation; the figure caption describes accuracy on an intervention budget of up to three edits.Source: §5.3 · pp. 8–9, Figure 3 · p. 9

6 RELATED WORK

Related work addresses explanation unfaithfulness, decomposition, and solution-space search; CausalCoT distinguishes itself by isolating node contexts through DAG routing.

Decomposition and Advanced Exploration Prompting

  • Factored Decomposition reduces some reasoning risks but leaves downstream steps exposed to prior answers and restores global context at final prediction [Radhakrishnan et al., 2023].CausalCoT instead restricts each node to explicitly routed parents; Tree of Thoughts and Graph of Thoughts use topologies to search paths, making them complementary approaches [Pandey et al., 2025].Source: §6 · p. 9

Faithfulness in Reasoning

  • Faithfulness research documents cases where explanations do not causally track model behavior, including a reported increase in unfaithfulness on simpler tasks [Xiong et al., 2025; Lanham et al., 2023; Emmons et al., 2025].The cited mitigation approaches include contrastive learning and deterministic solvers, the latter limited to formalizable domains.Source: §6 · p. 9
  • Inference-time factored decomposition methods are described as lacking safeguards against attention leakage, while CausalCoT uses isolated contexts and causal routing [Sovrano and Bacchelli, 2026].The cited discussion frames open-ended exploration methods as potentially useful for refining CausalCoT's topology before execution [Pandey et al., 2025].Source: §6 · p. 9

7 CONCLUSION

The conclusion presents CausalCoT as a way to retain competitive accuracy while improving faithfulness and enabling interventions on reasoning graphs. The appendices describe its inference procedure and approaches to evaluating explanation faithfulness.

7 CONCLUSION

  • CausalCoT maintains competitive accuracy while achieving higher faithfulness across model families and tasks, and its graph supports expert interventions that can improve final outputs.The authors caution that malformed initial DAGs bottleneck downstream accuracy, prompt-based DAG generation lacks guarantees, and context isolation does not guarantee mechanistic faithfulness or counterfactual robustness.Source: §7 · pp. 9–10

A CAUSAL CHAIN OF THOUGHT (CAUSALCOT) ALGORITHM

  • The inference algorithm generates a DAG, evaluates nodes in topological order using the input for root nodes and direct-parent answers for descendants, then returns the terminal node's answer.Each node is evaluated in an isolated environment.Source: A CAUSAL CHAIN OF THOUGHT (CAUSALCOT)… · p. 15

B ALTERNATIVE APPROACHES TO EVALUATING FAITHFULNESS

  • Faithfulness evaluation includes input perturbations and intermediate-step perturbations, which test whether predictions change when prompt biases or reasoning steps are modified.The described tests treat omitted prompt biases or unchanged predictions after step modification as evidence that the rationale may be causally disconnected from the decision.Source: B ALTERNATIVE APPROACHES TO EVALUATING… · p. 15

C CAUSALCOT AND FACTORED DECOMPOSITION COMPARISON

CausalCoT replaces Factored Decomposition’s implicit sequential dependencies and leaky recomposition with explicit graph routing, while defining its causal guarantee as procedural rather than formal causal inference. The paper also states limits on what this structure establishes and what its benchmarks can measure.

C CAUSALCOT AND FACTORED DECOMPOSITION COMPARISON

  • Factored Decomposition leaves dependencies implicit, sometimes evaluating questions in a contextual vacuum or through ad hoc sequential updates.CausalCoT instead maps dependencies in a DAG before evaluation.Source: C CAUSALCOT AND FACTORED DECOMPOSITION… · pp. 15–16
  • CausalCoT routes each parent’s answer directly to designated child prompts, providing specified context without exposing nodes to the global history.This is the paper’s structural alternative to unstructured updates or isolated evaluation without dependencies.Source: C CAUSALCOT AND FACTORED DECOMPOSITION… · p. 16
  • Factored Decomposition’s recomposition step concatenates the prompt, questions, and answers, reintroducing attention leakage and possible shortcut use.CausalCoT’s Markovian constraint also applies to the final predictive node, which receives only its direct parents’ content.Source: C CAUSALCOT AND FACTORED DECOMPOSITION… · p. 16

D FORMAL CAUSALITY IN LLMS

  • CausalCoT’s “causal” guarantee is procedural dependence between reasoning steps, not mechanistic causality or formal causal inference.The framework restricts inference architecture rather than claiming uniqueness for generated dependent reasoning steps.Source: D FORMAL CAUSALITY IN LLMS · p. 16
  • The authors take no position on whether LLMs answer causal queries by reciting learned correlations; their guarantee concerns whether stated reasoning produced the answer.They argue that possible recitation or post-hoc rationalization increases the importance of aligning displayed and executed plans.Source: D FORMAL CAUSALITY IN LLMS · p. 16
  • The DAG represents evidential use rather than causal semantics, and context isolation does not prevent nodes from answering through memorized associations.The authors describe this as a trade against SCM-style formalization in favor of free-form natural-language applicability.Source: D FORMAL CAUSALITY IN LLMS · p. 16
  • The benchmarks lack graph annotations, so the study does not measure whether generated decompositions cover a task’s true causal structure.Establishing such coverage would require ground-truth reasoning-structure datasets, which the authors leave to future work.Source: D FORMAL CAUSALITY IN LLMS · pp. 16–17
  • Whether question decomposition can express causal dependence in language generally, beyond procedural dependence within one inference, remains open.Source: D FORMAL CAUSALITY IN LLMS · p. 17

E OPTIMIZING GRAPH GENERATION VIA REINFORCEMENT LEARNING

The paper identifies fine-tuning graph generation as a possible optimization, while leaving it for future work. Its current contribution remains a training-free, model-agnostic inference framework.

E OPTIMIZING GRAPH GENERATION VIA REINFORCEMENT LEARNING

  • Multi-sample Pass@3 experiments show that resampling often yields a graph that reaches the correct final prediction, providing a potential downstream-accuracy reward signal.The authors describe fine-tuning the graph planner with reinforcement learning as a future optimization, not an evaluated method.Source: E OPTIMIZING GRAPH GENERATION VIA… · p. 17
  • A proposed GRPO-like objective could reward correct final answers and add structural penalties for DAG violations, graph density, depth, or node count.The zero-shot models had not empirically violated the DAG constraint.Source: E OPTIMIZING GRAPH GENERATION VIA… · p. 17
  • Fine-tuned graph generation remains future work; the paper’s contribution is a training-free, model-agnostic framework compatible with open-weight models and commercial APIs.Source: E OPTIMIZING GRAPH GENERATION VIA… · p. 17

F EXPERIMENTAL DETAILS

The experiments use three reasoning benchmarks and a diverse set of proprietary and open-weight models. The setup also specifies resampling, model-specific sampling configurations, and reproducibility resources.

F EXPERIMENTAL DETAILS

  • The evaluation samples 200 instances each from GPQA, MedQA, and Counterfactuals, with three independent resamples per instance.Interventions use up to three structural interventions per generation; standard errors capture resampling variability except for Pass@3, which uses dataset variability.Source: F EXPERIMENTAL DETAILS · p. 17
  • The model suite spans GPT and Claude proprietary families and open-weight Gemma models, with multiple size variants subject to hardware and API-budget limits.Open-weight scale was bounded by 48 GB VRAM; proprietary models were sampled within API budget constraints.Source: F EXPERIMENTAL DETAILS · p. 17
  • API models use temperature 0.7, while locally hosted Gemma models use temperature 1.0, top_k 64, top_p 0.95, and min_p 0.0.Source: F EXPERIMENTAL DETAILS · pp. 17–18
  • The experiments use Python 3.12.3 and open-source libraries, with code and evaluation scripts slated for public release under an MIT license.Local runs used GPUs up to 48 GB VRAM and approximately five days of GPU compute.Source: F EXPERIMENTAL DETAILS · p. 18

G IMPLEMENTATION DETAILS

CausalCoT supports local interventions and uses concise finding summaries to propagate context through its graph. A fallback can relax dependencies when an internal node reports insufficient context.

G IMPLEMENTATION DETAILS

  • To limit context-window overhead, each node passes children its question plus a concise conclusion summary prefixed “FINDING.”Source: G IMPLEMENTATION DETAILS · p. 18
  • When an internal node signals insufficient context, the fallback dissolves its parent dependencies and makes it a root with the full initial question.The authors say this is particularly effective at mitigating cascading errors in smaller, less performant models.Source: G IMPLEMENTATION DETAILS · p. 18

H.1 TEST-TIME INTERVENABILITY

  • 20.84 accuracy gain after three interventions for GPT-5.4 Nano on MedQA, versus 13.16 for CoT and 17.5 for FactoredDecomp.On GPQA, CausalCoT eventually outperforms both baselines after refinement despite starting below standard CoT; Claude Sonnet 4.6 FactoredDecomp intervention results were omitted due to budget constraints.Source: §H.1 · p. 18

H.2 REASONING ANALYSIS

These analyses probe how graph structure, context isolation, model capability, and evaluation choices shape CausalCoT’s accuracy, faithfulness, and practical costs. They also examine interventions, failure patterns, and extension beyond multiple-choice question answering.

H.2 REASONING ANALYSIS

  • Larger reasoning graphs generally maintain performance and sometimes outperform smaller ones, while deeper graphs reduce accuracy on MedQA and GPQA but slightly improve it on Counterfactuals.The optimal node count varies by task; larger and deeper graphs have wider error intervals because they occur less often. GPQA has the highest malformed-graph failure rates, while most models remain below 1%.Source: §H.2 · pp. 18, 19, 21, Figure 4 · p. 20, Figure 5 · p. 20

H.3 FAITHFULNESS ANALYSIS

  • CausalCoT’s faithfulness and Predictable Change rise when interventions target nodes nearer the final answer, while standard CoT’s Predictable Change falls below 20% on MedQA late in the sequence.CausalCoT’s parallel unperturbed branches can suppress early errors; Factored Decomposition’s Predictable Change stays flat because sub-answers are concatenated only at recomposition.Source: §H.3 · pp. 21–22, Figure 7 · p. 22, Figure 8 · p. 22

H.4 UNDERSTANDING PERFORMANCE DEGRADATION IN SMALLER MODELS

  • Transferring GPT-5.4-generated graphs improves smaller models’ accuracy, sometimes above their standard CoT baselines, but their results still trail the larger model.Adding the full input to the final node partly recovers accuracy while reducing causal faithfulness; remaining errors also reflect incomplete intermediate contexts.Source: §H.4 · p. 22, Table 7 · p. 23

H.5 DIRECT VALIDATION OF CONTEXT LEAKAGE: THE COUNTERFACTUAL SWAP EXPERIMENT

  • In counterfactual swaps, input-follow rises with GPT capability to nearly half of conflict cases for the strongest model, while the weakest follows the frozen trace in every case.The behavioral test does not reveal internal mechanisms; CausalCoT’s final node never receives the original input, so the analogous bypass cannot be constructed.Source: §H.5 · p. 23, Table 8 · p. 23

H.6 ABLATING CONTEXT ISOLATION: THE CAUSALCOT-NOTRIM CONTROL

  • Removing context isolation sharply reduces faithfulness; for GPT-5.4 Mini, Answer Change falls from 58.72 to 37.07 (NoTrim), below standard CoT’s 40.95.Predictable Change also falls from 66.53 to 49.87 (NoTrim), below CoT’s 52.70; restoring context partly recovers weaker-model accuracy but reduces faithfulness.Source: §H.6 · p. 24, Table 9 · p. 24, Table 10 · p. 25

H.7 RUN-TO-RUN VARIABILITY: PLAN SAMPLING VERSUS EXECUTION

  • For GPT models, fixing the graph removes 60%–78% of CausalCoT’s excess run-to-run variability over CoT; on Gemma 4B MedQA, variability changes little, from 0.493 to 0.463.The authors attribute most GPT-family excess variability to plan sampling, while Gemma’s residual appears to arise during execution.Source: §H.7 · pp. 24–25, Table 11 · p. 25

H.9 A TAXONOMY OF CAUSALCOT FAILURE MODES

  • An incorrect intermediate answer appeared in 354 of 360 failed evaluations, making errors visible in the trace; missing knowledge and graph-construction problems were common contributing factors.Automatic evaluation audits found expected-label validity between 91.7% and 100% across raters, while invalid edits tended to deflate absolute Answer Change similarly across methods.Source: §H.9 · p. 26, §H.8 · p. 26, Table 12 · p. 26

H.10 REPRODUCTION OF THE WRONG-PREFIX ALGEBRA EXPERIMENT

  • In forced-error algebra tests, CausalCoT followed incorrect intermediate values in 100% of cases; on 50 MedQA samples it took 3.43 minutes and 237 calls, versus Factored Decomposition’s 3.77 minutes and 300 calls.CausalCoT averaged 466 tokens per call versus CoT’s 948; the paper also sketches context-isolated execution and descendant-only recomputation for open-ended clinical plans.Source: §H.10 · p. 27, Figure 9 · p. 27, §H.11 · p. 27

K BROADER SOCIETAL IMPACTS

CausalCoT is presented as a way to improve faithfulness and interpretability, with potential benefits for human-in-the-loop correction in high-stakes reasoning. The authors also note computational overhead and a risk that planning could be used to manipulate evaluation.

K BROADER SOCIETAL IMPACTS

  • CausalCoT aims to improve LLM faithfulness and interpretability while supporting reliable human-in-the-loop corrections in high-stakes reasoning.The paper names medical reasoning (MedQA) as an example domain evaluated.Source: K BROADER SOCIETAL IMPACTS · p. 28
  • Explicit context isolation introduces greater computational overhead than standard autoregressive generation, potentially adding to inference’s environmental footprint at scale.Source: K BROADER SOCIETAL IMPACTS · p. 28
  • A sufficiently capable model could exploit the planning phase to generate a graph intended to manipulate evaluation or fool the user rather than pursue a faithful solution.The authors frame this as a theoretical alignment risk.Source: K BROADER SOCIETAL IMPACTS · p. 29

Related papers

Loading 2610.31639v1…