Source-linked AI summary
Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents
Sourena Khanzadeh
TL;DR
LLM reasoning traces may be post-hoc explanations rather than drivers of decisions, raising a transparency concern for high-stakes use. Project Ariadne models reasoning with Structural Causal Models and audits causal faithfulness; its reported audits find high violation density and Causal Decoupling.
Problem
It remains unclear whether human-readable reasoning traces are faithful drivers of LLM agents’ conclusions or post-hoc justifications, despite the safety importance of transparent decisions.
Source: §1 · p. 1Method
Project Ariadne uses a Structural Causal Model of agent reasoning to audit the causal integrity of reasoning traces.
Source: §1 · p. 1, §4.1 · p. 3Results
Across thirty distinct audits, Violation Density was ρ = 0.767, with Causal Decoupling attributed to terminal decisions driven by parametric priors rather than intermediate logical steps.
Source: §5.5 · p. 5, §6 · p. 6Takeaways & Limitations
Ariadne provides diagnostic tools for distinguishing agents that derive solutions from those that provide post-hoc justifications.
Source: §6 · p. 6The framework treats the query and model parameters as exogenous variables and represents reasoning steps and the answer through structural functions.
Source: §4.1 · p. 3
Abstract
from arXiv · showhide
As Large Language Model (LLM) agents are increasingly tasked with high-stakes autonomous decision-making, the transparency of their reasoning processes has become a critical safety concern. While \textit{Chain-of-Thought} (CoT) prompting allows agents to generate human-readable reasoning traces, it remains unclear whether these traces are \textbf{faithful} generative drivers of the model's output or merely \textbf{post-hoc rationalizations}. We introduce \textbf{Project Ariadne}, a novel XAI framework that utilizes Structural Causal Models (SCMs) and counterfactual logic to audit the causal integrity of agentic reasoning. Unlike existing interpretability methods that rely on surface-level textual similarity, Project Ariadne performs \textbf{hard interventions} ($do$-calculus) on intermediate reasoning nodes -- systematically inverting logic, negating premises, and reversing factual claims -- to measure the \textbf{Causal Sensitivity} ($\phi$) of the terminal answer. Our empirical evaluation of state-of-the-art models reveals a persistent \textit{Faithfulness Gap}. We define and detect a widespread failure mode termed \textbf{Causal Decoupling}, where agents exhibit a violation density ($\rho$) of up to $0.77$ in factual and scientific domains. In these instances, agents arrive at identical conclusions despite contradictory internal logic, proving that their reasoning traces function as "Reasoning Theater" while decision-making is governed by latent parametric priors. Our findings suggest that current agentic architectures are inherently prone to unfaithful explanation, and we propose the Ariadne Score as a new benchmark for aligning stated logic with model action.
1 Introduction
Project Ariadne addresses the gap between human-readable reasoning traces and the actual drivers of agents’ conclusions. It audits whether interventions on reasoning steps affect the final answer.
1 Introduction
- Project Ariadne models agent reasoning with Structural Causal Models to audit whether reasoning traces causally shape terminal conclusions.The framework addresses traces that may function as post-hoc justifications rather than generative drivers.Source: §1 · p. 1
- Hard interventions flip logical operators, negate factual premises, or invert causal directions, then measure changes in the counterfactual answer distribution.The approach treats the reasoning trace as discrete causal nodes rather than relying on surface-level textual similarity.Source: §1 · pp. 1–2
- Causal Sensitivity quantifies how strongly the output responds to perturbations, supporting audits that distinguish faithful reasoning from “reasoning theater.”The framework evaluates unfaithful reasoning across state-of-the-art agentic architectures.Source: §1 · p. 2
2 Related Work
The related work frames faithfulness as distinct from plausibility and situates Ariadne among causal and counterfactual approaches to interpretability. It also presents faithfulness auditing as a diagnostic for agentic reasoning.
2 Related Work
- The broader field’s faithfulness problem concerns whether reasoning traces reflect actual decisions rather than merely plausible narratives.The discussion situates this question within the evaluation of LLM agents.Source: §2 · p. 2, §2.1 · p. 2
2.1 The Faithfulness-Plausibility Gap
- Faithfulness asks whether a reasoning trace reflects an agent’s actual decision process, unlike plausibility, which concerns a human-convincing narrative.Prior work describes traces as post-hoc justifications [3], while studies report conclusions based on biased heuristics despite logical-looking explanations.Source: §2.1 · p. 2
2.2 Causal Interpretability and SCMs
- Project Ariadne uses Structural Causal Models to move from correlational interpretability toward interventional proof, grounded in Pearl’s do-calculus.The account models reasoning steps as causal dependencies and frames faithfulness as consistency between changed reasoning and the final answer.Source: §2.2 · p. 2
2.3 Counterfactual Interventions in LLMs
- Ariadne extends causal tracing from model weights to semantic interventions on reasoning steps, formalizing faithfulness through answer similarity.Its Faithfulness Score is ϕ = 1 − S(a, aι).Source: §2.3 · p. 2
2.4 Benchmarking Agentic Reasoning
- Ariadne contributes a diagnostic for faithfulness violations when an agent’s answer remains unchanged despite contradictory reasoning.The framework supports batch auditing and aggregate statistics such as Violation Rate and Average Faithfulness across task domains.Source: §2.4 · p. 2
3 Ariadne Framework Overview
The framework models agent generation as a Structural Causal Model and tests causal dependence by intervening on a reasoning step. It compares original and counterfactual answers to derive a faithfulness score.
3 Ariadne Framework Overview
- Ariadne treats the agent’s generation process as a Structural Causal Model and uses controlled counterfactual interventions to audit reasoning-output dependence.The process is illustrated in Figure 1.Source: §3 · pp. 2–3
- An intervention on target step sk creates a counterfactual path and answer, whose semantic distance from the original answer determines Causal Faithfulness Score ϕ.High answer similarity and low ϕ indicate Causal Decoupling, where the intervention has negligible effect on the outcome.Source: §3 · p. 3
4 Mathematical Framework
The mathematical framework represents agent reasoning as a causal graph of structural equations. It specifies how reasoning steps depend on query and prior steps, and treats the final answer as the terminal node.
4 Mathematical Framework
- The framework represents agentic reasoning as a directed computational graph and quantifies faithfulness through controlled semantic interventions.It defines the agentic process as an SCM M = ⟨U, V, F⟩, with variables and structural equations.Source: §4 · p. 3, §4.1 · p. 3
4.1 The Structural Causal Model (SCM) of Reasoning
- The SCM separates exogenous query and model parameters from endogenous reasoning steps and answer, with each variable defined by a function of its causal parents.The structural equations are denoted by F.Source: §4.1 · p. 3
4.1.1 Stepwise Dependency
- Each reasoning step si is generated conditioned on the query and preceding reasoning history, with stochastic noise representing autoregressive sampling.The history is s<i = {s1, . . . , si−1}.Source: §4.1.1 · p. 3
4.1.2 The Answer Function
- The final answer a is the terminal causal-chain node, determined by the query and complete reasoning trace.Source: §4.1.2 · p. 3
4.2 Counterfactual Interventions
Ariadne audits causal faithfulness by intervening on a reasoning step and comparing the original and counterfactual answers.
4.2 Counterfactual Interventions
- An intervention replaces step k with a counterfactual thought generated to contradict the original reasoning.Source: §4.2 · p. 3
4.2.1 The Intervened Distribution
- After intervention, Ariadne re-executes the agent from that step to generate a counterfactual answer, resampling later steps that may deviate from the original trace.Source: §4.2.1 · p. 3
4.2.2 Intervention Modalities
- Intervention modalities map steps to contradictory counterparts by type, while answer divergence quantifies faithfulness; named modalities include premise negation and causal inversion.Source: §4.2.2 · pp. 3–4, Figure 1 · p. 4
4.3 Quantifying Faithfulness and Causal Decoupling
Ariadne measures how much the terminal answer depends on intermediate reasoning and identifies violations when answers remain invariant despite substantive contradictions.
4.3 Quantifying Faithfulness and Causal Decoupling
- The Causal Sensitivity Score ϕ measures the terminal answer’s functional dependence on intermediate reasoning steps.Source: §4.3 · p. 4
4.3.2 Violation Detection
- Causal Decoupling is a faithfulness violation in which the answer remains invariant despite a substantive contradiction in the reasoning chain.Source: §4.3.2 · p. 4
- A violation is indicated when answer similarity exceeds τsim and intervention strength exceeds λ, the minimum strength expected to change the answer.Source: §4.3.2 · p. 4
4.4 Aggregate Metrics
The section introduces expected faithfulness and violation density as aggregate metrics for a dataset of m queries.
4.4 Aggregate Metrics
- For a dataset D of m queries, the section defines Expected Faithfulness (EF) and Violation Density (ρ).Source: §4.4 · p. 4
5 Experiments and Results
Across the audits, answers often remained similar despite contradictory reasoning, while trace length did not reliably increase causal grounding. In a 30-trace audit, agents often detected counterfactual contradictions but reverted to parametric priors.
5.1 Experimental Setup
- The 500-query evaluation used GPT-4o-based agents, with logic flips at the initial reasoning step and Claude 3.7 Sonnet judging answer similarity.Queries covered General Knowledge, Scientific Reasoning, and Mathematical Logic.Source: §5.1 · p. 5
5.3 Case Study: Post-hoc Justification
- In a Global Warming audit, the answer remained functionally identical (S = 0.9698) after the agent was forced to accept a premise denying human-induced climate change.The analysis characterizes the trace as a post-hoc justification rather than a generative driver, with the model bypassing its internal logic.Source: §5.3 · p. 5
5.4 Intervention Sensitivity vs. Trace Length
- Longer traces did not necessarily improve causal grounding; for General Knowledge, trace length correlated positively with answer similarity, potentially reflecting correction toward parametric bias.Figure 2 shows faithfulness-score distributions across task domains.Source: §5.4 · p. 5, Figure 2 · p. 5
5.5 Discussion: The Robustness of Parametric Priors
- Across 30 audited traces, violation density was ρ = 0.767; agents often identified a counterfactual contradiction and then reverted to a high-probability parametric prior.The authors describe this behavior as beneficial for accuracy but harmful to faithfulness, and state that intermediate reasoning can be nonessential for factual retrieval decisions.Source: §5.5 · pp. 5–6
5.2 Quantitative Results: The Faithfulness Gap
- Most audited responses retained high semantic similarity despite contradictory reasoning; Scientific Reasoning had ρ = 0.96, while Mathematical Logic had ¯ϕ = 0.329.Table 1 summarizes causal audit results across task categories.Source: §5.2 · p. 5, Table 1 · p. 6
6 Conclusion
The conclusion presents Project Ariadne as a causal audit of agent reasoning and identifies Causal Decoupling in its empirical results. The authors characterize the traces as narrative veneers when parametric priors, rather than intermediate logic, drive decisions.
6 Conclusion
- Project Ariadne formalizes and evaluates agent reasoning’s causal integrity using Structural Causal Models and do-calculus rather than surface-level textual evaluation.Source: §6 · p. 6
- Across thirty distinct audits, the reported violation density was ρ = 0.767, highlighting Causal Decoupling in current autoregressive architectures.Source: §6 · p. 6
- The authors state that terminal decisions in these instances are driven by parametric priors rather than intermediate logical steps, making Ariadne a diagnostic for post-hoc justifications.Source: §6 · p. 6
7 Future Work
The future-work agenda proposes extending Ariadne’s interventions, using faithfulness in training, testing models with more thinking time, and automating intervention targeting.
7 Future Work
- Future versions could perturb multiple reasoning nodes to map when a model abandons parametric bias for contextual logic.Source: §7 · p. 6
- The authors propose using Faithfulness Score ϕ as an RLHF or DPO reward signal to penalize decoupled responses during fine-tuning.Source: §7 · p. 6
- Future research could test whether greater test-time thinking increases causal faithfulness or produces more elaborate post-hoc justifications.Source: §7 · p. 6
- Automated saliency detection using attention weights or gradients could identify load-bearing trace steps for targeted interventions.Source: §7 · p. 6