Source-linked AI summary
When Do Causal World Models Help Modular LLM Agents
Xinyuan Song, Zekun Cai
TL;DR
Modular agents need intervention-time predictions, but observational traces may not identify cross-module causal effects. FedCausalCompose uses local intervention-response evidence to model those interfaces; diagnostic results show causal information is most actionable in structured tools, while other settings may need attention anchoring.
Problem
Method
FedCausalCompose uses local actions and intervention-response summaries to identify and compose causal interfaces between functional modules.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
Causal world models help when cross-module interfaces are identifiable and presented in a form agents can use at decision time.
Source: §8 · p. 7Most LLM-agent cells are single-seed point estimates with some wide split-level intervals, so results are mechanistic evidence rather than powered benchmark claims.
Source: §7 · pp. 6–7
Abstract
from arXiv · showhide
LLM agents increasingly act through modular systems, such as order, payment, inventory, and shipment services, where actions in one module change which transitions are valid in another. Standard world models usually fit observational traces, but this is not the quantity needed for intervention-time planning: a trace may show that payment precedes shipment without identifying whether payment authorizes shipment, inventory mediates the effect, or a hidden trigger explains both. We study this gap through FedCausalCompose, a causal world-model framework for modular LLM agents in which local actions provide intervention-response evidence for cross-module interfaces. We first show that observational world models incur an irreducible interventional error under unblocked back-door paths, that interface recovery improves with intervention-response coverage, and that an oracle causal composition can beat the non-causal lower bound when coverage and local mechanism errors are controlled. We then test the resulting prediction in diagnostic agent settings. Causal interfaces help most in structured tool environments, where API signatures expose preconditions and downstream effects. In contrast, dialogue and narrative environments often ignore raw edge lists unless a short attention anchor makes the causal information decision-relevant. These results identify a concrete condition for causal world models in LLM agents: causal structure helps when cross-module interfaces are both statistically identifiable and presented in a form the agent can use at action time.
1 Introduction
Modular LLM agents operate across coupled functional components, where observational traces alone may not identify cross-module causal effects. FedCausalCompose uses local intervention-response evidence to recover and compose those interfaces.
1 Introduction
- The paper frames each client as a functional module within one coupled system, rather than an independent environment replica.The shared object is an interface graph connecting local mechanisms.Source: §1 · p. 1
- Observing that payment precedes shipment cannot establish whether payment authorizes shipment, inventory mediates the effect, or a latent trigger explains both.Intervention-time planning requires P(· | do(Ak = a)), not merely P(· | Ak = a).Source: §1 · p. 1
- FedCausalCompose identifies local variables, actions, and interfaces; matches events and interventions to downstream responses; validates edges; and composes local mechanisms.The experiments use a centralized prototype, while the proposed federated evidence consists of interface-level summaries rather than raw traces or parameters.Source: §1 · pp. 1–2
- The paper combines oracle-side claims about back-door error and interface coverage with diagnostic experiments on when agents use causal information.Its stated boundary case is that causal gains can shrink when observed action features already proxy the confounder.Source: §1 · p. 2
2 Related Work
Prior work studies federated learning, causal discovery, world models, and stateful agent benchmarks, but FedCausalCompose centers causal interfaces between functional modules within a coupled system.
Federated reinforcement learning and world models
- Federated RL and model-based RL commonly treat clients as environment instances, whereas FedCausalCompose treats each client as a functional component in one coupled system.Its shared object is an interface graph connecting local mechanisms, not a parameter vector or latent transition model.Source: §2 · p. 2
Causal discovery with interventions and environments
- Federated causal-discovery methods address privacy and decentralization, but this setting has disjoint module states and naturally occurring intervention evidence from agent actions.The recovered graph is evaluated through world-model prediction and downstream control, not only structural accuracy.Source: §2 · p. 2
Causal and language-agent world models
- Causal world-model and LLM-agent work motivates sequence and edge-prompt baselines, but does not separate observational correlations from interventional effects across private functional modules.Stateful benchmarks contain implicit modules such as accounts, orders, household containers, apps, and permissions; this work makes their interfaces explicit.Source: §2 · pp. 2–3
Prompt attention as a deployment bottleneck
- Causal edges can be correct yet unused when prompts make them weakly signaled or poorly positioned for the current goal.The paper relates its attention-anchor result to long-context and tool-use prompting.Source: §2 · p. 3
3 Why Non-Causal World Models Fail
Observational world models can learn the wrong interventional object under unblocked back-door paths, and errors can compound across modular rollouts. Causal composition can beat that lower bound when interfaces are sufficiently covered and local mechanism errors are controlled.
3 Why Non-Causal World Models Fail
- More observational data cannot remove a back-door gap when the predictor has converged to the wrong conditional mean.The relevant failure is intervention-time prediction, not weak prediction on the training distribution.Source: §3 · p. 3
- In modular systems, a wrong early interface changes the state passed to later modules, so rollout error compounds along the interface path.The stated proposition assumes the non-causal predictor has a per-hop correctness ceiling.Source: §3 · p. 3
- The back-door gap grows with confounding strength and compounds across interface hops.Synthetic diagnostics report that Δγ grows with confounder shift and Δh grows with rollout depth.Source: Figure 2 · p. 4, §3 · p. 4
- Local interventions provide evidence for P(· | do(Ak = a)), while the federated graph identifies which local mechanism should consume it.With sufficient true-interface coverage and controlled local mechanism errors, causal composition can fall below the non-causal lower bound.Source: §3 · pp. 3–4
4 FedCausalCompose
FedCausalCompose estimates a causal interface graph from interface-level intervention-response summaries rather than raw trajectories. Under the theorem’s assumptions, its oracle causal composition is strictly below the non-causal interventional lower bound.
4 FedCausalCompose
- FedCausalCompose estimates the causal interface graph from interface-level intervention-response summaries rather than raw trajectories.For each candidate edge, the method defines Bernoulli matches.Source: §4 · p. 4
- The stated assumptions require Bernoulli verification independence conditional on shared confounders, positive verification bounds, bounded errors, and L-Lipschitz interface maps.Source: §4 · p. 4
- Under the theorem’s assumptions, oracle FEDCAUSALCOMPOSE is strictly below the non-causal interventional lower bound.The predictor uses causal adjustment.Source: §4 · p. 4
5 Experiments
The diagnostic experiments find the clearest benefit of causal interfaces in structured tools, while narrative settings often require an attention anchor for agents to use edge information. Synthetic sweeps probe theoretical conditions, but the empirical cells are intended as mechanistic diagnostics rather than powered benchmark claims.
Setup
- The experiments are diagnostic cells, not a large benchmark sweep, combining the seed-0 synthetic SCM study with targeted τ-bench retail, ALFWorld, ScienceWorld, APIBank Stateful, and stateful tool-use settings.Values are point estimates; several cells have wide Wilson intervals, and the goal is to identify where causal interface information is used, ignored, or harmful.Source: §5 · pp. 4–5
Narrative environments are the hardest setting
- Across ALFWorld, correct-edge fraction from 0 to 1 is nearly uncorrelated with task success, so raw edge correctness alone is not a reliable narrative-task control signal.The experiments also include ScienceWorld, where causal structure is encoded in natural language.Source: §5 · p. 5
The useful case: structured tools expose causal order
- APIBank Stateful success improves monotonically from sequence to discovered-edge to oracle-edge prompting, the strongest structured-tool signal across 582 evaluations.The table reports task success and ROUGE-L for APIBank Stateful.Source: §5 · p. 5, Table 1 · p. 6
The failure mode: raw edges are not automatically attended
- In τ-bench retail, causal-edge prompts underperform the sequence baseline without an attention anchor; a goal-focus instruction reverses the pattern for edge variants.The non-causal baseline does not benefit from the anchor.Source: §5 · p. 5
Theory-facing checks
- Synthetic sweeps show confounding and horizon gaps increasing and edge recovery saturating once pint ≥0.25; c denotes action-confounder proxy strength.As action carries more information about the latent confounder, the observational predictor partially adjusts through action.Source: §5 · pp. 5–6
Efficiency
- Causal prompting is not free: lower cost per successful run depends on added structure improving decisions enough to offset longer prompts or extra calls.Some cells reduce or preserve call counts without automatically improving cost per win.Source: §5 · p. 5, Table 2 · p. 6
6 When Does a Causal World Model Help?
Causal interfaces help when an environment makes their dependencies usable in action selection: structured tools expose typed arguments, preconditions, and verification steps. Narrative settings are more fragile because edge lists compete with other context; attention anchoring and the degree of confounding shape whether causal information helps.
Structured tools
- Structured tools are the clearest useful case because typed arguments, visible preconditions, and verification steps align causal edges with action choices.In this regime, the framework indicates which upstream state must be satisfied before a downstream operation.Source: §6 · p. 6
Narrative and dialogue
- Narrative and dialogue agents may ignore raw edge lists because they compete with goals, history, and tool text; a short metaprompt can make edges salient to the decision.Identifiability is necessary but not sufficient for deployment.Source: §6 · p. 6
Confounding boundary
- As action carries more confounder signal, the SEQ-WM–CAUSAL-ORACLE gap decreases monotonically because action partially adjusts for the latent confounder.Causal gains are largest when confounding is real, relevant, and not already encoded in observed action features.Source: §6 · p. 6
7 Limitations
The paper limits its empirical claims to mechanistic evidence, noting both uncertain estimates and prototype constraints. Its current system is not a full privacy-preserving federated deployment, and its module boundaries and oracle edges are author-designed.
7 Limitations
- Most LLM-agent cells are single-seed point estimates, and several split-level intervals are wide, so results are treated as mechanistic rather than powered benchmark evidence.The authors caution against deploying raw causal edges without attention anchoring, structured-tool validation, or a policy that explicitly consumes the graph.Source: §7 · pp. 6–7
- Experiments use a centralized trace harness, while a deployed federated system would need privacy accounting, secure aggregation, communication constraints, and asynchronous client updates.AndroidWorld was deferred because the required Android emulator support was unavailable in the compute environment.Source: §7 · p. 7
- The prototype uses author-designed modularization and oracle edges, so broader autonomy claims require learning module boundaries, edge lags, and response matches with less supervision.The authors describe the current design as suitable for testing the causal object and its prompt usability.Source: §7 · p. 7
8 Conclusion
The conclusion argues that causal interfaces can improve modular-agent planning when they are identifiable and usable at decision time. Structured tools can use edges directly, while dialogue and narrative settings often need attention anchoring.
8 Conclusion
- Observational world models retain irreducible interventional error under unblocked back-door paths, whereas covered and correctly composed causal interfaces can beat that lower bound.The experiments show that this theoretical target alone is insufficient: causal information must also be usable by the agent.Source: §8 · p. 7
- Structured tool environments can use causal edges directly, while narrative and dialogue agents often need attention anchoring before the edges influence behavior.Source: §8 · p. 7
- The paper concludes that causal world models require both identifiable cross-module interfaces and mechanisms that make those interfaces attendable at decision time.Source: §8 · p. 7
A.1 Notation and Standing Assumptions
The notation treats modules as components of a coupled system, each with local state, actions, interfaces, mechanisms, and noise. The standing assumptions include a modular DAG and conditions on confounding and intervention matching.
A.1 Notation and Standing Assumptions
- Each module is specified by local state and action variables, incoming and outgoing interfaces, a local mechanism, and exogenous noise.Source: §A.1 · p. 9
- The modular system is assumed to form a directed acyclic graph, with local exogenous noises conditionally independent given shared cross-module confounders.Source: §A.1 · p. 9
- The back-door assumption requires an unblocked path for some module pair, with no observed admissible adjustment set blocking every such path.Source: §A.1 · p. 9
- The assumptions also specify non-degenerate linear-Gaussian confounding and Bernoulli intervention matching, with trials independent across rounds and pairs conditional on shared confounders.Source: §A.1 · p. 9
A.2 Back-Door Lower Bound
The appendix states a squared-loss observational prediction result and conditions for recovering the true interface graph. It also describes horizon and global-error analyses, and gives a condition under which oracle causal composition strictly beats the non-causal interventional lower bound.
A.2 Back-Door Lower Bound
- Under squared loss, the observational Bayes predictor over σ(Z) is the conditional mean μobs(z).Source: §A.2 · p. 10
A.3 Horizon Compounding
- The horizon analysis denotes correctness at hop ℓ by Sℓ and considers a binomial lower envelope.Source: §A.3 · pp. 10–11
A.4 Edge Verification and Coverage
- With complete detection, identifiability conditions, false-positive control, and lag disambiguation, the recovered interface graph equals the true graph.Source: §A.4 · p. 12
A.5 Global Error Decomposition
- The global error decomposition uses a squared triangle or Minkowski bound.Source: §A.5 · p. 13
A.6 Strict Oracle Dominance
- Oracle causal composition is strictly below the non-causal interventional lower bound when coverage and local mechanism errors are controlled; if γ = 0, no strict-dominance threshold exists.Source: §A.6 · pp. 14–15