Understanding Causality with Large Language Models: Feasibility and Opportunities

1 Introduction

• Causal questions matter for AGI and require symbolic reasoning, high-precision prediction, human interaction to verify assumptions, knowledge discovery, and safety-aware real-world interaction.The paper presents these requirements as reasons to examine LLM causal reasoning and future opportunities.Source: §1 · p. 1
• The paper examines LLMs’ ability to answer causal questions, using causal reasoning as an example that may generalize to other reasoning skills.It focuses initially on autoregressive, next-token-generation models and notes that human-feedback training is still in its infancy.Source: §1 · p. 1
• Current LLMs can answer causal questions rooted in common knowledge or known tools, but remain insufficient for deep, high-precision reasoning and expanding existing knowledge.The paper identifies optimizing future actions and discovering beyond current knowledge as examples of the harder tasks.Source: §1 · p. 2
• The authors see opportunities to enable LLMs to answer deeper causal questions through new machine-learning paradigms, with the potential to augment human skills and broaden real-world AI use.They connect this direction to the models’ limitations in advanced mathematical reasoning.Source: §1 · p. 2

2.1 Causal questions

• Type 1 questions identify causal relationships using domain knowledge; Type 2 discover new knowledge from data; and Type 3 quantitatively estimate the consequences of actions.Examples of Type 3 questions include choosing 100 of 1,000 dealers to maximize revenue growth and deciding a dose after two prior 12mn betamethasone injections.Source: §2.1 · p. 2
• Current LLMs can answer Type 1 questions well because of their large knowledge stores, though the authors say answers are not yet fully reliable.They believe these questions will eventually be solved by LLMs.Source: §2.1 · p. 3
• Current token-generation approaches limit Type 2 and Type 3 answers; prompting can produce algorithms, but models often choose unsuitable methods and available causal methods remain limited.These questions require discovering causal mechanisms or recommending interventions with quantitative effects.Source: §2.1 · p. 3

2.2 Challenges

• LLMs are limited in advanced symbolic and mathematical reasoning and can misread assumptions needed to select or develop causal methods.They also make intermediate-step errors or skip steps in advanced mathematical reasoning.Source: §2.2 · p. 3
• Explicit assumptions and step-by-step explanations can support trust, while opaque or flawed reasoning makes current answers difficult to trust in high-stakes decisions.The paper says a way to earn trust is needed for high-stakes causal questions.Source: §2.2 · p. 3
• Type 3 decisions may demand high-precision numerical optimization in highly contextual settings, including accurate estimates of individual treatment effects.The examples require choosing dealers for revenue growth or selecting an exact treatment and dose for a patient.Source: §2.2 · p. 3
• External APIs leave LLMs subject to existing causal-method limits, while complete-information requirements raise privacy and continual-learning concerns.The paper notes that identical symptoms may indicate different things for different patients.Source: §2.2 · p. 3

• Causal capabilities could enlarge LLMs’ impact on new domains, while causal consideration in RLHF may improve language question-answering, generalization, fairness, interpretability, computation efficiency, and sample efficiency.The paper frames these as two distinct potential impacts.Source: §3 · p. 4
• LLMs could help address causal methods’ difficulty communicating domain knowledge, assumptions, and results, and may also advance causal methods themselves.The paper notes that causal methods have yet to realize their full potential at scale and need advances in scalability, flexibility, and generality.Source: §3 · p. 4
• Two proposed directions are causal modules that interact with LLMs and new training paradigms that expand reasoning while keeping the model foundational.The paper says these directions are not limited to causality and aligns them with symbolic reasoning and other functional competences.Source: §3 · p. 4
• Causal modules could use existing causal APIs or have LLMs generate modules themselves, rather than training LLMs to understand causality intrinsically.Source: §3 · p. 4
• Using modules requires eliciting assumptions from users, choosing a suitable causal model, and summarizing intermediate and final outputs with step-wise assumptions to build trust.The paper describes this as non-trivial even if causal methods already have easy-to-interface APIs.Source: §3 · p. 4
• Causal modules remain limited to existing causal machine-learning methods, making efficiency, real-world scalability, and generality important constraints.The paper describes causal machine learning as still at an early stage.Source: §3 · p. 4
• A causal-aware training paradigm could let one foundational model reason about causality directly or generate novel methods, but RL alone may be insufficient when real-world interventions are costly or infeasible.The paper notes that naive RL thrives when interventions are cheap, such as in simulators.Source: §3 · p. 4
• The proposed opportunities include distilling causal models so LLMs can build methods internally and adding assumption understanding and model verification for trustworthy answers.Source: §3 · p. 5

• Scott and Shaw (2021) frame AI’s value as removing constraints and pairing AI’s strengths with human strengths to pursue solutions to important problems.Source: §4 · p. 5
• The paper envisions AI solving causal questions beyond current human ability and providing insights that augment human ability.Source: §4 · p. 5
• The envisioned applications include new discoveries and optimizing decision-making in the physical world.Source: §4 · p. 5

• The appendix demonstrates GPT-4 behavior using new Bing with the default balanced conversation setting.Source: A Appendix · p. 7
• The examples were generated in March 2023, and green captions mark correct answers while orange captions mark limitations.Source: A Appendix · p. 7
• Although outputs are nondeterministic, the authors report that answer quality across question types did not change over multiple trials and present one exemplar per question.Source: A Appendix · p. 7

A.1 Type 1 question behaviours

• LLMs answer questions using known causal knowledge quite well, from identifying relationships to reasoning about consequences of actions.The examples include medical causal relationships and the consequences of an intervention.Source: §A.1 · p. 7, Figure 1 · p. 7, Figure 3 · p. 8
• Performance remains quite stable when the same region is described with different wording.One tested variation replaced “shoulder area” with “muscles close to the upper front collarbone area.”Source: Figure 2 · p. 8

A.2 Type 2 question behaviours

• Current LLMs without API access cannot discover new knowledge from data, and they may recommend unsuitable causal-discovery methods.Examples include recommending nonlinear or time-series methods despite a stated linear assumption, and a method that is inefficient at the stated scale.Source: §A.2 · p. 9, Figure 5 · p. 10
• GPT-4 may explain supplied assumptions without answering the causal question.The example shows that adding assumptions did not yield an answer to the discovery question.Source: Figure 4 · p. 9
• GPT can provide generic explanations but not concrete recommendations requiring causal inference and more real-world evidence.For choosing between two programs, the answer would need to identify their effects on revenue and note that more information is required.Source: Figure 6 · p. 11

A.3 Type 3 question behaviours

• GPT currently fails at high-precision causal reasoning and may hallucinate with further prompting, but correct ITE context can support useful answers.The paper illustrates Type 3 examples involving decision-making and medical guidance.Source: §A.3 · p. 12
• Bing’s revenue-growth computation used revenue multiplied by discount, and the authors suggest it may not have understood the header.This is one example of a decision task requiring causal reasoning.Source: Figure 7 · p. 12
• A correct example selects the highest options, but performance on this task was not stable as of March 2023.A related example gives a confusing, confident answer about choosing two companies.Source: Figure 8 · p. 13, Figure 9 · p. 13
• In a medical example, GPT-4 gives a general dose-guidance answer and appropriate referrals for allowed categories, showing stronger safety guards than ChatGPT.The passage reports this even when ITE information is provided.Source: Figure 10 · p. 14