PaperlayerView exact revision on arXiv

Source-linked AI summary

Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?

Anthony GX-Chen, Dongyan Lin, Mandana Samiei, Doina Precup, Blake A. Richards, Rob Fergus, Kenneth Marino

arXiv:2505.09614v3cs.AIcs.CL

TL;DR

The paper examines whether language-model agents can explore environments and infer causal relationships without systematic bias. It evaluates them with the Blicket Test and proposes test-time hypothesis sampling; the authors report adult-like disjunctive bias and improved performance with the intervention.

  • Problem

    It remains unclear whether language-model agents can discover causal relationships and whether their reasoning inherits human-like biases from training shaped by human data.

    Source: §1 · pp. 1–2

  • Method

    The authors evaluate language-model agents with the Blicket Test and propose test-time sampling that constructs a flatter prior and prompts the model to eliminate causal hypotheses.

    Source: §1 · p. 2

  • Results

    Language-model agents show adult-like disjunctive bias and perform poorly at causal exploration and inference, while the proposed test-time procedure significantly improves performance.

    Source: §1 · p. 2, §3.1 · p. 4

  • Takeaways & Limitations

    The findings advance language models toward more scientifically rigorous causal reasoning, while generalization beyond Blicket tests remains a future question.

    Source: §6 · p. 9

    The text-based task approximates the Blicket Test, and similar behavior between language models and humans does not establish identical internal mechanisms.

    Source: §6 · p. 9

Abstract

from arXiv · show

Language model (LM) agents are increasingly used as autonomous decision-makers which need to actively gather information to guide their decisions. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world -- key to robust, scientifically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclusions. In this work, we examine LMs' ability to explore and infer causal relationships, using the well-established Blicket Test paradigm from developmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the unusual, yet equally (or sometimes even more) evidenced conjunctive ones. This "disjunctive bias" persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their training data. To this end, we quantify similarities between LMs and humans, finding that LMs exhibit adult-like inference profiles (but not child-like). Finally, we propose a test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning.

1 Introduction

The paper asks whether language-model agents can discover causal relationships and whether their reasoning reflects human-like biases. It adapts the Blicket Test to study their exploration and inference, compare their behavior with humans, and test a hypothesis-sampling intervention.

1 Introduction

  • The study asks whether language models trained to imitate human behavior inherit human-like biases and heuristics when reasoning about causal relationships.The authors frame causal discovery as a central ability for intelligent agents and note that human data shapes language models.Source: §1 · pp. 1–2
  • The authors use the Blicket Test, a developmental-psychology paradigm for studying how people infer causal relationships.Prior work described in the paper finds that adult causal inferences can deviate from rational norms, while infants and toddlers can reason with less bias.Source: §1 · p. 2
  • They adapt the test into a text-based sequential decision-making game in which agents actively explore before inferring causal relationships from accumulated experience.The experience history is stored and returned to the agent in prompts.Source: §1 · p. 2
  • The authors report that language-model agents explore poorly, can fail to infer causal relationships even with perfect exploration data, and show adult-like rather than child-like reasoning biases.They propose test-time hypothesis sampling to address the bias and report that it significantly improves performance.Source: §1 · p. 2

2 Experimental Set-Up

The experimental setup turns the Blicket Test into an interactive task for evaluating causal exploration and inference. It measures exploration by information gain and compares language-model agents with non-language-model baselines across task sizes.

2.1 The Blicket Test

  • Agents must collect informative observations during exploration to identify which objects are Blickets and whether the machine follows a conjunctive or disjunctive rule.The text-based task has an exploration phase, followed by questions about each object's identity based on the full observation history.Source: §2.1 · p. 3

2.2 Information Gain

  • Information gain measures how much an action's observed outcome reduces uncertainty over hypotheses about the machine's input-output rule.With a uniform prior over non-zero hypotheses, maximizing information gain corresponds to eliminating the greatest number of hypotheses; expected information gain handles outcomes not known in advance.Source: §2.2 · p. 3

2.3 Models and Baselines

  • The study evaluates language models in four-object and eight-object environments, each with two Blickets, and allows up to 32 steps per trial.The setup considers two rules: conjunctive and disjunctive.Source: §2.3 · pp. 3–4
  • The InfoGain Oracle selects the action with maximum expected information gain as an approximate upper-bound baseline, while a random agent provides a lower-bound baseline.The random agent takes random exploration actions and guesses randomly during the question-and-answer phase.Source: §2.3 · p. 4

3 LM Agents Exhibit Cognitive Biases

LM agents struggle to identify Blickets, especially as the task grows or uses conjunctive rules. Their exploration and inference show a disjunctive bias, while performance correlates with how effectively they reduce uncertainty.

3.1 LM agents all perform poorly in the Blicket Test

  • LMs perform worse with eight objects and conjunctive rules; the InfoGain Oracle solves the hypothesis space and reaches Q&A accuracy 1.The conjunctive disadvantage also appears across models and prompting techniques.Source: Figure 2 · p. 4, §3.1 · p. 4

3.2 Which factors contribute to success?

  • Information gain has the strongest correlation with performance; unique state visitation correlates more weakly, while more steps and longer responses correlate with lower performance.Step count proxies when an LM believes it has gathered enough information; response length proxies output tokens, including reasoning tokens for reasoning models.Source: §3.2 · p. 5

3.3 All LM agents do not explore efficiently

  • The InfoGain Oracle reduces hypotheses to one efficiently, while LMs perform worse, especially with more objects or conjunctive rules.GPT-4o explores best among the tested LMs, while Gemma3 and deepseek-reasoner sometimes do worse than random exploration.Source: §3.3 · pp. 5–6, Figure 4 · p. 5

3.4 Most LM agents show reasoning bias when inferring causal relationships

  • With the same exploration data in the eight-object setting, oracle data improves conjunctive Q&A, yet most models remain below near-perfect accuracy and disjunctive bias persists.The passage attributes a causal role to poor exploration in some weak Q&A performance, but reports residual inference difficulty even with oracle data.Source: §3.4 · p. 6, Figure 5 · p. 6

4 LM’s cognitive biases appear similar to human adults

The section compares LM agents’ causal inferences and exploration with human data, finding adult-like disjunctive bias and rule-dependent exploration unlike children’s.

4 LM’s cognitive biases appear similar to human adults

  • The human comparison tests inference under ambiguous evidence and exploration trajectories, using a three-object, two-Blicket setup for consistency with prior psychology studies.The inference task presents rule-hint exploration data before ambiguous data for new objects.Source: §4 · pp. 6–7, §4.1 · p. 7

4.1 Most LM agents prefer disjunctive answers given uncertain data (like adults)

  • LMs show an adult-like preference for disjunctive inferences under ambiguous evidence, while 4-year-olds infer the conjunctive rule when evidence supports it.The comparison uses the three-object, two-Blicket setting to match the psychology literature.Source: §4.1 · p. 7, §4 · p. 7

4.2 LM agents spend less time exploring disjunctive settings (unlike children)

  • Unlike children, LMs try fewer unique object combinations and explore for less time in disjunctive than other causal settings.Children’s exploration was not significantly affected by the underlying causal rule.Source: §4.2 · p. 7, Figure 7 · p. 8

5 Making LMs more scientific through inference time hypothesis sampling

The hypothesis-sampling agent constructs a flatter prior from unique LM hypotheses and selects actions to eliminate hypotheses. With more initial unique samples, agents explore better and show no conjunctive-versus-disjunctive performance decrease.

5 Making LMs more scientific through inference time hypothesis sampling

  • The method samples distinct hypotheses from the LM prior and assigns them uniform weight to construct a flatter prior q(F).Its entropy increases with each additional unique sample.Source: §5 · p. 8, Figure 8 · p. 8
  • Under q(F), the agent is prompted to choose actions that eliminate as many hypotheses as possible, updating the set after observations and repeating.The procedure maximizes expected information gain under the constructed prior.Source: §5 · p. 8, Figure 8 · p. 8
  • Figure 9 presents the hypothesis-sampling agent in the eight-object environment.The reported results associate more initial unique samples with better exploration and Q&A performance.Source: Figure 9 · p. 9, §5 · p. 9
  • With more initial unique samples, agents explore better and perform better in Q&A, with no performance decrease in conjunctive versus disjunctive settings.The hypothesis-sampling agent also no longer shows the initially observed disjunctively biased exploration pattern.Source: §5 · p. 9

6 Conclusion

The paper concludes that LM agents’ disjunctive causal-reasoning bias resembles adults’ more than children’s, and that test-time hypothesis sampling reduces this bias. It frames the findings within prior work on LM exploration and human causal-reasoning biases.

6 Conclusion

  • LM agents exhibit systematic causal-reasoning biases, especially a disjunctive bias resembling adults’ reasoning more than children’s.The conclusion reports that test-time hypothesis sampling significantly reduces this bias.Source: §6 · p. 9
  • The proposed link between human-generated training data and LM disjunctive bias remains speculative, and the authors call for research to test whether a causal link exists.The paper also identifies additional human exploration data as a way to clarify the priors adults and children bring to causal discovery.Source: §6 · p. 9
  • The paper treats its text-based Blicket evaluation as an approximation; similar LM and human behavior does not establish identical internal cognitive mechanisms.The authors note that stronger claims about mechanisms would require human-like sensory modalities, embodiment, and developmental learning trajectories.Source: §6 · p. 9

A Related Works

  • Related work covers LM exploration and reasoning, human disjunctive biases, and a Blicket-like hypothesis-sampling study that this work complements by examining more actions and models.The paper positions its study as addressing LM biases in causal inference using the Blicket paradigm.Source: A Related Works · pp. 14–15

B Experimental Details

The appendix details an oracle information-gain agent, the evaluated language-model configurations and prompts, and examples of agent behavior. It also cautions against treating intermediate tokens as reliable evidence of reasoning.

B.1 The Oracle InfoGain Agent

  • The oracle agent evaluates each one-action candidate by expected information gain and chooses the candidate with the highest value.Its hypothesis set covers possible Blicket identities and disjunctive or conjunctive rules; the prior is uniform over hypotheses consistent with observations.Source: §B.1 · p. 15

B.2 Language Model Access

  • The model selection spans reasoning capability, capacity, openness, and architecture, including GPT, DeepSeek, QwQ, and Gemma models.GPT-4o and GPT-4o-mini support a size comparison within a similar architecture; DeepSeek models offer reasoning and non-reasoning comparisons.Source: §B.2 · pp. 15–16

B.3 Statistical Validation

  • For Figure 2, each model, prompt variant, and system prompt combination used at least 16 independent trials.Figure 2a reports the best system message and prompt for each model, averaged over random seeds.Source: §B.3 · p. 16

B.4 System Message

  • Three system-message variants frame Blickets with common-sense language, “blicketness” semantics, or a mathematical definition.The math-definition variant defines a Blicket through non-independence between an object's state and the machine's state.Source: §B.4 · p. 16, §B.4.3 · p. 17

B.5 Prompting Techniques

  • Non-reasoning models were tested with ReAct, Reflexion, and Chain-of-Thought prompting, using zero-shot exploration and Q&A prompts.The prompting technique applies both when the agent acts and when it answers True/False questions.Source: §B.5 · pp. 17–18

B.6 Example Trajectory

  • The example trajectory shows a three-object conjunctive task in which objects 1 and 2 are Blickets and the machine activates when both are present.The prompt gives the agent its exploration history before asking whether object 2 is a Blicket.Source: §B.6 · pp. 18–19

B.7 Example intermediate tokens during Q&A

  • The intermediate-token examples are presented qualitatively, and the authors avoid treating them as definitive evidence of the model's reasoning.They note that intermediate tokens may not need to be correct for the final answer to be correct.Source: §B.7 · p. 19

B.7.1 GPT-4o, react prompting (correct answer)

  • The example marks false the conclusion that object 5 is not a blicket: object 5 alone leaves the light off, while object 5 with object 2 and object 2 alone turn it on.Source: §B.7.2 · p. 19

B.7.3 DeepSeek-R1 (correct answer)

In this DeepSeek-R1 example, the model correctly identifies object 2 as a Blicket in a conjunctive task.

B.7.3 DeepSeek-R1 (correct answer)

  • DeepSeek-R1 uses the observation that objects 2 and 3 activate the machine together, while object 3 alone does not, to answer that object 2 is a Blicket.The example's stated ground truth is that objects 2 and 3 are Blickets under a conjunctive rule.Source: §B.7.3 · p. 20

B.7.4 DeepSeek-R1 (incorrect answer)

In this DeepSeek-R1 example, the model incorrectly rejects object 5 as a Blicket in a conjunctive task.

B.7.4 DeepSeek-R1 (incorrect answer)

  • DeepSeek-R1 concludes object 5 is not a Blicket because object 7 appears necessary to activate the machine, although the stated ground truth includes both objects 5 and 7.The example describes the light turning on only after object 7 is added and turning off when it is removed.Source: §B.7.4 · p. 20

B.8 Human inference experiments

The human-inference experiment adapts a developmental-psychology task to text interaction, testing how prior causal-rule evidence shapes judgments about new objects.

B.8 Human inference experiments

  • The experiment presents training evidence for a disjunctive rule, a conjunctive rule, or an ambiguous baseline before testing inference on new objects.The test objects are new, but the machine retains the causal rule introduced in training.Source: §B.8 · pp. 20, 22
  • The test trial is designed so that object A is a Blicket under the conjunctive-rule possibility but not under the disjunctive-rule possibility.The two alternatives are both A and C as Blickets under a conjunctive rule, or only C under a disjunctive rule.Source: §B.8 · p. 23
  • Participants judge each new object's Blicket identity, and the experiment uses the propensity to classify A as a Blicket as a proxy for cognitive bias.The setup translates the psychological study into text-based interaction for language models.Source: §B.8 · p. 23

C Additional Results

Appendix analyses show that the exploration bias persists after normalization against random actions, while higher reasoning effort lags in conjunctive tasks. The appendix also reports trajectory lengths by exploration data source and rule type.

C.1 Causal Exploration Progress

  • Even after normalization against random actions, the strongest models retain a disjunctive bias in hypothesis-elimination efficiency.Figure 10 presents raw and normalized progress; the normalized measure compares model progress with progress from random actions.Source: §C.1 · p. 24, Figure 10 · p. 24

C.2 Effect of reasoning effort on exploration

  • In conjunctive tasks, higher-reasoning GPT-o-mini models consistently lag behind lower-reasoning variants and the infoGain oracle.The authors offer overfitting to verifiable tasks and amplified disjunctive priors as hypotheses, not established explanations.Source: §C.2 · p. 24, Figure 12 · p. 26

C.3 Inference on causal relationships

  • Table 2 reports trajectory lengths by exploration data source and rule type for Figure 5.The table reports mean trajectory lengths with standard deviations.Source: §C.3 · p. 25, Table 2 · p. 25

C.4 Human exploration characteristics

Children explored conjunctive and disjunctive machines similarly across the reported conditions and measures. The sampling procedure represents candidate causal rules as hypotheses, eliminates those contradicted by observations, and prompts actions to test the remaining hypotheses.

C.4 Human exploration characteristics

  • Across conditions and exploration measures, children explored conjunctive and disjunctive machines similarly.The measures include attempted object combinations, machine checks, and time spent exploring; Figures 13 and 14 display these statistics.Source: §C.4 · p. 26, Figure 13 · p. 26, Figure 14 · p. 27

C.5.1 Details of Procedure

  • The sampling agent approximates InfoGain by prompting actions to reduce entropy over the hypothesis distribution.Rather than evaluating candidate next observations to compute information gain, it prompts the LM to minimize H(F).Source: §C.5.1 · p. 27

1. Hypotheses sampling to form set of active hypotheses

  • When it needs hypotheses, the agent generates diverse Python-function rules over object states, restricted to Boolean operations and one or all Blickets being required.The instructions exclude sum operations and previously generated or eliminated hypotheses.Source: §1 · pp. 27–28

2. Eliminate hypotheses based on observation history

  • Each active hypothesis that fails to predict every past observation is moved to the eliminated list.Source: §2 · p. 28

3. Prompt agent to take actions that eliminate more hypotheses

  • Given observations and active hypotheses, the agent chooses an action intended to disprove a hypothesis, then repeats the exploration procedure.The procedure also prompts the agent to answer questions using the remaining hypotheses.Source: §3 · pp. 28–29

C.5.2 Proof of Remark

The appendix proves that the entropy of an empirical distribution formed from unique samples increases with each added sample. It also reports hypothesis-sampling results for both causal rules and a reduction in rule-dependent exploration differences.

C.5.2 Proof of Remark

  • The empirical distribution’s entropy monotonically increases as each additional unique hypothesis is sampled.The remark considers sampling without replacement from a discrete prior.Source: §C.5.2 · p. 29

C.5.3 Exploration for Both Rules

  • Figure 15 presents hypothesis-elimination performance for the sampling agent with eight objects under both OR and AND rules.Source: Figure 15 · p. 29

C.5.4 Hypothesis sampling agent resolves biased exploration patterns

  • Hypothesis sampling significantly reduced the exploration-pattern difference between conjunctive and disjunctive conditions.Figure 16 measures exploration by unique object combinations attempted and steps taken before exiting, before and after sampling.Source: §C.5.4 · p. 29, Figure 16 · p. 30

Related papers

Loading 2505.09614v3…