Source-linked AI summary
Language models show human-like content effects on reasoning tasks
Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, Felix Hill
TL;DR
The paper asks whether language models mix semantic content into logical reasoning as humans do, a question relevant to characterizing both models and human cognition. It evaluates models and humans across three logical tasks using newly constructed, comparable stimuli. The models reproduce human content effects, and the authors argue that such effects can emerge from training a transformer on language produced by human culture.
Problem
The paper investigates whether language models’ prior knowledge causes content to interact with logic as it does in humans, informing debates about abstract and grounded intelligence.
Method
The authors evaluate large language models and humans on natural language inference, syllogism validity, and Wason selection using new datasets and directly comparable human stimuli.
Results
Language models reproduce human content effects across three logical reasoning tasks and mirror humans’ biased and consistent reasoning depending on content.
Takeaways & Limitations
Content effects can emerge from simply training a large transformer to imitate language produced by human culture, without explicitly incorporating human-specific mechanisms.
Takeaways & Limitations
The findings do not precisely ascertain how content effects in reasoning emerge from predictive learning on naturalistic data.
Abstract
from arXiv · showhide
Reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks, but exhibit many imperfections. However, human abstract reasoning is also imperfect. For example, human reasoning is affected by our real-world knowledge and beliefs, and shows notable "content effects"; humans reason more reliably when the semantic content of a problem supports the correct logical inferences. These content-entangled reasoning patterns play a central role in debates about the fundamental nature of human intelligence. Here, we investigate whether language models $\unicode{x2014}$ whose prior expectations capture some aspects of human knowledge $\unicode{x2014}$ similarly mix content into their answers to logical problems. We explored this question across three logical reasoning tasks: natural language inference, judging the logical validity of syllogisms, and the Wason selection task. We evaluate state of the art large language models, as well as humans, and find that the language models reflect many of the same patterns observed in humans across these tasks $\unicode{x2014}$ like humans, models answer more accurately when the semantic content of a task supports the logical inferences. These parallels are reflected both in answer patterns, and in lower-level features like the relationship between model answer distributions and human response times. Our findings have implications for understanding both these cognitive effects in humans, and the factors that contribute to language model performance.
Introduction
The paper asks whether language models, like humans, blend semantic content with logical reasoning. Across three tasks, it tests whether content effects can arise from learned language-model knowledge rather than task-specific mechanisms.
- Human content effects make reasoning more accurate for familiar, believable, or grounded situations than for unfamiliar, unbelievable, or abstract ones.
- These effects are widespread across reasoning tasks, challenging the ideal of abstract reasoning as independent of content and informing debates about human intelligence.
- The study examines whether language models’ learned prior knowledge produces human-like interactions between semantic content and logical reasoning.
- The authors create new datasets modeled on prior cognitive studies and collect directly comparable human data across natural language inference, syllogism validity, and Wason selection tasks.
- Models reproduce human content effects across tasks: content effects are modest in natural language inference, conclusion believability biases syllogism judgments, and realistic framing improves Wason performance.
- The paper adds item-level analyses and examines how content relates to continuous measures of model and human responses.
Results
Across natural language inference, syllogism validity, and Wason selection, humans and language models showed similar content-sensitive reasoning patterns. Content effects were small for NLI, substantial for syllogisms, and substantial on the more difficult Wason task, where models generally outperformed humans.
- Results: Across all three tasks, humans and models showed similar levels of accuracy and similar content effects.Content effects were measured as the advantage when reasoning content supported the logical inference.
- Natural Language Inference: NLI produced high accuracy and relatively little content sensitivity in both humans and models.The study found no statistically significant content effect on accuracy in humans or any evaluated language model.
- Syllogisms: Syllogism judgments showed moderate accuracy and substantial content effects in humans and models.Both groups tended to judge belief-consistent conclusions as valid and belief-violating conclusions as invalid, often regardless of logical validity.
- Wason Selection Task: On the Wason task, humans performed poorly overall, while slower human participants exceeded chance mainly for realistic rules.Human accuracy was significantly associated with response time, and the slow group performed above chance in the Realistic condition but near chance elsewhere.
- Model Confidence: Model confidence tracked correctness, content variables, and human response times, with stronger relationships on NLI and syllogisms than on Wason.For NLI and syllogisms, models were more confident on correct and belief-consistent problems and on items answered more rapidly by humans.
Discussion
The discussion argues that language models reproduce many human-like content effects in reasoning, while also showing important divergences and unresolved questions about their origins.
- Discussion: Across tasks, models reproduce human-like response patterns, including relationships between model confidence and human response times.Model confidence tends to be higher for correct answers and when prior content expectations are consistent with the logical structure.
- Discussion: The core effects are generally robust across models with different training and tuning paradigms and across different prompts.This supports the possibility that content biases are a fairly general phenomenon of predictive models learning from human-generated data.
- Discussion: Language models mirror human reliance on semantic content, showing biased yet more consistent reasoning depending on context and task difficulty.The paper frames abstract reasoning as a graded, content-sensitive capacity rather than purely symbolic computation.
- Discussion: A single transformer can exhibit both biased and consistent reasoning, suggesting an internally learned, context-dependent mechanism for arbitrating between competing responses.The discussion proposes that explicit augmentation with a second system may not be necessary, although a more explicit control network remains a possible design direction.
- Discussion: The experiments show that content effects can emerge from training a large transformer on language produced by human culture, but not which training-data features produce them.The authors identify causal manipulation of training objectives and experience as an important direction for future work.
- Discussion: Model behavior does not perfectly match human behavior: differences appear on Wason tasks, syllogism interactions, task instructions, and experiential grounding.The authors note that human–model comparisons are difficult to align perfectly and that prompting and framing may influence responses.
Materials and Methods
The paper builds matched reasoning datasets for three tasks, varying semantic content while preserving logical structure, then evaluates language models and human participants with comparable procedures. Analyses use likelihood-based model scoring and mixed-effects logistic regression.
- Dataset construction: The authors generate multiple stimulus versions for each task while keeping the logical structure fixed and manipulating the entities or semantic content.Conditions include realistic, arbitrary, violate, and nonsense formulations across the task designs.
- Dataset construction: Wason stimuli include realistic, arbitrary, nonsense, shuffled-realistic, and violate-realistic rules, each paired with instances matching or violating antecedents and consequents.Answer choices are presented in both possible orders to reduce ordering biases.
- Models and evaluation: Several language-model families are evaluated with task instructions and answer likelihoods, using Domain-Conditional PMI to compare contextualized answer probabilities.NLI uses maximum likelihood directly because prior correction would control for the bias under study.
- Statistical analyses: Mixed-effects logistic regression models predict responses from task conditions while controlling for random effects of items and, where applicable, models.Exact participant and item counts vary slightly because assignment was not precisely balanced and some data were lost or timed out.
A.1.1 NLI task generation
The NLI dataset is generated from model-produced comparisons and converted into matched logical problems whose conclusions either agree with, conflict with, or lack real-world semantic beliefs. Models select the answer with higher likelihood.
- Stimulus generation: The authors generate 100 real-world comparison sentences and manually remove false or subjective examples.Examples follow the form “[entity] is/are [comparison] than [other entity]”.
- Stimulus generation: The comparisons are converted into NLI prompts with premises and hypotheses that preserve the intended logical relationship.For example, “seas are bigger than puddles” supports “puddles are smaller than seas.”
- Evaluation: Models are evaluated by comparing the log probabilities assigned to the two answer options and selecting the higher-likelihood continuation.The task is presented as a forced choice between alternatives such as “smaller than” and “bigger than.”
- Stimulus generation: The violate condition makes the logically correct answer conflict with the model’s prior real-world belief.In the example, the correct answer is “bigger than,” despite the prior belief that puddles are smaller than seas.
- Stimulus generation: The nonsense condition replaces meaningful nouns with nonsense words while retaining the comparison structure and logical conclusion.This removes the intended real-world semantic associations.
A.1.2 Syllogisms data generation
The syllogism and Wason datasets use controlled manipulations of logical validity, semantic plausibility, and entity familiarity. Models answer forced-choice questions about validity or which cards to flip.
- Syllogisms data generation: The syllogism set includes realistic arguments based on plausible beliefs and nonsense arguments created by replacing realistic entities with nonsense words.Examples include arguments about librarians, guns, electronics, trees, flowers, and other entities.
- Syllogisms data generation: Syllogism stimuli are organized into twelve clusters, each containing four arguments crossing validity with belief consistency.The design uses the same three entities within a cluster and approximately balances low-level properties across conditions.
- Syllogisms evaluation: Models judge each syllogism through a forced choice between “The argument is valid” and “The argument is invalid.”Few-shot examples, when used, come from distinct clusters and are separated by blank lines.
- Wason data generation: Wason problems present a contextual rule, instances represented by cards or analogous objects, and a forced choice about which items must be flipped.Both answer-choice orders are tested to reduce ordering biases.
- Wason data generation: The Wason dataset contains realistic, arbitrary, and nonsense rules, with nonsense formulations designed to avoid negation while preserving comparable instance lengths.Nonsense rules use attributes such as “gluff” and “caft.”
- Wason data generation: Shuffled-realistic rules recombine antecedents and consequents without an obvious rationale, whereas violate-realistic rules reverse the consequent to make the rule implausible.The violate condition retains a card associated with the original consequent.
- Wason controls: Control analyses replace cards with sheets of paper or coins, and results are similar across these formats.This tests whether the observed effects are specific to cards.
- Stimulus validation: Participants found consistent and realistic stimuli more believable than stimuli from other conditions.This provides a validation of the intended semantic content of the datasets.
B.2 Robustnesss of the main language model results to raw-likelihood scoring and few-shot prompting
The appendix tests whether content effects depend on prompting, scoring, answer order, or basic proposition difficulty. Across these manipulations, content effects generally remain, though accuracy and error distributions can change.
- Robustness overview: Content effects remain robust across several manipulations of the evaluation context.The appendix examines instruction removal, scoring method, answer order, and few-shot prompting.
- B.2.1 Removing instruction prompts: Removing instruction prompts usually produces similar results, but Chinchilla shows stronger NLI content effects without instructions.A few models exhibit notable performance changes under this manipulation.
- B.2.2 Raw likelihoods versus DC-PMI: Raw-likelihood scoring creates substantial validity bias in syllogisms, with several models answering “valid” for every argument and performing much worse overall.For models retaining response variability, belief-consistent conclusions are still more likely to be judged valid than belief-violating conclusions.
- B.2.2 Raw likelihoods versus DC-PMI: Without prior correction, Wason accuracy changes in opposite directions across models, while above-chance models still perform better on Realistic than Arbitrary rules.PaLM-2 L exceeds 75% performance in every condition.
- B.2.3 Scoring and answer order: Scoring method substantially changes Wason error distributions even when accuracy is similar, whereas answer order has smaller effects.The comparison varies both DC-PMI versus raw likelihood scoring and antecedent-first versus consequent-first presentation.
- B.2.4 Few-shot prompting: Five-shot prompting leaves syllogism content effects slightly weaker but still robust for Chinchilla.The type of prompt examples does not strongly affect syllogism performance.
- B.2.4 Few-shot prompting: Five-shot prompting exaggerates Wason content effects because Chinchilla improves more on Realistic than Arbitrary rules.Realistic examples provide the greatest overall benefit and can enhance content effects.
- B.3 Proposition difficulty: Basic antecedent and consequent propositions show no substantial difficulty differences across Wason rule types.Arbitrary premises are numerically slightly easier, but the differences are not significant, so proposition verification is unlikely to explain the observed effects.
B.4 Additional recombined realistic conditions for the Wason tasks
The Wason tasks separate realistic rules, shuffled realistic rules, and rules that violate realistic expectations to test how content structure affects reasoning. Models perform above chance on shuffled rules, while performance declines toward chance when rules violate beliefs; humans generally perform poorly on both variants.
- The study contrasts realistic rules with shuffled realistic rules that combine realistic components nonsensically and violate realistic rules that contradict expected relationships.Examples include pairing overseas travel with an MD requirement, or replacing a passport requirement with a driver’s license requirement.
- Models perform well above chance on shuffled realistic rules, and PaLM 2 performs better on shuffled than realistic rules.
- When rules strongly violate beliefs, model performance is generally close to chance, whereas humans perform poorly on either rule variant.
- Models reason more accurately about realistic propositions, especially when relationships between propositions are also realistic.This advantage persists to some degree when realistic components are shuffled without directly violating expectations.
B.5 Human performance on the Wason tasks, in our original sample & replication
The replication tested whether a performance bonus improved human Wason-task accuracy. The bonus effect was marginal, but overall performance remained low and the content effect did not differ significantly between samples.
- An additional participant sample received a performance bonus intended to increase Wason-task performance.
- The bonus sample showed a marginally significant performance improvement in an additive mixed-effects model.
- Overall performance remained low, with no significant difference in the content effect between the original and bonus samples.
- The replication figure reports a significant advantage for slower humans in the Realistic condition, while the bonus did not clearly improve performance.
B.6 Human response time distributions on the Wason tasks
Human response times are longer on Realistic Wason questions, and this difference alone does not explain their accuracy advantage. Response time has no clear effect in NLI, while slower responses may help when syllogism content contradicts logic.
- Participants spend about 12 seconds longer on average answering Realistic than other Wason questions.The extra time may reflect reading context or deeper engagement, but the response-time difference alone does not explain the Realistic-condition advantage.
- NLI shows little effect of response time on accuracy, while syllogisms show no main effect but a marginally significant interaction with content.Slower responses are more helpful on syllogisms where content contradicts logic.
- Human success rates significantly predict language-model success rates for NLI and syllogisms after controlling for task variables.The associations are significant for NLI and syllogisms, but not for Wason, where human performance is low overall.
- Across individual Wason rules, models and slower humans perform well on one arbitrary French-word rule, while all perform poorly on some realistic rules.The authors note that it is not clear why the arbitrary rule is easier.
B.8 Model answer log-probability distributions
Model answer log-probabilities vary with task content, model scale, tuning, and scoring method. Chinchilla can identify valid syllogism conclusions accurately, but Domain-Conditional PMI produces substantially lower accuracy than raw likelihood scoring.
- Larger models show clearer separation between chosen and other answers, while instruction-tuned models produce larger-magnitude prior-corrected log-probabilities.The comparison specifically contrasts PaLM 2-L with PaLM 2-M and highlights Flan-PaLM 2 and GPT-3.5.
- Chosen and unchosen NLI answers have fairly separated log-probability distributions for most models, with additional content effects.
- Prior-corrected probability separation varies by model, validity, and content across syllogisms and Wason tasks.Wason chosen-versus-unchosen separation is generally lower than in other tasks, while content separation can be striking for GPT-3.5.
- Chinchilla achieves high accuracy when selecting the valid syllogism conclusion from all possible conclusions and shows relatively little bias.Bias increases in the few-shot condition.
- Without Domain-Conditional PMI correction, syllogism accuracy is around 35-40% regardless of belief consistency, although qualitative patterns remain similar.Raw log-likelihood scoring yields much higher accuracy than the correction in this task design.
C.1 NLI
The NLI analyses found no significant content effects for humans or the evaluated language models, although some model regressions failed to converge because of ceiling effects.
- Logistic regressions were generally used to analyze NLI correctness, with random effects for participant or model identity.
- PaLM 2-L and Flan-PaLM 2 regressions failed to converge because of ceiling effects, so χ2 tests were also reported.
- The reported χ2 tests likewise found no significant content effects on this simple task.
- No significant content effects were found for humans or the evaluated models on the NLI tasks.
C.2 Syllogisms
The syllogism analyses found significant content effects across humans and models, with most models also showing different sensitivity to logic depending on content consistency.
- The analysis encoded content consistency as whether the logical answer matched the conclusion’s believability.
- Three nested mixed-effects logistic regressions tested content, consistency, and their interaction, with model selection based on BIC.
- All evaluated humans and language models showed a significant content effect when judging syllogism validity.
- All models except Chinchilla and PaLM 2-M showed a significant interaction between content consistency and logical sensitivity.
- PaLM 2-L required χ2 tests because its regressions failed to converge, yet it showed significant content effects and a significant interaction.
C.3 Wason
The Wason analyses found content effects in most humans and models, while response-time controls showed that these effects were not solely explained by speed differences.
- A significant content effect appeared in most human and model analyses of the Wason task.
- Fast humans showed no significant content effect, while PaLM 2-M and Flan-PaLM 2 showed only marginally significant effects due to high item-level variance.
- The human content effect remained significant after controlling for log response time.
- Both content and z-scored response time remained significant when response time was standardized within condition.
C.3.2 Multinomial regression of the response patterns on the Wason tasks
Additional analyses linked model probability patterns to human response times on NLI and syllogism tasks while controlling for task variables, correctness, and random effects.
- The comparison split problems into cases where humans and models were both correct or both incorrect to control for accuracy.
- This procedure omitted problems that models or humans alone answered correctly.
- Model log-probability differences were significantly related to human response times on both NLI and syllogism tasks.
- On items where humans responded more slowly, models showed smaller differences in log-probabilities.
- The regressions controlled for task variables, answer correctness, and random effects of items and language models.