Source-linked AI summary
A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences
Leonardo Bertolazzi, Albert Gatt, Raffaella Bernardi
TL;DR
This paper asks whether LLMs can reason deductively from syllogistic form despite reported content effects, reluctance to identify invalid inferences, and difficulties with longer reasoning. It compares Zero-Shot CoT, ICL, and SFT, analyzing model conclusions as well as accuracy. ICL improves performance on valid inferences but increases inconsistency, whereas SFT lets evaluated medium-sized models approach ceiling performance while remaining consistent.
Problem
Prior work reports content effects, reluctance to answer that no conclusion follows, and difficulty with human-challenging and multi-step syllogistic reasoning.
Source: §1 · p. 1Method
Results
ICL improves valid-inference performance but increases inconsistency, while SFT is effective for evaluated small and medium models and lets medium models approach ceiling performance while remaining consistent.
Source: §8 · p. 8Takeaways & Limitations
The authors find that cognitive-science heuristics predict LLaMA-3 8B’s Zero-Shot CoT answers, and that SFT mitigates most reasoning biases without harming consistency in the evaluated models.
Source: §8 · p. 8The study covers only Pythia and LLaMA 2/3, and resource limits excluded larger models, including LLaMA-3 70B, from ICL and SFT analysis.
Source: §9 · p. 9
Abstract
from arXiv · showhide
The reasoning abilities of Large Language Models (LLMs) are becoming a central focus of study in NLP. In this paper, we consider the case of syllogistic reasoning, an area of deductive reasoning studied extensively in logic and cognitive psychology. Previous research has shown that pre-trained LLMs exhibit reasoning biases, such as $\textit{content effects}$, avoid answering that $\textit{no conclusion follows}$, display human-like difficulties, and struggle with multi-step reasoning. We contribute to this research line by systematically investigating the effects of chain-of-thought reasoning, in-context learning (ICL), and supervised fine-tuning (SFT) on syllogistic reasoning, considering syllogisms with conclusions that support or violate world knowledge, as well as ones with multiple premises. Crucially, we go beyond the standard focus on accuracy, with an in-depth analysis of the conclusions generated by the models. Our results suggest that the behavior of pre-trained LLMs can be explained by heuristics studied in cognitive science and that both ICL and SFT improve model performance on valid inferences, although only the latter mitigates most reasoning biases without harming model consistency.
1 Introduction
The introduction frames syllogistic inference as a test of whether LLMs can reason from logical form rather than content, against known human-like biases and difficulties. The paper compares prompting and fine-tuning strategies and analyzes generated conclusions as well as accuracy.
1 Introduction
- The motivation is to determine whether LLMs possess latent reasoning skills that prompting can elicit or whether other methods are needed.The paper situates this question within the broader debate over whether LLMs acquire reasoning skills alongside language proficiency.Source: §1 · p. 1
- The paper compares Zero-Shot CoT, ICL, and SFT on syllogisms with believable or unbelievable conclusions and with multiple premises.It examines generated conclusions for consistency and compatibility with cognitive-science heuristic theories, alongside accuracy.Source: §1 · p. 2
- Open-access models show human-like reluctance to answer “nothing follows” and content effects when responding to invalid syllogisms.These behaviors echo earlier findings by Eisape et al. (2024) and Lampinen et al. (2023).Source: §1 · p. 2
- ICL improves performance on valid inferences but does not remove content effects or difficulty with invalid syllogisms, and increases contradictory conclusions.The paper reports that ICL also increases model inconsistency.Source: §1 · p. 2
- SFT is effective for the evaluated small and medium models, with medium models approaching near-perfect performance while remaining consistent.How well this ability transfers to real-life scenarios remains an open question.Source: §1 · p. 2
2 Related Work
Related work traces the study of language-model reasoning from NLI and prompting to generative syllogistic tasks. The paper extends prior comparisons by jointly examining learning strategies and model conclusions in depth.
2 Related Work
- Earlier NLI research mainly treated inference as classification with encoder-only models, while CoT and ICL later drew attention to reasoning in decoder-only LLMs.The paper uses syllogisms as a test bed for the interplay of content and form.Source: §2 · p. 2
- Prior work found that instruction tuning, SFT, ICL, and voting improve LLaMA’s accuracy on GLoRE, while raising the possibility of superficial pattern use.The authors respond by analyzing accuracy and human-performance correlation, then examining outputs through cognitive heuristic theories.Source: §2 · pp. 2–3
- Lampinen et al. (2023) reported content effects in discriminative syllogism tasks, while Eisape et al. (2024) found human-like biases and difficulty recognizing invalid syllogisms in generative tasks.The present work extends this research by systematically comparing CoT, ICL, and SFT across world-knowledge conditions and multiple premises.Source: §2 · p. 3
- Prior work found that CoT and ICL can generalize to unseen rules but fail on longer proofs; this paper focuses instead on content-form interplay and generated conclusions.The authors also note that in-depth output analysis was missing from the current literature.Source: §2 · p. 3
3 Syllogisms
Syllogisms combine premise moods and figures to determine possible conclusions, and can be composed into longer inference chains. Their difficulty varies for both humans and models, including on invalid inferences and believable conclusions.
3 Syllogisms
- Each syllogism has two premises and a conclusion; premise moods and the order of terms determine its schema, which specifies the inference pattern.The four moods are A and I (affirmative) and E and O (negative); there are four figures, defined by the order of nouns in the premises.Source: §3 · p. 3
- Of 64 possible premise-mood and figure combinations, 37 are invalid and 27 yield at least one conclusion.Schemas can be composed into longer chains requiring multi-hop reasoning, with multiple possible intermediate conclusions.Source: §3 · pp. 3–4
- Human responses show content effects and difficulty deciding that no valid conclusion follows, while accuracy varies sharply by schema.The cited meta-review reports 90% accuracy on AI2 and 1% on AE2 schemas.Source: §3 · p. 4
4 Learning Strategies
The study compares zero-shot prompting, in-context examples, and supervised fine-tuning to examine whether models can acquire syllogistic schemas. The strategies differ in whether examples change model weights and whether training updates the model.
4 Learning Strategies
- The study frames syllogistic reasoning as a multiple-choice task and compares in-context with supervised learning to test latent or learnable schema use.The model is given premises and selects a conclusion from candidate answers.Source: §4 · p. 4
- The baseline is Zero-Shot CoT, a two-stage prompt that asks a model to reason before giving a final answer, without task examples.The study uses pretrained Pythia and LLaMA models, comparing model size and, when possible, instruction tuning.Source: §4 · p. 4
- ICL presents five task examples without changing model weights, with test schemas either matching or differing from the example schemas.The study evaluates both ICLin and ICLout, using randomly selected examples.Source: §4 · p. 4
- SFT trains models for multiple epochs using cross-entropy loss and next-token prediction, showing one instance of each schema per epoch.The terms instantiating the schema change across epochs.Source: §4 · p. 4
5 Task and Datasets
The study evaluates syllogistic conclusions in a multiple-choice task and builds datasets for world-knowledge effects and longer inference chains.
5 Task and Datasets
- The task presents two premises and nine conclusions, including “Nothing follows”; models must generate at least one correct conclusion.Options are randomized, and a schema can have one to four correct conclusions.Source: §5 · p. 4
- Training data and ICL examples use pseudo-words to preserve syllogistic form while removing meaningful content.The vocabulary contains 4,000 pseudo-words; the development set uses 1,000 unseen pseudo-words.Source: §5 · p. 4
- ICLin uses examples from the test example’s schema, whereas ICLout uses examples from different schemas; SFT training spans all schemas.Source: Figure 3 · p. 5
- Believable syllogisms use premises true in the real world, while unbelievable syllogisms have correct conclusions that violate world knowledge.Unbelievable examples are constructed only for valid schemas; the sets contain 640 believable and 270 unbelievable items.Source: §5 · p. 5
- Longer inferences replace one A-premise with a chain of two or three A-premises, producing three- and four-premise test sets.Each longer-premise set contains 280 items from schemas with at least one A-premise.Source: §5 · p. 5
6 Results based on task-accuracy
The results compare zero-shot CoT with ICL and SFT on valid and invalid syllogisms, including tests with more premises. SFT is strongest across validity and content-bias measures, while longer premise chains remain challenging.
6.1 Zero-shot CoT Setting
- In zero-shot CoT, LLaMA-2 reaches human accuracy on valid syllogisms and LLaMA-3 surpasses it, but no model reaches human accuracy on invalid ones.Instruction tuning improves invalid-inference performance at the cost of valid-inference performance; LLaMA-3 70B instruction-tuned is closest on invalids at 32.50%.Source: §6.1 · p. 5
- Most models lose accuracy on valid syllogisms when correct conclusions violate world knowledge, extending reported content effects from discriminative to generative tasks.The results also align with prior reports of human-score correlations and reluctance to answer “nothing follows” (Eisape et al., 2024).Source: §6.1 · p. 6
6.2 Learning strategies performance
- SFT performs well on valid and invalid inferences across both models and overcomes content bias and the difficulty of generating “nothing follows”; ICL does not.Among ICL settings, ICLin outperforms ICLout on valid conclusions.Source: §6.2 · p. 6
- Despite a greater performance drop than ICL, SFT retains higher accuracy on three- and four-premise tests; ICLout drops less than ICLin as premise count grows.Performance decreases as the number of premises increases.Source: §6.2 · p. 6
7 Beyond task accuracy
Beyond whether a model produces a correct conclusion, the analysis examines contradictory outputs and whether heuristic predictions match generated conclusions. It finds that consistency and heuristic patterns differ by model and learning strategy.
Consistency analysis
- ICL increases contradictory answers for both models, especially ICLin; SFT further reduces LLaMA contradictions, while 73.91% of Pythia SFT answers are contradictory.For Pythia SFT, 70.47% of contradictions involve “nothing follows”; LLaMA-3 8B ZS-CoT has 1.09% contradictory answers versus Pythia’s 18.91%.Source: §7 · p. 6
Heuristic analysis
- The Atmosphere Theory predicts 62.50% of correct valid conclusions but 0% on invalid schemas, and the analysis focuses on this theory as most insightful.Source: §7 · pp. 6, 8
- The Atmosphere Theory predicts 92% of correct valid conclusions generated by LLaMA ZS-CoT, as well as its errors in invalid schemas (82.97%) and valid schemas (62.79%).Source: §7 · p. 8
Model differences
- Pythia SFT struggles more with unseen premise counts and becomes inconsistent as its accuracy increases, unlike LLaMA SFT.The authors suggest model size and pretraining data may contribute to the differences.Source: §7 · p. 8
Limitations
- The authors cannot exclude that LLaMA SFT performance reflects sophisticated pattern matching rather than learning inference rules.Its comparatively large drop on unseen premise counts supports this possibility.Source: §7 · p. 8
8 Conclusion
The paper compares prompting and fine-tuning for syllogistic reasoning and analyzes generated conclusions through cognitive heuristics. ICL improves valid-inference performance but increases inconsistency, whereas SFT can approach ceiling performance while remaining consistent.
8 Conclusion
- The Atmosphere Theory was a very good predictor for LLaMA-3 8B ZS-CoT; the authors conjecture that zero-shot LLMs rarely generate “nothing follows” because premise quantifiers serve as cues.Source: §8 · p. 8
- ICL improves valid-inference performance but retains content effects and difficulty with invalid syllogisms, while increasing inconsistency; SFT lets medium-sized models approach ceiling performance while remaining consistent.The authors suggest content-based reasoning can be learned through pretraining, whereas form-based reasoning needs appropriate training.Source: §8 · pp. 8–9
- SFT LLaMA produces only 0.31% contradictory AO or EI answers and never pairs “nothing follows” with other conclusions, while SFT makes Pythia more inconsistent.The table contrasts contradiction patterns for the two fine-tuned models.Source: Table 4 · p. 9
9 Limitations
The limitations and appendices specify the study’s model and measurement boundaries, syllogism dataset construction, and training and inference procedures.
9 Limitations
- The study examined Pythia and LLaMA 2/3, restricted ICL/SFT analysis to small and medium open models, added appendix completeness analysis, but did not measure whether models generated all valid conclusions.Further work could use adversarial studies, real-word strategies for disrupting lexical content, and ablations.Source: §9 · p. 9
A.1 Valid and Invalid Syllogisms.
- Of 64 syllogistic schemas, 27 have valid conclusions and 37 have none, assuming nonempty terms and allowing conclusions in either term direction.The assumptions match those used in the cited psychological literature.Source: §A.1 · p. 11
A.2 Believable and Unbelievable sets.
- Believable and unbelievable sets use 30 hierarchical terms, rearranged across triples to make syllogisms true or false in the real world.The authors manually changed term order or swapped terms between triples.Source: §A.2 · pp. 11, 13
A.3 Syllogisms with 3 and 4 premises.
- Three- and four-premise datasets extend schemas by replacing A premises with transitive chains, using pseudowords to isolate longer inference from semantic-content biases.The replacement uses sequences of two or three A premises, respectively.Source: §A.3 · p. 13
B.1 Computational resources
- Experiments used one server with three 24GB NVIDIA GPUs, and fine-tuning plus evaluation took approximately 72 hours.The GPUs were two RTX A5000s and one Quadro P6000.Source: §B.1 · p. 13
B.2 Training
- SFT minimizes cross-entropy on premises, randomized options, and correct conclusions; models ran 100 epochs, with checkpoints selected by development accuracy.The development set used unseen pseudowords across the 64 schemas; the authors used LoRA with rank 32.Source: §B.2 · p. 13
B.3 Inference
- Inference uses greedy decoding; models generate multiple conclusions, with 20 answer tokens and longer reasoning-chain allowances for CoT and multi-premise tasks.ZS-CoT chains use 50 tokens for pretrained models and 70 for instruction-tuned models; three- and four-premise tasks use 70 tokens.Source: §B.3 · pp. 13–14
B.4 Prompts
- Non-fine-tuned models receive task instructions, while ICL adds five pseudoword examples that either share or differ from the test schema.ICLin examples share the schema; ICLout examples are chosen not to instantiate it.Source: §B.4 · p. 14
C Experimental Details
The appendix details accuracy and human-correlation measures, tests content bias, and examines completeness alongside consistency. Across settings, the two models show different consistency and completeness patterns.
C Experimental Details
- Standard accuracy counts an answer as successful if it contains at least one correct conclusion; Top 1 accuracy uses only the first generated answer and the best of three runs per setting.The Top 1 results are reported in Table 6.Source: C Experimental Details · pp. 14–15, Table 6 · p. 15
- A correlation of 0.80 accompanies LLaMA-3 70B accuracy of 58.89% and human accuracy of 44.63%, reflecting shared relative schema difficulty despite different overall accuracy.The correlation is Spearman’s coefficient computed over schema accuracies; higher correlation means humans and models struggle with the same schemas.Source: C Experimental Details · p. 15
- All settings except SFT showed more B|U than U|B, indicating content bias; manual annotation of 441 conclusions matched human judgment with 93.42% accuracy.The analysis used a chi-square test for independence with Yates’ continuity correction to assess content-effect performance differences.Source: C Experimental Details · p. 15, Table 7 · p. 15
D Inconsistency vs. Incompleteness
- Completeness was assessed by whether models produced both a conclusion and its converse for symmetric relationships, not by requiring every valid conclusion and no invalid ones.The symmetric conclusion types are I (Some a are b) and E (No a are b).Source: D Inconsistency vs. Incompleteness · p. 16
- LLaMA’s ICL settings improved completeness while slightly reducing consistency; across settings, both models produced more incomplete answers than under the consistency analysis.The reported completeness patterns differed between the fine-tuned models, and LLaMA generally outperformed Pythia except in ZS-CoT.Source: D Inconsistency vs. Incompleteness · p. 16
- For LLaMA-3 8B, accuracy and completeness rank settings SFT > ICL > CoT; for Pythia 1.4B, accuracy ranks likewise, but consistency ranks CoT > ICL > SFT.Figure 9 plots inconsistent answers on the x-axis and incomplete answers on the y-axis; distance from the origin represents a complete and consistent model.Source: D Inconsistency vs. Incompleteness · p. 16
E Heuristics Theories of Syllogistic Reasoning
The section describes heuristic theories of syllogistic reasoning and compares their predictions with model conclusions. The Atmosphere theory accounts for many LLaMA-3 8B ZS-CoT conclusions, while the theories have notable limits in capturing correct conclusions and invalid inferences.
E Heuristics Theories of Syllogistic Reasoning
- Atmosphere derives conclusions from premise quantity and polarity signs, but it never predicts an invalid syllogism with no valid conclusion; its feature matching requires no premise-meaning understanding.Its hypothesis is that reasoners favor conclusions fitting the premises’ mood; Revlis (1975) developed a rule-based feature-selection model.Source: E Heuristics Theories of Syllogistic… · pp. 16–17
- Matching selects end-term conclusions aligned with the more conservative premise, while Illicit Conversion treats A and O statements as symmetric and Probability Heuristics substitutes probabilistic for logical validity.Matching, Illicit Conversion, and Probability Heuristics are reviewed in Khemlani and Johnson-Laird (2012); Illicit Conversion was proposed by Chapman and Chapman (1959).Source: E Heuristics Theories of Syllogistic… · pp. 17–18
- For LLaMA-3 8B ZS-CoT, Atmosphere predicts 92% of correct valid conclusions, 82.97% of mistakes in invalid schemas, and 62.79% of mistakes in valid schemas.The authors identify Atmosphere as the only theory among those examined that explains high proportions in all three categories.Source: E Heuristics Theories of Syllogistic… · p. 18
- Conversion predicts 58.33% of LLaMA-3 and 50.70% of Pythia SFT mistakes on valid syllogisms, but its other predictions do not establish that the models use this heuristic.Further analysis is needed to determine whether the models follow Conversion heuristics.Source: E Heuristics Theories of Syllogistic… · p. 18
- All examined heuristics except Conversion fail to capture “Nothing follows” for invalid syllogisms; Atmosphere predicts at most 62.50% of correct conclusions on valid syllogisms.These percentages compare heuristic predictions with ground-truth conclusions.Source: E Heuristics Theories of Syllogistic… · p. 18
F Example Mistakes
The examples illustrate mistakes by LLaMA-3 8B in ICLin and ZS-CoT. The accompanying prompts distinguish in-context demonstrations with the test schema from demonstrations using a different schema.
F Example Mistakes
- In the ICLin example, LLaMA-3 8B generates a conclusion different from “nothing follows” despite seeing same-schema demonstrations with that answer.Figure 11 substitutes <shuffled-options> for the nine options in each demonstration; the demonstrations are highlighted.Source: F Example Mistakes · p. 18, Figure 7 · p. 21, Figure 11 · p. 23
- In Figure 12’s ZS-CoT setting, LLaMA-3 8B makes a mistake: the correct conclusion would be nothing follows, but the model produces three wrong ones.The caption says the nine options for each in-context example are substituted with the special token <shuffled-options> for ease of presentation.Source: F Example Mistakes · p. 18, Figure 12 · p. 24
- ICLout demonstrations use a different schema from the test syllogism, whereas ICLin demonstrations use the same schema; each prompt includes five demonstrations.Both prompt examples begin with a task instruction and have the model answer after “Answer:”.Source: Figure 6 · p. 20, Figure 7 · p. 21
- Figure 10 summarizes the proportions of correct conclusions and mistakes predicted by heuristic theories.Table 10 gives proportions of ground-truth answers predicted by each heuristic; Table 11 lists the theories’ predicted conclusions.Source: Table 10 · p. 18, Table 11 · p. 19, Figure 10 · p. 23