Source-linked AI summary
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models
Mingzi Cao, Xingwei Tan, Mahmud Elahi Akhter, Marco Valentino, Maria Liakata, Xi Wang, Nikolaos Aletras
TL;DR
The paper asks how deduction, induction, and abduction individually contribute to LLM reasoning generalization, a question complicated by prior entanglement with world knowledge. It collects knowledge-independent symbolic reasoning trajectories, induces these skills using several model-training approaches, and evaluates realistic out-of-domain tasks. The induced models achieve substantial transferable gains, with deduction generally strongest and upcycling best overall, while the study is limited by its English-only evaluation and one representative task per paradigm.
Problem
Prior work leaves the separate contributions of deduction, induction, and abduction to LLM generalization unclear, especially when reasoning data is entangled with world knowledge.
Method
The authors collect teacher-generated trajectories for three world-knowledge-independent symbolic paradigms and induce student LLMs using fine-tuning, up-scaling, and upcycling.
Results
The induced models show substantial transferable out-of-domain gains, up to 14.60, with deduction generally strongest and upcycling best overall.
Takeaways & Limitations
Fundamental reasoning skills induced from symbolic trajectories can improve LLM reasoning on realistic tasks beyond the symbolic training setting.
Takeaways & Limitations
The evaluation uses English data and one representative symbolic task for each reasoning paradigm.
Abstract
from arXiv · showhide
Deduction, induction, and abduction are fundamental reasoning paradigms, core for human logical thinking. Although improving Large Language Model (LLM) reasoning has attracted significant research efforts, the extent to which the fundamental paradigms induce generalization has yet to be systematically explored. In this study, we shed light on how the interplay between these core paradigms influences LLMs' reasoning behavior. To this end, we first collect a new dataset of reasoning trajectories from symbolic tasks, each targeting one of the three fundamental paradigms, to abstract from concrete world knowledge. Then, we investigate effective ways for inducing these skills into LLMs. We experiment with a battery of methods including simple fine-tuning, and more complex approaches to increase model depth, or transform a dense model to a mixture-of-experts. We comprehensively evaluate induced models on realistic out-of-domain tasks, that are entirely formulated in natural language and contain real-world knowledge. Our results reveal that our approach yields strong generalizability with substantial performance gains (up to $14.60$) across realistic tasks.
1 Introduction
The paper isolates deduction, induction, and abduction from world knowledge to study how each paradigm affects LLM reasoning and generalization. It introduces trajectory-based induction and reports substantial gains on realistic out-of-domain tasks.
- Prior work often conflates the three paradigms and mixes reasoning with world knowledge, leaving their individual effects on LLM generalization unclear.
- The authors enrich symbolic problems targeting deduction, induction, and abduction with teacher-generated reasoning trajectories that abstract inference patterns from world knowledge.
- The study evaluates fine-tuning, up-scaling, and upcycling across symbolic and realistic natural-language tasks in two model families.
- ≈17K symbolic problems are enriched with more than 160K reasoning trajectories generated by two teacher LLMs from different families.
- Up to 14.60 performance gains are obtained across realistic out-of-domain tasks and different inducing methods and LLM families.
- Deduction generally provides the highest out-of-domain gains, while upcycling achieves the best overall results and cross-paradigm improvements are observed.
2 Related Work
Related work has examined reasoning paradigms and synthetic trajectories, but often combines paradigms or leaves their separate contributions unresolved. This paper instead studies each paradigm individually using teacher-generated trajectories.
- Existing studies compare or merge reasoning paradigms in ways that make the individual impact of deduction, induction, and abduction difficult to isolate.
- The paper learns each fundamental paradigm separately from reasoning trajectories to provide a clearer, fine-grained understanding of its effect on LLM reasoning.
- Synthetic teacher-model trajectories are increasingly used because human-curated reasoning data can be noisy and difficult to scale.
3 Fundamental Reasoning Data
The authors construct trajectory data for three symbolic reasoning paradigms by enriching an existing question-answer dataset with teacher-model demonstrations. They sample, filter, and split trajectories for training and evaluation.
- Dataset construction: The dataset contains 3,600 deductive, 4,500 inductive, and 9,000 abductive questions, originally without reasoning trajectories.
- Symbolic tasks: Deduction uses satisfiable Boolean Satisfiability questions, with formulas converted to CNF and answers expressed in JSON for trajectory collection and evaluation.
- Symbolic tasks: Induction predicts the next number in sequences containing repeating addition, subtraction, or multiplication patterns.
- Symbolic tasks: Abduction works backward from assumed goals to hypothesize premises and trace known-atom truth values, addressing omissions in the original task design.
- Data splits: Each task type reserves 100 examples for development and 100 for testing, while trajectories are collected only for training examples.
- Trajectory sampling: Five zero-shot responses are sampled per question from Qwen3-30B-Instruct and Llama-3.3-70B-Instruct, excluding responses shorter than 20 words.
- Trajectory sampling: The final training set contains ∼16,000 questions with ∼82,000 Llama-generated and ∼60,000 Qwen-generated trajectories.
- Trajectory sampling: Qwen trajectories are longer than Llama trajectories, especially for inductive and abductive tasks, and frequently repeat inputs or use tables and tool-related special tokens.
4 Inducing Fundamental Reasoning Skills
The study compares standard parameter updates with capacity-expansion methods for inducing fundamental reasoning skills. These methods range from low-rank adaptation to added depth and mixture-of-experts width expansion.
- The inducing-method comparison spans standard fine-tuning and more sophisticated approaches that structurally modify the model.
- Fine-tuning: Full fine-tuning updates all model parameters through next-token prediction on collected reasoning trajectories.
- Fine-tuning: LoRA fine-tuning updates only low-rank matrices A and B rather than the existing weight matrix W.
- Capacity expansion: Up-scaling inserts additional layers and updates only those layers, increasing capacity while preserving the pretrained model’s original parametric knowledge.
- Capacity expansion: Upcycling converts dense Transformer layers into mixture-of-experts layers with multiple expert MLPs and a router selecting experts during inference.
5 Experimental Setup
The experiments separately induce deduction, induction, abduction, or their combination in two student LLMs using symbolic reasoning trajectories. Evaluation compares in-domain and symbolic/realistic out-of-domain performance against vanilla and teacher models.
- The study trains LLMs separately on deduction, induction, and abduction trajectories, while also testing a Mix condition containing all three paradigms.
- The experiments use Llama-3.1-8B-Instruct and Qwen3-8B as student models, comparing induced models with vanilla students and trajectory-generating teachers.
- Inducing methods include Full FT, LoRA, up-scaling through added layers, and upcycling dense layers into Mixture-of-Experts layers.
- Evaluation settings: In-domain symbolic evaluation tests the same reasoning type used during training, while Mix training evaluates all three reasoning types in-domain.
- Evaluation settings: Symbolic OOD evaluation tests reasoning tasks not seen during training, including cross-paradigm train–test combinations.
- Evaluation settings: Realistic OOD evaluation covers natural-language reasoning datasets including True Detective, αNLI, WinoWhy, and RECV, with final-answer accuracy judged by Qwen3-30B-Instruct.
6 Results
Inducing fundamental reasoning skills improves symbolic and realistic OOD performance, with deduction transferring most broadly and upcycling often performing best. Gains can depend on the model, teacher family, and target task, and induced students can surpass their teachers.
- Symbolic results: Specialist induction substantially outperforms vanilla students in five of six in-domain symbolic scenarios, with deduction producing the largest improvements.Llama-3.1-8B gains 56.00 via up-scaling, while Qwen3-8B gains 12.33 via upcycling.
- Symbolic results: Deduction transfers most strongly across symbolic paradigms, improving induction and abduction in nine cases each, while abduction transfers poorly.Across 16 symbolic cases per paradigm, deduction gains transfer in 12 cases; abduction benefits deduction in only four cases.
- Inducing methods: Upcycling generally provides the most robust gains, although the best inducing method remains model-dependent.For Qwen3-8B, upcycling is highest across paradigms; Llama-3.1-8B peaks with up-scaling for deduction and Full FT for induction.
- Teacher effects: Stronger Qwen teachers improve symbolic performance for both students, whereas realistic OOD performance improves when teacher and student belong to the same model family.The realistic-task finding is attributed to sensitivity to stylistic mismatches between teacher and student models.
- Realistic OOD results: Deduction also yields the strongest realistic OOD performance, reaching average scores of 70.14 via upcycling and 58.70 via up-scaling with Llama teacher traces.Induction is slightly lower, while abduction produces more modest gains; Mix can dilute single-paradigm inductive bias on less-structured realistic tasks.
- Realistic OOD results: Knowledge-free induction can improve students beyond their teachers on realistic tasks, with 13 of 16 induced Llama models outperforming the teacher and reaching 81.73.For Qwen students on αNLI, 13 of 16 cases also improve over the teacher, reaching 84.24 via inductive upcycling.
- Realistic OOD results: Realistic OOD gains concentrate on particular tasks and vary by teacher family, with αNLI and Folio showing more frequent improvements and RECV favoring Llama teachers.
- Inducing methods: Upcycling achieves the highest realistic performance for Qwen3-8B in seven of eight teacher–paradigm settings.
7 Qualitative Analysis
Qualitative examples show that induced structural reasoning can preserve uncertainty and prioritize stated logical premises over world knowledge. These behaviors prevent premature or content-driven conclusions in realistic reasoning tasks.
- Deductive up-scaling preserves uncertainty from a disjunctive premise, unlike vanilla and deductive Full FT models that commit through implicit assumptions.
- Deductive upcycling follows stated premises and logical rules instead of revising implausible premises according to world knowledge.
8 Conclusion & Takeaways
The paper isolates fundamental reasoning paradigms from world knowledge and evaluates multiple induction strategies for their generalization effects. It concludes that knowledge-free induction produces transferable OOD gains, with deduction most robust and upcycling strongest overall.
- The study trains on reasoning trajectories abstracted from world knowledge to isolate how deduction, induction, and abduction shape LLM reasoning generalization.
- Knowledge-free induction yields substantial and transferable OOD gains across the evaluated reasoning paradigms and tasks.
- Deduction is the most robust paradigm, while upcycling provides the largest overall improvements and can produce student models that surpass teachers on realistic tasks.
- Future work will investigate principled ways to combine multiple reasoning paradigms without diluting their individual contributions.
Limitations
The study is limited by its English-only data and evaluations and by using one representative symbolic task for each reasoning paradigm. It also documents evaluation-format deviations and a separate answer-verification procedure.
- Scope: All data and evaluations are in English, so cross-lingual differences in linguistic expression and cognitive strategies remain unexplored.The authors identify multilingual reasoning as an avenue for future work.
- Task coverage: The study explores only one representative symbolic task for deduction, induction, and abduction, limiting coverage of the broader task spectrum.Future work could examine how different symbolic data affect performance and generalisation.
- Evaluation: Some induced models produce correct answers while deviating from required output formats, especially JSON formatting for deduction and abduction.The evaluation therefore compares generated answers with gold answers using Qwen3-30B-Instruct to determine consistency.
D Full Results
Tables 4 and 5 report accuracy results for models induced with different methods and experimental settings, including models trained on trajectories from Llama-3.3-70B-Instruct and Qwen3-30B-Instruct.
- Results overview: Tables 4 and 5 present full results across different inducing methods and experimental settings.The table descriptions distinguish results by method and fundamental reasoning task.
- Table 4: Table 4 reports fine-tuned-model accuracy on fundamental reasoning tasks using trajectories collected from Llama-3.3-70B-Instruct.It identifies the base model, applied PEFT methods, fine-tuning task, and trainable versus total parameters.
- Table 5: Table 5 reports fine-tuned-model results using reasoning trajectories collected from Qwen3-30B-Instruct.The supplied caption identifies the trajectory source but does not provide numerical scores.
E Out-of-Domain Generation Example
The examples show that deductive induction can improve out-of-domain reasoning, while vanilla and mismatched inducing methods often follow world knowledge or make unsupported assumptions. Correct reasoning requires adhering to stated premises and preserving uncertainty when the premises do not determine a unique conclusion.
- FOLIO examples: Deductive up-scaling produces the correct FOL answer where vanilla Llama and deductive Full FT fail.The comparison is reported for Examples 1–3.
- FOLIO examples: Deductive up-scaling also succeeds where vanilla Qwen3-8B and abductive upcycling fail on another FOLIO question.The models differ in whether they follow the stated logical structure.
- FOLIO examples: Other generated reasoning derives a contradiction by combining conditional premises, then concludes that Bonnie must be a student and perform in talent shows.This chain appears in the reported generated output and ends with a True answer.
- FOLIO examples: Premise-based reasoning preserves uncertainty when the facts do not establish whether Bonnie is a student, avoiding a definitive performance conclusion.The premises support conditional conclusions but do not settle Bonnie’s student status.
- Geographic reasoning example: Some generated reasoning incorrectly treats real-world geography as grounds for rejecting a premise, rather than accepting the premises as given.The Barutin Cove example explicitly identifies the premise about Antarctica and the South Shetland Islands as confusing or incorrect in reality.
F Symbolic Generation Example
The examples compare untrained, inductive, and deductive Llama outputs on a sequence-completion task. Induced models identify the recurring operations and produce the correct answer, while one generated analysis becomes repetitive and fails to resolve the pattern.
- Untrained output: The untrained Llama examines inconsistent differences and multiple numerical regularities before failing to identify the intended sequence rule.Its analysis considers prime numbers, differences, multiples of 4, and conditional additions without reaching a stable solution until the final answer.
- Generation failure: One generated analysis becomes repetitive while proposing an unsupported multiplication-and-addition transformation, illustrating a failure to complete the reasoning process.The output repeats the same transformation discussion until reaching the maximum token limit.
- Inductive Full FT: Inductive Full FT Llama finds a repeating operation pattern and answers 446 for the next sequence value.The output describes multiplication by 4 alternating with additions of 3 and concludes 443 + 3 = 446.
- Final answer: The sequence-completion outputs ultimately report 446 as the value at the question mark.Both the inductive and deductive successful outputs explicitly provide 446 as the final answer.