Source-linked AI summary
Cognitive Foundations for Reasoning and Their Manifestation in LLMs
Priyanka Kargupta, Shuyue Stella Li, Haocheng Wang, Jinu Lee, Shan Chen, Orevaoghene Ahia, Dean Light, Thomas L. Griffiths, Max Kleiman-Weiner, Jiawei Han, Asli Celikyilmaz, Yulia Tsvetkov
TL;DR
LLMs can produce correct answers while failing on simpler variants, motivating a framework for distinguishing reasoning mechanisms from surface success. The paper synthesizes cognitive science into a 28-element taxonomy and analyzes model and human reasoning traces. It finds that models possess successful behavioral patterns but do not deploy them reliably, while automatically scaffolded test-time guidance improves performance by up to 66.7% on complex problems.
Problem
LLMs show high performance alongside failures on simpler variants, leaving unclear whether their outputs arise through mechanisms comparable to human reasoning.
Method
The paper synthesizes cognitive science into a 28-element taxonomy and evaluates these elements across model and human reasoning traces, papers, and test-time guidance interventions.
Results
Models possess behavioral patterns associated with success but fail to deploy them spontaneously, while test-time guidance improves performance by up to 66.7% on ill-structured problems.
Takeaways & Limitations
The framework provides a shared vocabulary for diagnosing reasoning failures and studying cognitive mechanisms in LLMs and humans.
Takeaways & Limitations
The study cannot yet predict which cognitive capabilities emerge from particular training procedures or whether observed patterns reflect genuine mechanisms rather than shortcuts.
Abstract
from arXiv · showhide
Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning. To understand this gap, we synthesize cognitive science research into a taxonomy of 28 cognitive elements spanning reasoning invariants, meta-cognitive controls, representations for organizing reasoning & knowledge, and transformation operations. We introduce a fine-grained evaluation framework and conduct the first large-scale empirical analysis of 192K traces from 18 models across text, vision, and audio, complemented by 54 human think-aloud traces, which we make publicly available. We find that models under-utilize cognitive elements correlated with success, narrowing to rigid sequential processing on ill-structured problems where diverse representations and meta-cognitive monitoring are critical. Human traces show more abstraction and conceptual processing, while models default to surface-level enumeration. Meta-analysis of 1.6K LLM reasoning papers reveals the research community concentrates on easily quantifiable elements (sequential organization: 55%, decomposition: 60%) but neglecting meta-cognitive controls (self-awareness: 16%) that correlate with success. Models possess behavioral repertoires associated with success but fail to deploy them spontaneously. Leveraging these patterns, we develop test-time reasoning guidance that automatically scaffold successful structures, improving performance by up to 66.7% on complex problems. By establishing a shared vocabulary between cognitive science and LLM research, our framework enables systematic diagnosis of reasoning failures and principled development of models that reason through robust cognitive mechanisms rather than spurious shortcuts, while providing tools to test theories of human cognition at scale.
1 Introduction
The paper unifies cognitive science and LLM reasoning research through a 28-element taxonomy, then evaluates how these elements appear in human and model traces. It finds that targeted test-time guidance can scaffold successful reasoning structures and improve performance on ill-structured problems.
- 1 Introduction: LLMs can master complex skills yet fail on simpler prerequisite tasks or trivial variants, raising questions about whether their successful outputs arise from human-like reasoning mechanisms.The paper frames this dissociation as a problem of reasoning generalization and mechanism identification.
- 1 Introduction: LLM reasoning research concentrates on sequential organization while giving comparatively little attention to diverse cognitive behaviors represented in human reasoning.The meta-analysis covers 1,598 arXiv reasoning papers and reports 55% examining sequential organization versus 8% studying backward chaining.
- 1 Introduction: The framework synthesizes cognitive science into 28 elements spanning reasoning invariants, meta-cognitive controls, representations, and reasoning operations.It provides a shared analytical vocabulary for evaluating machine reasoning beyond output performance.
- 1 Introduction: The study compares 170K model traces from 18 text, vision, and audio models with 54 human think-aloud traces using fine-grained span-level annotation.The analysis identifies which elements appear, where they occur, how they are sequenced, and which patterns correlate with correct outcomes.
- 1 Introduction: Up to 66.7% improvement on ill-structured problems came from test-time guidance that scaffolded successful cognitive structures, while well-structured performance remained at baseline.The guidance targeted sequences such as strategy selection, conceptual processing, and causal organization.
2 Formalizing Cognitive Foundations for Reasoning
The paper formalizes reasoning as the interaction of validity constraints, executive controls, organized representations, and operations that transform them. Drawing on cognitive theories and Marr’s levels of analysis, it uses this framework to diagnose reasoning processes rather than treating isolated behaviors as sufficient.
- 2 Formalizing Cognitive Foundations for Reasoning: Cognitive theories provide a starting point for defining reasoning because principled criteria are needed to distinguish reasoning from pattern matching.The paper uses Marr’s levels to organize fragmented theoretical perspectives into a shared framework for LLM evaluation.
- 2 Formalizing Cognitive Foundations for Reasoning: The taxonomy organizes 28 cognitive elements into reasoning invariants, meta-cognitive controls, reasoning representations, and reasoning operations.These dimensions respectively specify validity requirements, process regulation, knowledge organization, and procedures for constructing or transforming representations.
- 2 Formalizing Cognitive Foundations for Reasoning: Reasoning depends on dynamic interaction among goals, strategic choices, representations, and coordinated operations rather than isolated cognitive elements.The LEGO example illustrates how a building goal determines strategy and representations, which are then transformed while satisfying invariants.
- 2.1 Reasoning Invariants: Properties & Goals: Reasoning invariants define computational requirements including logical coherence, compositionality, productivity, and conceptual processing over abstract relations.Together, these constraints distinguish meaningful, generative reasoning from inconsistent or surface-level processing.
- 2.2 Meta-Cognitive Controls: Executive Regulation: Meta-cognitive controls select, monitor, and adapt reasoning processes, with self-awareness assessing knowledge, capability, and task solvability.These controls orchestrate reasoning rather than defining the validity conditions that reasoning must satisfy.
- 2.4 Reasoning Operations: Transformation Procedures: Reasoning operations include forward chaining, backward chaining, backtracking, verification, restructuring, pattern recognition, and abstraction.These operations navigate or modify representations and are flexibly deployed under meta-cognitive control.
3 Behavioral Manifestation in Humans and LLMs
The study compares cognitive behaviors across LLM and human reasoning traces using span-level annotation and structured problem types. It finds lower success on less-structured problems and evaluates how reasoning elements and their relationships manifest across modalities.
- Data sources and models: The analysis includes 16 open-weight text models, one open-weight audio-language model, image traces, and 18 human think-aloud participants across modalities.Human traces serve as qualitative reference points rather than a full benchmark, and audio evaluation uses BLAB, MMAR, and MMAU-Pro.
- Annotation validation: Two human annotators score each cognitive element from 0 to 2, with min-pooling producing conservative estimates used to refine automatic annotation prompts.Full-scale annotation then uses GPT-4.1 with temperature 0.6 after iterative human-in-the-loop prompt refinement.
- Annotation framework: Each trace receives span-level annotations identifying exact text segments that demonstrate the 28 cognitive capabilities.Guidelines combine cognitive-science definitions, behavioral indicators, three-level rubrics, examples, and character-index span boundaries.
- Problem coverage: The dataset spans 11 Jonassen problem types, with coverage decreasing as tasks become less structured and no strategic-performance instances.Algorithmic problems dominate, while rule-using problems are concentrated in image data.
- Reasoning structure construction: The reasoning-structure analysis represents annotated traces as heterogeneous transition graphs whose elements are connected by parallel, contains, or next relations.Spans are sorted by position, and greedy forward search begins from the most frequent initial element.
- Performance across problem types: 48.0% design-problem success contrasts with 62.2% on algorithms and 78.8% on story problems, while dilemmas range from 3.2% to 99.1%.Success declines with problem ill-structuredness, and dilemma performance varies substantially by model size.
3. How do reasoning structures differ between LLMs and human reasoners?
LLMs and humans differ in both which cognitive elements they deploy and how they organize them: models narrow into rigid sequential strategies on ill-structured problems, while humans use more abstraction, conceptual processing, and meta-cognitive control.
- Distribution of Cognitive Elements: Models narrow their behavioral repertoire on ill-structured problems, although successful traces require greater diversity across representations and operations.Average element presence falls from 0.397 ± 0.255 on algorithmic, story, and rule-using problems to 0.337 ± 0.261 on case analysis, design, and dilemma problems, while success becomes associated with hierarchical, network, spatial, and temporal representations and varied operations.
- Distribution of Cognitive Elements: Models often manifest cognitive elements without executing them effectively: logical coherence appears in 91% of traces but has average PPMI 0.091, while knowledge alignment appears in 20.2% with PPMI 0.234.Annotators observed that models identify contradictions but often fail to recognize or respond to them effectively, creating a gap between prevalence and success correlation.
- Distribution of Cognitive Elements: Meta-cognitive monitoring is prevalent in 53.5% of traces but has PPMI 0.031 and declines sharply on case-analysis and dilemma problems lacking clear ground truth.The pattern indicates limited self-assessment where answers are ambiguous or non-verifiable.
- Distribution of Cognitive Elements: Models favor sequential organization, decomposition, selective attention, integration, and forward chaining across problem types instead of adapting strategies to problem demands.Successful traces show substantially greater operational diversity, whereas models’ forward-chaining preference follows their broader sequential organizational bias.
- Distribution of Cognitive Elements: Across models and modalities, sequential organization and forward chaining remain frequent, while higher overall behavioral presence coincides with longer reasoning traces in the reported model comparison.Olmo-3-7B-Think has 48.4% presence and 17,416 average characters, while s1.1-32B has 59.7% presence and 13,210 average characters; well-structured problems also feature longer traces.
- Reasoning Structures: Successful diagnostic traces scope the problem before solution construction, whereas common traces rush directly into forward chaining with probability 0.748.The successful sequence is selective attention → sequential organization → knowledge alignment → forward chaining; the common pattern bypasses scoping and generates solutions before understanding validity constraints.
- Comparison with Humans: Human traces show more strongly present self-awareness and abstraction than LLM traces, at 49% versus 19% and 54% versus 36%, respectively.Humans also invoke conceptual processing and abstraction more quickly, while LLMs rely more on surface-level reiteration, enumeration, repeated verification, and backtracking.
4 Eliciting Cognitive Reasoning Structures
The paper converts empirically derived cognitive structures into test-time prompts and finds that their benefits depend on problem type and model capability.
- Methodology: The method linearizes each problem type’s consensus subgraph and converts it into an actionable prompt that scaffolds problem solving.
- Evaluation: Guidance is evaluated on balanced samples of previously correct and incorrect textual problems to test both improvement and performance preservation.Effectiveness is measured as percentage change in accuracy relative to baseline.
- Results and Analysis: Up to 66.7% accuracy gains occur on ill-structured problems when capable models receive automatically constructed cognitive-structure guidance.Reported gains include +66.7% for Qwen3-7B on diagnosis and +60.0% for Qwen3-14B and R1-Distill-Qwen-32B on dilemmas.
- Limitations: Smaller or less capable models can deteriorate sharply, with losses exceeding 50% in several categories and -72.0% on algorithmic problems for two models.The authors associate this pattern with insufficient reasoning flexibility and instruction-following ability.
- Results and Analysis: Benefits are strongest for complex open-ended problems, whereas well-structured problems show more modest or negative effects across models.
5 Cognitive Element Considerations in LLM Research Design
A meta-analysis shows that LLM reasoning research emphasizes easily evaluated linear behaviors while giving much less attention to flexible cognitive capabilities and their spontaneous deployment.
- Research Coverage: Research coverage is concentrated on context awareness at 70%, decomposition and integration at 60%, and knowledge structure alignment at 56%.
- Research Coverage: Flexible capabilities receive less attention: self-awareness appears in 16% of papers, spatial organization in 10%, and temporal organization in 22%.
- Design-Behavior Gaps: Compositionality, context alignment, and knowledge structure alignment are frequently targeted but appear inconsistently across model reasoning traces.
- Design-Behavior Gaps: Profundity appears in only 16% of papers yet manifests consistently in model outputs, indicating an emergent but undertheorized behavior.
- Neglected Capabilities: Self-awareness, temporal, ordinal, and spatial organization receive minimal attention and fail to emerge spontaneously in model behavior.
- Implications: The synthesis identifies a narrow conceptual vocabulary that privileges linear compositional behaviors over richer cognitive taxonomies.
6 Opportunities and Challenges
The paper frames cognitive analysis as actionable for improving reasoning, while identifying unresolved challenges in predicting, transferring, and validating cognitive capabilities.
- Validation Challenges: The taxonomy supports systematic characterization of cognitive elements, but it remains unclear whether guidance elicits latent capabilities or retrieves cached reasoning patterns.
- Predicting Capabilities: Cognitive scaffolding produces highly variable responses across models, and training or architecture details do not yet predict those differences.
- Generalization: Reasoning guidance improves performance through problem-specific templates but still depends on prior knowledge of diverse successful patterns.
- Generalization: Models can succeed in-distribution yet fail on superficial variants when they rely on shallow forward chaining rather than schema abstraction.
- Underlying Processes: Observable behavior alone cannot establish underlying cognitive processes because humans and models may reach correct answers through different mechanisms.
- Validation Challenges: Genuine capabilities should be tested through transfer, perturbation robustness, compositional deployment, and internal consistency rather than behavioral presence alone.
- Training Opportunities: Diverse behavioral repertoires correlate with success on ill-structured problems, where rigid strategies fail and meta-cognitive monitoring and representational flexibility matter.
- Training Opportunities: Outcome-based rewards provide sparse terminal signals, while process rewards still optimize accuracy rather than behavioral diversity that supports transfer.
A.1 Prompts for Fine-Grained Cognitive Element Annotation
The annotation framework operationalizes cognitive capabilities with definitions, observable indicators, calibrated scoring levels, and standardized output instructions.
- Annotation Guidelines: Each capability guideline includes a definition, trace indicators, a three-level rubric, and annotated examples for calibration.The abstraction example uses scores 0 for absent, 1 for partially present, and 2 for present.
- Abstraction: Abstraction is defined as extracting general principles from specific instances and applying concepts across contexts.
- Abstraction: Annotators assess abstraction through generalization from examples, concept formation, and movement between concrete and abstract levels.
- Scoring Rubric: The rubric distinguishes absent reasoning focused on concrete details, partial abstraction with inconsistent generalization, and present abstraction applied across domains.
- Output Format: Annotators receive a standardized format requiring an explanation followed by a score from 0 to 2.
A.2 Typology of Problems
The taxonomy classifies cognitive tasks along a continuum from well-structured problems with clear solutions to ill-structured problems with ambiguity and uncertainty.
- Problems range from well-structured tasks with clear goals, known solution paths, and predictable outcomes to ill-structured tasks with ambiguous goals, multiple paths, and uncertain outcomes.
A.2.1 Extension of Jonassen’s Taxonomy
The framework extends Jonassen’s 11-category problem-solving taxonomy with factual recall and creative/expressive tasks, producing 13 categories spanning the datasets’ cognitive demands.
- The taxonomy adds Factual Recall for retrieving stored knowledge without reasoning or problem-solving.
- It adds Creative/Expressive tasks that generate novel content judged by originality or aesthetic quality rather than convergence on a predetermined solution.
- These additions yield a 13-category taxonomy spanning the full spectrum of cognitive demands in the datasets.
A.2.2 Problem Type Definitions
Problem-solving is defined as a goal-directed cognitive activity that transforms an initial state into a desired goal state through systematic reasoning.
- Problem-solving transforms an initial state into a desired goal state through systematic reasoning.
Well-Structured Problems
Well-structured problem types include logical puzzles, algorithmic procedures, story problems, rule-using tasks, and decision-making among finite alternatives.
- Logical problems involve abstract reasoning puzzles with optimal solutions and minimal context.
- Algorithmic problems apply fixed procedures to similar variable sets to produce correct answers through prescribed methods.
- Story problems embed mathematical or scientific tasks in narratives requiring value extraction and formula application.
- Rule-using problems apply procedures constrained by rules while allowing multiple valid approaches to system-constrained answers.
- Decision-making problems require selecting and justifying one option from finite alternatives by weighing benefits and limitations.
Moderately Structured Problems
Accuracy varies substantially with problem structure and model scale: well-structured tasks are generally easier, while diagnosis-solution and other complex tasks expose larger capability differences.
- Performance by Problem Structure: 44.7% average accuracy on Diagnosis-Solution and 46.6% on Design trail well-structured Story Problems at 79.5%.Dilemma is an exception, reaching 82.4% despite being classified as most ill-structured.
- Frontier Model Performance: 88.0% average accuracy makes DeepSeek-R1-671B the performance ceiling across nearly all problem types.Its largest gains over average performance occur on Diagnosis-Solution (+43%), Case Analysis (+41%), and Design (+40%).
- Training Methodology Effects: 81.3% average accuracy from Qwen3-32B approaches frontier performance, while Qwen3-14B and Qwen3-8B score 77.7% and 74.5%.The series uses four-stage reinforcement-learning training, including supervised fine-tuning, reasoning reinforcement learning, thinking-mode fusion, and general reinforcement learning.
- Training Methodology Effects: OpenThinker-32B reaches 75.8% average accuracy using 114K verified examples, compared with DeepSeek’s 800K distillation corpus.The passage attributes this result to automated verification that filters incorrect reasoning traces and produces cleaner training signals.
- Problem Type Variability and Training Robustness: Diagnosis-Solution accuracy ranges from 2.4% to 88.0%, whereas Story Problems range from 57.5% to 92.0%.Smaller distilled models degrade sharply on Diagnosis-Solution but retain reasonable performance on Story Problems.
- Sample Size Considerations: Problem-type representation is highly uneven, making R1-671B’s 100% Dilemma score statistically meaningless with one sample.Creative/Expressive has only seven examples, so results for that category are unreliable.