PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Model Reasoning Failures

Peiyang Song, Pengrui Han, Noah Goodman

arXiv:2602.06176v1cs.AIcs.CLcs.LG

TL;DR

Reasoning failures in LLMs remain fragmented and underexplored despite their broad capabilities. This survey unifies prior work through a two-axis taxonomy and identifies systematic weaknesses, robustness issues, and mitigation directions.

  • Problem

    Research on LLM reasoning failures remains fragmented and underexplored as a unified area, limiting broader understanding of common patterns.

    Source: §1 · p. 2

  • Method

    The survey organizes reasoning failures using a taxonomy of embodied versus non-embodied reasoning and fundamental, application-specific, versus robustness failures.

    Source: §2.1 · p. 2, §1 · p. 2

  • Results

    The survey unifies fragmented observations into a structured framework that identifies failure patterns, underlying causes, mitigation strategies, and persistent robustness concerns.

    Source: §1 · p. 2, §4.2 · p. 9, §6 · p. 13

  • Takeaways & Limitations

    Categorizing reasoning failures provides a foundation for a mature subfield focused on anticipating, detecting, and mitigating LLM reasoning failures.

    Source: §6 · p. 14

    LLM-driven embodied agents remain sensitive to prompt phrasing, vulnerable to adversarial manipulation, and unsafe for some real-world deployments.

    Source: §5.3 · p. 12

Abstract

from arXiv · show

Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, we present the first comprehensive survey dedicated to reasoning failures in LLMs. We introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, we classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, we provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies. By unifying fragmented research efforts, our survey provides a structured perspective on systemic weaknesses in LLM reasoning, offering valuable insights and guiding future research towards building stronger, more reliable, and robust reasoning capabilities. We additionally release a comprehensive collection of research works on LLM reasoning failures, as a GitHub repository at https://github.com/Peiyang-Song/Awesome-LLM-Reasoning-Failures, to provide an easy entry point to this area.

1 Introduction

Although LLMs have achieved impressive results across diverse reasoning tasks, reasoning failures remain underexplored as a unified research area. This survey consolidates fragmented studies, identifies patterns and causes, discusses mitigation strategies, and releases a continuously updated research collection.

Motivation

  • LLMs have achieved significant success across linguistic, mathematical, scientific, and reasoning tasks, while debate remains over whether their reasoning is human-like.The survey explicitly avoids settling this debate and instead focuses on reasoning failures.Source: §1 · pp. 1–2
  • Studying failures can reveal fundamental elements of LLM behavior and provide insights for improvement, paralleling psychological research on learning from human failures.The authors argue that this principle may similarly benefit the study and development of LLMs.Source: §1 · p. 2

Survey contribution

  • The survey unifies fragmented, case-by-case studies of LLM reasoning failures, identifying patterns, analyzing causes, and discussing mitigation strategies to improve reasoning robustness and reliability.It is presented as the first comprehensive survey dedicated to LLM reasoning failures and includes a continuously updated GitHub collection of related research works.Source: §1 · p. 2

2 Definition and Formulation

The survey defines reasoning through embodied and non-embodied forms, with non-embodied reasoning divided into informal and formal reasoning. It categorizes LLM reasoning failures as fundamental, application-specific, or robustness-related, using behavioral evaluation to organize research and mitigation.

2.1 Fundamentals of Reasoning

  • The taxonomy distinguishes reasoning as embodied or non-embodied, with non-embodied reasoning further divided into informal and formal reasoning.This two-axis reasoning distinction structures the survey’s analysis of LLM reasoning failures.Source: §2.1 · p. 2
  • Informal reasoning uses intuitive judgments shaped by biases and heuristics, whereas formal reasoning explicitly manipulates symbols through logic, mathematics, and code.The distinction separates everyday and social decision-making from rule-based symbolic processes.Source: §2.1 · p. 2
  • Embodied reasoning relies on physical interaction, spatial intelligence, and real-time feedback to interpret dynamics and perform goal-directed behaviors under physical laws.It includes predicting physical dynamics and acting within real-world constraints.Source: §2.1 · p. 2

2.2 LLM Reasoning Failures & Common Research Practice

  • Because LLMs remain largely black-box systems, their reasoning abilities are typically assessed behaviorally through outputs on carefully designed prompts and tasks.This practice follows advances in interpretability research but retains behavioral evaluation as the standard approach.Source: §2.2 · p. 2
  • LLM reasoning failures are responses that significantly diverge from expected logical coherence, contextual relevance, or factual correctness.They include decisive poor performance and apparently adequate but unstable performance that reveals hidden vulnerabilities.Source: §2.2 · pp. 2–3
  • Fundamental failures arise broadly from LLM architectures, whereas application-specific limitations are tied to particular domains despite expected model competence.Together with robustness issues, these categories form a mutually consistent framework for classifying failures.Source: §2.2 · p. 3
  • Research commonly progresses from intuitive tests exposing glaring failures to larger systematic evaluations that assess their generality and impact.The survey uses explicit categorization and representative Appendix E examples to unify findings and guide mitigation efforts.Source: §2.2 · p. 3

3 Reasoning Informally in Intuitive Applications

The section surveys LLM failures in informal reasoning, spanning individual cognitive abilities and biases, implicit social understanding, and explicit multi-agent coordination. These failures reflect architectural and training limitations, producing brittle performance and motivating interventions from prompting to structured multi-agent safeguards.

3.1 Individual Cognitive Reasoning

  • LLMs exhibit informal-reasoning failures in individual cognitive skills and biases, including limited working memory, weak inhibition and flexibility, abstract-reasoning deficits, and sensitivity to content and presentation.These failures are linked to self-attention, next-token prediction, text-only learning, training-data biases, architectural predispositions, and alignment processes.Source: §3.1 · pp. 4–5

3.2 Implicit Social Reasoning

  • LLMs’ implicit social reasoning remains brittle: even advanced models struggle with ToM tasks, degrade under minor phrasing changes, and perform poorly on higher-order, emotional, and dynamic conversational reasoning.GPT-4 struggles with classic false-belief, perspective-taking, and unexpected-content tasks, while newer models solve many standard tests but remain unstable under perturbations and weaker than humans on dynamic benchmarks.Source: §3.2 · p. 6
  • LLMs apply social norms and moral values inconsistently across rewordings, languages, generalization, and cultural contexts, creating application-specific safety and robustness risks that surface-level alignment methods do not eliminate.Reported mitigations include prompting, activation steering, and curated fine-tuning, but these remain vulnerable to prompt manipulation and jailbreaks.Source: §3.2 · pp. 6–7

3.3 Explicit Social Reasoning

  • In multi-agent systems, LLMs fail at long-horizon planning, communication and ToM, robustness, adaptability, verification, and termination, causing misinterpretations, strategic misalignment, and cascading errors.Mitigations use belief tracking, structured communication with verification, inspector agents, and context engineering, but remain task-specific and require integrated safety mechanisms.Source: §3.3 · p. 7

4 Reasoning Formally in Logic

This section surveys formal reasoning failures from natural-language logic through benchmark robustness and arithmetic, emphasizing systemic weaknesses, their causes, and mitigation strategies. It shows that failures persist in bidirectional inference, composition, logically equivalent task variants, counting, arithmetic, and mathematical word problems.

4 Reasoning Formally in Logic

  • Formal logic provides rigor for ensuring premises support conclusions, while LLM failures can produce flawed reasoning and harmful decisions across increasingly explicit representations.The section progresses from implicit natural-language structures to symbolic and mathematical formalisms.Source: §4 · p. 7

4.1 Logic in Natural Languages

  • LLMs exhibit reversal curse failures by not inferring “B is A” after learning “A is B,” even when the fact is restated as a question.The failure is attributed to directional training objectives, structural asymmetry, and antecedent-prediction limits; mitigations expose models to bidirectional formulations through augmented data.Source: §4.1 · pp. 7–8
  • LLMs often solve individual facts or problems yet fail to integrate them, with performance worsening for deeper compositions and distractors across reasoning, mathematics, and claim verification.Chain-of-thought prompting makes steps explicit, while graph-structured data, attention-module editing, and related methods aim to improve latent compositional reasoning.Source: §4.1 · p. 8
  • Purpose-built datasets expose targeted weaknesses in converse relations, syllogisms, causal inference, and shallow questions, while separating factual inference from logical entailment adds application-specific complexity.Synthetic symbolic templates and seeded failure expansion support quantitative analysis and direct fine-tuning, although root causes are harder to isolate.Source: §4.1 · p. 8

4.2 Logic in Benchmarks

  • Performance drops under logic-preserving perturbations—such as reordered options, rearranged premises, or edited irrelevant contexts—reveal robustness failures hidden by strong static benchmark scores.These transformations preserve task semantics, so inconsistent results indicate reduced trustworthiness; training diversification and evolving or private datasets are proposed mitigations.Source: §4.2 · pp. 8–9
  • Math and coding benchmarks enable structured perturbations, but small logical changes expose deeper limitations; state-of-the-art reasoning models also face accuracy collapse as Tower of Hanoi complexity increases.Coding tests rename variables, alter control flow, and introduce adversarial changes, while meta-theorem-generated problems challenge even large reasoning models; the Tower of Hanoi finding is disputed on design grounds.Source: §4.2 · p. 9

4.3 Arithmetic & Mathematics

  • Robust counting remains elusive even for reasoning models, reflecting tokenization, positional encoding, and training-data limitations rather than superficial prompting or task framing.The weakness extends to character-level operations and downstream applications; supervised fine-tuning and engaged reasoning help, but the section argues for architectural innovation.Source: §4.3 · pp. 9–10
  • LLMs rely on superficial heuristics in arithmetic, fail as operands grow, and show inconsistent digit reasoning, while math word problems remain difficult when unsolvable, faulty, or requiring process assessment.Proposed remedies include step-by-step data, digit-order reversals, self-improvement, and neuro-symbolic augmentation; current MWP work generally prioritizes broad reasoning methods over individual failures.Source: §4.3 · p. 10

5 Reasoning in Embodied Environments

Embodied reasoning requires perceiving, predicting, and acting within the physical world, yet LLMs lack direct physical grounding and therefore make systematic errors. The survey organizes these failures across text-based, perception-based, and real-world interactive modalities, alongside modality-specific mitigations.

5.1 1D – Text-Based Physical Reasoning Failures

  • Text-based models lack grounded knowledge of object properties, spatial relations, physical laws, and affordances, while also struggling with formal physics and scientific problem-solving.Mitigations include structured physical-knowledge fine-tuning, chain-of-thought prompting, and external code or physics tools for verification and simulation.Source: §5.1 · pp. 10–11

5.2 2D – Perception-Based Physical Reasoning Failures

  • VLMs fail at anomaly detection, counting, overlap, spatial relations, visual physics, and composing basic spatial knowledge in dynamic interactive tasks.Proposed remedies include balanced visual and physics data, spatially grounded attention, reinforcement learning, and external physical simulation.Source: §5.2 · pp. 11–12

5.3 3D – Real-World Physical Reasoning Failures

  • In 3D environments, agents produce infeasible plans, misjudge affordances and spatial relations, fail at tool use, and remain vulnerable to unsafe or prompt-sensitive behavior.Feedback and error handling reduce autoregressive mistakes, while stronger internal world models should improve spatial memory, causal dynamics, and quantitative spatial understanding.Source: §5.3 · pp. 12–13

6 Discussions & Conclusion

The survey’s two-axis taxonomy reveals cross-cutting fundamental, domain-specific, and robustness failures while highlighting gaps in root-cause understanding and evaluation. It calls for persistent, dynamic, interactive benchmarks and sustained failure analysis to build LLMs that fail more gracefully, transparently, and recoverably.

Along the Failure Axis

  • Fundamental failures span reasoning types, whereas application-specific failures cluster by domain and robustness issues arise from inconsistent responses to minor, meaning-preserving perturbations.Examples include reversal and cognitive-bias failures, Theory of Mind instability, mathematical generalization limits, and embodied affordance errors; domain-specific mitigations include physics simulators and symbolic augmentation.Source: §6 · p. 13
  • Perturbation-based stress testing offers a transferable way to detect robustness problems across coding benchmarks, social reasoning, and other domains.Reordering answer options, renaming code variables, and paraphrasing moral dilemmas can produce large, inconsistent output shifts.Source: §6 · p. 13

Suggestions for Future Directions

  • Root-cause analyses remain incomplete for compositional reasoning, higher-order ToM, physical commonsense, and multi-agent planning failures, requiring links between behavior and internal mechanisms.Suggested mechanisms include faulty attention-head coordination and insufficient intermediate-representation alignment.Source: §6 · p. 13
  • Unified persistent benchmarks, failure injection, private or evolving suites, and event-driven evaluations could broaden coverage, track failure persistence, and reduce short-term overfitting.The survey cites Malek et al. (2025), Phan et al. (2025), Rajore et al. (2024), Zhang et al. (2024d), Jain et al. (2024a), White et al. (2024), Zheng et al. (2025), and AIMO 2 as examples.Source: §6 · p. 13

A Broad Picture

  • Because interactive and multi-turn settings remain underrepresented, broader benchmarks and sustained failure analysis are needed to support resilient LLMs that fail gracefully, transparently, and recoverably.The survey frames failure categorization as foundational to anticipating, detecting, and mitigating reasoning failures, paralleling fault-tolerance and safety-critical incident analysis.Source: §6 · pp. 13–14

A Taxonomy

The survey organizes LLM reasoning failures along two complementary axes: reasoning type and failure type, with dedicated taxonomies for informal, formal, and embodied reasoning.

Overview

  • The visualized taxonomy clarifies the survey’s overall structure and provides an accessible introduction to LLM reasoning failures.The taxonomy directly reflects how the survey divides its categories.Source: A Taxonomy · p. 47

Reasoning-type taxonomy

  • Figure 2 divides reasoning failures into embodied and non-embodied settings, with non-embodied failures split into informal, instinct-based reasoning and formal, logic-based reasoning.The survey provides dedicated taxonomies for informal, formal, and embodied reasoning in Figures 3, 4, and 5, respectively.Source: A Taxonomy · p. 47

Failure-type taxonomy

  • A secondary axis classifies failures into three failure types, supporting additional analysis and more nuanced discussion of the field’s complexity.The taxonomy is described as complete and mutually exclusive on each axis.Source: A Taxonomy · p. 47

B Artifacts

The paper releases a comprehensive, continuously updated collection of research on LLM reasoning failures as a GitHub repository, providing an accessible entry point for future research.

B Artifacts

  • The authors release a comprehensive research collection on LLM reasoning failures as a continuously updated GitHub repository, intended to provide an easy entry point and facilitate future research.Repository: https://github.com/Peiyang-Song/Awesome-LLM-Reasoning-FailuresSource: B Artifacts · p. 47

C Other Emerging Areas of Reasoning

Emerging reasoning research is expanding toward diverse media, general-purpose frameworks, and formally verifiable systems, but systematic analyses of generalizable failures remain limited. The survey argues that applying failure-analysis methods as these areas mature can support more reliable reasoning and deployment.

Emerging Areas

  • Because these research areas are nascent, systematic investigations of generalizable failure modes remain limited, motivating early efforts to apply the survey’s failure-analysis methodology.The methodology is expected to become increasingly valuable as emerging reasoning fields mature.Source: C Other Emerging Areas of Reasoning · p. 47

Toward Broad Applications: Reasoning in Diverse Media

  • Multimodal progress is enabling emerging paradigms including video, audio, and music reasoning, whose future failure analyses may follow established language-vision research patterns.These domains remain in early foundational stages, with systematic failure analysis expected to guide later incremental improvements.Source: C Other Emerging Areas of Reasoning · p. 50

Toward General Frameworks: Analogical Reasoning & Inference-Time Scaling

  • Inference-time scaling generates intermediate thoughts, while analogical frameworks retrieve and reuse prior examples to support deeper, more systematic problem solving.The survey calls for evaluating whether these frameworks mitigate established reasoning failures and improve robust real-world deployment.Source: C Other Emerging Areas of Reasoning · pp. 50–51

Toward Verifiable Reasoning: Formal Math and Science Validations

  • Formal validation grounds LLM reasoning in proof assistants, filtering outputs for correctness and reducing hallucinations across mathematics, science, and safety-critical verification.Applications include theorem proving, auto-formalization, proof search, agentic tools, automated conjecturing, and software and hardware verification.Source: C Other Emerging Areas of Reasoning · p. 51

D Other Important LLM (Non-Reasoning) Failures

Beyond reasoning failures, LLMs exhibit serious trustworthiness, fairness, and safety limitations that warrant unified investigation and mitigation. These include hallucinations, social biases, security and privacy vulnerabilities, and unreliable watermarking.

Scope and Future Directions

  • Because these non-reasoning failures affect general reliability, the survey calls for systematic research to identify, mitigate, and improve them beyond its primary scope.Improving grounding, uncertainty estimation, fairness, security, privacy, and watermarking remains important for responsible real-world deployment.Source: D Other Important LLM (Non-Reasoning)… · pp. 51–52

Trustworthiness: Hallucinations & Over-Confidence in Generations

  • Hallucinations produce fluent, confident, but factually incorrect or fabricated outputs, especially threatening accuracy-critical legal, scientific, and medical applications.Retrieval augmentation grounds generation in external sources, while calibration aligns expressed confidence with correctness, but both approaches remain imperfect.Source: D Other Important LLM (Non-Reasoning)… · p. 51

Fairness: Harmful Ethical & Social Biases

  • LLMs inherit harmful social and ethical biases from human-generated data, producing unfair outputs in high-stakes recruitment, healthcare, and law-enforcement settings.Benchmarking, mitigation, and regulation have advanced, yet adversarial prompts, new modalities, and implicit representations continue to complicate debiasing.Source: D Other Important LLM (Non-Reasoning)… · pp. 51–52

Safety: AI Security, Privacy & Watermarking

  • LLMs remain vulnerable to adversarial attacks, prompt injections, and private-data leakage, while watermarking struggles against removal attacks and can reduce output quality and fluency.Safer deployments therefore require more secure, privacy-preserving architectures and improved methods for detecting generated content.Source: D Other Important LLM (Non-Reasoning)… · p. 52

E Examples

This section presents representative examples and case studies showing how the survey’s LLM reasoning failures manifest across informal, formal, and embodied reasoning. The examples are organized by the survey’s sections and subsections in tables, with concrete demonstrations of instability, misunderstanding, perception errors, action failures, and safety risks.

Examples and Case Studies

  • The examples are organized in tables by survey section and subsection to make each reasoning failure concrete.They cover informal reasoning, formal reasoning, and embodied reasoning failures.Source: E Examples · p. 52

Formal Reasoning

  • LLMs fail to grasp mathematical problem structure and become unstable when irrelevant context is added.They may solve one numerically varied instance but fail another, or solve an original problem and fail its irrelevant-context variant.Source: E Examples · p. 60
  • Subtle code changes expose robustness failures, while models may prefer statistically common but incorrect continuations over unusual correct ones.Failures arise after changing a docstring or function name, and when builtin functions are swapped in a Python prompt.Source: E Examples · p. 61
  • Faulty mathematical assumptions can produce unreasonable answers from non-reasoning models and endless thinking from reasoning models.The models fail to identify the issue in the original mathematical word problem.Source: E Examples · p. 62

Embodied Reasoning

  • Embodied examples show failures in physical perception, including ignoring anomalous scenes, failing simple visual tests, and misgrounding spatial language.BLIP-2 captions a skater on parquet as being on an ice rink; models also fail basic visual and light physical-scene understanding, while “cylinder” remains grounded to the upright cylinder despite prompt context.Source: E Examples · pp. 64, 66
  • Embodied systems also exhibit action-sequencing, distance-estimation, and safety failures, including wrong or missing steps, uncertain spatial judgments, and harmful jailbreak behaviors.Trajectory errors include wrong order, missing steps, affordance errors, and additional steps; examples also include estimating whether a robot fits between furniture and recording someone showering or stealing private information.Source: E Examples · p. 67

Related papers

Loading 2602.06176v1…