PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Large Language Models for Mathematical Reasoning

Peng-Yuan Wang, Tian-Shuo Liu, Chenyang Wang, Yi-Di Wang, Shu Yan, Cheng-Xing Jia, Xu-Hui Liu, Xin-Wei Chen, Jia-Cheng Xu, Ziniu Li, Yang Yu

arXiv:2506.08446v1cs.AIcs.CL

TL;DR

Mathematical reasoning remains a demanding frontier, and LLM capabilities have advanced through better comprehension and answer generation. This survey organizes those advances across pretraining, prompting, fine-tuning, and inference-time scaling, while identifying persistent challenges in capacity, efficiency, and generalization.

  • Problem

    Mathematical reasoning demands abstract understanding and formal solution construction, while robust capacity, efficiency, and generalization remain challenging for LLMs.

  • Method

    The survey organizes LLM mathematical reasoning into comprehension and generation, then reviews pretraining, prompting, fine-tuning, test-time scaling, self-improvement, and future directions.

  • Results

    83.9% on AIME 2024 placed Grok 3 Beta within the top 2.5% of nationwide participants, while the survey synthesizes broader progress across reasoning methods.

  • Takeaways & Limitations

    CoT is presented as enabling structured procedural reasoning, while extended CoT and test-time scaling support more comprehensive reasoning and correction of inconsistencies.

  • Takeaways & Limitations

    Robust generalization of mathematical meta-skills to complex open-ended domains remains unresolved.

Abstract

from arXiv · show

Mathematical reasoning has long represented one of the most fundamental and challenging frontiers in artificial intelligence research. In recent years, large language models (LLMs) have achieved significant advances in this area. This survey examines the development of mathematical reasoning abilities in LLMs through two high-level cognitive phases: comprehension, where models gain mathematical understanding via diverse pretraining strategies, and answer generation, which has progressed from direct prediction to step-by-step Chain-of-Thought (CoT) reasoning. We review methods for enhancing mathematical reasoning, ranging from training-free prompting to fine-tuning approaches such as supervised fine-tuning and reinforcement learning, and discuss recent work on extended CoT and "test-time scaling". Despite notable progress, fundamental challenges remain in terms of capacity, efficiency, and generalization. To address these issues, we highlight promising research directions, including advanced pretraining and knowledge augmentation techniques, formal reasoning frameworks, and meta-generalization through principled learning paradigms. This survey tries to provide some insights for researchers interested in enhancing reasoning capabilities of LLMs and for those seeking to apply these techniques to other domains.

1 Introduction

Mathematical reasoning is a demanding benchmark for AI because it requires symbolic manipulation, abstract understanding, formal argument construction, and transfer across contexts. This survey reviews LLM progress through comprehension, generation, and methods including prompting, fine-tuning, and test-time scaling.

  • Mathematical problem-solving rigorously evaluates machine reasoning through symbols, abstract concepts, formal arguments, and transfer to varied contexts.
  • Semantic parsing mapped problem text to logical forms but relied heavily on human engineering and struggled with diverse mathematical tasks.
  • 83.9% on AIME 2024 placed Grok 3 Beta within the top 2.5% of nationwide participants.
  • LLM mathematical reasoning combines comprehension of mathematical concepts with step-by-step solution generation.
  • The survey reviews mathematical reasoning advances, covering LLM background, enabling elements, prompting, fine-tuning, test-time scaling, self-improvement, limitations, and future directions.

2 Background

LLMs are trained through pre-training on broad corpora and adapted through supervised fine-tuning and reinforcement learning. Their reasoning can also be elicited with prompting methods that progress from zero-shot instructions to structured multi-path search.

  • 2.1 Pre-Training: Pre-training typically optimizes next-token prediction over large corpora to develop general language understanding and domain knowledge.
  • 2.1 Pre-Training: High-quality, diverse data and filtering are important because corpus composition shapes reasoning, comprehension, and generalization abilities.
  • 2.2 Fine-Tuning: Supervised Fine-Tuning: Supervised fine-tuning adapts LLMs to downstream tasks or human instructions by training on instructional inputs and desired outputs.
  • 2.2 Fine-Tuning: Supervised Fine-Tuning: SFT trains models to mimic golden responses, while augmentation, synthesis, and distillation can improve dataset quality, diversity, fluency, consistency, and efficiency.
  • 2.3 Fine-Tuning: Reinforcement Learning: Reinforcement learning refines model behavior through reward signals, including human-preference feedback, when SFT alone may not ensure alignment, factuality, or reduced harmful behavior.
  • 2.3 Fine-Tuning: Reinforcement Learning: In the LLM MDP formulation, states are generated token sequences, actions are next-token selections, transitions append tokens, and rewards provide scalar feedback.
  • Prompting: Prompting ranges from zero-shot and few-shot instructions to Chain-of-Thought prompts that elicit intermediate reasoning steps.
  • Prompting: Tree-of-thoughts and graph-of-thoughts represent reasoning as searchable structures, supporting exploration of solution paths and flexible backtracking.

3 LLMs’ Mathematical Reasoning from Comprehension and Generation

The survey organizes LLM mathematical reasoning into comprehension and answer generation: models acquire mathematical knowledge from diverse data, then generate solutions through increasingly extended Chain-of-Thought reasoning. It reviews the progression from manually engineered solvers and direct generation toward scalable training, multimodal inputs, and test-time reasoning, while noting unresolved limitations in CoT reliability.

  • Comprehension and answer generation: The framework separates mathematical reasoning into comprehension of mathematical structures and answer generation through direct prediction or step-by-step CoT.Comprehension spans arithmetic, geometry, theorem formalisms, and textual or visual representations.
  • Comprehension and answer generation: Earlier mathematical solvers relied on manually written rules, pattern matching, or semantic parsing, limiting coverage and scalability across mathematical tasks.These approaches depended heavily on human intervention and handled only predefined scenarios or struggled with task diversity.
  • Comprehension: Large-scale mathematical pretraining and curated datasets expose LLMs to diverse problem-solving patterns, supporting mathematical comprehension and generalization.Examples include OpenWebMath with 14.7B tokens and MathPile with approximately 9.5B tokens; data distribution adjustments can also improve higher-level reasoning.
  • Comprehension: Multimodal inputs add images, audio, and video that can provide essential reasoning information while reducing reliance on verbose textual descriptions.The passage also notes that multimodal data introduce their own challenges, though the supplied text does not specify them.
  • Answer generation: CoT prompting generates intermediate reasoning steps, and long CoT extends this process through iterative exploration, self-reflection, multiple solution paths, and backtracking.Test-time scaling supports inconsistency detection and corrective measures during inference.
  • Answer generation: Despite performance gains, generated CoT can contain unreliable intermediate reasoning: competition-level tasks show process error rates reaching 51.8%, and correct answers may coexist with flawed steps.The survey therefore examines CoT mechanisms from theoretical and empirical perspectives while identifying reliability as an unresolved issue.

4 Methods for Boosting Reasoning

The survey organizes reasoning improvements around pretraining, supervised fine-tuning, reinforcement learning, and extended Chain-of-Thought generation. These methods improve instruction following, exploration, reward-guided reasoning, and long-form solution generation, while DPO retains important limitations under distribution shift.

  • Fine-tuning improves pretrained models’ instruction-following and generation capabilities, addressing their difficulty producing contextually appropriate responses.
  • Supervised Fine-Tuning: High-quality supervised demonstrations provide structural priors that constrain generation and support effective exploration during later reinforcement learning.
  • Supervised Fine-Tuning: Synthetic data from strong LLMs, program-based rationales, generated questions, and error-correction examples broaden training signals for mathematical reasoning.
  • Extended CoT: Extended Chain-of-Thought training and inference improve reasoning accuracy by allocating additional computational resources to longer reasoning sequences.
  • Reinforcement Learning: Rule-based rewards provide deterministic, task-specific feedback that supports stable reinforcement learning and mitigates reward hacking.
  • Reinforcement Learning: Long-CoT reinforcement learning uses rule-based rewards to enhance extended reasoning, with DeepSeek R1 demonstrating comparable or superior performance to OpenAI’s o1.
  • Direct Preference Optimization: DPO offers computational efficiency but lacks exploration, is sensitive to SFT quality, and can generalize poorly under distribution shifts.Reported out-of-distribution accuracy drops reached 7% compared with explicit reward models.

4.3 Test Time Inference with Structural Search

Test-time scaling enhances mathematical reasoning by increasing generation and search effort. The survey contrasts majority voting with structural tree search and connects self-improvement, external tools, and retrieval to more reliable reasoning and verification.

  • Test-Time Scaling: Increasing test-time generation burden improves mathematical reasoning, while complex problems with large token budgets motivate tree search beyond majority voting.
  • Structural Search: Tree-of-Thoughts integrates depth-first and breadth-first search to maintain and explore multiple candidate reasoning paths.
  • Structural Search: Monte Carlo Tree Search addresses efficiency limits of heuristic search by constructing trees whose nodes represent reasoning states and exploring promising paths.
  • Structural Search: LLM-based MCTS systems use language models as world models or reasoning agents, while value models provide process supervision and step-level evaluation.
  • Self-Improvement: Self-improvement methods generate and select higher-quality responses through rationale generation, self-consistency, verification, and process-reward guidance.
  • External Knowledge: External tools and retrieval-augmented generation address computational, factual, and real-time information limits by supplying executable operations and relevant knowledge.
  • External Knowledge: Empirical studies report that retrieved knowledge improves proof accuracy, faithfulness, grounding, and mathematical word-problem performance.

5 Discussions

The survey identifies important gains and boundaries in mathematical reasoning, showing that CoT and reinforcement learning can organize or expose existing capabilities but face theoretical, training, evaluation, and model-capacity constraints.

  • Reasoning capabilities: Chain-of-Thought reasoning can teach models structured procedures through step-by-step decomposition, positioning it as a blueprint for algorithmic problem-solving.Evidence spans navigation, web-agent, and other multi-step tasks.
  • Reasoning capabilities: CoT expressiveness is bounded by the number of intermediate steps, while looped transformers offer a possible efficiency-oriented alternative.Logarithmic, linear, and polynomial step budgets correspond to different computational capabilities.
  • Training paradigms: Supervised fine-tuning improves instruction following but can reduce generation diversity, limiting exploration for downstream reinforcement learning.Direct RL may preserve exploratory capabilities and improve reasoning-task performance without SFT.
  • Training paradigms: Process reward models outperform outcome reward models, but high-quality reward signals remain costly and automated annotation remains vulnerable to reward hacking.Automated methods reduce annotation costs but struggle to produce precise reward scores.
  • Limits of improvement: Vanilla RL tends to improve Pass@1 without changing Pass@k, whereas long-CoT RL improves Pass@k by exploring more reasoning pathways.The reported distinction links longer generated sequences with broader response exploration.
  • Limits and future directions: Across studies, RL primarily activates latent base-model abilities rather than creating fundamentally new reasoning capacity, making pretraining and architecture central targets for future progress.Structural search adds systematic exploration and backtracking, but base-model capabilities remain a fundamental constraint.

6 Potential Directions in Mathematical Reasoning for LLMs

The survey identifies capacity, efficiency, and generalization as central challenges for mathematical reasoning in LLMs. It highlights reinforcement-learning exploration, knowledge augmentation, verification-aware optimization, and open-domain generalization as key directions.

  • LLMs still face challenges in extending mathematical reasoning performance, improving efficiency, and generalizing beyond mathematics.The section frames these as three critical research directions.
  • Extending LLM performance boundaries: Long-CoT reinforcement learning advances multi-step mathematical reasoning, but current implementations remain below theoretical performance ceilings.DeepSeek-R1 illustrates the potential of long-CoT reinforcement learning for competition-level mathematics and complex logical derivations.
  • Enhancing reasoning efficiency: Compact latent action spaces may make reinforcement-learning exploration more efficient while preserving computational tractability.BWArea and CoLA use future information to infer latent actions in lower-dimensional spaces for more comprehensive information gathering.
  • Knowledge-augmented reasoning: External tools and retrieval-augmented generation can augment models on problems beyond their parametric knowledge, including open problems and mathematical conjectures.A major challenge is learning when and how to invoke calculators, code interpreters, geometric visualizers, and other resources.
  • Optimality of reasoning paths: Outcome-based rewards can accept correct answers despite sub-optimal or logically flawed reasoning paths.Verification-aware optimization with formal systems such as Lean and intermediate-step tool validation is proposed to address this limitation.
  • Reasoning generalization to open domains: Mathematical cross-domain transfer does not yet ensure robust generalization to open-ended domains because triggering and using learned meta-skills remains unresolved.CoT prompting offers limited task adaptation but may leave models reliant on surface-level statistical correlations rather than foundational reasoning principles.
  • Reward modeling: Reward model scaling and generative reward models are proposed as potential routes toward more generalizable higher-order reasoning.The proposed approaches aim to address the difficulty of designing nuanced reward functions outside mathematics' clearer binary feedback.

7 Conclusion

The survey unifies research on mathematical reasoning in LLMs around comprehension and solution generation, covering methods from prompting through inference-time scaling. It identifies efficient exploration, reasoning-path quality, and open-domain generalization as the field’s central challenges and points to corresponding research directions.

  • The survey organizes mathematical reasoning around comprehension of problems and generation of solutions.It examines advances from training-free prompting to fine-tuning and inference-time scaling.
  • Three frontier challenges are efficient exploration under constrained RL sampling budgets, sub-optimal reasoning trajectories, and limited transfer from mathematics to open domains.These challenges define the survey’s account of the current research frontier.
  • Promising directions include compact representation spaces, verification-aware optimization, and scalable reward modeling.The survey presents these as responses to the identified exploration, trajectory-quality, and generalization challenges.
  • The survey aims to serve researchers and newcomers by clarifying achievements, limitations, and future directions in mathematical reasoning for AI.It positions mathematical reasoning as a benchmark for evaluating genuine understanding in the pursuit of AGI.

A Mathematical Reasoning Evaluation Benchmarks

The survey frames mathematical-reasoning evaluation as an assessment of capabilities relative to human levels. It proposes categorizing datasets by corresponding human levels to support targeted evaluation of different reasoning abilities.

  • Mathematical-reasoning benchmarks are intended to assess whether models reach or exceed comparable human capabilities.The evaluation framework focuses on capabilities relevant to mathematical reasoning.
  • Categorizing datasets by human level enables targeted evaluation of distinct aspects of mathematical reasoning.The passage presents human-level categorization as an organizing principle for benchmark assessment.

A.1 Elementary Level

Elementary-level benchmarks cover foundational mathematical problem solving across arithmetic, algebra, proportional reasoning, and related topics. The section describes datasets spanning English, Chinese, and broader STEM-oriented tasks.

  • MAWPS contains 3,320 elementary math word problems covering basic arithmetic, algebra, and proportional reasoning.It is presented as an online repository for evaluating models on elementary word problems.
  • ASDiv-A consists mainly of elementary-level English word problems spanning arithmetic, algebra, and basic geometry.The dataset covers several foundational school mathematics topics.
  • SVAMP builds on ASDiv-A and contains elementary word problems typically taught in grade four or lower.Its problems target early-grade mathematical reasoning.
  • Math23K contains 23,161 Chinese elementary word problems for evaluating translation from natural language into mathematical expressions and equations.The dataset targets mathematical reasoning and problem-solving capabilities.
  • MMLU contains 231,400 entries, including STEM tasks in mathematics, physics, and computer science.It extends beyond elementary mathematics to broader STEM evaluation.
  • CMATH contains 1.7k elementary-school math word problems with detailed annotations.The passage identifies it as a Chinese elementary-level dataset.

A.2 Middle/High School Level

Middle/high-school benchmarks evaluate LLM mathematical reasoning on grade-school problems, especially multi-step problems involving numbers, operations, and basic concepts.

  • GSM8K contains 8.5K linguistically diverse grade-school word problems for evaluating multi-step mathematical reasoning.The benchmark targets understanding and manipulation of numbers, operations, and basic mathematical concepts.

A.3 University Level

University-level benchmarks cover operational reasoning, code synthesis, undergraduate mathematics, theorem application, and formal theorem proving. They range from natural-language questions to formal proofs and include tests of generalization to novel theorems.

  • MathQA annotates AQuA-RAT with fully specified operational programs, while MathQA-Python evaluates code synthesis for mathematically complex natural-language problems.
  • TheoremQA assesses whether models can apply domain-specific theorems to solve complex problems.
  • LeanDojo evaluates theorem-proving generalization to novel theorems and contains 98,734 theorems and proofs extracted from mathlib.Its coverage is primarily at the university level.
  • miniF2F contains 488 Olympiad-level formal mathematics problems for evaluating neural theorem-proving capabilities.

A.4 Competition Level

Competition-level benchmarks evaluate advanced mathematical reasoning across topics and difficulty levels, including competition mathematics, Olympiad problems, and the AIME.

  • MATH contains 12,500 competition-level problems spanning algebra, geometry, calculus, and number theory.
  • miniF2F provides an Olympiad-level benchmark for formal mathematics.
  • AIME24 contains 30 problems from the 2024 American Mathematics Invitational for evaluating performance on complex mathematical problems.
  • OlympiadBench is listed as another competition-level mathematical reasoning benchmark.
Loading 2506.08446v1…