Source-linked AI summary
Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, Zhenru Zhang
TL;DR
Mathematical reasoning models can be improved through specialized data and training strategies rather than model scaling alone. Qwen2.5-Math applies self-improvement across pre-training, post-training, and inference, and reports strong bilingual CoT and TIR benchmark performance. Its flagship instruct model improves over Qwen2-Math-Instruct across English and Chinese evaluations.
Problem
The report addresses how to automatically generate reliable CoT and TIR supervision and use it effectively for SFT and reinforcement learning.
Method
Qwen2.5-Math uses synthesized mathematical pre-training data, iterative reward-model-guided SFT and reinforcement learning, and bilingual CoT/TIR data.
Results
Qwen2.5-Math-Instruct improves over Qwen2-Math-Instruct across English and Chinese benchmarks; Qwen2.5-Math-72B-Instruct gains 4.4 English points and 6.1 Chinese points on average.
Takeaways & Limitations
The reported results show that improved training data and strategy can provide an alternative pathway to gains beyond simply increasing parameter scale.
Abstract
from arXiv · showhide
In this report, we present a series of math-specific large language models: Qwen2.5-Math and Qwen2.5-Math-Instruct-1.5B/7B/72B. The core innovation of the Qwen2.5 series lies in integrating the philosophy of self-improvement throughout the entire pipeline, from pre-training and post-training to inference: (1) During the pre-training phase, Qwen2-Math-Instruct is utilized to generate large-scale, high-quality mathematical data. (2) In the post-training phase, we develop a reward model (RM) by conducting massive sampling from Qwen2-Math-Instruct. This RM is then applied to the iterative evolution of data in supervised fine-tuning (SFT). With a stronger SFT model, it's possible to iteratively train and update the RM, which in turn guides the next round of SFT data iteration. On the final SFT model, we employ the ultimate RM for reinforcement learning, resulting in the Qwen2.5-Math-Instruct. (3) Furthermore, during the inference stage, the RM is used to guide sampling, optimizing the model's performance. Qwen2.5-Math-Instruct supports both Chinese and English, and possess advanced mathematical reasoning capabilities, including Chain-of-Thought (CoT) and Tool-Integrated Reasoning (TIR). We evaluate our models on 10 mathematics datasets in both English and Chinese, such as GSM8K, MATH, GaoKao, AMC23, and AIME24, covering a range of difficulties from grade school level to math competition problems.
1 INTRODUCTION
Qwen2.5-Math develops math-specific models through self-improvement across pre-training, post-training, and inference, with bilingual CoT and TIR capabilities. Across English and Chinese benchmarks, the models improve substantially over earlier Qwen systems and achieve strong results across model sizes.
- Self-improvement pipeline: Qwen2.5-Math applies self-improvement during pre-training, post-training, and inference to develop math-specific language models.Qwen2-Math-Instruct synthesizes pre-training data, reward models guide iterative SFT and reinforcement learning, and reward-guided sampling is used during inference.
- Capabilities: Qwen2.5-Math-Instruct adds Chinese support and Tool-Integrated Reasoning data to the English CoT data used for Qwen2-Math-Instruct.This expands the training strategy beyond English chain-of-thought reasoning.
- Evaluation: 91.6, 55.4, and 57.6 are Qwen2.5-Math-7B base scores on GSM8K, MATH, and GaoKao Math Cloze, exceeding Qwen2-72B scores of 89.5, 51.1, and 55.9.The comparison is reported on the same three benchmarks.
- Evaluation: 66.8 on MATH makes Qwen2.5-Math-72B base state of the art, improving 5.3 points over Qwen2-Math-72B and 15.7 over Qwen2-72B.The reported result concerns the MATH benchmark.
- Evaluation: The 72B instruct model improves over Qwen2-Math-72B-Instruct by average margins of 4.4 English points and 6.1 Chinese points.In TIR mode, the 72B model approaches 90 on MATH while the 1.5B model scores around 80.
2 QWEN2.5-MATH PRE-TRAINING
Qwen2.5-Math pre-training centers on a curated mathematical corpus assembled from diverse sources and improved through recall, filtering, synthesis, deduplication, and mixture optimization. Qwen2-Math-Instruct then supplies additional high-quality data, especially Chinese data, for the larger Qwen Math Corpus v2.
- Corpus construction: The Qwen Math Corpus v1 combines math web texts, code, encyclopedias, exam questions, and synthetic mathematical data.Its construction includes data recall, deduplication, filtering, synthesis, and optimization of the data mixture.
- Motivation: General language models’ weak mathematical reasoning is attributed to insufficient mathematical data during pre-training.The pipeline therefore begins by recalling mathematical data from web sources such as Common Crawl.
- Data quality: A language-model-based filter scores candidate mathematical entries so higher-quality data are prioritized for inclusion.Qwen2-0.5B-Instruct is augmented with prompt engineering for this evaluation.
- Pre-training: 700 billion tokens comprise Qwen Math Corpus v1, used to continuously pre-train Qwen2-Math-1.5B/7B/72B with 4K context length.The corpus mixture was selected after ablation studies using Qwen2-Math-1.5B.
- Corpus upgrade: Qwen2-Math-72B-Instruct synthesizes additional high-quality data for Qwen Math Corpus v2, alongside more recalled data with particular emphasis on Chinese sources.Corpus v2 is used for Qwen2.5-Math-1.5B/7B/72B pre-training.
3 QWEN2.5-MATH POST-TRAINING
Qwen2.5-Math post-training targets reliable CoT and TIR supervision through iterative response generation, rejection sampling, reward modeling, and reinforcement learning. The reward model supplies granular reasoning-quality signals for SFT data selection and later RL training.
- Objectives: Post-training addresses automatic generation of reliable CoT and TIR annotations and their use in SFT and reinforcement learning.The two target capabilities are step-by-step natural-language reasoning and external-tool use for complex mathematical or algorithmic tasks.
- CoT data: The CoT dataset contains 580K English and 500K Chinese mathematical problems from annotated and synthesized sources.Chinese K-12 problem collections are added to strengthen Chinese reasoning capabilities.
- CoT response construction: Iterative rejection sampling expands candidate reasoning paths, retaining correct top-k paths for annotated problems and using weighted majority voting for synthesized problems.The current best model generates multiple pathways at each iteration.
- TIR motivation: CoT alone faces computational-accuracy and complexity challenges, including quadratic roots and matrix eigenvalues, motivating tool-integrated reasoning.TIR is intended to improve precise calculation, symbolic manipulation, and algorithmic reasoning.
- TIR data: The TIR dataset contains 190K annotated and 205K synthesized problems from mathematical benchmark and generated sources.Its response construction uses online rejection fine-tuning, deduplication, and majority voting.
- Reward modeling: Qwen2.5-Math-RM trains on 361K English and 257K Chinese problems, each paired with 6 responses, to support Chinese and TIR reasoning.The reward models provide supervision beyond final answers for SFT selection and subsequent RL training.
- Reward modeling: Responses are labeled positive or negative by final-answer correctness, while all-correct and all-incorrect cases are filtered out.The reward model is initialized from the SFT model and replaces its language-modeling head with a scalar-value head.
- Reinforcement learning: GRPO uses the average reward of sampled response groups as the advantage baseline and avoids additional value-function approximation.The sequence-level advantage is applied to each token in the response.
4 DECONTAMINATION
The decontamination procedure removes potentially overlapping training samples before evaluation. It combines normalized 13-gram matching with a longest-common-subsequence threshold to identify contamination.
- Decontamination: Potentially contaminated training samples are excluded using 13-gram matching, following prior work.Text normalization removes irrelevant punctuation and symbols before matching.
- Decontamination: A sample is considered contaminated when its longest-common-subsequence ratio exceeds 0.6.This additional criterion targets false negatives, particularly for common mathematical expressions.
5 EVALUATION
The evaluation covers English and Chinese mathematical benchmarks, including widely used datasets and more challenging examination and competition problems. Base-model results show strong performance across both languages and multiple difficulty levels.
- The evaluation includes English benchmarks such as GSM8K and MATH, alongside Chinese benchmarks including CMATH, GaoKao Math Cloze, and GaoKao Math QA.
- Qwen2.5-Math-1.5B outperforms all specific baselines on GSM8K, MATH, CMATH, GaoKao Math Cloze, and GaoKao Math QA.
- Qwen2.5-Math-7B scores 91.6 on GSM8K and 55.4 on MATH, exceeding Qwen2-72B and Llama-3.1-405B on both reported benchmarks.
- Qwen2.5-Math-72B achieves state-of-the-art results on MATH, CMATH, GaoKao Math Cloze, and GaoKao Math QA, including 66.8 on MATH.
BENCHMARK EN ZH GSM8K MATH MMLU
Across English, Chinese, and competition benchmarks, Qwen2.5-Math-Instruct improves over earlier Qwen2-Math models and benefits from reward-model sampling and tool-integrated reasoning. The reported results span standard benchmarks through AIME 2024 and AMC 2023.
- English benchmarks: English evaluations report CoT, PoT, majority-voting, and reward-model best-of-N results across multiple instruct-model benchmarks.
- English benchmarks: Qwen2.5-Math-Instruct-1.5B and 7B achieve results comparable to Qwen2-Math-Instruct-7B and 72B, respectively, in traditional CoT mode.
- English benchmarks: Qwen2.5-Math-72B-Instruct averages 2.5 points ahead of the current best model and 6.2 points higher than GPT-4o on English benchmarks.
- Chinese benchmarks: Chinese-specific post-training data substantially improves Chinese performance, with Qwen2.5-Math-72B-Instruct exceeding GPT-4o by 17.5 points.
- Chinese benchmarks: In Chinese benchmarks, RM@N consistently surpasses Maj@N, whereas TIR does not show a significant advantage over CoT.
- Competition benchmarks: On AMC 2023, Qwen2.5-Math-1.5B-Instruct solves 29 of 40 problems using RM@256 in CoT mode, outperforming NuminaMath-72B CoT.
- Competition benchmarks: On AIME 2024, Qwen2.5-Math-72B-Instruct solves 9 problems with greedy CoT and 12 with TIR, while Qwen2.5-Math-7B-Instruct solves up to 21 with RM support.
6 CONCLUSION
The report concludes that Qwen2.5-Math combines synthesized data, reward-model-guided training, bilingual support, and CoT/TIR capabilities. It also includes several mathematical problem statements and solution claims.
- Qwen2.5-Math uses synthesized mathematical data, iterative reward-model-guided fine-tuning and reinforcement training, and bilingual CoT/TIR support.
- The report presents Qwen2.5-Math as an advanced open-source math model series, with 1.5B and 7B instruct models compared against larger predecessors.
- The IMO problem asks for the smallest positive integer t for which a sum of cubes equals 20022002.
- For the digit-sum polynomial problem, the stated conclusion is that s(k) and s(P(k)) cannot have the same parity for every positive integer k.
- The supplied functional-equation solution claims that the only function satisfying the positive-real condition is f(x)=1/x.
Qwen2-Math-72B-Instruct Solution
The supplied solution considers integer-valued functions satisfying a two-variable functional equation and concludes that all solutions are linear functions of the form f(x)=kx.
- Step 1: Initial Substitution: Initial substitutions yield f(3y)=f(y)+f(2y) and f(4x)=f(3x)+f(x).
- Step 4: Generalizing the Function: The solution hypothesizes a linear form f(x)=kx for an integer k.
- Verification: Substitution verifies that f(x)=kx makes both sides equal for all x and y.
- Conclusion: The conclusion states that the only solutions are functions of the form f(x)=kx for integer k.
Problem From Baltic Way 2023
The supplied passages contain multiple unrelated functional-equation materials and a Martian population problem, rather than a complete coherent section for the named Baltic Way problem.
- One functional equation is presented over the reals, with the displayed relation involving f(f(x)), xf(y), f(xy+y), and f(x).
- The supplied derivation obtains f(c+y)=f(y)+c after setting c=f(0).
- The stated conclusion for that equation is that the only solutions are f(x)=0 and f(x)=x.
- A separate Martian problem asks whether two target populations can be reached from two survivors of each colour using the transformation machine.
Qwen2-Math-72B-Instruct Solution
The solutions analyze three problems using parity, move numbering, and combinatorial graph arguments. They conclude that the Martian target in part (a) is possible, part (b) is impossible, Penny wins the stone game, and k=512 in the course problem.
- Martian problem: Parity remains even for all three Martian colors after any number of transformations.The solution uses the initial even counts and the parity behavior of each transformation.
- Martian problem: 2022 red, 2022 green, and 2022 blue Martians are possible after finitely many transformations.The stated conclusion follows because 2022 is even in every color.
- Martian problem: 2021 red and 2023 blue Martians are impossible because both target counts are odd.The solution states that each color count remains even, ruling out both odd targets.
- Stone game: Penny wins the 5000-stone game because the solution identifies the 5000th move as the final move and assigns even-numbered moves to Penny.The supplied solution assumes the game lasts 5000 moves.
- Course problem: The course-combination argument concludes that the maximum number of selected students is k=512.It models nonempty course combinations as graph vertices and uses a dominating-set formulation.
Qwen2-Math-72B-Instruct Solution
The solutions address integer-length conditions in a right triangle and an orthocenter angle construction. They conclude that the listed triangle segments cannot all be integral and that ∠CGH=90°.
- Integer-length triangle: The triangle setup makes BD and CE angle bisectors, so their intersection I is the incenter of right triangle ABC.The construction is then analyzed using coordinates and incenter distances.
- Integer-length triangle: An example with a=3 and b=4 is given, yielding AB=3, AC=4, BC=5, and BI=CI=5.The cited passage presents these values while discussing the integer-length conditions.
- Integer-length triangle: The solution concludes that AB, AC, BI, ID, CI, and IE cannot all have integer lengths simultaneously.Its preceding argument imposes integer conditions on the legs and derived segment lengths.
- Orthocenter angle: The orthocenter construction defines D, E, and F as altitude feet and G as the intersection of the line through D parallel to AB with EF.H is the orthocenter lying on the three altitudes.
- Orthocenter angle: ∠CGH=90° in the stated orthocenter configuration.The derivation uses parallel-line angle relations and the cyclic quadrilateral AEHF.
B PROMPTS USED IN THE EVALUATION
The evaluation appendix specifies prompts for base and instruct models across English and Chinese mathematics tasks. The examples include direct-answer, chain-of-thought, multiple-choice, and tool-integrated-reasoning formats.
- Prompt coverage: Figures 5–10 provide prompts for evaluating the base models, while Figures 11–14 cover instruct-model reasoning prompts.The instruct prompts include Chain-of-Thought Reasoning and Tool-Integrated Reasoning.
- Example tasks: The examples span English and Chinese mathematical questions, including arithmetic, algebra, sequences, propositions, radicals, and trigonometry.The passages show both English and Chinese prompt-answer examples.
- Base-model prompts: The base-model examples include GSM8K, MATH, MMLU STEM, CMATH, GaoKao Math Cloze, and GaoKao Math QA prompts.These prompts are identified by the corresponding figure captions and examples.
- Instruct-model prompts: The instruct-model prompts include zero-shot Chain-of-Thought, Tool-Integrated Reasoning, MMLU-STEM, and multiple-choice GaoKao evaluations.The relevant prompt captions are assigned to Figures 11–14.