PaperlayerView exact revision on arXiv

Source-linked AI summary

MathScale: Scaling Instruction Tuning for Mathematical Reasoning

Zhengyang Tang, Xingxing Zhang, Benyou Wang, Furu Wei

arXiv:2403.02884v1cs.CLcs.AIcs.LG

TL;DR

LLMs remain inadequate at mathematical problem solving, while available reasoning datasets are limited and evaluation coverage is uneven. MathScale generates concept-diverse training data through concept extraction and graph-based question generation, and MathScaleQA improves open-source LLM performance on MWPBENCH. MathScale-7B surpasses equivalent-size peers by 42.9% in micro average accuracy and 43.7% in macro average accuracy.

  • Problem

    LLMs remain inadequate at mathematical problem solving, and existing reasoning datasets are limited while common evaluations leave gaps across difficulty levels.

  • Method

    MathScale extracts concepts from seed questions, builds a concept graph, and uses sampled concepts to generate new questions; the resulting data trains open-source LLMs and MWPBENCH evaluates them across datasets.

  • Results

    MathScale-7B achieves state-of-the-art performance across MWPBENCH, surpassing equivalent-size peers by 42.9% in micro average accuracy and 43.7% in macro average accuracy.

  • Takeaways & Limitations

    MathScale provides a scalable source of mathematical reasoning data and a broad, standardized benchmark for comparing instruction-tuned LLMs.

  • Takeaways & Limitations

    The authors note a small possibility that MWPBENCH test examples leaked into GPT-3.5-Turbo or LLaMA-2 training data despite decontamination.

Abstract

from arXiv · show

Large language models (LLMs) have demonstrated remarkable capabilities in problem-solving. However, their proficiency in solving mathematical problems remains inadequate. We propose MathScale, a simple and scalable method to create high-quality mathematical reasoning data using frontier LLMs (e.g., {\tt GPT-3.5}). Inspired by the cognitive mechanism in human mathematical learning, it first extracts topics and knowledge points from seed math questions and then build a concept graph, which is subsequently used to generate new math questions. MathScale exhibits effective scalability along the size axis of the math dataset that we generate. As a result, we create a mathematical reasoning dataset (MathScaleQA) containing two million math question-answer pairs. To evaluate mathematical reasoning abilities of LLMs comprehensively, we construct {\sc MwpBench}, a benchmark of Math Word Problems, which is a collection of ten datasets (including GSM8K and MATH) covering K-12, college, and competition level math problems. We apply MathScaleQA to fine-tune open-source LLMs (e.g., LLaMA-2 and Mistral), resulting in significantly improved capabilities in mathematical reasoning. Evaluated on {\sc MwpBench}, MathScale-7B achieves state-of-the-art performance across all datasets, surpassing its best peers of equivalent size by 42.9\% in micro average accuracy and 43.7\% in macro average accuracy, respectively.

1. Introduction

MathScale addresses limited mathematical reasoning data by extracting concepts from seed questions, building a concept graph, and generating new questions with frontier LLMs. The resulting MathScaleQA dataset supports improved reasoning performance for open-source models, while MWPBENCH broadens evaluation across mathematical levels and datasets.

  • Limited mathematical reasoning datasets constrain instruction tuning, with GSM8K and MATH each containing around 7.5K training examples.
  • MathScale extracts topics and knowledge points from seed questions, builds a concept graph, and generates questions from sampled concept combinations.This reduces dependence on original questions and supports greater generation diversity than augmentation-based methods.
  • MWPBENCH combines ten math word problem datasets spanning elementary school through college and competition-level difficulty.
  • MWPBENCH standardizes evaluation across datasets with a unified protocol to promote consistent and fair model comparisons.
  • 2 million math question-answer pairs comprise MathScaleQA, which is used to fine-tune open-source LLMs and improve mathematical reasoning.

2. MWPBENCH Evaluation Framework

MWPBENCH addresses gaps in mathematical-reasoning evaluation by covering college-level mathematics and using a unified protocol across varied datasets. It combines established datasets with CollegeMath and evaluates instruction-tuned LLMs in a zero-shot accuracy setting.

  • Existing Datasets: Existing datasets span different problem types, including math word problems and multiple-choice questions.
  • CollegeMath: CollegeMath fills a coverage gap by targeting college-level mathematics, which requires analytical thinking, logical reasoning, and quantitative analysis.
  • CollegeMath: CollegeMath is curated from nine textbooks covering algebra, pre-calculus, calculus, vector calculus, probability, linear algebra, and differential equations.
  • Inconsistent evaluation metrics and protocols across prior work make benchmarking LLM mathematical reasoning difficult.
  • MWPBENCH Evaluation Protocol: MWPBENCH uses a unified zero-shot evaluation protocol with accuracy as the metric and the Alpaca template as the default prompt format.

3. MathScale: Scaling Instruction Tuning for Mathematical Reasoning

MathScale builds a scalable mathematical-reasoning data pipeline from seed questions by extracting concepts, composing them through a graph, and prompting GPT-3.5 to generate new question-answer pairs. The pipeline produces MathScaleQA after decontamination and validation choices are applied.

  • 3. MathScale: Scaling Instruction Tuning for Mathematical Reasoning: MathScale is presented as a four-step process for generating a large-scale mathematical reasoning dataset by prompting ChatGPT.
  • 3.1. Concept Extraction: The pipeline begins with seed questions and extracts topics and knowledge points as mathematical metadata using GPT-3.5.Topics are broad subject or chapter names, while knowledge points are finer-grained concepts such as theorems and problem-solving skills.
  • 3.1. Concept Extraction: The extraction process uses diverse MWPBENCH training questions and removes concepts appearing only once, yielding around 2K topics and 8K knowledge points.
  • 3.2. Concept Graph Construction: The concept graph contains topic and knowledge-point nodes connected by topic-topic, topic-KP, and KP-KP edges based on co-occurrence.Its edges form topic, topic-KP, and KP sub-graphs; formally, edges are retained when the adjusted co-occurrence weight is positive.
  • 3.2. Concept Graph Construction: MathScale adjusts raw co-occurrence counts with a small constant and samples concept compositions through graph random walks.The process samples topics, walks one to two steps through the topic graph, samples a knowledge point through the hybrid graph, and then expands through the KP graph for zero to four steps.
  • 3.3. Mathematical Reasoning Data Generation: The sampled topics and knowledge points are supplied to GPT-3.5, which generates question-answer pairs using few-shot seed examples selected by knowledge-point similarity.The prompts ask GPT-3.5 to follow the sampled topics and combine the sampled knowledge points.
  • 3.3. Mathematical Reasoning Data Generation: The generated data undergoes decontamination by removing all MWPBENCH test-set questions before use.
  • 3.4. Validation: A GPT-4 validation step was tested because some generated solutions were incorrect, but it did not improve results and was removed from the final pipeline.The authors state that some incorrect solutions remained useful for learning GPT-3.5 model distributions.

4. Experiments

The experiments generate MathScaleQA through concept-based synthesis, fine-tune several open-source backbones, and evaluate them against diverse mathematical reasoning models on MWPBENCH. MathScale-7B achieves the strongest average performance among equivalent-size open-source models.

  • Data Generation: 2 million unique concept compositions are converted into question-answer pairs, then combined with MWPBENCH training data to form MathScaleQA.The pipeline extracts 2,018 topics and 8,892 knowledge points from around 20K seed questions and excludes MWPBENCH test questions during decontamination.
  • Model Training: MathScale models use LLaMA-2 7B, LLaMA-2 13B, and Mistral 7B backbones, trained on MathScaleQA for 3 epochs.Training uses batch size 128 and learning rate 2e-5.
  • Evaluation: All models are evaluated on MWPBENCH using the Alpaca prompt and reported with individual-dataset, micro-average, and macro-average accuracies.The benchmark contains 10 mathematical reasoning datasets, and comparisons use chain-of-thought natural-language solutions.
  • Main Results: 35.0% micro-average and 37.5% macro-average accuracy make MathScale-7B the strongest equivalent-size model on MWPBENCH.It surpasses its best equivalent-size peers by 42.9% and 43.7% on the two metrics, respectively.
  • Main Results: MathScale-7B also outperforms other open-source models on test sets without training data, while MathScale-Mistral matches GPT-3.5-Turbo on both average metrics.The training-data-free evaluations are GaokaoBench-Math, AGIEval-Gaokao-MATH, and AGIEval-SAT-MATH.

5. Analysis and Discussions

MathScale’s performance grows nearly logarithmically as MathScaleQA expands, while ablations show that seed diversity and knowledge-point coverage materially affect results. Validation experiments question the value of correcting synthetic solutions, and leakage remains a limitation despite decontamination.

  • Scaling Property: Performance grows nearly logarithmically across MWPBENCH datasets as MathScaleQA scales in size.The scaling study uses LLaMA-2 7B as the base model.
  • Ablation on Concept Extraction: Using only 10K seed questions lowers MWPBENCH macro average by 2.9%, while restricting seeds to GSM8K and MATH lowers it by 3.5%.These results support using a larger and more diverse seed-question pool.
  • Ablation on Concept Extraction: Removing 50% of knowledge points decreases macro average by 8.6%, compared with a 2.3% decrease when removing 50% of topics.The comparison highlights the stronger contribution of knowledge points in the concept-extraction pipeline.
  • On Validating Generated Data: GPT-4 identifies 26% of solutions in a 5K sample as incorrect, yet using original GPT-3.5 solutions produces similar results to validation and correction.The authors therefore omit validation and correction from the final pipeline.
  • Limitations: Although MathScaleQA is decontaminated against MWPBENCH tests, leakage into GPT-3.5-Turbo or LLaMA-2 training data cannot be ruled out.This leaves a residual threat to the independence of benchmark evaluation.
  • Performance on a Fresh Math Dataset: MathScale consistently surpasses WizardMath and MetaMath on Fresh-GaokaoMath-2023, supporting performance on newly collected questions.The comparison uses LLaMA-2 7B-based models and also includes GPT-3.5-Turbo and GPT-4.

6. Related Work

Related work uses ChatGPT-based data synthesis and, in some cases, tool integration for mathematical instruction tuning. MathScale differs by focusing solely on natural-language reasoning rather than program-based tool usage.

  • ChatGPT-based Instruction Tuning: WizardMath and MetaMath use ChatGPT-based strategies to synthesize or evolve mathematical instruction-tuning data.WizardMath applies reinforced evol-instruct, while MetaMath bootstraps questions through multiple transformations.
  • Tool-Integration Instruction Tuning: ToRA integrates natural-language reasoning with program-based tool usage to synthesize trajectory data.Its trajectories concatenate reasoning, programming, and program outputs until reaching a final answer.
  • Tool-Integration Instruction Tuning: MathScale focuses solely on natural-language reasoning and leaves tool integration for future research.The paper presents tool integration as an unexplored extension of its pipeline.

7. Conclusions

The paper introduces MathScale for scalable synthetic mathematical-reasoning data generation and MWPBENCH for broad, unified evaluation. MathScale-7B achieves state-of-the-art performance across the benchmark and substantially exceeds equivalent-size peers.

  • Contributions: MathScale generates high-quality mathematical reasoning data with a simple, scalable method using frontier LLMs.The method is paired with MWPBENCH, a benchmark spanning K-12, college, and competition-level math word problems.
  • Results: 42.9% higher micro average accuracy and 43.7% higher macro average accuracy distinguish MathScale-7B from its best equivalent-size peers on MWPBENCH.The paper reports state-of-the-art performance across all datasets.

Broader Impact

The paper aims to advance mathematical reasoning with scalable synthetic data and unified evaluation, while acknowledging that model biases were not examined. It identifies bias assessment and alignment with societal values as important future concerns.

  • Scope and Risks: The study does not examine potential model biases in its mathematical-capability evaluation.The broader-impact discussion flags bias assessment as outside the study’s scope.
  • Scope and Risks: The paper identifies addressing biases and ensuring alignment with societal values as essential concerns.These concerns accompany the proposed use of synthetic data and evaluation benchmarks.

A.1. MWPBENCH: Transform Non-Word Problems into Word Problems

MWPBENCH converts selected non-word math problems into word problems to reduce problem-format effects and focus evaluation on mathematical problem-solving ability.

  • Multiple-choice problems from TAL-SCQ, GaokaoBench-Math, and AGIEval were converted into word problems.The conversion targets the influence of problem type on evaluation.
  • The transformation is intended to concentrate measurement on LLMs’ intrinsic ability to address mathematical problems.
  • The resulting benchmark represents these datasets in a unified word-problem format.

A.1.1. FILTERING QUESTIONS

The appendix describes preprocessing steps for constructing a unified mathematical reasoning corpus, including filtering multiple-choice items, normalizing languages and formats, and extracting textbook questions and answers. It also reports topic-level evaluation patterns, including strong out-of-distribution performance but limited success on differential equations.

  • FILTERING QUESTIONS: Questions relying heavily on multiple-choice formatting were identified with indicative keywords and filtered out.The filtering procedure includes checking each question for predefined keyword matches.
  • FILTERING QUESTIONS: Remaining multiple-choice items were paired with correct answer choices and reformatted as word problems followed by solutions.
  • DATA NORMALIZATION: Chinese problems from four datasets were translated into English with GPT-3.5-Turbo, with a few examples excluded after parsing errors.
  • COLLEGEMATH CONSTRUCTION: CollegeMath was built by using GPT-3.5-Turbo to extract questions and answers from raw, segmented LaTeX textbook exercises and solutions.The extraction produced a comprehensive question-answer collection forming the dataset’s foundation.
  • TOPIC-LEVEL EVALUATION: MathScale performs robustly on out-of-distribution vector calculus, probability, and linear algebra tests, while differential equations remain challenging for all models.
Loading 2403.02884v1…