PaperlayerView exact revision on arXiv

Source-linked AI summary

MuMath-Code: Combining Tool-Use Large Language Models with Multi-perspective Data Augmentation for Mathematical Reasoning

Shuo Yin, Weihao You, Zhilong Ji, Guoqiang Zhong, Jinfeng Bai

arXiv:2405.07551v1cs.CLcs.AI

TL;DR

The paper addresses the limited integration of math-question augmentation and external tool use for open LLMs. It combines multi-perspective augmented questions, code-nested solutions, and two-stage training; MuMath-Code reports strong results across math benchmarks.

  • Problem

    Few methods had combined data augmentation and external tool use, two approaches used to improve open LLMs’ mathematical reasoning.

    Source: §1 · p. 2

  • Method

    MuMath-Code pairs multi-perspective augmented math questions with code-nested solutions and trains first on pure CoT data, then on code-nested data.

    Source: §1 · pp. 2–3

  • Results

    MuMath-Code-70B scores 90.7% on GSM8K and 55.1% on MATH, with reported state-of-the-art performance among open LLMs across in-domain and out-of-domain datasets.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Ablation studies support the effectiveness of prefix CoT, code debugging, and pseudo-answer guidance filtering in data synthesis.

    Source: §6 · p. 10

    LLMs have inherent logical-reasoning and numerical-computation limitations; solving math solely with their capabilities requires substantially more data than tool-use methods.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

The tool-use Large Language Models (LLMs) that integrate with external Python interpreters have significantly enhanced mathematical reasoning capabilities for open-source LLMs, while tool-free methods chose another track: augmenting math reasoning data. However, a great method to integrate the above two research paths and combine their advantages remains to be explored. In this work, we firstly include new math questions via multi-perspective data augmenting methods and then synthesize code-nested solutions to them. The open LLMs (i.e., Llama-2) are finetuned on the augmented dataset to get the resulting models, MuMath-Code ($\mu$-Math-Code). During the inference phase, our MuMath-Code generates code and interacts with the external python interpreter to get the execution results. Therefore, MuMath-Code leverages the advantages of both the external tool and data augmentation. To fully leverage the advantages of our augmented data, we propose a two-stage training strategy: In Stage-1, we finetune Llama-2 on pure CoT data to get an intermediate model, which then is trained on the code-nested data in Stage-2 to get the resulting MuMath-Code. Our MuMath-Code-7B achieves 83.8 on GSM8K and 52.4 on MATH, while MuMath-Code-70B model achieves new state-of-the-art performance among open methods -- achieving 90.7% on GSM8K and 55.1% on MATH. Extensive experiments validate the combination of tool use and data augmentation, as well as our two-stage training strategy. We release the proposed dataset along with the associated code for public use.

1 Introduction

The paper addresses the limited integration of math-question augmentation and external tool use for open LLMs. It combines multi-perspective augmented questions with code-nested solutions and two-stage training, reporting strong results on GSM8K and MATH.

1 Introduction

  • Few methods had combined the individually successful paths of tool use and data augmentation for mathematical reasoning.Source: §1 · p. 2
  • Training first strengthens pure-language mathematical reasoning, then teaches code generation so MuMath-Code can use Python execution during inference.The first stage uses MuMath-Data; the second trains on MuMath-Code-Data, and inference returns execution results for subsequent reasoning or code generation.Source: §1 · pp. 2–3
  • MuMath-Code-Data combines multi-perspective augmented math questions with code-nested solutions, using prefix CoT, code debugging, and pseudo-answer guidance filtering.Prefix CoT provides natural-language analysis before code generation; GPT-4 debugs inexecutable code, and majority-voting answers guide filtering when altered questions lack ground truth answers.Source: §1 · pp. 2–3
  • MuMath-Code-7B scores 83.8 on GSM8K and 52.4 on MATH, while MuMath-Code-70B scores 90.7% on GSM8K and 55.1% on MATH.The paper reports state-of-the-art performance among open LLMs across in-domain and out-of-domain math reasoning datasets.Source: §1 · p. 3

2 Related Work

The related work contrasts data augmentation without tools against tool-use methods that often omit question augmentation. MuMath-Code combines these directions with diverse question generation and additional solution-quality procedures.

2.1 Tool-Free LLMs for Math

  • Tool-free methods expand training questions and solutions, but LLM limits in logical reasoning and numerical computation mean relying only on model capabilities requires substantially more data than tool-use methods.MuMath-Code adopts MuMath’s multi-perspective augmentation, including rephrasing, alteration, and other methods; altered questions can lack ground-truth answers for filtering.Source: §2.1 · p. 3

2.2 Tool-Use LLMs for Math

  • PAL and PoT use external tools, while MAmmoTH separates CoT and PoT by sample and ToRA interleaves them; these methods do not augment questions.MathCoder also interleaves code and reasoning and adds interpolation problems, but its new questions are narrower in scope and diversity than MuMath-Code’s multi-perspective augmentation.Source: §2.2 · pp. 3–4
  • MuMath-Code interleaves Python code and natural-language reasoning, adding prefix CoT, code debugging, and pseudo-answer filtering alongside multi-perspective question augmentation.The paper states that its broader question diversity exposes models to a wider scope of novel questions and enhances generalization.Source: §2.2 · p. 4

3 Preliminaries

The preliminaries introduce augmented question sets and two corresponding datasets: one for pure natural-language mathematical reasoning and one for code-based tool interaction. The question set combines original questions with several transformations.

3 Preliminaries

  • The method uses MuMath-augmented questions and synthesizes code-nested solutions, with prefix CoT, code debugging, pseudo-answer filtering, and two-stage training to support multi-turn reasoning and tool interaction.Source: §3 · p. 4

3.1 MuMath Augmented Questions

  • Augmentation starts from GSM8K and MATH training questions and includes rephrasing, five alteration types, expression replacement, FOBAR reverse questions, and BF-Trans backward-forward questions.Alteration and expression replacement can change a question’s intrinsic meaning, so their answer is not definitely known; FOBAR masks a condition and uses the original answer as a new condition, while BF-Trans reframes that backward question as a direct request.Source: §3.1 · p. 4
  • The resulting question set combines the original, rephrased, altered, replacement, FOBAR, and BF-Trans sets to generate separate reasoning and code-interaction datasets.Source: §3.1 · p. 4

3.2 MuMath-Data

  • MuMath-Data contains about 750K pure-CoT samples, while majority sampling retains solutions with the prevailing answer for altered and replacement questions lacking reference answers.Each majority answer serves as a pseudo-answer for its question.Source: §3.2 · p. 5

4 Methodology

MuMath-Code-Data combines natural-language reasoning, Python code, and execution outputs in multi-turn solutions, with prefix CoT, debugging, and pseudo-answer guidance used to shape the data. Training proceeds from pure CoT reasoning to interleaved code-and-CoT data, with execution outputs masked in Stage 2.

4.1 MuMath-Code-Data

  • MuMath-Code-Data solutions interleave natural-language reasoning, Python code, and execution outputs across multiple turns.The synthesized paths are built by extracting and executing code, appending each result, and iterating when correction is needed.Source: §4.1 · p. 5
  • Prefix CoT adds a thorough natural-language analysis before code generation to help models consider the analysis while writing code.The prompt asks for relevant knowledge points that benefit problem solving.Source: §4.1 · p. 5
  • When generated code fails, the synthesis process prompts GPT-4 to debug it, retains the failed code and error information, and repeats until execution succeeds or requests run out.The retained failures are intended to equip finetuned models with debugging ability.Source: §4.1 · pp. 5–6
  • Pseudo-answer guidance uses majority-sampled answers for augmented questions without reference answers to select code-nested solutions.The authors say this improves synthesized-solution correctness and overall augmented-data quality.Source: §4.1 · p. 6

4.2 Two-Stage Training

  • Stage 1 trains on MuMath-Data for pure CoT mathematical reasoning, which the authors describe as a preliminary task for Stage 2.The stage is intended to endow models with fairly strong mathematical reasoning capability.Source: §4.2 · p. 6
  • Stage 2 trains on interleaved PoT-CoT data for Python-interpreter interaction and masks execution-output losses because those outputs should not be learned.The authors state that this stage simulates inference, where generated code is executed and the returned result is used for continued generation.Source: §4.2 · p. 6

5 Experiments

Across the reported scales and datasets, MuMath-Code outperforms open-source baselines, while ablations support the two-stage strategy and three data-synthesis methods. The experiments also find continued gains as training data volume increases in single-stage scaling tests.

5.1 Experimental Setup

  • The experiments use GSM8K and MATH seeds, 750K MuMath examples for Stage 1, and 600K filtered code-nested samples for Stage 2.LLaMA and CodeLlama model sizes and training implementations are also specified in the experimental setup.Source: §5.1 · pp. 6–7

5.2 Comparison Results

  • MuMath-Code-L 7B scores 83.8 on GSM8K, and MuMath-Code-CL 7B scores 52.4 on MATH; the approach outperforms open-source models across scales and datasets.On MATH, MuMath-Code-CL 34B scores 55.0 and 70B scores 55.1.Source: §5.2 · p. 7, Table 1 · p. 8

5.5 Scaling Study

  • All tested MuMath-Code-Data subsets continue to improve performance as sample volume increases, with no observed saturation in the single-stage LLaMA 7B experiments.For two-stage scaling, the passage points readers to Appendix B.Source: Figure 3 · p. 9, §5.5 · p. 9

5.3 Effectiveness of the Two-Stage Training Strategy

  • Two-stage training outperforms training on Dµ-code alone and one-stage training that merges both stages’ data; MetaMath and Xwin-Math initial checkpoints also outperform single-stage Llama training.The comparison varies Dµ-code data volume; the alternative first-stage checkpoints demonstrate compatibility with different CoT datasets.Source: §5.3 · p. 7, Table 2 · p. 9

5.4 Ablation Studies

  • Removing prefix CoT or code debugging from the synthetic solutions lowers test accuracy.The ablations remove preliminary analysis before code writing, or retain only final successfully executed code and omit debugging attempts.Source: §5.4 · p. 7, Table 3 · p. 9
  • Pseudo-answer-filtered training data performs better than random sampling across data volumes from 30K to 180K.Pseudo-answers are used for questions without definitive correct answers, including Qalter and Qreplace.Source: Table 4 · p. 9, §5.4 · p. 9

6 Conclusion

MuMath-Code-Data combines multi-perspective math questions with solutions containing natural-language analysis and multi-turn code generation and execution. The paper reports that two-stage-trained models outperform open methods and some proprietary models across in-domain and out-of-domain reasoning datasets.

6 Conclusion

  • MuMath-Code-Data pairs multi-perspective math questions with solutions containing multi-turn code generation, code execution, and natural-language analysis.The paper describes the dataset as integrating code into mathematical reasoning solutions.Source: §6 · p. 10
  • Two-stage-trained MuMath-Code models outperform state-of-the-art open methods and some proprietary models across in-domain and out-of-domain reasoning datasets.The conclusion names GSM8K and MATH as in-domain datasets.Source: §6 · p. 10
  • Ablation studies support the effectiveness of prefix CoT, code debugging, and pseudo-answer guidance filtering in data synthesis.Source: §6 · p. 10

A Examples

The example solves a system of congruences by using Python code and revising errors encountered during execution. The resulting smallest whole number is 37.

A Examples

  • The example frames the problem as finding a positive integer satisfying three congruences and proposes using Sympy’s solve_congruence function.Source: A Examples · p. 13
  • The worked solution demonstrates debugging an undefined function and correcting string inputs to integers before execution.Source: A Examples · pp. 13–14
  • The corrected program returns 37, the smallest whole number meeting the stated remainder conditions.Source: A Examples · pp. 13–14

B Additional Experimental Results

Table 5 compares training on Dµ-code alone with two-stage training across data volumes; the appendix also reports that scaling the MuMath-Code-Data subsets yields trends similar to Figure 3.

B Additional Experimental Results

  • Training solely on Dµ-code is consistently inferior to two-stage training across all data volumes.The comparison is reported for varied Dµ-code volumes.Source: Table 5 · p. 14
  • Table 5 varies the data volumes of Dµ-code.The passage does not specify the individual volumes.Source: Table 5 · p. 14
  • After finetuning on MuMath-Data, scaling all MuMath-Code-Data subsets produces curves with trends similar to Figure 3.The passage describes the trends as very similar.Source: Figure 4 · p. 15

Related papers

Loading 2405.07551v1…